feat(go): add filter push-down and predicate API for Go binding#216
Open
luoyuxia wants to merge 5 commits intoapache:mainfrom
Open
feat(go): add filter push-down and predicate API for Go binding#216luoyuxia wants to merge 5 commits intoapache:mainfrom
luoyuxia wants to merge 5 commits intoapache:mainfrom
Conversation
3608a37 to
4e3ace4
Compare
Add complete predicate support to the Go binding, enabling filter push-down for table scans. This includes: - C FFI layer: predicate construction functions (equal, not_equal, less_than, greater_than, is_null, is_in, etc.), combinators (and/or/not), with_filter on ReadBuilder, and fix scan to propagate filters via TableScanState - Go binding: Predicate type, Go-native value types (Date, Time, Timestamp, LocalZonedTimestamp, Decimal, Bytes), automatic type inference from Go literals (int, string, bool, etc.), and ReadBuilder.WithFilter method - Tests: TestReadWithFilter with shared test helpers Closes apache#215 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix empty string predicate rejection: allow str_data=null with str_len=0 in datum_from_c, matching Bytes branch semantics - Add integer datum auto-coercion in C FFI layer so Go int literals work on BIGINT columns (and vice versa with range checking) - Change predicate combinators to method style: pred.And(other), pred.Or(other), pred.Not() instead of package-level functions - Expand Decimal to full i128 via Lo/Hi int64 fields with NewDecimal convenience constructor - WithFilter(nil) is now a no-op instead of panicking - Add regression subtests for empty string predicates and nil filter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8e0e972 to
09d4d64
Compare
- Rename And/Or/Not to PredicateAnd/PredicateOr/PredicateNot - OrPredicate test: assert matching rows are present instead of exact count, since filter pushdown prunes at split level not row level Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TableReadState was missing data_predicates, so paimon_table_read_to_arrow reconstructed TableRead with an empty predicate set, bypassing the row-level parquet filtering added in apache#208. - Add data_predicates field to TableReadState - Add TableRead::data_predicates() accessor - Pass stored predicates when constructing TableRead in to_arrow - Restore exact row assertion in OrPredicate test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e1b69ee to
ceff52f
Compare
Compound predicates (Or/And/Not) are not yet supported at the reader level, so the OrPredicate test was silently returning all rows. Replace with a single-leaf EqualById test that exercises filter push-down with correct results. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ceff52f to
2f47899
Compare
JingsongLi
reviewed
Apr 7, 2026
Contributor
JingsongLi
left a comment
There was a problem hiding this comment.
Can you create a documentation for Go API?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add filter push-down and predicate API support to the Go binding, enabling Go callers to filter table scans with native Go values.
Linked issue: close #214
Brief change log
paimon_predicate,paimon_datum(tagged flat struct with Timestamp/Decimal/Bytes fields),ReadBuilderState.filter,TableScanStatepaimon_result_predicatewith_filter, fix scan to propagate filters viaTableScanState; add integer datum auto-coercion across integer-family types; fix empty string datum handlingPredicateBuildertype with short methods (Eq,NotEq,Lt,Le,Gt,Ge,IsNull,IsNotNull,In,NotIn);Predicate.And/Or/Notcombinator methods (return errors, no panics); Go value types (Date,Time,Timestamp,LocalZonedTimestamp,Decimalwith full i128,Bytes); automatic type inference from Go literals; FFI wrappersAnd/Orconvenience functions andNotwrapperReadBuilder.WithFiltermethod (nil-safe, no-op)Table.PredicateBuilder()factory methodTestReadWithFilterwith subtests, refactor shared test helpersTests
TestReadWithFilter/EqualById: filtersid = 1, verifies 1 matching rowTestReadWithFilter/EmptyStringEqual: verifiesEq("name", "")is accepted and returns 0 rowsopenTestTableandreadRowshelpersAPI and Format
Go API usage:
Documentation
N/A