Search before asking
Description
Motivation
The Rust client supports predicate pushdown through TableScan::filter(), allowing the Fluss server to prune Arrow log RecordBatches using column statistics.
However, the C++ binding does not expose the predicate model or a filter API Currently. Native C++ consumers therefore cannot use this capability and must fetch log batches without filter-based pruning.
The C++ binding should provide predicate semantics consistent with the Rust client. This also enables native integrations such as StarRocks to use predicate pushdown without going through JNI.
This work builds on #3971.
Solution
Expose typed predicate builders and add TableScan::Filter() to the C++ binding.
Example:
fluss::LogScanner scanner;
auto predicate =
fluss::Col("amount")
.GreaterOrEqual(100)
.And(fluss::Col("region").In({"CN", "SG"}));
auto result = table.NewScan()
.Filter(std::move(predicate))
.ProjectByName({"order_id", "amount"})
.CreateRecordBatchLogScanner(scanner);
The C++ API should support:
- comparison predicates:
=, !=, <, <=, >, and >=;
IS NULL and IS NOT NULL;
IN and NOT IN;
- string prefix, infix, and suffix predicates;
AND and OR;
- scalar literal types supported by the Rust predicate model;
- predicate pushdown together with column projection.
The C++ predicate tree will be passed through the FFI and reconstructed as a Rust Core Predicate. The Rust client remains responsible for schema validation and protocol encoding through TableScan::filter().
Filters should be rejected by scanners that do not support predicate pushdown rather than being silently ignored.
Predicate pushdown is conservative and operates at RecordBatch level. Returned batches may still contain non-matching rows, so callers must retain and evaluate the original predicate as a residual filter.
This feature is limited to Arrow log scans. Row-level filtering, primary-key lookup, lake reads, and UnionRead are outside the scope of this issue.
Anything else?
The change should include C++ integration tests covering predicate pushdown with projection, invalid filter columns, and unsupported scanner types. The C++ API reference and log-table examples should also be updated.
Willingness to contribute
Search before asking
Description
Motivation
The Rust client supports predicate pushdown through
TableScan::filter(), allowing the Fluss server to prune Arrow log RecordBatches using column statistics.However, the C++ binding does not expose the predicate model or a filter API Currently. Native C++ consumers therefore cannot use this capability and must fetch log batches without filter-based pruning.
The C++ binding should provide predicate semantics consistent with the Rust client. This also enables native integrations such as StarRocks to use predicate pushdown without going through JNI.
This work builds on #3971.
Solution
Expose typed predicate builders and add
TableScan::Filter()to the C++ binding.Example:
The C++ API should support:
=,!=,<,<=,>, and>=;IS NULLandIS NOT NULL;INandNOT IN;ANDandOR;The C++ predicate tree will be passed through the FFI and reconstructed as a Rust Core
Predicate. The Rust client remains responsible for schema validation and protocol encoding throughTableScan::filter().Filters should be rejected by scanners that do not support predicate pushdown rather than being silently ignored.
Predicate pushdown is conservative and operates at RecordBatch level. Returned batches may still contain non-matching rows, so callers must retain and evaluate the original predicate as a residual filter.
This feature is limited to Arrow log scans. Row-level filtering, primary-key lookup, lake reads, and UnionRead are outside the scope of this issue.
Anything else?
The change should include C++ integration tests covering predicate pushdown with projection, invalid filter columns, and unsupported scanner types. The C++ API reference and log-table examples should also be updated.
Willingness to contribute