Skip to content

[c++] Create C++ bindings for log predicate pushdown #4035

Description

@naivedogger

Search before asking

  • I searched in the issues and found nothing similar.

Description

Motivation

The Rust client supports predicate pushdown through TableScan::filter(), allowing the Fluss server to prune Arrow log RecordBatches using column statistics.

However, the C++ binding does not expose the predicate model or a filter API Currently. Native C++ consumers therefore cannot use this capability and must fetch log batches without filter-based pruning.

The C++ binding should provide predicate semantics consistent with the Rust client. This also enables native integrations such as StarRocks to use predicate pushdown without going through JNI.

This work builds on #3971.

Solution

Expose typed predicate builders and add TableScan::Filter() to the C++ binding.

Example:

fluss::LogScanner scanner;

auto predicate =
        fluss::Col("amount")
                .GreaterOrEqual(100)
                .And(fluss::Col("region").In({"CN", "SG"}));

auto result = table.NewScan()
                      .Filter(std::move(predicate))
                      .ProjectByName({"order_id", "amount"})
                      .CreateRecordBatchLogScanner(scanner);

The C++ API should support:

  • comparison predicates: =, !=, <, <=, >, and >=;
  • IS NULL and IS NOT NULL;
  • IN and NOT IN;
  • string prefix, infix, and suffix predicates;
  • AND and OR;
  • scalar literal types supported by the Rust predicate model;
  • predicate pushdown together with column projection.

The C++ predicate tree will be passed through the FFI and reconstructed as a Rust Core Predicate. The Rust client remains responsible for schema validation and protocol encoding through TableScan::filter().

Filters should be rejected by scanners that do not support predicate pushdown rather than being silently ignored.

Predicate pushdown is conservative and operates at RecordBatch level. Returned batches may still contain non-matching rows, so callers must retain and evaluate the original predicate as a residual filter.

This feature is limited to Arrow log scans. Row-level filtering, primary-key lookup, lake reads, and UnionRead are outside the scope of this issue.

Anything else?

The change should include C++ integration tests covering predicate pushdown with projection, invalid filter columns, and unsupported scanner types. The C++ API reference and log-table examples should also be updated.

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions