[client] Expose Arrow changelog batches

[jordepic]

Generated-by: Codex (GPT-5) following the guidelines

Purpose

Linked issue: close #4005

Expose the Arrow representation that Fluss already stores in ARROW primary-key log batches through the Java client, including the per-row changelog sidecar.

StreamFusion needs this path for its Flink Fluss source so live-log batches can remain Arrow-backed instead of being materialized as RowData first. Snapshot records can still use the normal conversion path initially; this PR provides the direct Arrow path for the unbounded changelog.

Brief change log

• Promote LogScanner.pollRecordBatch and its result types to public evolving APIs.
• Decode and retain the stored change-type vector for primary-key Arrow batches.
• Support existing server-side non-empty top-level projection pushdown on the Arrow batch path.
• Expose per-row ChangeType access and a read-only view of the raw sidecar.
• Keep Arrow rows and change types aligned across projection, offset slicing, and truncation.
• Use a default unsupported scanner implementation to preserve compatibility for existing LogScanner implementations.
• Remove the internal implementation cast from the Flink tiering reader.

Tests

• Added LogScannerITCase#testPollArrowBatchesWithPrimaryKeyChangelog covering insert, update-before, update-after, delete, projected Arrow columns, and mid-batch offset slicing.
• Ran the new test together with testPollArrowBatchesWithSchemaEvolution: 2 tests passed.
• Built fluss-flink-common and its dependencies with test compilation enabled: success.
• Spotless and Checkstyle passed for affected modules as part of the Maven builds.

API and Format

This adds public-evolving Java client APIs for Arrow batch polling and changelog access. Existing LogScanner implementations remain compatible through a default method.

There is no storage-format change: the implementation exposes the change-type vector already stored before the Arrow IPC payload in primary-key log batches and reuses the existing server-side Arrow projection path.

Caller-provided Arrow allocator support is intentionally left for follow-up because allocator ownership and lifecycle need a separate API decision.

Documentation

Public API Javadocs describe format restrictions, ownership, close requirements, and changelog behavior. No user-facing storage or configuration documentation changes are required.

[wuchong]

cc @luoyuxia, do you think we need anything else if we want to expose ArrowBatch as a public API in LogScanner?

[woodsbrandon044-hash]

cc @luoyuxia, do you think we need anything else if we want to expose ArrowBatch as a public API in LogScanner?

Who you talking about

[Copilot]

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Expose Arrow record-batch polling (including per-row changelog sidecar for PK tables) through the Java client so downstream consumers can keep live log batches Arrow-backed.

Changes:

• Promote Arrow batch polling APIs to public-evolving (LogScanner#pollRecordBatch, ArrowScanRecords, ArrowBatchData) and remove internal implementation casts.
• Decode and propagate stored per-row change types for primary-key Arrow batches, keeping them aligned across slicing/truncation.
• Add an integration test covering PK changelog Arrow batches, projection, and mid-batch offset slicing.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/tiering/source/TieringSplitReader.java	Stops downcasting to LogScannerImpl, relying on the new public API.
fluss-common/src/main/java/org/apache/fluss/record/LogRecordReadContext.java	Extends Arrow batch creation to accept optional change-type sidecar.
fluss-common/src/main/java/org/apache/fluss/record/DefaultLogRecordBatch.java	Reads changelog sidecar bytes for non-append-only Arrow batches and passes them through.
fluss-common/src/main/java/org/apache/fluss/record/ArrowRecordBatchContext.java	Updates internal Arrow batch access interface to carry optional change types.
fluss-common/src/main/java/org/apache/fluss/record/ArrowBatchData.java	Makes Arrow batches public-evolving and adds change-type access + slicing propagation.
fluss-client/src/test/java/org/apache/fluss/client/table/scanner/log/LogScannerITCase.java	Adds IT coverage for PK changelog Arrow batches, projection, and offset slicing; removes LogScannerImpl casts.
fluss-client/src/main/java/org/apache/fluss/client/table/scanner/log/LogScannerImpl.java	Implements the now-public Arrow polling method and relaxes prior restrictions.
fluss-client/src/main/java/org/apache/fluss/client/table/scanner/log/LogScanner.java	Adds default pollRecordBatch method to keep compatibility for existing implementations.
fluss-client/src/main/java/org/apache/fluss/client/table/scanner/log/ArrowScanRecords.java	Promotes Arrow scan records wrapper to public-evolving API.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jordepic]

Resolved the Copilot review comments in a7e9371:\n\n- Clarified Arrow 15 slice ownership: VectorSchemaRoot.slice already uses per-vector splitAndTransfer, so the sliced root remains valid after closing the original.\n- Reworked Arrow payload length calculation to use batch-relative values.\n- Clarified the default unsupported LogScanner implementation error and Javadoc.\n- Kept changelog test offsets as long values without narrowing.\n- Confirmed UpsertWriter is not AutoCloseable; its shared WriterClient is owned and closed by FlussConnection.\n\nValidation:\n- FileLogInputStreamTest: 9 passed\n- LogScannerITCase: 13 passed\n- Spotless, Checkstyle, RAT, and affected reactor compilation passed\n\nAll six review threads have been replied to and resolved.