Skip to content

branch-4.1: [chore](paimon) remove paimon-cpp dependency - #67378

Open
suxiaogang223 wants to merge 1 commit into
apache:branch-4.1from
suxiaogang223:codex/remove-paimon-cpp-phase1
Open

branch-4.1: [chore](paimon) remove paimon-cpp dependency#67378
suxiaogang223 wants to merge 1 commit into
apache:branch-4.1from
suxiaogang223:codex/remove-paimon-cpp-phase1

Conversation

@suxiaogang223

@suxiaogang223 suxiaogang223 commented Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: N/A

Related PR: #67385

Problem Summary:

This is phase one of removing paimon-cpp. It removes the source, build, link, reader, configuration, and test dependencies while keeping the existing Arrow prebuilt artifacts reusable.

  • Paimon logical splits now use the JNI reader; physical Parquet/ORC splits keep the existing native routing.
  • TQueryOptions.enable_paimon_cpp_reader field 201 and TPaimonReaderType.PAIMON_CPP remain as deprecated wire values. New FE plans no longer emit or set them.
  • Arrow validation keeps the current semantic fingerprint and accepts the existing legacy marker. Extra paimon-cpp files in an old prebuilt archive are ignored.
  • The existing Arrow patch remains in phase one so the shared prebuilt fingerprint does not change before replacement artifacts are published.
  • Release-branch Thrift rebuilds overlay their own source headers before compilation. This prevents the FE UT image's Thrift 0.24 headers from being mixed with branch-4.1's Thrift 0.16 sources.

Release note

Remove the paimon-cpp reader and third-party build dependency.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
      • DORIS_GCC_HOME=/usr ./run-fe-ut.sh --run org.apache.doris.datasource.paimon.source.PaimonScanNodeTest (45 tests passed)
    • Manual test (add detailed scripts or steps below)
      • Rebuilt Thrift 0.16 with clang 20 after injecting the Thrift 0.24 TOutput.h; build completed and selected Thrift 0.16.0.
      • thirdparty/test/arrow-lifecycle-test.sh
      • thirdparty/test/adbc-jni-config-test.sh
      • Shell syntax checks for changed build scripts
      • Workflow YAML parsing and git diff --check
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes. Paimon logical splits no longer select paimon-cpp and use JNI instead.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223 suxiaogang223 mentioned this pull request Sep 1, 2026
Closed
@suxiaogang223

suxiaogang223 commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

run buildall

@suxiaogang223 suxiaogang223 changed the title [chore](paimon) remove paimon-cpp dependency [branch-4.1][chore](paimon) remove paimon-cpp dependency Sep 1, 2026
@suxiaogang223 suxiaogang223 mentioned this pull request Sep 1, 2026
Open
5 tasks
@hello-stephen

hello-stephen commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.58% (25801/43307)
Line Coverage 44.18% (264914/599586)
Region Coverage 40.08% (210043/524094)
Branch Coverage 41.55% (96967/233356)

@hello-stephen

hello-stephen commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 74.32% (31317/42138)
Line Coverage 58.52% (348119/594841)
Region Coverage 55.22% (290107/525329)
Branch Coverage 56.06% (130685/233114)

@suxiaogang223 suxiaogang223 changed the title [branch-4.1][chore](paimon) remove paimon-cpp dependency branch-4.1: [chore](paimon) remove paimon-cpp dependency Sep 2, 2026
@suxiaogang223
suxiaogang223 marked this pull request as ready for review September 2, 2026 02:02
@suxiaogang223
[chore](paimon) remove paimon-cpp dependency
5478d34 
Remove the paimon-cpp source, build, reader, configuration, and test dependencies while retaining deprecated Thrift wire values for rolling-upgrade compatibility.

Keep existing Arrow prebuilts reusable, preserve the ADBC JNI lifecycle check from branch-4.1, and make release-branch Thrift rebuilds use their own headers when the build image contains a newer Thrift.
@suxiaogang223
suxiaogang223 force-pushed the codex/remove-paimon-cpp-phase1 branch from 79747a2 to 5478d34 Compare September 2, 2026 02:18
@suxiaogang223

suxiaogang223 commented Sep 2, 2026

Copy link
Copy Markdown
Member Author

run buildall

suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Sep 2, 2026
@suxiaogang223
[chore](paimon) Remove paimon-cpp dependency
6624286 
### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#67378

Problem Summary:

Doris still downloads, builds, links, and carries the deprecated paimon-cpp reader even though master plans serialized Paimon splits for the JNI reader. Remove the paimon-cpp thirdparty package, BE reader implementation, obsolete session-variable plumbing, and related tests. Keep the deprecated PaloInternalService field, TPaimonReaderType.PAIMON_CPP enum value, and IGNORE_PAIMON_CPP option for rolling-upgrade and configuration compatibility. Preserve Arrow as an independent shared thirdparty component so existing prebuilts remain usable and future automation artifacts no longer include paimon-cpp.

### Release note

Remove the deprecated paimon-cpp reader and its build dependency. Paimon reads continue through the JNI and native readers.

### Check List (For Author)

- Test: Unit Test and manual test
  - mvn -f fe/pom.xml -pl :fe-connector-paimon -am install -Dmaven.build.cache.enabled=false (553 tests passed, 1 live test skipped)
  - thirdparty Arrow and ADBC lifecycle tests
  - thirdparty download/mirror/checksum/retry script tests
  - build-support/check-build-hygiene.sh
  - shell syntax, YAML parsing, and git diff --check
- Behavior changed: Yes. The paimon-cpp reader and enable_paimon_cpp_reader session variable are removed; deprecated wire fields remain for compatibility.
- Does this need documentation: No
@hello-stephen

hello-stephen commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.34% (1942/2479)
Line Coverage 64.91% (34550/53231)
Region Coverage 52.39% (33115/63212)
Branch Coverage 55.03% (10174/18488)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei Awaiting requested review from yiguolei yiguolei is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@suxiaogang223 @hello-stephen