[python] Enable native writes with external data paths

[JingsongLi]

Purpose

Enable native writes for tables with data-file.external-paths, using the Rust implementation in apache/paimon-rust#981.

Dependency merged: Rust PR #981 is merged. CI continues building apache/paimon-rust@main.

• Remove the external-path fallback from native writer selection for supported Parquet append, primary-key and data-evolution tables.
• Resolve literal Java path strings for Python file I/O without decoding escaped partition characters. Preserve the original external path in persisted metadata and share physical-path and sidecar resolution between readers, writers and commit cleanup.
• Delete external index sidecars along with their data files when aborting native commit messages through the Python committer.
• Match Java's positive 32-bit external-path weights and trailing-comma handling.
• Document native external writes. Retain the existing Blob writer fallback.

Tests

With the companion Rust build and all five native test flags enabled:

• 1593 passed, 2 skipped, 74 subtests passed across affected plan/read/write/update, data-directory, external-path, Blob, split-serialization and commit-conflict suites.
• 75 new parameterized cases cover five external-path strategies; append/PK/data-evolution; partitioned and unpartitioned tables; Python and native writers/planners/readers; batch and stream writes; changed destinations; escaped partition values; updates/upserts/deletes; historical reads; abort cleanup; and existing Blob fallback.
• Native execution counters: 1453 plans, 2023 reads, 1014 writes, 5 commits; updates exercised row-id, grouped, predicate, upsert and incremental routes.
• Flake8 for all changed Python files and git diff --check passed.

Local test environment: Python 3.13 and PyArrow 18.1.0. GitHub CI supplies the Linux/Python 3.11 Native matrix.

CI follow-up

• Replace the buffered-writer metadata stub with real DataFileMeta so abort tests exercise shared path resolution. Cover cleanup of an unpublished external file and its index sidecar after a Blob/vector close failure.
• Python 3.10–3.13 CI had the same obsolete-stub failure. The previous Native job also built Rust commit 603ecdec, before [core] Add addColumn refactoring method for SchemaChange class #981 merged.
• Reproduced the stub failure locally, then passed 104 ordinary-path tests and 706 Native regression tests (8 subtests in each run) against Rust main commit fc27163. All five Native execution counters were exercised. Flake8 and git diff --check passed.

[leaves12138]

Reviewed 9e79d1ffe9d8c4f212b4c666c2854b9a0b43af3c together with apache/paimon-rust#981 (b63e4d99290eb781e308840874124aecdd9ef72c).

Found and fixed a regression in the existing abort test: TestAbortCommitMessages.test_reconstructs_local_path_after_wire_decode used a bare Mock for the data file. After cleanup delegates to physical_path() / collect_files(), that mock produces a non-iterable mock instead of file paths, so the expected deletion never occurs. Reproduced the failure before the fix and pushed 9e79d1f to the PR branch.

The fix uses real DataFileMeta instances instead of weakening production cleanup or skipping the test. Added coverage for missing runtime file paths plus aligned sidecars, and actual local-file deletion through a literal external file: path with escaped partition characters. The tests also verify persisted metadata is unchanged and external cleanup does not consult the current table destination.

Validation:

• Full selected paired-native regression with all five native flags enabled: 1,836 passed, 3 skipped, 155 subtests passed. Includes native external writes, data directories, plan/read/write/update/commit, Blob, serialization, and commit/abort tests.
• Python-backend commit/Blob/write/Daft-source regression: 311 passed, 1 skipped, 76 subtests passed.
• Additional Python path/serialization regression: 233 passed, 51 skipped, 8 subtests passed (native-only cases excluded in this lane).
• Reproduced Python- and native-written escaped external partitions through the public Daft reader successfully.
• Repository-configured flake8, Python 3.6 grammar parsing for changed files, and git diff --check passed.

No remaining blocking code-review findings for these paired revisions. Keep this PR draft until apache/paimon-rust#981 merges; then rerun Native CI against upstream Rust main and require it to pass before merging. Local native validation used the companion Rust revision, not the currently published upstream-main dependency. CI configuration and the existing merge-order requirement are unchanged. LGTM for the reviewed code.