[osipovartem]

Which issue does this PR close?

Performance follow-up for Rustice #1112; this does not close its downstream qualification gates.

Rationale for this change

Nested views can introduce adjacent column projections that change only output metadata. Existing metadata barriers retain the whole chain even though these expressions cannot inspect their input metadata. Remove the redundant execution layers without weakening barriers around computed expressions or changing observable Arrow IPC dictionary properties.

What changes are included in this PR?

• Collapse adjacent column-only projections across metadata overrides while retaining the original outer metadata.
• Preserve computed-expression barriers, including when an intermediate collapse exposes a metadata-observing expression.
• Check root dictionary IDs/ordering both for identity removal and for replacement plans returned by datasource pushdown. Arrow schema equality alone does not cover these IPC properties.
• Add real-row, NULL, nested-dictionary, datasource and metadata-observer regressions, plus a bounded Criterion benchmark with unchanged-path controls.

No row-by-row conversion, new dependency, public API, or configuration option.

Are these changes tested?

Local qualification on rebased HEAD 841da53e0, based on 95e77b795:

• cargo +1.97.0 fmt --all: passed.
• cargo +1.97.0 clippy --offline --locked --profile ci --all-targets --all-features -- -D warnings: passed.
• Required extended workspace suite (avro/json/backtrace/extended_tests/recursive_protection/parquet_encryption, excluding examples/benchmarks/CLI): 10,756 passed, zero failed, 10 ignored; separately 509/509 SQLLogicTest files completed successfully.
• Independent read-only review approved source correctness, regressions, IPC/API behavior, performance controls and test coverage. Review findings on dictionary-property preservation were confirmed with failing tests and fixed before the final runs.

The rebase leaves the complete four-file patch and Cargo.lock byte-identical to the independently reviewed measured patch. No expected-result snapshots were adjusted.

Local performance screen

The independently reviewed frozen comparison is bc0b207 -> 901719559, before the later integration rebase. Rust 1.97, release-nonlto, identical lockfile/benchmark and 267 dependency records; four alternating paired blocks with fresh processes, one-CPU affinity, and independent correctness preflights. All raw samples and adverse blocks were retained. This is a local physical-plan microbenchmark, not SQL/SPCS latency or a statistical equivalence claim.

8,192 rows, depth 8, metadata overrides	Median paired candidate/baseline cost	Matched block range	Reciprocal cost ratio
Optimize + collect, 1 column	0.3422	0.3344-0.3596	2.92x
Optimize + collect, 16 columns	0.5310	0.4979-0.5427	1.88x
Execute optimized plan, 1 column	0.1296	0.1174-0.1419	7.72x
Execute optimized plan, 16 columns	0.1267	0.1179-0.1284	7.90x

All six unchanged case/phase controls and both one-row override controls passed the predeclared gates: median cost ratio <= 1.03, at least three of four ratios <= 1.05, and no ratio > 1.10. The largest control median increase was 0.68%; the worst individual increase was 4.33%. These increases are reported, not discarded. Independent review recomputed all 48 ratios from 96 timed datasets / 960 samples.

The integration rebase leaves all four changed files and Cargo.lock byte-identical, but its dependencies were not timed again. A downstream Rustice repin and the actual view-query cost/wire screen remain separate acceptance gates.

To reconstruct the measured pair from public sources, start both arms at bc0b2074b. Apply only this PR's benchmark file and Cargo registration to the baseline, and its complete four-file diff to the candidate (effective source tree 419d9af484e77c3e3be177bdc6837ec36f109cd4). Run cargo bench --profile release-nonlto -p datafusion-physical-plan --bench projection_metadata on each arm with the same settings. The older local commit ID above identifies the frozen measurement, not the rebased PR HEAD.

Are there any user-facing changes?

No public API changes. Metadata-only column projection chains can execute with fewer operators; output values, metadata, and IPC dictionary properties must remain unchanged.