fix(datafusion): use caller session for filter planning

[lance-gatefixer]

Summary

• forward the DataFusion session from TableProvider scans through Lance planning
• build pushed-down and refinement physical filters with the originating session so foreign UDFs remain callable
• add an FFILanceTableProvider regression test using a custom Python UDF

Root cause

Lance discarded the Session supplied to TableProvider::scan and rebuilt physical filter expressions with its default planner. Across FFI, that lost the caller-owned UDF and session configuration, so pushed-down custom UDF filters failed during planning or execution.

Validation

• cargo fmt --all
• cargo check -p lance-datafusion -p lance
• cargo clippy --all --tests --benches -- -D warnings
• cd python && make build
• cd python && uv run make lint
• cd python && uv run pytest python/tests/test_table_provider.py::test_custom_udf_filter -vv
• cd python && uv run pytest python/tests/test_table_provider.py -q

Fixes #5144

[lance-gatekeeper]

✅ Gate recommendation: approve.

Forwarding the originating DataFusion session when physicalizing Lance filters fixes the demonstrated FFI UDF/configuration failure while preserving the default Dataset scanner path. Using the session directly is preferable to copying selected registry or configuration fields because it retains the complete caller planning contract.

Please mark this PR with the breaking-change label.

[lance-gatefixer]

No code change was needed; the requested breaking-change label is now applied to this Ready PR at head 7d1ef0c04.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!