Parse only the columns a query needs (#296)

[chrisrichards]

Implements #296: the query engine now parses only the columns a query references, in two phases — the WHERE clause's columns for every candidate row, and the remaining projected (plus, when sorting, ORDER BY) columns only once a row matches. For selective queries over wide tables, most rows never parse most columns.

How it works

• SqlColumnCollector walks the bound expression tree once at bind/prepare time to collect filter ordinals; the post-filter set is (projection ∪ sort keys) − filter columns.
• DbfRecord gains an internal subset mode: TryParseValues(int[] ordinals) parses just those values (additively within a row), and per-row version stamps track what's valid. Accessing an unparsed ordinal throws a clear InvalidOperationException instead of exposing the previous row's content through the reused value objects — the stale-data hazard the issue called out. The eager Read path is untouched; the guard only activates for records the engine puts in subset mode, costing one null check.
• Applied everywhere rows are filtered or projected: DbfQueryDataReader (sequential + index row sources, sync + async, and the ORDER BY buffer now snapshots only needed ordinals), DbfQuery<T> (Execute, async enumeration, Count), and both COUNT(*) read-and-filter executors.
• Unreferenced memo columns are never parsed, so selective queries on memo tables also skip memo-file I/O.

Measurements

BenchmarkDotNet, 50,000-row generated table (4 columns), version-pinned packages (before = main with #303):

Scenario	Before	After
equality seek, full scan	21,634 µs / 6,262 KB	17,328 µs / 1,184 KB
character seek, full scan	21,584 µs / 7,048 KB	17,944 µs / 1,977 KB
range scan, full scan	22,419 µs / 8,218 KB	17,763 µs / 1,193 KB
count(*) filtered, full scan	22,374 µs / 8,215 KB	17,767 µs / 1,184 KB
top 10 order by desc, scan + sort	51,760 µs / 54,124 KB	45,831 µs / 53,467 KB
count(*) status scan / index paths	unchanged	unchanged

On the 16-column tl_2019_01_place fixture, a selective scan projecting one column runs 3.6× faster with 11× fewer allocations (1,737 → 483 µs, 341 → 31 KB), and filtered COUNT(*) is 1.9× faster.

Benchmark infrastructure fix this surfaced

While validating, the before/after BDN runs produced byte-identical allocation numbers — because BenchmarkDotNet rebuilds the benchmarks project in a generated child project that does not inherit -p: MSBuild properties. -p:DbfDataReaderVersion=x.y.z has therefore been silently benchmarking the csproj default all along. Fixed by documenting the environment-variable form (DbfDataReaderVersion=x.y.z dotnet run …), which reaches every build, and by making each benchmark print the informational version of the DbfDataReader assembly it actually loaded (// DbfDataReader under benchmark: 1.1.0+41ded65…) so a wrong package is impossible to miss. The published benchmarks.md tables were re-verified against correctly-pinned packages: true 1.1.0 measures 643.8 µs / 704.4 KB vs the recorded 611.1 µs / 704.41 KB — within noise, so the recorded data stands.

Tests

501 tests pass (21 new): record-level guard tests (unparsed access throws, parsing is additive, stamps reset on row advance, full reads mark everything parsed), and full-parse-oracle differentials for every path — projection/filter on disjoint columns, SELECT * with filter, ORDER BY on a non-projected column, LIMIT, COUNT, async readers, memo tables, the typed builder (sync/async/Count), and index-vs-scan equality on the CDX fixtures.

Closes #296

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud