[prrao87]

Why

A CSR slot held only the neighbour key, so it couldn't point back to the edge row it came from. Any hop that named its relationship (-[k:KNOWS]->) fell back to hash joins, even to read one property. Refresh also had to read deleted rows from the old dataset version, so once that version was cleaned up, the engine rebuilt the whole CSR.

This is Stage 1 of #72 and closes #73: each CSR slot now records its edge's row, which keeps named hops on the CSR and lets refresh work without the old version's files. Stage 2 (persisting the CSR) builds on it in a later PR.

What changed

Edge rows in the CSR. Each neighbour now carries the row its edge came from: the Lance row address, or the row index for in-memory tables. Reading these rows is compiled out of hops that don't need them.

Named relationships stay on the CSR. A hop such as (a)-[k:KNOWS]->(b) fetches only the columns of k that the query reads, by row. explain() shows them as edge=k[k__since]. Parallel edges keep separate rows and properties, as on the join path. Inline property maps (-[k:KNOWS {since: 2020}]->) still use the join path.

Filters on relationship columns run inside the expansion. A WHERE condition that reads only k's own columns moves into the CSR node (edge_filter= in explain()), next to the existing target-filter pushdown:

• a small hop fetches just the filter's columns for each batch of expanded edges and drops failures before fetching anything else;
• once a hop has expanded more than 1/256 of the relationship's edges, it reads the filter columns once, in order, into a bitmap of passing rows and skips failing edges during expansion. The partitions of a query share one bitmap.

Refresh no longer needs the old dataset version. Removed, rewritten and newly deleted rows are matched against the rows the CSR stores, so only added fragments are read. Compaction followed by cleanup_old_versions() now applies as a delta instead of forcing a rebuild.

count(r) works on relationships. It failed with No field named r__id on both the CSR and join paths. It now counts the rows where r matched, so an unmatched OPTIONAL MATCH counts zero. count(DISTINCT r) compares all of r's columns, so parallel edges with identical properties count once, the same rule relationship uniqueness already follows.

Rust API marked experimental. The crate docs, crate READMEs and the Rust guide now say breaking changes are possible over the next few weeks. This PR itself changes CsrIndexBuilder::add_edge, Adjacency::build and with_changes, EdgeChanges and CsrExpandNode::try_new. The Python API is unchanged.

Costs. Peak memory while building the CSR goes from about 62 to about 94 bytes per edge (10M edges), and about 32 bytes per edge stay resident. The CSR build is roughly 1.35 times slower. The docs now say about 95 bytes per edge.

Validation

• Rust: 671 tests pass, plus clippy with -D warnings and cargo fmt --check. New tests cover edge rows in the CSR, deltas by row, the per-batch filter, bitmaps from in-memory and multi-fragment Lance data, and the count(r) rewrite.
• Python: 1653 tests pass (in parallel), plus ruff and pyright. New cases compare CSR and join answers for named relationships (directed, undirected, cycles, cross-hop property filters, reuse, duplicate edges, null endpoints), relationship filters crossing the bitmap switch over many small Lance fragments, deltas after compaction plus cleanup_old_versions() (this test rebuilds on main), and count(r) with OPTIONAL MATCH.
• Docs: strict Zensical build, doc snippets and typos pass.
• CI: all 8 checks pass.

Benchmarks. Release wheels from main (1dc63aa) and this branch (0124f41), run with the harness in CONTRIBUTING.md: both suites, both modes, main and branch alternating twice. All 88 runs passed against the expected answers. Apple M5, 24 GB, macOS 26.6.2, on AC power with no other heavy work running (load average 2.0 at the start, 4.6 at the end); graph-benchmark-ldbc-complex f3e63fd, graph-benchmark-ldbc b649bc1.

Suite, mode	Branch vs main (geomean)	Median gap between main runs
IC1 to IC14, memory	-1.3%	2.6%
IC1 to IC14, lazy	+0.4%	0.6%
Custom 30, memory	+0.2%	0.7%
Custom 30, lazy	-2.2%	2.0%

Queries that changed beyond the gap between main runs:

Query	Memory: main / branch	Lazy: main / branch
IC5	72.3 / 95.3 ms (+32%)	104.8 / 176.9 ms (+69%)
IC7	21.7 / 16.5 ms (-24%)	32.2 / 23.2 ms (-28%)
IC11	3.6 / 3.1 ms (-15%)	5.8 / 5.4 ms (-7%)
IC12	20.4 / 17.7 ms (-13%)	27.0 / 24.7 ms (-9%)
IC3	61.0 / 66.9 ms (+10%)	412.7 / 410.8 ms (-0.5%)

IC5 is the one regression. Its hasMember hop used to fall back to joins only because it names membership. On the CSR, its relationship filter now costs what a plain scan does (15.4 ms against 14.4 ms for the filtered count in lazy mode). What remains is fetching forum columns one batch of scattered rows at a time across about 800k expanded edges. Unnamed hops already have that weakness on main, and #86 tracks the fix. The docs describe the case.

IC3 doesn't name a relationship and is flat in lazy mode, but came out about 10% slower in memory mode in this run and the previous one. That is inside the 15 to 20% run-to-run range CONTRIBUTING.md treats as noise; it's listed so a reviewer can watch for it.

None of the 30 custom queries names a relationship; they stay within 2.2% of main in both modes.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying lance-csr with    Cloudflare Pages

Latest commit:	5ee7774
Status:	✅  Deploy successful!
Preview URL:	https://05d29fd5.lance-csr.pages.dev
Branch Preview URL:	https://feat-csr-edge-rows.lance-csr.pages.dev

View logs