[prrao87]

Why

IC5 got about 20% slower in memory mode when #87 moved named relationship hops onto the CSR (#92). Profiling by stage showed two causes. This PR fixes the first, the relationship filter on IC5's hasMember hop, and is part of #92 rather than closing it.

(friend)<-[membership:hasMember]-(forum) WHERE membership.joinDate > $minDate expands about 824K edges. Once it had expanded 1/256 of the relationship, CsrExpandExec switched to a mask over all 1.6M edges, built by one partition while the others waited on its lock. 0.4.0 instead filtered the edge table in parallel and hash-joined. The switch exists for Lance datasets, where scattered reads of the filter column cost about a full read each. For in-memory tables it only serialized the work.

What changed

• In-memory relationship tables never switch to the mask. Each partition gathers its own chunk's filter column from the loaded arrays and evaluates the filter there, in parallel. Lance datasets keep the 1/256 threshold.
• The CSR guide's paragraph on relationship filters says so.
• User impact: none beyond speed. Results are unchanged.

IC5's stages in memory mode (median ms, release wheels):

Stage	0.4.0	main	branch
Friends, then the hasMember hop with its joinDate filter	6.7	9.9	7.6
Plus WITH forum, collect(friend)	11.5	20.5	16.5
IC5	77.6	97.1	89.8

IC5's plan computes the hop and its grouping twice (once for the collect, once to seed the OPTIONAL MATCH), so each saving counts double.

What we learned from a second step, not included

The remaining gap is the grouping after the hop. I tried grouping a node by its id and carrying its other columns with first_value:

• It's a big win when the node's properties go unused afterwards: IC5 up to its last grouping went from about 96 to 75 ms.
• It doesn't help when they're used, as IC5's forum.title is. Carrying a string through a grouping of 800K rows costs about the same 7 to 9 ms whichever way it's done (max, head(collect()), or grouping by it), and today's grouping by all columns is already the cheapest.
• DataFusion 54's first_value only has a grouped accumulator when it has an ORDER BY. Without one, IC5 went to 130 ms.

So the fix is late materialization: group by the id and fetch the node's properties after the grouping, for 77K forums instead of 800K rows. That needs node ids known to be unique, and is tracked in #99.

Validation

• cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, make test (1660 passed) and make lint pass locally, as do the strict docs build and typos. CI is pending.
• Existing tests cover both filter paths: the per-chunk and mask paths in csr_expand.rs, and test_relationship_filter_over_many_batches, which compares against the join path over many batches in memory and from Lance.

Benchmarks

Apple M5 (10 cores), 24 GB, macOS 26.6.2, other apps closed. Release wheels built from main at 29f3dcf and from this branch, loaded with uv run --with; main and branch alternated twice per mode, each run started below a 1-minute load of 3. Benchmark repositories: graph-benchmark-ldbc 5f528da, graph-benchmark-ldbc-complex f3e63fd. Every result check passed.

Suite	Mode	Geomean branch/main	Median main run gap
graph-benchmark-ldbc	memory	1.003	2.6%
graph-benchmark-ldbc	lazy	1.007	2.4%
graph-benchmark-ldbc-complex	memory	0.979	2.5%
graph-benchmark-ldbc-complex	lazy	1.000	1.0%

IC5 in memory mode: 97.1 to 89.1 ms (-8.2%). Every other change is within run-to-run noise. Lazy mode doesn't run the changed code, so its ±4% movements (IC5 -4.1%, IC1 +3.4%) are noise too.

Every query, both suites and modes (mean ms)

graph-benchmark-ldbc: memory

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	0.63	0.63	-0.0%	1.2%	1.5%
query2	0.53	0.54	+2.0%	4.4%	2.8%
query3	0.65	0.65	+0.5%	5.0%	2.3%
query4	0.50	0.51	+1.3%	4.3%	0.9%
query5	0.78	0.78	+0.1%	1.6%	1.1%
query6	0.43	0.43	+1.0%	1.7%	2.6%
query7	1.03	1.04	+1.3%	3.0%	2.2%
query8	0.63	0.64	+1.6%	2.6%	1.2%
query9	0.81	0.83	+1.7%	3.5%	1.2%
query10	0.72	0.73	+0.7%	4.1%	0.2%
query11	1.54	1.57	+2.2%	2.9%	1.4%
query12	0.75	0.76	+1.2%	4.0%	0.5%
query13	0.80	0.81	+1.2%	4.5%	0.4%
query14	0.53	0.54	+1.8%	3.2%	0.2%
query15	0.57	0.57	+0.6%	3.3%	0.4%
query16	0.67	0.67	+0.4%	3.5%	0.9%
query17	0.92	0.91	-0.9%	2.4%	0.7%
query18	0.70	0.70	+0.2%	3.7%	0.2%
query19	0.94	0.93	-0.6%	2.4%	1.1%
query20	2.23	2.05	-8.4%	0.4%	6.2%
query21	0.78	0.77	-1.8%	1.1%	0.4%
query22	0.72	0.72	-0.4%	1.4%	1.6%
query23	0.74	0.74	+0.7%	2.4%	1.2%
query24	0.70	0.70	-0.3%	2.3%	0.5%
query25	0.81	0.81	+0.6%	2.4%	0.5%
query26	0.75	0.75	+0.5%	2.9%	1.3%
query27	0.86	0.89	+2.8%	1.9%	2.1%
query28	1.43	1.43	+0.0%	2.4%	0.7%
query29	0.69	0.69	-0.1%	1.5%	1.7%
query30	42.99	42.33	-1.5%	0.5%	8.4%

Geomean branch/main 1.003; median main run gap 2.6%

graph-benchmark-ldbc: lazy

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	1.16	1.17	+1.4%	2.6%	9.4%
query2	1.37	1.40	+2.2%	2.8%	6.3%
query3	0.79	0.81	+1.7%	5.7%	4.7%
query4	0.78	0.80	+1.7%	4.9%	5.4%
query5	2.18	2.26	+3.7%	0.8%	9.7%
query6	0.72	0.73	+0.9%	2.9%	3.6%
query7	1.08	1.09	+0.5%	6.1%	3.1%
query8	1.74	1.72	-0.8%	0.9%	1.5%
query9	1.34	1.34	+0.2%	3.6%	3.0%
query10	0.93	0.94	+1.0%	2.4%	4.5%
query11	1.97	2.01	+2.1%	4.7%	5.0%
query12	6.80	7.04	+3.6%	4.2%	6.9%
query13	1.04	1.06	+1.8%	3.2%	3.9%
query14	0.68	0.69	+1.2%	2.3%	3.3%
query15	0.76	0.76	+0.6%	3.6%	1.9%
query16	1.09	1.10	+0.9%	2.4%	1.1%
query17	1.39	1.41	+1.4%	1.8%	1.5%
query18	1.13	1.13	+0.0%	2.1%	0.5%
query19	1.34	1.35	+0.8%	3.4%	0.6%
query20	17.49	16.73	-4.3%	2.3%	0.7%
query21	0.85	0.85	+0.9%	4.3%	4.2%
query22	0.93	0.97	+3.7%	1.3%	11.0%
query23	1.23	1.24	+1.2%	1.5%	0.5%
query24	1.06	1.06	+0.2%	1.2%	1.0%
query25	1.20	1.20	+0.2%	1.3%	1.0%
query26	1.08	1.07	-0.7%	0.6%	2.3%
query27	1.30	1.31	+0.3%	0.5%	1.4%
query28	2.14	2.08	-3.0%	5.5%	0.6%
query29	1.05	1.03	-1.9%	2.0%	1.1%
query30	42.98	42.81	-0.4%	0.5%	2.7%

Geomean branch/main 1.007; median main run gap 2.4%

graph-benchmark-ldbc-complex: memory

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	25.72	25.19	-2.1%	1.1%	2.1%
query2	2.70	2.65	-1.9%	18.8%	5.2%
query3	57.19	55.07	-3.7%	1.4%	5.9%
query4	6.20	6.26	+1.0%	1.5%	0.3%
query5	97.08	89.08	-8.2%	1.9%	4.5%
query6	23.04	23.71	+2.9%	6.1%	4.1%
query7	16.59	16.50	-0.6%	2.5%	0.6%
query8	2.28	2.26	-1.1%	2.8%	1.5%
query9	40.99	39.91	-2.6%	6.9%	8.0%
query10	101.91	103.15	+1.2%	2.7%	0.9%
query11	3.08	3.02	-2.1%	1.1%	0.2%
query12	20.16	18.19	-9.8%	6.2%	7.6%
query13	1.10	1.10	-0.4%	0.2%	0.5%
query14	51.78	51.06	-1.4%	2.0%	2.1%

Geomean branch/main 0.979; median main run gap 2.5%

graph-benchmark-ldbc-complex: lazy

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	31.60	32.67	+3.4%	0.9%	2.6%
query2	25.89	25.72	-0.7%	0.5%	5.6%
query3	166.69	172.43	+3.4%	4.5%	2.8%
query4	13.13	13.06	-0.5%	0.9%	0.3%
query5	116.35	111.56	-4.1%	1.0%	2.6%
query6	24.25	25.87	+6.7%	9.2%	3.6%
query7	23.23	23.10	-0.6%	1.3%	0.2%
query8	3.34	3.35	+0.4%	1.0%	1.1%
query9	307.27	310.19	+1.0%	0.1%	1.9%
query10	135.37	134.75	-0.5%	1.2%	0.4%
query11	5.48	5.60	+2.1%	0.7%	0.2%
query12	22.40	20.11	-10.2%	10.3%	4.5%
query13	1.46	1.45	-0.6%	1.0%	0.1%
query14	51.68	52.50	+1.6%	3.3%	1.6%

Geomean branch/main 1.000; median main run gap 1.0%

🤖 Generated with Claude Code