[prrao87]

Why

Grouping by a node (WITH forum, collect(friend) AS friends) grouped by every column of that node and carried them through the aggregate for every input row. After an expansion that's many rows per node: in IC5, about 800K hasMember rows carry the forum's title into 77K groups. While working on #92, carrying a string through that grouping cost 7 to 9 ms however it was done (grouping by it, max, head(collect())), which kept IC5 behind 0.4.0. #99 proposed the fix: group by the node's id and fetch its other properties afterwards, once per node.

This implements it for in-memory node tables. It addresses #99 and #92's remaining memory-mode gap; #99 stays open for Lance node tables (see below).

What changed

• Grouping by a node with an in-memory node index uses the id alone. When b's label has a node index (unique integer ids) and its table is in memory, WITH b, ... groups by b's id, and a new NodeLookup operator adds b's other columns after the aggregate. With unique ids the groups are identical to grouping by every column.
• NodeLookup looks rows up through the node index, the same id-to-row map the CSR's fused target lookup uses, and gathers the wanted columns per batch. A null or unknown id gets null columns, so the null group from an unmatched OPTIONAL MATCH keeps null properties. explain() shows it as NodeLookupExec: key=b__id, take=b[b__name].
• Unused lookup columns are pruned by the same optimizer rule that prunes the fused target lookup's columns, and a lookup whose columns are all unused is dropped.
• Lance node tables keep the plain grouping. By the time a query groups by a node, its rows already hold the node's properties, read from the dataset by an earlier step, so grouping by all of them reads nothing more. Looking them up after the grouping would read them from disk a second time: with the lookup on, IC1 came out 11% slower in lazy mode. Doing it well for Lance needs a read-once switch like the CSR's target lookup has, which is why Late-materialize a grouped node's properties from Lance node tables #99 stays open.
• User impact: none beyond speed. Results are unchanged; explain() shows the new operator. The CSR guide gains a "Grouping by a node" section.

Two variants tried first

Query	Join back to the node table	Lookup, all tables	Lookup, in-memory only (this PR)
IC3, memory	-50%	-53%	-52%
IC5, memory	-11%	-7%	-7%
IC10, memory	+22%	-0.5%	+0.6%
IC1, lazy	+10%	+11%	-1.1%

Joining the grouped ids back to the node table scans the whole table for every fetch, which IC10 and IC1 (6 and 10 fetches in their plans) feel. Looking ids up through the node index fixed that; restricting it to in-memory tables removed the lazy-mode IC1 cost.

Validation

• cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, make test (1665 passed), make lint, docs/check_snippets.py, zensical build --strict and typos pass locally. CI is pending.
• New tests: the lookup with null and unknown ids (Rust); the plan uses NodeLookupExec with a node index and not without; an unmatched OPTIONAL MATCH groups with null columns; 20K groups in memory and from Lance with deleted rows, each compared against the join path. Existing WITH tests run with use_csr on and off against the same expected rows.

Benchmarks

Apple M5 (10 cores), 24 GB, macOS 26.6.2, other apps closed. Release wheels built from main at 20c1a36 and from this branch, loaded with uv run --with; main and branch alternated twice per mode, each run started below a 1-minute load of 3. Benchmark repositories: graph-benchmark-ldbc 5f528da, graph-benchmark-ldbc-complex f3e63fd. Every result check passed.

Suite	Mode	Geomean branch/main	Median main run gap
graph-benchmark-ldbc-complex	memory	0.930	1.5%
graph-benchmark-ldbc-complex	lazy	0.994	1.1%
graph-benchmark-ldbc	memory	1.000	3.2%
graph-benchmark-ldbc	lazy	0.992	0.7%

In memory: IC3 53.4 to 25.4 ms (-52%), IC7 16.2 to 14.4 ms (-11%), IC5 85.1 to 78.9 ms (-7%; 0.4.0: 77.6 ms), IC1 -6%, IC14 -5%. Nothing got slower beyond run-to-run noise in either mode. Lazy plans are unchanged from main.

Every query, both suites and modes (mean ms)

graph-benchmark-ldbc: memory

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	0.65	0.62	-5.4%	11.8%	0.4%
query2	0.53	0.52	-0.3%	3.0%	4.2%
query3	0.64	0.62	-2.6%	1.5%	2.4%
query4	0.50	0.50	+0.0%	3.3%	4.2%
query5	0.78	0.76	-1.4%	0.3%	0.6%
query6	0.42	0.42	-0.4%	3.0%	3.2%
query7	1.03	1.03	+0.3%	3.2%	2.1%
query8	0.63	0.62	-0.8%	2.0%	3.0%
query9	0.81	0.81	+0.5%	3.4%	3.1%
query10	0.72	0.72	-0.1%	4.0%	1.5%
query11	1.58	1.51	-3.9%	3.5%	1.5%
query12	0.75	0.75	-0.4%	1.8%	2.2%
query13	0.83	0.80	-4.1%	4.8%	3.2%
query14	0.53	0.53	+0.4%	2.7%	3.8%
query15	0.55	0.56	+1.2%	4.1%	3.1%
query16	0.65	0.66	+1.6%	4.0%	2.8%
query17	0.90	0.90	+0.9%	3.2%	0.9%
query18	0.68	0.70	+2.9%	2.5%	1.9%
query19	0.92	0.93	+1.6%	3.7%	1.5%
query20	2.08	1.98	-4.6%	1.3%	0.8%
query21	0.74	0.75	+0.8%	0.1%	0.5%
query22	0.70	0.71	+1.6%	2.3%	0.2%
query23	0.72	0.74	+2.7%	4.9%	1.1%
query24	0.67	0.69	+3.0%	4.7%	0.8%
query25	0.78	0.80	+2.1%	3.7%	1.2%
query26	0.73	0.74	+1.9%	3.1%	1.2%
query27	0.85	0.86	+1.5%	3.2%	1.7%
query28	1.37	1.39	+1.4%	4.4%	1.3%
query29	0.68	0.68	+1.2%	2.9%	2.0%
query30	40.81	40.09	-1.8%	0.2%	1.8%

Geomean branch/main 1.000; median main run gap 3.2%

graph-benchmark-ldbc: lazy

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	1.15	1.12	-2.6%	3.2%	2.0%
query2	1.37	1.37	-0.0%	0.2%	1.7%
query3	0.80	0.78	-1.4%	1.4%	3.4%
query4	0.78	0.77	-0.6%	0.2%	1.1%
query5	2.13	2.09	-1.8%	1.7%	2.0%
query6	0.72	0.71	-0.6%	0.2%	0.1%
query7	1.08	1.07	-0.5%	1.0%	0.5%
query8	1.71	1.71	-0.2%	3.6%	3.0%
query9	1.32	1.32	-0.3%	1.2%	0.4%
query10	0.92	0.92	+0.3%	0.8%	0.3%
query11	1.96	1.97	+0.6%	0.2%	2.1%
query12	6.73	6.63	-1.4%	1.3%	3.5%
query13	1.03	1.03	-0.9%	0.8%	0.9%
query14	0.68	0.68	-0.4%	0.8%	0.3%
query15	0.75	0.74	-0.5%	0.8%	0.0%
query16	1.08	1.07	-0.5%	0.1%	0.4%
query17	1.37	1.37	-0.5%	0.2%	0.0%
query18	1.12	1.12	-0.5%	0.0%	0.6%
query19	1.33	1.32	-0.7%	0.0%	0.1%
query20	17.33	16.35	-5.7%	0.2%	0.8%
query21	0.83	0.82	-0.5%	1.0%	0.9%
query22	0.91	0.91	-0.5%	0.2%	0.0%
query23	1.22	1.21	-0.5%	0.1%	0.3%
query24	1.04	1.04	-0.3%	0.3%	0.9%
query25	1.18	1.17	-0.5%	0.0%	0.6%
query26	1.06	1.05	-0.7%	0.1%	0.4%
query27	1.28	1.27	-0.7%	0.3%	0.7%
query28	2.02	2.00	-1.2%	1.0%	0.6%
query29	1.01	1.01	-0.5%	0.7%	0.5%
query30	40.43	40.59	+0.4%	0.7%	1.5%

Geomean branch/main 0.992; median main run gap 0.7%

graph-benchmark-ldbc-complex: memory

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	25.11	23.63	-5.9%	1.5%	0.3%
query2	2.43	2.42	-0.2%	2.3%	1.6%
query3	53.41	25.43	-52.4%	1.7%	0.4%
query4	5.83	5.75	-1.3%	0.7%	0.1%
query5	85.14	78.86	-7.4%	1.3%	1.0%
query6	21.32	21.36	+0.2%	1.7%	0.4%
query7	16.15	14.40	-10.8%	1.0%	1.4%
query8	2.24	2.25	+0.5%	0.3%	0.5%
query9	37.98	37.93	-0.1%	2.3%	1.6%
query10	98.03	98.64	+0.6%	0.6%	0.5%
query11	2.96	2.98	+0.7%	0.3%	0.6%
query12	17.41	17.90	+2.8%	3.0%	0.1%
query13	1.10	1.09	-1.0%	1.7%	2.0%
query14	49.50	47.20	-4.7%	0.5%	0.6%

Geomean branch/main 0.930; median main run gap 1.5%

graph-benchmark-ldbc-complex: lazy

Query	main (ms)	branch (ms)	Change	main run gap	branch run gap
query1	31.61	31.27	-1.1%	2.0%	0.4%
query2	25.18	25.58	+1.6%	0.3%	2.5%
query3	164.76	164.58	-0.1%	4.4%	0.4%
query4	12.76	12.60	-1.2%	0.7%	0.5%
query5	108.34	109.96	+1.5%	3.1%	1.7%
query6	22.92	22.71	-0.9%	0.5%	5.7%
query7	22.67	22.69	+0.1%	0.4%	0.5%
query8	3.27	3.25	-0.6%	1.3%	0.4%
query9	307.29	303.70	-1.2%	0.5%	3.9%
query10	128.84	129.01	+0.1%	1.4%	0.2%
query11	5.33	5.32	-0.2%	1.1%	0.3%
query12	20.97	19.73	-5.9%	1.5%	0.4%
query13	1.44	1.43	-0.4%	0.7%	0.8%
query14	49.98	50.10	+0.2%	0.8%	0.1%

Geomean branch/main 0.994; median main run gap 1.1%

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying lance-csr with    Cloudflare Pages

Latest commit:	b830f1a
Status:	✅  Deploy successful!
Preview URL:	https://93571d58.lance-csr.pages.dev
Branch Preview URL:	https://perf-late-node-properties.lance-csr.pages.dev

View logs