Repository navigation
fix(planner): plan optional match pattern predicates that read earlier variables - #107
Merged
Merged
Conversation
…rn predicate OPTIONAL MATCH (friend)<-[:CREATED]-(post) WHERE (post)-->()<--(person) failed to plan with 'No field named person' when person was bound by an earlier clause. The optional side's seed carries the earlier variables its WHERE reads, but they were collected with boolean_variables, which skips pattern predicates. Also collect the variables named in pattern predicates, keeping only those bound earlier.
DataFusion's physical planner formats every extension node with Debug while planning it. The derived Debug of CsrExpandNode, NodeLookupNode, ShortestPathNode and ApplyNode printed the whole input plan, which made this quadratic in the plan's depth. Print the node's explain line instead, as PathMatchNode already does.
Deploying lance-csr with
|
| Latest commit: |
e5d026f
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://5653a8cf.lance-csr.pages.dev |
| Branch Preview URL: | https://fix-optional-match-pattern-v.lance-csr.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A pattern predicate in an
OPTIONAL MATCH ... WHEREthat reads a variable bound by an earlier clause fails to plan. This is how Ladybug's runner phrases LDBC IC10, and it surfaced while profiling that query:On
mainthis returnsPlanError: Failed to build projection: Schema error: No field named person. The same profile also showed physical planning spending time formatting whole input plans, which this PR removes while it is in the same area.What changed
WHEREreads. Those were collected by a helper that deliberately skips pattern predicates (it was written for selectivity scoring), sopersonabove was never carried. The planner now also collects the variables named in pattern predicates, keeping only those bound by earlier clauses, so variables local to the predicate's pattern are never carried. A pattern predicate nested inside a value (CASE WHEN (a)-->(b) ...) is still not visited; that is noted in the code.Debugwhile planning it. The derivedDebugofCsrExpandNode,NodeLookupNode,ShortestPathNodeandApplyNodeprinted the entire input plan, which made this quadratic in plan depth. They now print theirEXPLAINline, asPathMatchNodealready did. Nothing relied on the old output.User impact: queries of the shape above now run instead of failing. No API changes and no documentation changes; the docs already list
WHEREinsideOPTIONAL MATCHas supported.Validation
New regression test
optional_match_where_pattern_reads_an_earlier_variable, with and without the CSR. It fails onmainwith the error above and passes here. It covers two people with different interests (so each row must use its ownperson), a post matching through two tags (counted once), and a friend with no posts (count 0).cargo test --workspace --locked,make test(1665 passed),cargo clippy --workspace --all-targets -- -D warnings,cargo fmt --all --checkanduvx typos@1.26.0: all pass.Expected answers in both benchmark suites, with a release wheel of this branch: graph-benchmark-ldbc 30/30 and graph-benchmark-ldbc-complex 14/14, in both
memoryandlazymode.Planning time, release build, SF1 in memory, mean of 50 plan-only calls, two alternating runs of
mainand this branch. Apple M5, 24 GiB, macOS 26.6.2:main(ms)End-to-end execute times for these three queries moved by less than run-to-run noise.
Not yet run: the full main-versus-branch benchmark harness from
CONTRIBUTING.md(both suites, both modes, alternating runs). This stays a draft until those tables are added.🤖 Generated with Claude Code