feat(sql): make tree-sitter-sql a core dependency so default installs parse .sql (#1745) - #2959
feat(sql): make tree-sitter-sql a core dependency so default installs parse .sql (#1745)#2959egarcia74 wants to merge 4 commits into
Conversation
A default install (uv tool install graphifyy / pipx install graphifyy) shipped without SQL parsing: tree-sitter-sql lived behind the [sql] extra, so every .sql file silently contributed nothing until the user found the extra or ran pipx inject (Graphify-Labs#1745). The grammar ships prebuilt abi3 wheels for every supported platform from 0.3.9, so the reason tree-sitter-dm stays optional does not apply. - promote tree-sitter-sql>=0.3.9,<0.4 into [project.dependencies]; keep the [sql] extra as an alias so existing install scripts resolve - drop .sql from _EXTRA_FOR_EXTENSION: a missing grammar now means a broken install, so the Graphify-Labs#1745 warning must not point at an extra - reword the extractor's missing-grammar error to say the install is incomplete and name the direct repair Bumps version to 0.9.49. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new test_sql_core_dependency.py: tree-sitter-sql must sit in
[project.dependencies] with an upper bound, .sql must not map to an
optional extra, and a table/view/procedure corpus must produce
structural nodes plus contains/references/reads_from edges
- convert every importorskip("tree_sitter_sql") to a hard import: the
grammar is core now, so its absence must fail the suite, not skip it
- update the Graphify-Labs#1745 warning test: the dev environment now ships the
grammar, so genuine absence is simulated by blanking find_spec, and
the message must say "core dependency" instead of hinting at
graphifyy[sql]
Each packaging guard was mutation-tested (dep removed from pyproject,
.sql re-added to the extras map) and failed as required.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the sql row from the optional-extras table, add an upgrade note (existing extra/pipx-inject users just run the normal upgrade), and add the 0.9.49 changelog entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream cuts versions and batches changelog entries in maintainer commits (see 'chore: bump to 0.9.48'), so a contributor PR should not claim the next version number. Comments and messages now describe the grammar as a core dependency without naming a release.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Promotes tree-sitter-sql from the sql extra to a core dependency in pyproject.toml, so .sql files extract on a default install (#1745); the sql extra stays as a harmless alias for old install scripts. Drops .sql from _EXTRA_FOR_EXTENSION and rewords extract_sql's missing-grammar error to name a broken install / direct reinstall rather than an optional extra, with README updated to match. Replaces the importorskip/_extract_sql_or_skip skips in the SQL tests with hard import tree_sitter_sql so absence fails instead of silently skipping.
Worth a look
- SQL dependency is only added to the lockfile, not the packaging source —
uv.lock:1338· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1897 functions depend on the 710 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 476 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_js()— 80 callers, 3 callees - new:
dispatch_command()— 2 callers, 119 callees - new:
_get_extractor()— 26 callers, 6 callees - new:
run_pipeline()— 8 callers, 13 callees - new:
collect_files()— 17 callers, 6 callees - …and 25 more — each is listed as a finding
Verification — 1897 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1749 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify extract\_sql.
The verifier did not have enough to check extract\_sql, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
· 33 more finding(s) on lines outside this diff (see the check run).
|
On the Graphify-review finding ("SQL dependency is only added to the lockfile, not the packaging source", flagged high at |
Problem
A default
uv tool install graphifyy/pipx install graphifyycontributes nothing for.sqlfiles: the grammar lives behind the[sql]extra, soextract_sqlbails with an error result and the #1745 warning is the only signal. SQL is a mainstream corpus language — schemas, migrations, stored procedures — and the failure mode is exactly the silent-skip shape #1745 was about: the operator sees a graph, just one with an entire language missing, and nothing tells them at install time.We hit this in the field: a 391-file T-SQL data-migration corpus produced zero SQL object nodes on a fresh default install, and the fix was discoverable only by reading extractor source.
Why core rather than a better hint
tree-sitter-sqlships prebuilt abi3 wheels for every supported platform (Windows/macOS/Linux glibc+musl, x86_64+arm64) from 0.3.9, so promoting it keeps the default install toolchain-free — the reasontree-sitter-dmstays optional (#1104) does not apply here. The pin>=0.3.9,<0.4carries an upper bound like every other grammar pin.What this PR does
tree-sitter-sql>=0.3.9,<0.4moves from the[sql]extra into[project.dependencies], with a comment recording the wheel-availability rationale so the packaging policy stays legible.sqlextra is kept as an alias, so existinggraphifyy[sql]install commands and scripts keep resolving; it now adds nothing beyond the core install..sqlis removed from_EXTRA_FOR_EXTENSION, so the .sql files silently extract 0 nodes when the [sql] extra is missing #1745 warning no longer sends users to a now-redundant extra; the extractor's missing-grammar error now says the install is incomplete and names the direct repair.sqlrow is replaced with a note explaining the move and that a normal upgrade is all an existing[sql]/pipx injectinstall needs.chore: bump to 0.9.48), so comments and messages describe the grammar as a core dependency without naming a release.Tests
New
tests/test_sql_core_dependency.pypins the fix at the packaging layer so a regression (the dependency sliding back into an extra) fails the suite rather than resurfacing as a field report:[project.dependencies], with an upper bound;.sqlmust not be mapped to an optional extra in the hint map;contains/references/reads_fromedges, and no dangling edge sources.Existing SQL tests in
tests/test_multilang.pyconvert fromimportorskip(which would silently skip the whole SQL surface if the grammar went missing — the exact failure mode this PR removes) to a hard import that fails instead.test_extract_warns_when_sql_grammar_missingnow simulates genuine absence via bothsys.modulesand afind_specmonkeypatch, and asserts the new message names the direct repair, not the extra.Validation
tree-sitter-sqltransitively; a SQL fixture produces nodes/edges with no extra orpipx injectinvolved; the missing-grammar warning path proven live by blanking the module.6 failed, 4835 passed, 28 skipped— the 6 (test_ollama.py,test_ollama_retry_cap.py) fail identically on unmodifiedv8in this environment (local Ollama config), none touch SQL or packaging.ruff check graphify/ tests/: all checks passed.uv.lockregenerated.Note
Independent of my companion PR #2960 fixing T-SQL routine recovery in
graphify/extractors/sql.py; both touchtests/test_multilang.py, and a local merge of the two branches resolves trivially (we run the combined branch in production against the 391-file corpus).