Skip to content

feat(sql): make tree-sitter-sql a core dependency so default installs parse .sql (#1745) - #2959

Open
egarcia74 wants to merge 4 commits into
Graphify-Labs:v8from
egarcia74:fix/sql-core-dependency
Open

feat(sql): make tree-sitter-sql a core dependency so default installs parse .sql (#1745)#2959
egarcia74 wants to merge 4 commits into
Graphify-Labs:v8from
egarcia74:fix/sql-core-dependency

Conversation

@egarcia74

@egarcia74 egarcia74 commented Aug 23, 2026

Copy link
Copy Markdown

Problem

A default uv tool install graphifyy / pipx install graphifyy contributes nothing for .sql files: the grammar lives behind the [sql] extra, so extract_sql bails with an error result and the #1745 warning is the only signal. SQL is a mainstream corpus language — schemas, migrations, stored procedures — and the failure mode is exactly the silent-skip shape #1745 was about: the operator sees a graph, just one with an entire language missing, and nothing tells them at install time.

We hit this in the field: a 391-file T-SQL data-migration corpus produced zero SQL object nodes on a fresh default install, and the fix was discoverable only by reading extractor source.

Why core rather than a better hint

tree-sitter-sql ships prebuilt abi3 wheels for every supported platform (Windows/macOS/Linux glibc+musl, x86_64+arm64) from 0.3.9, so promoting it keeps the default install toolchain-free — the reason tree-sitter-dm stays optional (#1104) does not apply here. The pin >=0.3.9,<0.4 carries an upper bound like every other grammar pin.

What this PR does

  • tree-sitter-sql>=0.3.9,<0.4 moves from the [sql] extra into [project.dependencies], with a comment recording the wheel-availability rationale so the packaging policy stays legible.
  • The sql extra is kept as an alias, so existing graphifyy[sql] install commands and scripts keep resolving; it now adds nothing beyond the core install.
  • .sql is removed from _EXTRA_FOR_EXTENSION, so the .sql files silently extract 0 nodes when the [sql] extra is missing #1745 warning no longer sends users to a now-redundant extra; the extractor's missing-grammar error now says the install is incomplete and names the direct repair.
  • README extras table updated; the sql row is replaced with a note explaining the move and that a normal upgrade is all an existing [sql]/pipx inject install needs.
  • Version bump and changelog entry are deliberately not included — those look like maintainer release-commit territory here (chore: bump to 0.9.48), so comments and messages describe the grammar as a core dependency without naming a release.

Tests

New tests/test_sql_core_dependency.py pins the fix at the packaging layer so a regression (the dependency sliding back into an extra) fails the suite rather than resurfacing as a field report:

  • the grammar must be in [project.dependencies], with an upper bound;
  • .sql must not be mapped to an optional extra in the hint map;
  • the dev environment itself must satisfy the dependency;
  • an end-to-end table/view/procedure corpus must produce object nodes, contains/references/reads_from edges, and no dangling edge sources.

Existing SQL tests in tests/test_multilang.py convert from importorskip (which would silently skip the whole SQL surface if the grammar went missing — the exact failure mode this PR removes) to a hard import that fails instead. test_extract_warns_when_sql_grammar_missing now simulates genuine absence via both sys.modules and a find_spec monkeypatch, and asserts the new message names the direct repair, not the extra.

Validation

  • Fresh-venv wheel install pulls tree-sitter-sql transitively; a SQL fixture produces nodes/edges with no extra or pipx inject involved; the missing-grammar warning path proven live by blanking the module.
  • Full suite: 6 failed, 4835 passed, 28 skipped — the 6 (test_ollama.py, test_ollama_retry_cap.py) fail identically on unmodified v8 in this environment (local Ollama config), none touch SQL or packaging.
  • ruff check graphify/ tests/: all checks passed.
  • uv.lock regenerated.

Note

Independent of my companion PR #2960 fixing T-SQL routine recovery in graphify/extractors/sql.py; both touch tests/test_multilang.py, and a local merge of the two branches resolves trivially (we run the combined branch in production against the 391-file corpus).

egarcia74 and others added 4 commits August 23, 2026 00:13
A default install (uv tool install graphifyy / pipx install graphifyy)
shipped without SQL parsing: tree-sitter-sql lived behind the [sql]
extra, so every .sql file silently contributed nothing until the user
found the extra or ran pipx inject (Graphify-Labs#1745). The grammar ships prebuilt
abi3 wheels for every supported platform from 0.3.9, so the reason
tree-sitter-dm stays optional does not apply.

- promote tree-sitter-sql>=0.3.9,<0.4 into [project.dependencies];
  keep the [sql] extra as an alias so existing install scripts resolve
- drop .sql from _EXTRA_FOR_EXTENSION: a missing grammar now means a
  broken install, so the Graphify-Labs#1745 warning must not point at an extra
- reword the extractor's missing-grammar error to say the install is
  incomplete and name the direct repair

Bumps version to 0.9.49.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new test_sql_core_dependency.py: tree-sitter-sql must sit in
  [project.dependencies] with an upper bound, .sql must not map to an
  optional extra, and a table/view/procedure corpus must produce
  structural nodes plus contains/references/reads_from edges
- convert every importorskip("tree_sitter_sql") to a hard import: the
  grammar is core now, so its absence must fail the suite, not skip it
- update the Graphify-Labs#1745 warning test: the dev environment now ships the
  grammar, so genuine absence is simulated by blanking find_spec, and
  the message must say "core dependency" instead of hinting at
  graphifyy[sql]

Each packaging guard was mutation-tested (dep removed from pyproject,
.sql re-added to the extras map) and failed as required.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the sql row from the optional-extras table, add an upgrade note
(existing extra/pipx-inject users just run the normal upgrade), and add
the 0.9.49 changelog entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream cuts versions and batches changelog entries in maintainer
commits (see 'chore: bump to 0.9.48'), so a contributor PR should not
claim the next version number. Comments and messages now describe the
grammar as a core dependency without naming a release.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Promotes tree-sitter-sql from the sql extra to a core dependency in pyproject.toml, so .sql files extract on a default install (#1745); the sql extra stays as a harmless alias for old install scripts. Drops .sql from _EXTRA_FOR_EXTENSION and rewords extract_sql's missing-grammar error to name a broken install / direct reinstall rather than an optional extra, with README updated to match. Replaces the importorskip/_extract_sql_or_skip skips in the SQL tests with hard import tree_sitter_sql so absence fails instead of silently skipping.

Worth a look

  • SQL dependency is only added to the lockfile, not the packaging sourceuv.lock:1338 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1897 functions depend on the 710 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract() — 476 callers, 42 callees
  • new: _rebuild_code() — 98 callers, 50 callees
  • new: extract_xaml() — 19 callers, 17 callees
  • new: extract_js() — 80 callers, 3 callees
  • new: dispatch_command() — 2 callers, 119 callees
  • new: _get_extractor() — 26 callers, 6 callees
  • new: run_pipeline() — 8 callers, 13 callees
  • new: collect_files() — 17 callers, 6 callees
  • …and 25 more — each is listed as a finding

Verification — 1897 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1749 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_sql.

The verifier did not have enough to check extract\_sql, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 33 more finding(s) on lines outside this diff (see the check run).

@egarcia74

Copy link
Copy Markdown
Author

On the Graphify-review finding ("SQL dependency is only added to the lockfile, not the packaging source", flagged high at uv.lock:1338): this doesn't match the diff. tree-sitter-sql>=0.3.9,<0.4 is added to [project.dependencies] in pyproject.toml (line 50 on the branch tip) — the lockfile change is downstream of that. The PR also ships test_tree_sitter_sql_is_a_core_dependency, which pins exactly this invariant by parsing pyproject.toml and fails if the dependency ever slides back into an extra; it passes on this branch. (The review itself notes the finding was "agreed but NOT verified".) Happy to adjust if a maintainer sees something I'm missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant