feat(kg): index package declarations for Java, Kotlin, and Scala - #14
Conversation
The indexer minted a `package` entity for Go only. Go's package clause is a
bare identifier ("auth"), while the JVM languages declare a dotted namespace
("com.depop.auth.client") — so on a Scala/Java estate the graph contained no
package entities at all.
That gap is why cross-layer package linking (#7) cannot work on the corpus it
was designed for: matching an import to a package needs a namespace specific
enough to be worth matching, and a bare Go identifier never is. Verified this
handling has never existed here — `git log -S package_declaration` and
`-S package_header` return nothing across all branches, langConfig had no
package field, and EntityTypePackage was written at exactly one site.
Generalise rather than add three more special cases. langConfig gains
PackageNodeTypes and an optional extractPackageName; the switch arm that was
hardcoded to Go's "package_clause" now consults the config like every other
node kind. One default extractor covers all four languages, because each spells
the name as a single named child and they differ only in its type:
Go package_clause -> package_identifier
Scala package_clause -> package_identifier
Java package_declaration -> scoped_identifier
Kotlin package_header -> identifier
Node types were confirmed by parsing real sources through tree-sitter rather
than read off the grammars — Scala reusing Go's node names, and Kotlin using a
plain "identifier" for a dotted name, are both easy to get wrong from docs.
Go behaviour is unchanged; it is now declared rather than hardcoded.
Tests index a real file per language end to end, so each case covers the
grammar node type, the extractor, and the config wiring together. All three
mutations caught: removing either JVM node type fails that language, and
narrowing the extractor to package_identifier fails Java and Kotlin while Go
and Scala still pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Review Summary
- Files reviewed:
src/kg/internal/knowledge/indexer_treesitter.go,src/kg/internal/knowledge/indexer_package_test.go(plus surrounding context:kg_types.go,indexer.go,indexer_treesitter_test.go,.github/workflows/ci.yml) - Overall verdict: APPROVE
This is a clean, well-scoped generalization of a hardcoded Go-only special case into config-driven behavior. Traced the full path: langConfig.PackageNodeTypes/extractPackageName (indexer_treesitter.go:36-45) → dispatch via slices.Contains(cfg.PackageNodeTypes, nodeType) (line 634, correctly mutually exclusive with the other node-type switch arms since no language reuses a package node type elsewhere) → extractPackageDeclName (line 113) using the same writeEntity/RelBelongsTo plumbing the old Go-only arm used, so dedup and codeDerivedIDPrefixes clearing (kg_types.go:39, already covering package:) continue to work unchanged. Verified slices was already imported, so no new import risk. Confirmed via gh pr view 14 --json statusCheckRollup that the kg module's CI leg (go test -race ./..., no -short, so the new end-to-end test actually executes) passed — that test would fail immediately if any of the four claimed grammar node types (Go/Scala package_identifier, Java scoped_identifier, Kotlin identifier) were wrong, which gives good confidence in the core empirical claim I couldn't verify myself (sandboxed Bash here can't run go build/go test).
Critical Issues (must fix before merge)
None.
Major Issues (should fix)
None.
Minor Issues (optional)
src/kg/internal/knowledge/indexer_treesitter.go:634-645— SincewalkNoderecurses into all children regardless of the switch arm taken, a source file with multiple/chained package clauses (Scala'spackage a.b/package cchaining idiom, or a Javapackage-info.javaedge case) would visit eachpackage_clause/package_declarationindependently and mint a separate, fragmented package entity per clause rather than one composed namespace. I couldn't reproduce this (no test-execution access in this environment) — flagging as a plausible gap for a human/CI check, not a confirmed defect, and it's a pre-existing style of ambiguity the PR doesn't claim to solve.src/kg/internal/knowledge/indexer_treesitter.go:39-40— Doc comment saysPackageNodeTypesis "Empty for languages with no package concept (C, JS/TS, Python)," but Groovy (also configured in this file,.groovyat line 521) does have a package concept and was left out of this PR's scope (title says Java/Kotlin/Scala only). Worth a one-line comment clarifying Groovy is deliberately out of scope for now rather than "no package concept," so a future reader doesn't assume it was considered and rejected.
Security Findings
None.
Positive Observations
- Single shared
extractPackageDeclNameextractor cleanly covers all four languages by exploiting the fact each spells the name as one named child, differing only in child type — good use of the existinglangConfigextensibility pattern (mirrorsextractFuncName/extractImportPath). - New test indexes real source end-to-end (not just unit-testing the extractor), which actually exercises grammar-node-type + extractor + config wiring together — exactly the kind of test that would have caught the sort of doc-vs-grammar mismatch the PR description warns about.
codeDerivedIDPrefixesalready contained thepackage:prefix, so no follow-up needed there for re-index/clear correctness.- PR explicitly verifies "Go behaviour is unchanged" and the code confirms it:
NamedChildwalk +TrimSpaceis a strict superset of the oldChild-based scan for Go'spackage_identifiercase.
The indexer minted a
packageentity for Go only. Go's package clause is a bare identifier (auth); the JVM languages declare a dotted namespace (com.depop.auth.client). So on a Scala/Java estate the graph contained no package entities at all.Why this blocks #7
Cross-layer package linking matches an import to a package by dotted prefix, and requires a namespace specific enough to be worth matching (
minPackageSegments = 3). A bare Go identifier is one segment and never qualifies — soLinkPackagesreturnsDerived: 0against anything the indexer produced, and works only on hand-built fixtures.Verified this handling has never existed here:
git log -S "package_declaration"and-S "package_header"— nothing, across all brancheslangConfighad no package fieldEntityTypePackagewas written at exactly one site: the Gopackage_clausehandlerApproach
Generalise rather than add three special cases.
langConfiggainsPackageNodeTypesand an optionalextractPackageName; the switch arm that was hardcoded to Go's"package_clause"now consults the config like every other node kind.One default extractor covers all four languages, because each spells the name as a single named child and they differ only in its type:
package_clausepackage_identifierpackage_clausepackage_identifierpackage_declarationscoped_identifierpackage_headeridentifierNode types were confirmed by parsing real sources through tree-sitter, not read off the grammars — Scala reusing Go's node names, and Kotlin using a plain
identifierfor a dotted name, are both easy to get wrong from documentation.Go behaviour is unchanged; it is now declared rather than hardcoded.
Tests
Each case indexes a real source file end to end, so it covers the grammar node type, the extractor, and the config wiring together — not just the extractor in isolation.
All three mutations caught:
package_declarationfrom Java configpackage_headerfrom Kotlin configpackage_identifieronlyThe test asserts segment counts directly rather than importing
minPackageSegments, so it does not depend on #7 shipping.After this merges
#7's Critical is resolvable: re-index the estate and re-run the measurement against what the code actually produces. The design doc's Follow-up currently states the inverse of reality — that the indexers mint packages "only for dotted namespaces", when Go was the only source and is never dotted — and should be corrected alongside those numbers.
🤖 Generated with Claude Code