Skip to content

feat(kg): index package declarations for Java, Kotlin, and Scala - #14

Merged
bawoodruff merged 1 commit into
mainfrom
feat/kg-jvm-package-indexing
Aug 29, 2026
Merged

bawoodruff merged 1 commit into
mainfrom
feat/kg-jvm-package-indexing

Conversation

@bawoodruff

Copy link
Copy Markdown
Contributor

The indexer minted a package entity for Go only. Go's package clause is a bare identifier (auth); the JVM languages declare a dotted namespace (com.depop.auth.client). So on a Scala/Java estate the graph contained no package entities at all.

Why this blocks #7

Cross-layer package linking matches an import to a package by dotted prefix, and requires a namespace specific enough to be worth matching (minPackageSegments = 3). A bare Go identifier is one segment and never qualifies — so LinkPackages returns Derived: 0 against anything the indexer produced, and works only on hand-built fixtures.

Verified this handling has never existed here:

  • git log -S "package_declaration" and -S "package_header" — nothing, across all branches
  • langConfig had no package field
  • EntityTypePackage was written at exactly one site: the Go package_clause handler

Approach

Generalise rather than add three special cases. langConfig gains PackageNodeTypes and an optional extractPackageName; the switch arm that was hardcoded to Go's "package_clause" now consults the config like every other node kind.

One default extractor covers all four languages, because each spells the name as a single named child and they differ only in its type:

language package node name child
Go package_clause package_identifier
Scala package_clause package_identifier
Java package_declaration scoped_identifier
Kotlin package_header identifier

Node types were confirmed by parsing real sources through tree-sitter, not read off the grammars — Scala reusing Go's node names, and Kotlin using a plain identifier for a dotted name, are both easy to get wrong from documentation.

Go behaviour is unchanged; it is now declared rather than hardcoded.

Tests

Each case indexes a real source file end to end, so it covers the grammar node type, the extractor, and the config wiring together — not just the extractor in isolation.

All three mutations caught:

mutation result
remove package_declaration from Java config java case fails
remove package_header from Kotlin config kotlin case fails
narrow extractor to package_identifier only java + kotlin fail; go + scala still pass

The test asserts segment counts directly rather than importing minPackageSegments, so it does not depend on #7 shipping.

After this merges

#7's Critical is resolvable: re-index the estate and re-run the measurement against what the code actually produces. The design doc's Follow-up currently states the inverse of reality — that the indexers mint packages "only for dotted namespaces", when Go was the only source and is never dotted — and should be corrected alongside those numbers.

🤖 Generated with Claude Code

The indexer minted a `package` entity for Go only. Go's package clause is a
bare identifier ("auth"), while the JVM languages declare a dotted namespace
("com.depop.auth.client") — so on a Scala/Java estate the graph contained no
package entities at all.

That gap is why cross-layer package linking (#7) cannot work on the corpus it
was designed for: matching an import to a package needs a namespace specific
enough to be worth matching, and a bare Go identifier never is. Verified this
handling has never existed here — `git log -S package_declaration` and
`-S package_header` return nothing across all branches, langConfig had no
package field, and EntityTypePackage was written at exactly one site.

Generalise rather than add three more special cases. langConfig gains
PackageNodeTypes and an optional extractPackageName; the switch arm that was
hardcoded to Go's "package_clause" now consults the config like every other
node kind. One default extractor covers all four languages, because each spells
the name as a single named child and they differ only in its type:

	Go      package_clause      -> package_identifier
	Scala   package_clause      -> package_identifier
	Java    package_declaration -> scoped_identifier
	Kotlin  package_header      -> identifier

Node types were confirmed by parsing real sources through tree-sitter rather
than read off the grammars — Scala reusing Go's node names, and Kotlin using a
plain "identifier" for a dotted name, are both easy to get wrong from docs.

Go behaviour is unchanged; it is now declared rather than hardcoded.

Tests index a real file per language end to end, so each case covers the
grammar node type, the extractor, and the config wiring together. All three
mutations caught: removing either JVM node type fails that language, and
narrowing the extractor to package_identifier fails Java and Kotlin while Go
and Scala still pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

  • Files reviewed: src/kg/internal/knowledge/indexer_treesitter.go, src/kg/internal/knowledge/indexer_package_test.go (plus surrounding context: kg_types.go, indexer.go, indexer_treesitter_test.go, .github/workflows/ci.yml)
  • Overall verdict: APPROVE

This is a clean, well-scoped generalization of a hardcoded Go-only special case into config-driven behavior. Traced the full path: langConfig.PackageNodeTypes/extractPackageName (indexer_treesitter.go:36-45) → dispatch via slices.Contains(cfg.PackageNodeTypes, nodeType) (line 634, correctly mutually exclusive with the other node-type switch arms since no language reuses a package node type elsewhere) → extractPackageDeclName (line 113) using the same writeEntity/RelBelongsTo plumbing the old Go-only arm used, so dedup and codeDerivedIDPrefixes clearing (kg_types.go:39, already covering package:) continue to work unchanged. Verified slices was already imported, so no new import risk. Confirmed via gh pr view 14 --json statusCheckRollup that the kg module's CI leg (go test -race ./..., no -short, so the new end-to-end test actually executes) passed — that test would fail immediately if any of the four claimed grammar node types (Go/Scala package_identifier, Java scoped_identifier, Kotlin identifier) were wrong, which gives good confidence in the core empirical claim I couldn't verify myself (sandboxed Bash here can't run go build/go test).

Critical Issues (must fix before merge)

None.

Major Issues (should fix)

None.

Minor Issues (optional)

  • src/kg/internal/knowledge/indexer_treesitter.go:634-645 — Since walkNode recurses into all children regardless of the switch arm taken, a source file with multiple/chained package clauses (Scala's package a.b / package c chaining idiom, or a Java package-info.java edge case) would visit each package_clause/package_declaration independently and mint a separate, fragmented package entity per clause rather than one composed namespace. I couldn't reproduce this (no test-execution access in this environment) — flagging as a plausible gap for a human/CI check, not a confirmed defect, and it's a pre-existing style of ambiguity the PR doesn't claim to solve.
  • src/kg/internal/knowledge/indexer_treesitter.go:39-40 — Doc comment says PackageNodeTypes is "Empty for languages with no package concept (C, JS/TS, Python)," but Groovy (also configured in this file, .groovy at line 521) does have a package concept and was left out of this PR's scope (title says Java/Kotlin/Scala only). Worth a one-line comment clarifying Groovy is deliberately out of scope for now rather than "no package concept," so a future reader doesn't assume it was considered and rejected.

Security Findings

None.

Positive Observations

  • Single shared extractPackageDeclName extractor cleanly covers all four languages by exploiting the fact each spells the name as one named child, differing only in child type — good use of the existing langConfig extensibility pattern (mirrors extractFuncName/extractImportPath).
  • New test indexes real source end-to-end (not just unit-testing the extractor), which actually exercises grammar-node-type + extractor + config wiring together — exactly the kind of test that would have caught the sort of doc-vs-grammar mismatch the PR description warns about.
  • codeDerivedIDPrefixes already contained the package: prefix, so no follow-up needed there for re-index/clear correctness.
  • PR explicitly verifies "Go behaviour is unchanged" and the code confirms it: NamedChild walk + TrimSpace is a strict superset of the old Child-based scan for Go's package_identifier case.

@bawoodruff
bawoodruff merged commit 5211bc2 into main Aug 29, 2026
6 checks passed
@bawoodruff
bawoodruff deleted the feat/kg-jvm-package-indexing branch August 29, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant