kg: review follow-ups — observation-age stats, stats/health parity, embedding scan, hub install reconciliation - #4
Conversation
…nk repair, drop visibility Review of #4 found the scope-parity change regressed a case it did not intend. resolveScopeDB could not tell a scope named with --scope from one inherited out of config.json, so a stale defaultScope naming nothing — reachable, since SetDefaultScope does not verify the scope exists and config.json travels with a repo — turned `kg stats` from a legacy-database report into an error. The resolver now takes the requested scope separately: a NAMED scope that cannot load is still an error, while an inherited default falls back to the legacy database when the project has no scope configs at all. Two more from the same review: - The hub's reconcile pass collapsed "readlink failed" into "nothing to do", so a `current` symlink missing entirely — a partial copy, a restored backup, an operator — was left broken with every search 500ing, strictly worse than the split reconcile exists to fix. Missing links are now recreated from the registry; other readlink errors are logged, not silently skipped, as are registry entries failing path validation. - Dropping an embedding vector with unhandled component types is correct but as invisible as the zeroing it replaced, so the build logs the count once. New tests, each mutation-verified: kg stats routed through its own RunE (a test of the shared helper alone cannot show the caller still uses it) covering both the named-scope error and the inherited-default fallback; a missing-current reconcile case; humanAge's thresholds; and the report's age line asserted for order and real values rather than its label. Also from review: the median query builds its SKIP with strconv rather than embedding a shared constant in a format string, and the reconcile comment records the same-commit re-push residual it cannot detect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-review of #4 caught a severity escalation the last commit introduced. reconcileInstalls validates registry entries before joining them into paths, but dereferenced info.Commit to do it — and registry.json is hand-editable, where `{"graphs":{"g":null}}` unmarshals to a nil entry. Because the pass runs in the constructor, that panic aborted `kg hub serve` outright; before, the same entry only panicked inside a request, which net/http recovers per connection, so the hub kept serving every other graph. Nil entries are now skipped and logged. Also from the same review: - kg health no longer hard-fails when the observation count shifts mid-report. The median's SKIP offset came from counts taken by two earlier queries, so a concurrent kg index deleting rows made the offset overshoot and failed the entire run; collectObservationAge now takes its own count over the same predicate as the median query, and a vanished row set reports no age rather than an error. - reconcile logs the one state it cannot repair: current naming the registered commit whose directory is gone (searches fail, and the registry has no other answer). - printStats writes to an io.Writer like runHealth does, so the stats tests assert what was read — the previous fallback test passed even with stats bypassing the shared resolver entirely, which the reviewer demonstrated. - humanAge names negative durations instead of calling them "just now", and its year boundary is 12 30-day months so the ladder no longer prints "12mo" before "1y". - Fixture comments corrected to the six observations they now create. New tests for the null entry and the negative duration; the nil guard is mutation-verified (removing it panics the suite). Left as filed follow-ups, both from the reviewer: buildIndex's drop-on-reject branch stays uncovered until the loop body is extracted into a pure function, and the health/stats projectIDFromCwd divergence is provably cosmetic (findProjectRoot is idempotent) rather than a bug. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four real runs since the reviewer went live show 40 is too snug, and that turn count tracks how far the reviewer wanders rather than how big the diff is: PR lines turns outcome #8 584 24 approved #5 2,704 35 posted a review <- largest diff, fewest turns #4 848 41 hit the 40 cap #7 989 41 hit the 40 cap The largest PR by a factor of three used the fewest turns, so sizing the cap to the diff is the wrong model. Both failures also stopped exactly AT the cap rather than near it, so the real ceiling is unknown — a snug cap will keep catching runs that were about to finish. Treat the cap as a runaway guard rather than a budget. Reviews run on a subscription OAuth token, so the reported total_cost_usd is notional rather than billed, and the binding limit is the job's timeout-minutes. At the observed ~6s/turn, 120 turns is roughly 12 minutes against a 30-minute wall clock, so the timeout still catches a genuine runaway. This matters more than a short review would, because exceeding the cap fails as a non-zero exit with an empty result, not as a truncated review — the run produces nothing at all. #4 and #7 currently have no review for that reason. num_turns is logged on every run, so if reviews start landing near 120 the signal is there rather than showing up as an unexplained silence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four acknowledged-but-unfiled items from this week's reviews: - kg health gains ADR-009's observation-age stats: newest/oldest/median stored created_at over timestamped observations (zero/NULL rows excluded — they are the zero-timestamp count, and would report every legacy-bearing graph as centuries old), rendered as ages in the human report. Median is positional (SKIP (n-1)/2), pinned by a fixture with known 2020/2022 rows so a query grabbing first, last, or a legacy row fails the test. - kg stats shares resolveScopeDB with kg health: an explicitly named scope that cannot be loaded is now an error, never a silent fallback to the legacy database. - kglib's HNSW build accepts float64 embedding components (converted) and rejects vectors with any other component type instead of silently zeroing them — a zeroed component distorted every distance the vector participated in, invisibly. - The hub reconciles installs at construction: a hard kill between repointing `current` and writing the registry left search reading one commit's database with another commit's ProjectID — 200 with zero results, permanently. The registry write is the commit point, so an unregistered `current` target now rolls back to the registered commit; the orphaned directory is left for the next install's prune. All four are mutation-verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nk repair, drop visibility Review of #4 found the scope-parity change regressed a case it did not intend. resolveScopeDB could not tell a scope named with --scope from one inherited out of config.json, so a stale defaultScope naming nothing — reachable, since SetDefaultScope does not verify the scope exists and config.json travels with a repo — turned `kg stats` from a legacy-database report into an error. The resolver now takes the requested scope separately: a NAMED scope that cannot load is still an error, while an inherited default falls back to the legacy database when the project has no scope configs at all. Two more from the same review: - The hub's reconcile pass collapsed "readlink failed" into "nothing to do", so a `current` symlink missing entirely — a partial copy, a restored backup, an operator — was left broken with every search 500ing, strictly worse than the split reconcile exists to fix. Missing links are now recreated from the registry; other readlink errors are logged, not silently skipped, as are registry entries failing path validation. - Dropping an embedding vector with unhandled component types is correct but as invisible as the zeroing it replaced, so the build logs the count once. New tests, each mutation-verified: kg stats routed through its own RunE (a test of the shared helper alone cannot show the caller still uses it) covering both the named-scope error and the inherited-default fallback; a missing-current reconcile case; humanAge's thresholds; and the report's age line asserted for order and real values rather than its label. Also from review: the median query builds its SKIP with strconv rather than embedding a shared constant in a format string, and the reconcile comment records the same-commit re-push residual it cannot detect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-review of #4 caught a severity escalation the last commit introduced. reconcileInstalls validates registry entries before joining them into paths, but dereferenced info.Commit to do it — and registry.json is hand-editable, where `{"graphs":{"g":null}}` unmarshals to a nil entry. Because the pass runs in the constructor, that panic aborted `kg hub serve` outright; before, the same entry only panicked inside a request, which net/http recovers per connection, so the hub kept serving every other graph. Nil entries are now skipped and logged. Also from the same review: - kg health no longer hard-fails when the observation count shifts mid-report. The median's SKIP offset came from counts taken by two earlier queries, so a concurrent kg index deleting rows made the offset overshoot and failed the entire run; collectObservationAge now takes its own count over the same predicate as the median query, and a vanished row set reports no age rather than an error. - reconcile logs the one state it cannot repair: current naming the registered commit whose directory is gone (searches fail, and the registry has no other answer). - printStats writes to an io.Writer like runHealth does, so the stats tests assert what was read — the previous fallback test passed even with stats bypassing the shared resolver entirely, which the reviewer demonstrated. - humanAge names negative durations instead of calling them "just now", and its year boundary is 12 30-day months so the ladder no longer prints "12mo" before "1y". - Fixture comments corrected to the six observations they now create. New tests for the null entry and the negative duration; the nil guard is mutation-verified (removing it panics the suite). Left as filed follow-ups, both from the reviewer: buildIndex's drop-on-reject branch stays uncovered until the loop body is extracted into a pure function, and the health/stats projectIDFromCwd divergence is provably cosmetic (findProjectRoot is idempotent) rather than a bug. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2392fcf to
03cb73d
Compare
There was a problem hiding this comment.
Major Issues (should fix)
- src/kg/internal/knowledge/health.go:157-182 (
collectObservationAge, the max/min query) — Inconsistent race handling with the median query three lines below it. The function explicitly reasons that "a concurrentkg indexdeleting rows in between [the count and a follow-up query] would make the offset overshoot the result set" and handles that for the median query by returning(nil, nil)when!result.HasNext(). The max/min query is exposed to the exact same race (it also runs after the initialtimestampedcount, against the same non-transactional read connection), but itsif result.HasNext() { ... }block has noelse: if the aggregate returns no row in that narrow window,age.Newestandage.Oldestsilently stay at Go's zerotime.Time{}(year 1) instead of erroring or degrading tonil. That zero time then flows straight intoprintHealth'shumanAge(gen.Sub(oa.Newest)), rendering something like "newest ~2053y" — reproducing, in a report command whose entire point (ADR-009, per this PR's own summary) is to stop exactly this "reports every legacy-bearing graph as centuries old" failure mode. This is a live concurrency concern for a tool explicitly meant to be used by multiple agents/processes against the same graph. Fix: mirror the median query's handling — if!result.HasNext()after the max/min query, return(nil, nil)(or propagate a clear error) rather than falling through withagestill zero-valued.
Minor Issues (optional)
- src/kglib/hnsw_index.go:126-127 — The dropped-vectors log line (
"dropped %d of %d vectors read") computes the denominator asdropped+len(nodes), which excludes rows skipped earlier for missing/emptyrawEmb(if !ok || len(rawEmb) == 0 { continue }). The message therefore understates the true number of rows the query returned whenever some entities have no embedding at all, making the "of %d" figure not actually "vectors read" but "vectors read that had a non-empty embedding column." Consider tracking a separatereadcounter incremented once per row, or rewording to "of %d vectors with embedding data."
Positive Observations
resolveScopeDB's split between "user-named scope" (always errors) and "inherited default scope" (falls back to legacy when no scope configs exist) is implemented correctly and is exercised by both the new named-scope-error test and the inherited-default-fallback regression test — matches the documented parity contract exactly, including the subtle case where a named scope must error even when no scope configs exist at all.embeddingFromRawcorrectly replaces the old silent-zeroing type assertion with an explicit float32/float64 accept-list and drop-on-unknown-type, with solid unit coverage of the float64/mixed/reject cases.reconcileInstallsis careful and well-reasoned: it guards against nil registry entries and invalid path components before ever joining them into a filesystem path, distinguishes "current missing" from "current pointing at an unregistered commit" for logging purposes without a functional difference in the repair path, and correctly leaves data alone (rather than guessing) when the registered commit directory itself is missing orcurrentis an unreadable non-symlink. The accompanying tests (interrupted seed, missing symlink, null registry entry) exercise real crash-window scenarios rather than just the happy path.printStats/runStatsTorefactor cleanly threads anio.Writerthrough for testability, with no stale callers left on the old signature.
…eady handles Both findings from #4's first review. collectObservationAge runs two queries after its initial count, on the same non-transactional read connection, and both are exposed to a writer deleting rows in between. The median query says so in a comment and returns (nil, nil) when the row is gone. The max/min query above it had `if result.HasNext()` with no else, so in that window it fell through with Newest and Oldest still at the zero time. printHealth renders those as humanAge(gen.Sub(oa.Newest)) — roughly two thousand years. That is the "every legacy-bearing graph reads as centuries old" symptom this metric was added to expose, so the failure would have been indistinguishable from the bug it reports on. Mirror the median's handling. Hoisting the row read out of the if-block left the median's `row, err :=` declaring nothing new, so it becomes `=` with a note saying why. Also fixed the dropped-vector log's denominator. It computed dropped+len(nodes), but rows whose embedding column is missing or empty are skipped before `dropped` is incremented, so they appeared in neither term — the "of %d vectors read" figure silently excluded them. Count reads per row instead, so the number means what it says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Review Summary
- Files reviewed:
src/kg/health.go,src/kg/health_test.go,src/kg/internal/hub/install_test.go,src/kg/internal/hub/server.go(diff + surroundinginstall/checkGraphOwnership/handleGraphSearch/expandLayers/handleFederatedSearchcontext),src/kg/internal/knowledge/health.go,src/kg/stats.go,src/kg/stats_test.go,src/kglib/hnsw_index.go,src/kglib/hnsw_index_test.go - Overall verdict: REQUEST CHANGES-worthy issues not present; Major issues found → comment
- The two Major/Minor items flagged in the prior review round (median/max-min race asymmetry in
collectObservationAge, and thedropped/readdenominator in the hnsw log line) are both fixed correctly in this revision and are not re-raised.
Major Issues (should fix)
-
src/kg/internal/knowledge/health.go:176-190(collectObservationAge, the max/min query) — The newif !result.HasNext() { return nil, nil }guard almost certainly never fires for this query, so it doesn't actually close the race it's meant to close.RETURN max(...), min(...)with noGROUP BYis a full-aggregation query: under standard Cypher semantics (which Kuzu follows), aggregation over zero matched rows still returns exactly one row, withmax/minbound toNULL— it does not return zero rows the way the row-returning median query (ORDER BY ... SKIP ... LIMIT 1) does. So when the race actually happens (a concurrentkg indexdeletes the timestamped rows between the count and this query),result.HasNext()is stilltrue,row.GetValue()returns anilinterface for both columns, andtimeCell(line 244-254) hits itsv.(time.Time)type-assertion failure, returning an error liketimestamp cell has unexpected type <nil>. That error propagates up throughCollectHealthMetrics(fmt.Errorf("collect observation age: %w", err)) and fails the wholekg healthrun — a command whose own doc comment says it "always exits 0 — it is a report, not a gate." So the race this code believes it closed instead trades "silently wrong age" for "the entire health report errors out," in the narrow window it's meant to guard. Fix: after getting the row, treat anil/non-time.Timevalue fromGetValuethe same as the median query's!HasNext()case (return(nil, nil)) rather than routing it throughtimeCell's hard-error path — or havetimeCellaccept an "ok" return so the caller can distinguish "really missing" from "wrong type." -
src/kg/internal/hub/server.go:348, 393-394, 419, 480-484, 578-579, 730-731— This PR adds a nil-check forregistry.Graphsentries in the newly-addedreconcileInstalls(server.go:132-136) with a test proving{"graphs":{"g":null}}is a real, hand-editable state. But every other consumer ofreg.Graphs[name]in this same file only checksok, not whether the value itself isnil, and several dereference it unconditionally on the request path:handleGraphSearch(line 419info, ok := ...→ line 393info.ProjectIDinsidesearchGraph),expandLayers(line 480info, ok := ...→ line 484info.Layers, reachable fromhandleGraphSearch's layer expansion),handleFederatedSearch(line 578info := reg.Graphs[name]with no nil check at all, used at 579/info.Commit), andcheckGraphOwnership(line 730-731existing, ok := ...; existing.Repo, on the seed/push path). A registry hand-edited to hold a null entry for an existing graph name (not just the newly-added-and-not-yet-served case this PR tests) will nil-pointer-panic one of these request handlers instead of erroring cleanly — the same hazard this PR just spent a test and a fix on, one call away. Fix: giveRegistry(or these call sites) a helper likereg.Graphs[name]→(*GraphInfo, bool)that also treats a nil pointer as "not found," and use it at all five sites, or addinfo == nilchecks alongside the existing!okchecks.
Minor Issues (optional)
None beyond what's already fixed from the prior review round.
Security Findings
- None new. The nil-registry-entry panic above is a robustness/availability issue (self-inflicted via hand-edited
registry.json), not something remotely triggerable — not rated as a security finding.
Positive Observations
- The
collectObservationAgemedian query's own race handling (its!result.HasNext()branch) is correct as written — that query is a genuine row-returning query, unlike the max/min aggregate above it, so the asymmetry the prior review flagged is now resolved for that half. embeddingFromRawand thedropped/readcounters insrc/kglib/hnsw_index.goare correctly fixed from the prior round; the newTestEmbeddingFromRawcovers float32/float64/mixed/reject cases well.resolveScopeDB's split between "user-named scope" (always errors when unloadable) and "inherited default scope" (falls back to legacy only when no scope configs exist at all) is a real behavior fix over the oldstats.go, which used to fall back to legacy for any scope (including explicitly-named ones) wheneverlen(configs) == 0— the new logic and its tests (TestStatsErrorsOnUnloadableNamedScope,TestStatsFallsBackWhenInheritedDefaultScopeHasNoConfigs) correctly close that gap.reconcileInstalls's handling of null/invalid registry entries, missingcurrent, and unregisteredcurrenttargets is careful, and the accompanyinginstall_test.gotests exercise real crash-window scenarios (interrupted seed, missing symlink, null registry entry) rather than just the happy path.- The
printStats/runStatsTowriter-threading refactor is clean and leaves no stalefmt.Print*callers behind.
…is time
Two Majors from the second review round. The first is a correction to my own
previous fix.
The max/min guard I added last round never fires. `RETURN max(...), min(...)`
has no GROUP BY, so it is a full aggregation: over zero matched rows Kuzu
returns exactly ONE row with both columns NULL, not an empty result set.
Verified directly against the engine rather than reasoned about — a probe
against an empty store returns 1 row with nil in both columns.
So !HasNext() is not what the race looks like, and the NULLs went straight to
timeCell, whose type assertion turned them into "timestamp cell has unexpected
type <nil>". That error propagates out of CollectHealthMetrics and fails the
whole run — for a command whose own doc says it always exits 0, being a report
and not a gate. The previous fix therefore traded a silently wrong age for a
dead report in exactly the window it meant to protect.
Handle the NULL case where it actually occurs, in a new ageBoundsFromRow that
reports presence separately from error. A NULL means the rows went away, so it
degrades to no age like the median query does; a non-NULL value of the wrong
type stays a hard error, since that is a real defect and silent zeroing is the
bug this metric exists to catch. timeCell now delegates to a value-level
timeValue so an already-read cell can reuse it. Tested through the existing
row interface seam, covering all three outcomes, and mutation-checked.
Second: this PR added a nil guard to reconcileInstalls with a test proving
`{"graphs":{"g":null}}` is a real hand-editable state, but every other lookup
in server.go checked only `ok`. Graphs is map[string]*GraphInfo, so a null
entry is a present key with a nil value — ok is true and the next dereference
panics. Five request-path sites were exposed, one of them (federated search)
with no check at all before reading info.Commit.
Route every read through a Registry.graph helper that treats nil as absent.
The test for it needed a second attempt worth recording: asserting merely "not
200" passed even with the guard removed, because net/http recovers a handler
panic into a 500. Asserting 404 specifically is what makes it real — the
mutation now fails with 500 and the panic's error body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Review Summary
- Files reviewed:
src/kg/health.go,src/kg/health_test.go,src/kg/stats.go,src/kg/stats_test.go,src/kg/internal/knowledge/health.go,src/kg/internal/knowledge/health_test.go,src/kg/internal/hub/server.go,src/kg/internal/hub/registry.go,src/kg/internal/hub/install_test.go,src/kglib/hnsw_index.go,src/kglib/hnsw_index_test.go(plus surrounding context:install,checkGraphOwnership,handleGraphSearch/handleFederatedSearch/expandLayers,resolveScopeDBcall graph, and modulego.mod/replacewiring) - Both Major issues raised in the prior review rounds are verified fixed in this revision and are not re-raised:
collectObservationAge/ageBoundsFromRow(src/kg/internal/knowledge/health.go:176-291) now correctly treats Kuzu's one-row-of-NULLs full-aggregation result as "bounds absent" (checkingnewestRaw == nil || oldestRaw == nil) rather than relying on!HasNext(), which — as the second review round correctly diagnosed — never fires for aRETURN max(...), min(...)query. This is covered by a new direct unit test (TestAgeBoundsFromRow) exercising present/NULL/wrong-type cases without needing to race a real writer.- The null-registry-entry hazard (
src/kg/internal/hub/registry.go:30-45,server.go) is fixed with aRegistry.graph(name)helper that treats a nil pointer as absent, and every request-path read site (handleGetGraph,handleGraphSearch,expandLayers(both the outer and inner lookups),handleFederatedSearch(both the enumerate-all and named-list paths),checkGraphOwnership) now goes through it.reconcileInstallsalso guards its own loop. I traced all remaining directreg.Graphs[...]accesses and confirmed they are either writes (install,handleSeed) or test-only.
Critical Issues
None.
Major Issues
None.
Minor Issues (optional)
None beyond what was already fixed from prior rounds.
Security Findings
None.
Positive Observations
reconcileInstalls's doc comment carefully scopes what it can and cannot repair (explicitly calling out the one residual gap — a same-commit re-push interrupted before the registry write — as a known, deliberately-unhandled case) rather than overclaiming; the accompanying tests (TestReconcileRollsBackInterruptedSeed,TestReconcileRestoresMissingCurrent,TestReconcileSurvivesNullRegistryEntry,TestRequestHandlersSurviveNullRegistryEntry) exercise real crash-window states, not just happy paths.resolveScopeDB's consolidation ofkg health's andkg stats' previously-duplicated scope-resolution logic is a genuine simplification, with the intentional named-vs-inherited-default asymmetry preserved and independently tested for both callers.humanAge's threshold ladder and themedian/oldest/newestordering in the human-readable report are pinned with precise boundary tests (TestHumanAge,TestHealthHumanOutputAgeLine) rather than loose substring checks.embeddingFromRaw's float32/float64 accept-list with explicit rejection of other types (instead of silently zeroing unknown components) is correctly reasoned and unit-tested for the float64/mixed/reject cases, and thedropped/readcounter fix from the prior review round is applied correctly.
Summary
Clears the four acknowledged-but-unfiled follow-ups from the PR #3 review rounds and the 2026-08-23 audit:
kg healthobservation-age stats (the ADR-009 scope cut; see ADR-009) — newest/oldest/median storedcreated_at, excluding zero-timestamp rows, rendered as ages in the human report. The median is positional and pinned by fixture rows dated 2020/2022, so a query returning the first, last, or a legacy row fails the test.kg stats/kg healthscope parity — both now shareresolveScopeDB; an explicitly named scope that cannot be loaded errors instead of silently reporting the legacy database.float64components and rejects vectors with unhandled component types; the old bare type assertion silently zeroed them, distorting every distance the vector participated in.current-symlink rename and the registry write left a graph answering 200-with-zero-results permanently. Server construction now rollscurrentback to the registered commit (the registry write is install's commit point); the orphaned commit directory is left for the next install's prune.All four fixes are mutation-verified;
go build/vet/testgreen insrc/kgandsrc/kglib.🤖 Generated with Claude Code