fix(insights): reconcile only defined words (M19c)

PLAN.md

@@ -5491,7 +5491,8 @@ never looks like a fact.
   unranked word after, by text) and cut at `--terms`. The module prompt
   shows, per word, each type's `concern`, `sense` and `meaning`, or "no type
   defined it", read through `RecordSource.words(id)` (a `block -> '$.words'`
-  projection in the store) only when the prompt renders.
+  projection in the store) only when the prompt renders. *Since, a word no
+  type defined is not shown: "As built — only defined words reconciled".*
 - **Fingerprints** take, per member, the identifiers and words asked and a
   module's glossary words; never what a type said of them. The same
   `--terms` reuses every module, and another redoes exactly the modules whose
@@ -5683,6 +5684,32 @@ Their senses were grammar ("lookup criterion", "boolean predicate",
 - **The Java fixture's plan** is now 114 474 prompt and 52 415 completion
   tokens (was 114 483 and 52 900).
 
+**As built — only defined words reconciled (2026-11-03).** Of the ~50
+words the PetClinic modules were asked to reconcile, 2 no type had defined:
+`Pet` skipped `birth` (of `birthDate`), and the `owner` module paid for
+`birth`, "ambiguous: undefined term"; `BaseEntity` skipped `base`, and the
+`model` module wrote it a meaning anyway. The decisions:
+
+- **Asked = a type defined it.** `glossaryAsked(member)` in `context.ts` keeps
+  the candidates with at least one type entry; the prompt lists those, and on
+  receipt the module's glossary keeps only those. The "no type defined it"
+  line is gone. A module whose candidates all went undefined shows no
+  glossary section and keeps an empty `glossary`.
+- **The candidates stay hashed, not the asked.** The fingerprint takes the
+  cut list as before; which of it a type defined is read when the prompt
+  renders, like the summaries it quotes. No record goes stale: PetClinic's
+  `--dry-run` statuses are the same, unit by unit. A type that is
+  re-explained changes the module's fingerprint already, as its container.
+- **Cut, then filter.** `--terms N` still cuts the candidates; a module asks
+  at most N, fewer when some went undefined. Filtering first would make the
+  hashed cut a function of what the types said.
+- **The estimate** counts a candidate unless every type that spells it has
+  `words` and none defined it: a type not explained yet, or under prompt 1,
+  may still define it, so a cold plan counts every candidate as before. On
+  PetClinic's warm plan, 250 completion tokens fewer (`birth`, `base`). A cold
+  module prompt now prints no glossary lines at all, so the plan's input is a
+  little lower (the Java fixture: 114 136 prompt tokens, was 114 474).
+
 ### 20.4 Milestone split
 
 **M19a — the lexicon, the split, the artifact** (closes the profile half of #19)

docs/cli.md

@@ -337,7 +337,7 @@ which on a prepaid account is what keeps a low balance from refusing every call.
 Each unit is also asked for the meaning of the words its names spell (`words`:
 its own name, an operation's parameters, a type's fields, split as `vocabulary`
 splits them), and a module reconciles what its types said into a `glossary`
-of its heaviest `--terms N` words (default 60). Design record: [insights.md](insights.md).
+of its heaviest `--terms N` words (default 60) that a type defined. Design record: [insights.md](insights.md).
 
 ### `insights`
 Asks the insights store what `explain` has written — and opens nothing else:

docs/insights.md

@@ -117,7 +117,8 @@ and `in` is restated from the split. A module is also shown what its types
 said of the words they spell — read from the store through a `words(id)`
 projection, as summaries are, and only when its prompt renders — heaviest
 first by the vocabulary's module weights, cut at `--terms N`, and answers a
-`glossary` that reconciles them, flagging a word its types disagree on as
+`glossary` that reconciles those a type defined (a word its types all left
+undefined is not asked), flagging a word its types disagree on as
 `polysemous`. Templated records carry no `words`.
 
 `PROMPT_VERSION` is `"2"`. A record explained under prompt 1 keeps decoding and
@@ -332,10 +333,11 @@ is handed an OPEN database, so the path, the file and `node:sqlite` itself
 Every record carries a sha256 over what it was computed from: the prompt
 version, the model slug, the level, the source slices, the comments, the
 signatures, a digest of the facts shown, the words its prompt asks to define
-and a module's glossary words (so a lexicon correction re-explains exactly the
-units whose words it changed, and another `--terms` exactly the modules whose
-cut moved), the **fingerprints of the units it depended on**, and the ids of
-dependencies that had no record. Merkle-style:
+and a module's glossary candidates (so a lexicon correction re-explains the
+units whose words it changed, and through the next input what depends on them,
+and another `--terms` exactly the modules whose cut moved; which candidates a
+type defined is not hashed), the **fingerprints of the units it depended on**,
+and the ids of dependencies that had no record. Merkle-style:
 change one leaf's source and exactly its transitive dependents and containers
 change; nothing else does. The explanation text is deliberately **not**
 hashed — a non-deterministic answer must never cascade re-runs.

packages/cli/test/explain.test.ts

@@ -62,7 +62,7 @@ function glossaryAsked(user: string): string[] {
   const head = user.indexOf("Words its types define");
   if (head < 0) return [];
   return user.slice(head).split("\n").slice(1).flatMap((line) => {
-    const match = /^- ([^:\s]+)(?:: no type defined it)?$/u.exec(line);
+    const match = /^- (\S+)$/u.exec(line);
     return match === null ? [] : [match[1] ?? ""];
   });
 }

packages/insights/src/context.ts

@@ -65,6 +65,17 @@ export interface GlossaryWord {
   readonly word: string;
   /** Read from the record source on first use: a type's `words` entry for this word, by type name. */
   readonly entries: readonly { readonly type: string; readonly entry: WordEntry }[];
+  /** A type that spells it has no `words` yet (not explained, or under prompt 1): it may still define it. */
+  readonly pending: boolean;
+}
+
+/**
+ * The glossary words a module is asked to reconcile: those a type defined. A
+ * word its types all left undefined has nothing to reconcile, and asking
+ * paid for an "undefined term" (PetClinic's `birth`). Read when it renders.
+ */
+export function glossaryAsked(member: MemberPack): readonly GlossaryWord[] {
+  return member.glossary.filter((g) => g.entries.length > 0);
 }
 
 export interface DependencySummary {
@@ -275,31 +286,40 @@ function typeNames(env: ContextEnv, dossier: TypeDossier): NameWords[] {
 
 /**
  * The words a module's types spell, heaviest first by the speller's ranking
- * (an unranked word after, by text), cut at `terms`. What the types said of
- * them is read once, when a prompt first asks.
+ * (an unranked word after, by text), cut at `terms` — the candidates, which
+ * are hashed. What the types said of them is read once, when a prompt first
+ * asks, and decides which are asked (`glossaryAsked`).
  */
 function glossaryOf(env: ContextEnv, types: readonly TypeDossier[], moduleId: EntityId): GlossaryWord[] {
-  const spelled = new Set(types.flatMap((t) => typeNames(env, t).flatMap((n) => n.words)));
+  const spelledBy = new Map<string, EntityId[]>();
+  for (const t of types) {
+    for (const word of new Set(typeNames(env, t).flatMap((n) => n.words))) spelledBy.set(word, [...(spelledBy.get(word) ?? []), t.id]);
+  }
   const rank = new Map(env.speller.ranked(moduleId).map((word, i) => [word, i]));
-  const ordered = [...spelled].sort((a, b) => (rank.get(a) ?? Infinity) - (rank.get(b) ?? Infinity) || (a < b ? -1 : a > b ? 1 : 0));
-  let said: Map<string, { type: string; entry: WordEntry }[]> | undefined;
-  const read = (): Map<string, { type: string; entry: WordEntry }[]> => {
-    if (said === undefined) {
-      said = new Map();
+  const ordered = [...spelledBy.keys()].sort((a, b) => (rank.get(a) ?? Infinity) - (rank.get(b) ?? Infinity) || (a < b ? -1 : a > b ? 1 : 0));
+  let read: { said: Map<string, { type: string; entry: WordEntry }[]>; wordless: Set<EntityId> } | undefined;
+  const records = (): NonNullable<typeof read> => {
+    if (read === undefined) {
+      read = { said: new Map(), wordless: new Set() };
       for (const t of types) {
-        for (const entry of env.records.words(t.id) ?? []) {
-          const list = said.get(entry.word) ?? [];
+        const words = env.records.words(t.id);
+        if (words === undefined) read.wordless.add(t.id);
+        for (const entry of words ?? []) {
+          const list = read.said.get(entry.word) ?? [];
           list.push({ type: t.name ?? t.id, entry });
-          said.set(entry.word, list);
+          read.said.set(entry.word, list);
         }
       }
     }
-    return said;
+    return read;
   };
   return ordered.slice(0, Math.max(0, env.terms)).map((word) => ({
     word,
     get entries() {
-      return read().get(word) ?? [];
+      return records().said.get(word) ?? [];
+    },
+    get pending() {
+      return (spelledBy.get(word) ?? []).some((id) => records().wordless.has(id));
     },
   }));
 }

packages/insights/src/prompt.ts

@@ -1,4 +1,4 @@
-import type { ContextPack, DependencySummary, MemberPack } from "./context.js";
+import { glossaryAsked, type ContextPack, type DependencySummary, type MemberPack } from "./context.js";
 import { CONCEPT_DEFINITIONS, DOMAIN_CONCEPTS, ENFORCEMENTS, EVENT_KINDS, FIELD_KINDS, INTERFACE_ROLES, OPERATION_OWNERS } from "./ddd.js";
 import { responseSchemaName, type Level } from "./schema.js";
 
@@ -170,11 +170,11 @@ function memberSection(member: MemberPack, full: boolean): string[] {
   if (member.names.length > 0) {
     lines.push("Words to define in `words` (each once, with every identifier it appears in):", ...member.names.map((n) => `- \`${n.identifier}\`: ${n.words.join(", ")}`));
   }
-  if (member.glossary.length > 0) {
+  const glossary = glossaryAsked(member);
+  if (glossary.length > 0) {
     lines.push("Words its types define, heaviest first (reconcile each in `glossary`):");
-    for (const { word, entries } of member.glossary) {
-      if (entries.length === 0) lines.push(`- ${word}: no type defined it`);
-      else lines.push(`- ${word}`, ...entries.map(({ type, entry }) => `  - ${type}: ${entry.concern}, "${oneLine(entry.sense)}": ${oneLine(entry.meaning)}`));
+    for (const { word, entries } of glossary) {
+      lines.push(`- ${word}`, ...entries.map(({ type, entry }) => `  - ${type}: ${entry.concern}, "${oneLine(entry.sense)}": ${oneLine(entry.meaning)}`));
     }
   }
   return lines;
@@ -184,11 +184,15 @@ function oneLine(text: string): string {
   return text.replace(/\s+/gu, " ").trim();
 }
 
-/** Distinct words over the members a prompt fully shows, and the glossary entries it asks for. */
+/**
+ * Distinct words over the members a prompt fully shows, and the glossary
+ * entries it asks for — counting, in a plan, a word a type still to explain
+ * may define, since the prompt sent after it may list it.
+ */
 function askedOf(members: readonly MemberPack[]): Prompt["asked"] {
   return {
     words: members.reduce((sum, m) => sum + new Set(m.names.flatMap((n) => n.words)).size, 0),
-    glossary: members.reduce((sum, m) => sum + m.glossary.length, 0),
+    glossary: members.reduce((sum, m) => sum + m.glossary.filter((g) => g.entries.length > 0 || g.pending).length, 0),
   };
 }
 

packages/insights/src/run.ts

@@ -1,5 +1,5 @@
 import type { z } from "zod";
-import { contextPackFor, type ContextEnv, type MemberPack } from "./context.js";
+import { contextPackFor, glossaryAsked, type ContextEnv, type MemberPack } from "./context.js";
 import type { Unit } from "./order.js";
 import { unitFingerprint, type PlanEnv, type PlanStep, type RunPlan } from "./plan.js";
 import { renderPrompts, type Prompt } from "./prompt.js";
@@ -230,7 +230,7 @@ function restateWords(block: Block, member: MemberPack | undefined): Block {
   };
   const words = once(block.words, (word) => asked.has(word)).map((entry) => clampConfidence({ ...entry, in: asked.get(entry.word) ?? [] }));
   if (!("glossary" in block)) return { ...block, words };
-  const listed = new Set(member.glossary.map((g) => g.word));
+  const listed = new Set(glossaryAsked(member).map((g) => g.word));
   return { ...block, words, glossary: once(block.glossary, (word) => listed.has(word)) };
 }
 

packages/insights/test/prompt.test.ts

@@ -202,9 +202,10 @@ describe("words to define (PLAN.md §20.3)", () => {
   }
 
   const entry = (word: string, sense: string, meaning: string): WordEntry => ({ word, in: [], concern: "domain", sense, meaning, confidence: 0.8 });
-  const typeRecord = (id: string, words: WordEntry[]): InsightRecord => ({
+  /** Without `words`, a record explained under prompt 1. */
+  const typeRecord = (id: string, words: WordEntry[] | undefined): InsightRecord => ({
     t: "i", id, level: "type", kind: "class", origin: "llm", fingerprint: "1".repeat(64),
-    block: { name: id, description: "A type.", concept: "entity", eventKind: null, interfaceRole: null, aggregateRoot: null, containedIn: null, syncPattern: null, identity: null, fields: [], invariants: [], stateMachine: null, relatesTo: [], exposes: [], dependsOn: [], domainTerms: [], words, confidence: 0.7 } satisfies TypeBlock,
+    block: { name: id, description: "A type.", concept: "entity", eventKind: null, interfaceRole: null, aggregateRoot: null, containedIn: null, syncPattern: null, identity: null, fields: [], invariants: [], stateMachine: null, relatesTo: [], exposes: [], dependsOn: [], domainTerms: [], ...(words === undefined ? {} : { words }), confidence: 0.7 } satisfies TypeBlock,
   });
 
   it("asks an operation for the words of its name and of its parameters, one line per identifier, never their class", () => {
@@ -240,15 +241,35 @@ describe("words to define (PLAN.md §20.3)", () => {
         '  - SortOrder: domain, "a sequence": The order rows are listed in.',
         "- line",
         '  - OrderLine: domain, "an item": One product of an order.',
-        "- price: no type defined it",
       ].join("\n"),
     );
-    expect(glossary.indexOf("- sort:")).toBeGreaterThan(glossary.indexOf("- price:"));
     expect(glossary).not.toContain("- shop");
 
-    const cut = renderPrompts(shop(records, 2).pack(SHOP), 12)[0]!.user;
-    expect(cut).toContain("- line\n");
-    expect(cut).not.toContain("- price");
+    const cut = renderPrompts(shop(records, 1).pack(SHOP), 12)[0]!.user;
+    expect(cut).toContain("- order\n");
+    expect(cut).not.toContain("- line");
+  });
+
+  it("asks a module to reconcile no word its types left undefined, and estimates the words a type still to explain may define", () => {
+    const listed = (records: InsightRecord[]) => {
+      const user = renderPrompts(shop(records).pack(SHOP), 12)[0]!.user;
+      const head = user.indexOf("Words its types define");
+      return head < 0 ? [] : user.slice(head).split("\n").flatMap((line) => /^- ([^\s:]+)/u.exec(line)?.slice(1) ?? []);
+    };
+    const asked = (records: InsightRecord[]) => renderPrompts(shop(records).pack(SHOP), 12)[0]!.asked.glossary;
+    const line = typeRecord(LINE, [entry("order", "a purchase", "Bought."), entry("line", "an item", "One product.")]);
+    const sort = typeRecord(SORT, [entry("order", "a sequence", "How rows are listed.")]);
+
+    // Both explained: `price` and `unit` (OrderLine's), `sort` and `key` (SortOrder's) were left undefined.
+    expect(listed([line, sort])).toEqual(["order", "line"]);
+    expect(asked([line, sort])).toBe(2);
+    // SortOrder not explained yet, or under prompt 1: what it spells may still be defined.
+    expect(listed([line])).toEqual(["order", "line"]);
+    expect(asked([line])).toBe(4);
+    expect(asked([line, typeRecord(SORT, undefined)])).toBe(4);
+    // A cold plan counts every candidate; it lists none, since no type has said anything.
+    expect(listed([])).toEqual([]);
+    expect(asked([])).toBe(6);
   });
 
   it("asks a module whose types spell no word for no glossary", () => {

packages/insights/test/run.test.ts

@@ -84,7 +84,7 @@ function glossaryAsked(user: string): string[] {
     .split("\n")
     .slice(1)
     .flatMap((line) => {
-      const match = /^- ([^:]+)(?:: no type defined it)?$/u.exec(line);
+      const match = /^- (\S+)$/u.exec(line);
       return match === null ? [] : [match[1] ?? ""];
     });
 }
@@ -379,6 +379,24 @@ describe("executeRun", () => {
     expect(module?.level === "module" && module.block.glossary?.map((g) => g.word)).toEqual(["total"]);
   });
 
+  it("asks a module about no word its types left undefined, and drops an entry for one", async () => {
+    const completer = fakeCompleter({ words: (id) => (id === T ? [] : undefined) });
+    const inner = completer as Completer;
+    const insisting: Completer = async (request) => {
+      const completion = await inner(request);
+      if (!request.schemaName.startsWith("codegraph_module")) return completion;
+      const block = completion.json as ModuleBlock;
+      const total = { word: "total", concern: "ambiguous", sense: "undefined term", meaning: "No type defines it.", polysemous: false, senses: [] };
+      return { ...completion, json: { ...block, glossary: [...(block.glossary ?? []), total] } };
+    };
+    const { result } = await run(new Map(), {}, Object.assign(insisting, { requests: completer.requests }));
+    expect(result.records.find((r) => r.id === T)?.block.words).toEqual([]);
+    const prompt = completer.requests.find((r) => r.user.startsWith(`# Unit: module ${P}`));
+    expect(prompt?.user).not.toContain("Words its types define");
+    const module = result.records.find((r) => r.id === P);
+    expect(module?.level === "module" && module.block.glossary).toEqual([]);
+  });
+
   it("concurrency does not change the record set", async () => {
     const serial = await run(new Map(), {}, fakeCompleter(), 1);
     const parallel = await run(new Map(), {}, fakeCompleter(), 4);

website/content/docs/discover/meaning.md

@@ -48,9 +48,9 @@ codegraph explain app.jsonl --src ~/src/app/src/main/java --dry-run
 explain plan: 77 units in 8 layers
   operation     57 units     48 calls ~78146 prompt tokens
   type          17 units     17 calls ~31000 prompt tokens
-  module         3 units      3 calls ~5328 prompt tokens
+  module         3 units      3 calls ~4990 prompt tokens
   statuses: llm 68, template 9, reuse 0, skip-scope 0, skip-budget 0
-  total: 68 calls, ~114474 prompt tokens in, ~52415 completion tokens out
+  total: 68 calls, ~114136 prompt tokens in, ~52415 completion tokens out
 ```
 
 ```bash

website/content/docs/how-to/explain-cost.md

@@ -21,10 +21,10 @@ codegraph explain model.jsonl --src fixtures/java/src --dry-run
 explain plan: 77 units in 8 layers
   operation     57 units     48 calls ~78146 prompt tokens
   type          17 units     17 calls ~31000 prompt tokens
-  module         3 units      3 calls ~5328 prompt tokens
+  module         3 units      3 calls ~4990 prompt tokens
   statuses: llm 68, template 9, reuse 0, skip-scope 0, skip-budget 0
   models: openai/gpt-5.6-luna (operations), openai/gpt-5.6-luna (types, modules); depth 1
-  total: 68 calls, ~114474 prompt tokens in, ~52415 completion tokens out
+  total: 68 calls, ~114136 prompt tokens in, ~52415 completion tokens out
 
 llm         L0 operation java:com.acme.order.adapter/LedgerAdapter.<init>() calls=1 ~1461tok
 template    L0 operation java:com.acme.order.legacy/List.head()
@@ -45,8 +45,8 @@ codegraph explain model.jsonl --src fixtures/java/src \
   level        calls   input tokens  output tokens
   operation       48         78 146         30 120
   type            17         31 000         15 470
-  module           3          5 328          6 825
-  total           68        114 474         52 415
+  module           3          4 990          6 825
+  total           68        114 136         52 415
 
   cost at $0.1/M in, $0.6/M out: $0.0429
   input ≈ rendered prompts at 4 characters per token; output ≈ measured on a live run: operation 470, type 530, module 530 tokens per block asked for, plus 120 per word and 125 per glossary entry asked to define (gpt-5.6-luna, reasoning included).