diff --git a/.agents/skills/tldrgraph-init/SKILL.md b/.agents/skills/tldrgraph-init/SKILL.md index 7ddfa53..175bf3b 100644 --- a/.agents/skills/tldrgraph-init/SKILL.md +++ b/.agents/skills/tldrgraph-init/SKILL.md @@ -5,18 +5,32 @@ description: Build or continue this repository's TLDRGraph architecture graph (l # TLDRGraph: build this repository's architecture graph -One command, run repeatedly until it says DONE. `tldrgraph init` never guesses: -it stops and tells you exactly what it needs. +In Claude Code or Cursor, invoke `/tldrgraph-init`. In Codex CLI, type `/skills` +and select `tldrgraph-init`, or mention `$tldrgraph-init` directly. + +One command handles layer design, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -Read the `NEXT ACTION` block it prints, do what it says, then run `tldrgraph init` -again. Repeat until the output says `status: done`. There are only three things -it can ask for. +By default TLDRGraph detects `claude`, `cursor-agent`, or `gemini`, asks once before enrichment token spend, processes every +eligible node in batches of 200, and downloads/builds the local embedding model. +Use `--yes` for non-interactive approval, `--batch N` to override the batch size, +`--embeddings off|auto|on` to override embeddings, or `--no-agent-cli` for the +manual file handoff. + +After the user approves the full run, use exactly `tldrgraph init --yes`. The +approval is saved for the current candidate set, so later `tldrgraph init` calls +must continue without asking again. `--batch 200` means all nodes in 200-node +batches; `--limit 200` means stop after only 200 nodes. Never add `--limit` or +`--embeddings off` unless the user explicitly requests a partial or no-embedding run. -## 1. `status: needs_layers` +If no supported agent is available or dense embeddings cannot be built, `init` +preserves the graph and prints a resumable status. It never guesses source intent +or architectural layers. + +## `status: needs_layers` TLDRGraph ships **no layer templates** and will not invent an architecture. Design one from this repository. @@ -47,25 +61,7 @@ Design one from this repository. } ``` -4. Run `tldrgraph init` again. - -### What a layer set looks like - -Sketches from other codebases, to show the *shape* of an answer. They are not a -menu and none of them will fit this repository -- read the code and name what you -actually find. - -- A web app might split presentation from request handling from domain logic - from persistence, with background jobs and deployment config as their own tiers. -- A CLI tool might split the command surface from the processing engine from - local state, with adapters to outside systems separate again. -- A library might split its public API from the core implementation from its - data types, with backend adapters separate. -- A data pipeline might split ingestion from transformation from model training - from serving. - -The useful question is not "which of these is it?" but "where does responsibility -change hands in *this* code, and what would a new engineer need named?" +4. Run `tldrgraph init` again; it continues with enrichment and embeddings. ### Rules that hold for any answer @@ -78,17 +74,17 @@ change hands in *this* code, and what would a new engineer need named?" - Derive rules from paths and symbol names you actually saw. A rule matching nothing is worse than no rule; a rule matching everything collapses the map. -## 2. `status: needs_confirmation` +## `status: needs_confirmation` The output shows how many nodes need enrichment and how many agent round-trips that implies. **Ask the user whether to proceed, and show them that estimate.** Do not decide for them. -- They agree: `tldrgraph init --yes` +- They agree: `tldrgraph init --yes` saves approval for the full campaign - Smaller first pass: `tldrgraph init --yes --limit 100` - They decline: stop. The graph is already built and queryable. -## 3. `status: needs_enrichment` +## `status: needs_enrichment` 1. Read `.tldrgraph/enrichment_request.yaml`. 2. **Open the source file of every node in it.** This is the entire point: an @@ -106,8 +102,14 @@ Do not decide for them. calls: [ApplicationsService, pension_cases] ``` -4. Run `tldrgraph init --yes` again. It applies the response and hands you the - next batch, until there is nothing left. +4. Run `tldrgraph init` again. Approval is already saved. If another + `needs_enrichment` batch appears, process it immediately and repeat this loop + without asking the user again. Continue until `status: done`. + +Inside an existing Codex/Claude/Cursor session, nested-agent protection may stop +the CLI from launching a second agent. In that case **you are the enrichment +agent**: process every 200-node batch yourself. A `needs_enrichment` status is a +continuation instruction, not a reason to stop or request confirmation. **Copy every `id` verbatim.** A constructed id matches nothing, is dropped, and gets reported back to you -- but the work is wasted. diff --git a/.claude/commands/tldrgraph-init.md b/.claude/commands/tldrgraph-init.md index 7ddfa53..175bf3b 100644 --- a/.claude/commands/tldrgraph-init.md +++ b/.claude/commands/tldrgraph-init.md @@ -5,18 +5,32 @@ description: Build or continue this repository's TLDRGraph architecture graph (l # TLDRGraph: build this repository's architecture graph -One command, run repeatedly until it says DONE. `tldrgraph init` never guesses: -it stops and tells you exactly what it needs. +In Claude Code or Cursor, invoke `/tldrgraph-init`. In Codex CLI, type `/skills` +and select `tldrgraph-init`, or mention `$tldrgraph-init` directly. + +One command handles layer design, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -Read the `NEXT ACTION` block it prints, do what it says, then run `tldrgraph init` -again. Repeat until the output says `status: done`. There are only three things -it can ask for. +By default TLDRGraph detects `claude`, `cursor-agent`, or `gemini`, asks once before enrichment token spend, processes every +eligible node in batches of 200, and downloads/builds the local embedding model. +Use `--yes` for non-interactive approval, `--batch N` to override the batch size, +`--embeddings off|auto|on` to override embeddings, or `--no-agent-cli` for the +manual file handoff. + +After the user approves the full run, use exactly `tldrgraph init --yes`. The +approval is saved for the current candidate set, so later `tldrgraph init` calls +must continue without asking again. `--batch 200` means all nodes in 200-node +batches; `--limit 200` means stop after only 200 nodes. Never add `--limit` or +`--embeddings off` unless the user explicitly requests a partial or no-embedding run. -## 1. `status: needs_layers` +If no supported agent is available or dense embeddings cannot be built, `init` +preserves the graph and prints a resumable status. It never guesses source intent +or architectural layers. + +## `status: needs_layers` TLDRGraph ships **no layer templates** and will not invent an architecture. Design one from this repository. @@ -47,25 +61,7 @@ Design one from this repository. } ``` -4. Run `tldrgraph init` again. - -### What a layer set looks like - -Sketches from other codebases, to show the *shape* of an answer. They are not a -menu and none of them will fit this repository -- read the code and name what you -actually find. - -- A web app might split presentation from request handling from domain logic - from persistence, with background jobs and deployment config as their own tiers. -- A CLI tool might split the command surface from the processing engine from - local state, with adapters to outside systems separate again. -- A library might split its public API from the core implementation from its - data types, with backend adapters separate. -- A data pipeline might split ingestion from transformation from model training - from serving. - -The useful question is not "which of these is it?" but "where does responsibility -change hands in *this* code, and what would a new engineer need named?" +4. Run `tldrgraph init` again; it continues with enrichment and embeddings. ### Rules that hold for any answer @@ -78,17 +74,17 @@ change hands in *this* code, and what would a new engineer need named?" - Derive rules from paths and symbol names you actually saw. A rule matching nothing is worse than no rule; a rule matching everything collapses the map. -## 2. `status: needs_confirmation` +## `status: needs_confirmation` The output shows how many nodes need enrichment and how many agent round-trips that implies. **Ask the user whether to proceed, and show them that estimate.** Do not decide for them. -- They agree: `tldrgraph init --yes` +- They agree: `tldrgraph init --yes` saves approval for the full campaign - Smaller first pass: `tldrgraph init --yes --limit 100` - They decline: stop. The graph is already built and queryable. -## 3. `status: needs_enrichment` +## `status: needs_enrichment` 1. Read `.tldrgraph/enrichment_request.yaml`. 2. **Open the source file of every node in it.** This is the entire point: an @@ -106,8 +102,14 @@ Do not decide for them. calls: [ApplicationsService, pension_cases] ``` -4. Run `tldrgraph init --yes` again. It applies the response and hands you the - next batch, until there is nothing left. +4. Run `tldrgraph init` again. Approval is already saved. If another + `needs_enrichment` batch appears, process it immediately and repeat this loop + without asking the user again. Continue until `status: done`. + +Inside an existing Codex/Claude/Cursor session, nested-agent protection may stop +the CLI from launching a second agent. In that case **you are the enrichment +agent**: process every 200-node batch yourself. A `needs_enrichment` status is a +continuation instruction, not a reason to stop or request confirmation. **Copy every `id` verbatim.** A constructed id matches nothing, is dropped, and gets reported back to you -- but the work is wasted. diff --git a/.cursor/commands/tldrgraph-init.md b/.cursor/commands/tldrgraph-init.md index 7ddfa53..175bf3b 100644 --- a/.cursor/commands/tldrgraph-init.md +++ b/.cursor/commands/tldrgraph-init.md @@ -5,18 +5,32 @@ description: Build or continue this repository's TLDRGraph architecture graph (l # TLDRGraph: build this repository's architecture graph -One command, run repeatedly until it says DONE. `tldrgraph init` never guesses: -it stops and tells you exactly what it needs. +In Claude Code or Cursor, invoke `/tldrgraph-init`. In Codex CLI, type `/skills` +and select `tldrgraph-init`, or mention `$tldrgraph-init` directly. + +One command handles layer design, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -Read the `NEXT ACTION` block it prints, do what it says, then run `tldrgraph init` -again. Repeat until the output says `status: done`. There are only three things -it can ask for. +By default TLDRGraph detects `claude`, `cursor-agent`, or `gemini`, asks once before enrichment token spend, processes every +eligible node in batches of 200, and downloads/builds the local embedding model. +Use `--yes` for non-interactive approval, `--batch N` to override the batch size, +`--embeddings off|auto|on` to override embeddings, or `--no-agent-cli` for the +manual file handoff. + +After the user approves the full run, use exactly `tldrgraph init --yes`. The +approval is saved for the current candidate set, so later `tldrgraph init` calls +must continue without asking again. `--batch 200` means all nodes in 200-node +batches; `--limit 200` means stop after only 200 nodes. Never add `--limit` or +`--embeddings off` unless the user explicitly requests a partial or no-embedding run. -## 1. `status: needs_layers` +If no supported agent is available or dense embeddings cannot be built, `init` +preserves the graph and prints a resumable status. It never guesses source intent +or architectural layers. + +## `status: needs_layers` TLDRGraph ships **no layer templates** and will not invent an architecture. Design one from this repository. @@ -47,25 +61,7 @@ Design one from this repository. } ``` -4. Run `tldrgraph init` again. - -### What a layer set looks like - -Sketches from other codebases, to show the *shape* of an answer. They are not a -menu and none of them will fit this repository -- read the code and name what you -actually find. - -- A web app might split presentation from request handling from domain logic - from persistence, with background jobs and deployment config as their own tiers. -- A CLI tool might split the command surface from the processing engine from - local state, with adapters to outside systems separate again. -- A library might split its public API from the core implementation from its - data types, with backend adapters separate. -- A data pipeline might split ingestion from transformation from model training - from serving. - -The useful question is not "which of these is it?" but "where does responsibility -change hands in *this* code, and what would a new engineer need named?" +4. Run `tldrgraph init` again; it continues with enrichment and embeddings. ### Rules that hold for any answer @@ -78,17 +74,17 @@ change hands in *this* code, and what would a new engineer need named?" - Derive rules from paths and symbol names you actually saw. A rule matching nothing is worse than no rule; a rule matching everything collapses the map. -## 2. `status: needs_confirmation` +## `status: needs_confirmation` The output shows how many nodes need enrichment and how many agent round-trips that implies. **Ask the user whether to proceed, and show them that estimate.** Do not decide for them. -- They agree: `tldrgraph init --yes` +- They agree: `tldrgraph init --yes` saves approval for the full campaign - Smaller first pass: `tldrgraph init --yes --limit 100` - They decline: stop. The graph is already built and queryable. -## 3. `status: needs_enrichment` +## `status: needs_enrichment` 1. Read `.tldrgraph/enrichment_request.yaml`. 2. **Open the source file of every node in it.** This is the entire point: an @@ -106,8 +102,14 @@ Do not decide for them. calls: [ApplicationsService, pension_cases] ``` -4. Run `tldrgraph init --yes` again. It applies the response and hands you the - next batch, until there is nothing left. +4. Run `tldrgraph init` again. Approval is already saved. If another + `needs_enrichment` batch appears, process it immediately and repeat this loop + without asking the user again. Continue until `status: done`. + +Inside an existing Codex/Claude/Cursor session, nested-agent protection may stop +the CLI from launching a second agent. In that case **you are the enrichment +agent**: process every 200-node batch yourself. A `needs_enrichment` status is a +continuation instruction, not a reason to stop or request confirmation. **Copy every `id` verbatim.** A constructed id matches nothing, is dropped, and gets reported back to you -- but the work is wasted. diff --git a/.tldrgraph/AGENT_CONTRACT.md b/.tldrgraph/AGENT_CONTRACT.md index d9c48ba..3f6ccc9 100644 --- a/.tldrgraph/AGENT_CONTRACT.md +++ b/.tldrgraph/AGENT_CONTRACT.md @@ -1,6 +1,6 @@ # TLDRGraph Agent Contract -**Audience: the coding agent with this repository open** (Claude Code, Cursor, Antigravity). +**Audience: the coding agent with this repository open** (Codex, Claude Code, Cursor, Antigravity). TLDRGraph builds an architectural graph from the graphify AST export. The layer set itself is designed by you, reading this repository. TLDRGraph ships no layer templates. @@ -18,21 +18,22 @@ and a path (`snippet` is never populated), so it guesses. You do not have to gue ## Start here: `tldrgraph init` -One command does everything, and it is resumable: +One command handles layers, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -It runs every deterministic step — extraction, classification, indexing, applying whatever -you last wrote — then stops with a `NEXT ACTION` block the moment it needs judgement only -you can supply. Do what the block says and run it again. Repeat until `status: done`. +It asks once before enrichment token spend. Full approval is persisted for the current +candidate set until enrichment finishes, so continuation runs must not ask again. By +default it uses 200-node batches and builds dense embeddings. | status | what it wants | | --- | --- | | `needs_layers` | Read the code and design this repository's architecture. **TLDRGraph ships no layer templates**; nothing will be applied for you. The request carries sketches of how other kinds of codebase divide — for shape only, never to copy. | -| `needs_confirmation` | Enrichment costs the user tokens. Show them the estimate and ask. Then `tldrgraph init --yes`. | -| `needs_enrichment` | A batch to open, read and describe, per the schema below. | +| `needs_confirmation` | Show the estimate and ask once. Approval via `tldrgraph init --yes` persists until the current campaign is done. | +| `needs_enrichment` | Open, read, and describe this batch, then continue immediately without asking again. | +| `needs_embeddings` | Enrichment finished but the required dense model/index could not be built. Fix model access and rerun init. | | `done` | Nothing left. Use `query` / `trace` / `layers`. | `--json` gives you the same thing machine-readably. The sections below document the file @@ -47,13 +48,20 @@ dropped, and will be reported back to you — but the work is wasted. ## The loop ```bash -tldrgraph queue-enrichment --limit 50 # 1. writes .tldrgraph/enrichment_request.yaml -# 2. you read it, read the SOURCE, and write -# .tldrgraph/enrichment_response.yaml -tldrgraph apply-enrichment # 3. merges into the graph, cache and index -tldrgraph queue-enrichment --limit 50 # 4. repeat -- the queue advances automatically +tldrgraph init --yes # 1. approve every current candidate; writes a 200-node request +# 2. read every requested source and write enrichment_response.yaml +tldrgraph init # 3. applies it and emits the next batch; approval is remembered +# 4. repeat steps 2-3 without asking until status: done ``` +Inside an existing coding-agent session, nested-agent protection can prevent the CLI +from launching another agent. In that case **you are the enrichment agent** and must +process every batch yourself. Do not stop at `needs_enrichment`. + +`--batch 200` means process all candidates in chunks of 200. `--limit 200` means stop +after only 200 candidates and is only for an explicitly requested partial run. Never add +`--limit` or `--embeddings off` unless the user explicitly requests that behavior. + Request and response are **separate files**. Never write your answer back into `enrichment_request.yaml`; it is regenerated on every run and your work would be lost. @@ -62,6 +70,7 @@ Request and response are **separate files**. Never write your answer back into | `.tldrgraph/enrichment_request.yaml` (or `enrichment_request.json`) | `queue-enrichment` | you | | `.tldrgraph/enrichment_response.yaml` (or `enrichment_response.json`) | **you** | `apply-enrichment` | | `.tldrgraph/enrichment_cursor.json` | both commands | both commands | +| `.tldrgraph/enrichment_approval.json` | `init --yes` | later `init` runs | | `.tldrgraph/pending_enrichment.json` | *(legacy)* | `apply-enrichment`, only if no response file exists | --- @@ -76,8 +85,8 @@ contract: .tldrgraph/AGENT_CONTRACT.md progress: total_candidates: 1873 # un-enriched, non-utility nodes already_enriched: 12 # nodes that already carry an intent - queued_now: 50 # entries in "nodes" below - remaining_after: 1823 # still waiting after this batch is applied + queued_now: 200 # entries in "nodes" below + remaining_after: 1673 # still waiting after this batch is applied nodes: - id: backend_src_applications_applications_controller_applicationscontroller label: ApplicationsController @@ -187,6 +196,10 @@ valid and useful. 5. **Answer only the nodes in the request.** Extra ids are ignored; missing ids just come back in a later batch. +6. **After full approval, never ask again for the same campaign.** Continue processing + `needs_enrichment` batches until `status: done`. Do not silently add `--limit` or + `--embeddings off`. + --- ## Priority order in the queue diff --git a/AGENTS.md b/AGENTS.md index 3c0db18..247352d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,10 +24,10 @@ tldrgraph dead-code # review candidates, never a dele Those are read-only and never trigger enrichment. -**To build or continue the graph**, run `tldrgraph init`, do what the `NEXT ACTION` -block prints, and run it again -- repeat until `status: done`. It has no template -fallback: if this repository has no architecture yet, it will stop and ask you to -design one from the code. Do not skip reading the files. +**To build or refresh the graph**, run `tldrgraph init`. It automatically handles +layer design, extraction, source-aware enrichment in 200-node batches, and dense +embeddings when a supported agent CLI is available. If it prints a `NEXT ACTION` +fallback, follow that handoff without guessing from symbol names. Full workflow: `.claude/commands/tldrgraph-init.md` (identical copies live in every other agent directory). Schema: `.tldrgraph/AGENT_CONTRACT.md`. diff --git a/AGENT_CONTRACT.md b/AGENT_CONTRACT.md index d9c48ba..3f6ccc9 100644 --- a/AGENT_CONTRACT.md +++ b/AGENT_CONTRACT.md @@ -1,6 +1,6 @@ # TLDRGraph Agent Contract -**Audience: the coding agent with this repository open** (Claude Code, Cursor, Antigravity). +**Audience: the coding agent with this repository open** (Codex, Claude Code, Cursor, Antigravity). TLDRGraph builds an architectural graph from the graphify AST export. The layer set itself is designed by you, reading this repository. TLDRGraph ships no layer templates. @@ -18,21 +18,22 @@ and a path (`snippet` is never populated), so it guesses. You do not have to gue ## Start here: `tldrgraph init` -One command does everything, and it is resumable: +One command handles layers, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -It runs every deterministic step — extraction, classification, indexing, applying whatever -you last wrote — then stops with a `NEXT ACTION` block the moment it needs judgement only -you can supply. Do what the block says and run it again. Repeat until `status: done`. +It asks once before enrichment token spend. Full approval is persisted for the current +candidate set until enrichment finishes, so continuation runs must not ask again. By +default it uses 200-node batches and builds dense embeddings. | status | what it wants | | --- | --- | | `needs_layers` | Read the code and design this repository's architecture. **TLDRGraph ships no layer templates**; nothing will be applied for you. The request carries sketches of how other kinds of codebase divide — for shape only, never to copy. | -| `needs_confirmation` | Enrichment costs the user tokens. Show them the estimate and ask. Then `tldrgraph init --yes`. | -| `needs_enrichment` | A batch to open, read and describe, per the schema below. | +| `needs_confirmation` | Show the estimate and ask once. Approval via `tldrgraph init --yes` persists until the current campaign is done. | +| `needs_enrichment` | Open, read, and describe this batch, then continue immediately without asking again. | +| `needs_embeddings` | Enrichment finished but the required dense model/index could not be built. Fix model access and rerun init. | | `done` | Nothing left. Use `query` / `trace` / `layers`. | `--json` gives you the same thing machine-readably. The sections below document the file @@ -47,13 +48,20 @@ dropped, and will be reported back to you — but the work is wasted. ## The loop ```bash -tldrgraph queue-enrichment --limit 50 # 1. writes .tldrgraph/enrichment_request.yaml -# 2. you read it, read the SOURCE, and write -# .tldrgraph/enrichment_response.yaml -tldrgraph apply-enrichment # 3. merges into the graph, cache and index -tldrgraph queue-enrichment --limit 50 # 4. repeat -- the queue advances automatically +tldrgraph init --yes # 1. approve every current candidate; writes a 200-node request +# 2. read every requested source and write enrichment_response.yaml +tldrgraph init # 3. applies it and emits the next batch; approval is remembered +# 4. repeat steps 2-3 without asking until status: done ``` +Inside an existing coding-agent session, nested-agent protection can prevent the CLI +from launching another agent. In that case **you are the enrichment agent** and must +process every batch yourself. Do not stop at `needs_enrichment`. + +`--batch 200` means process all candidates in chunks of 200. `--limit 200` means stop +after only 200 candidates and is only for an explicitly requested partial run. Never add +`--limit` or `--embeddings off` unless the user explicitly requests that behavior. + Request and response are **separate files**. Never write your answer back into `enrichment_request.yaml`; it is regenerated on every run and your work would be lost. @@ -62,6 +70,7 @@ Request and response are **separate files**. Never write your answer back into | `.tldrgraph/enrichment_request.yaml` (or `enrichment_request.json`) | `queue-enrichment` | you | | `.tldrgraph/enrichment_response.yaml` (or `enrichment_response.json`) | **you** | `apply-enrichment` | | `.tldrgraph/enrichment_cursor.json` | both commands | both commands | +| `.tldrgraph/enrichment_approval.json` | `init --yes` | later `init` runs | | `.tldrgraph/pending_enrichment.json` | *(legacy)* | `apply-enrichment`, only if no response file exists | --- @@ -76,8 +85,8 @@ contract: .tldrgraph/AGENT_CONTRACT.md progress: total_candidates: 1873 # un-enriched, non-utility nodes already_enriched: 12 # nodes that already carry an intent - queued_now: 50 # entries in "nodes" below - remaining_after: 1823 # still waiting after this batch is applied + queued_now: 200 # entries in "nodes" below + remaining_after: 1673 # still waiting after this batch is applied nodes: - id: backend_src_applications_applications_controller_applicationscontroller label: ApplicationsController @@ -187,6 +196,10 @@ valid and useful. 5. **Answer only the nodes in the request.** Extra ids are ignored; missing ids just come back in a later batch. +6. **After full approval, never ask again for the same campaign.** Continue processing + `needs_enrichment` batches until `status: done`. Do not silently add `--limit` or + `--embeddings off`. + --- ## Priority order in the queue diff --git a/README.md b/README.md index 3d9a053..8a96942 100644 --- a/README.md +++ b/README.md @@ -49,34 +49,41 @@ wrong everywhere it looks right. ```bash pip install tldrgraph ``` -*(For optional local ONNX neural embeddings: `pip install "tldrgraph[embeddings]"`)* +### 2. Build the graph — through your coding agent -### 2. Build the graph — one command -```bash -tldrgraph init -``` +**Do not run `tldrgraph init` manually in a terminal.** Start the +`tldrgraph-init` workflow in whichever coding agent you use; the agent reads the +repository, runs `init`, and handles every required follow-up. In Claude Code or +Cursor, run `/tldrgraph-init`. In Codex, open `/skills` and select +`tldrgraph-init`, or invoke `$tldrgraph-init`. + +The agent designs the repository-specific layers, extracts the graph, asks once +before enrichment token spend, enriches every eligible node in 200-node batches, +and downloads/builds local dense embeddings. That approval is remembered for the +current candidate set until enrichment is complete. -`init` is a resumable state machine. It runs every step it can — extraction, -classification, indexing, enrichment — and stops with a `NEXT ACTION` block the -moment it needs judgement only an agent can supply. Do what the block says, run -it again, repeat until it prints `status: done`. +`--batch 200` controls chunk size while still processing everything. `--limit +200` intentionally stops after 200 total nodes. Embeddings remain enabled unless +you explicitly pass `--embeddings off`. -It can ask for exactly three things: +If no supported agent is usable, it preserves everything already built and +prints a `NEXT ACTION` handoff. Follow that handoff and rerun the same command; +TLDRGraph never invents architecture or source intent. + +It can report four resumable states: | status | what it needs | | --- | --- | | `needs_layers` | Read the code and design the architecture. No template will be applied for you. | | `needs_confirmation` | Shows how many nodes need enrichment and how many agent rounds that is. **Your agent asks you before spending tokens.** | | `needs_enrichment` | A batch of nodes to open, read, and describe. | +| `needs_embeddings` | Enrichment is complete, but the required dense model/index could not be built. | -```bash -tldrgraph init --yes # proceed past the estimate -tldrgraph init --yes --limit 100 # smaller first pass -tldrgraph init --json # machine-readable status for agents -``` - -Your agent can drive the whole thing with the installed `/tldrgraph-init` -command. `scan` and `enrich` are aliases for `init`, kept for existing scripts. +Your agent drives the whole process with the installed `tldrgraph-init` +workflow. Give it any scope or batch-size constraints you need; it will choose +the appropriate `init` options. `scan` and `enrich` are aliases for `init`, kept +for existing scripts, and should likewise be run by the agent rather than +manually. ### 3. Explore the Architecture Visually ```bash @@ -93,7 +100,7 @@ Opens the interactive canvas: ```bash tldrgraph query "pension application approval flow" ``` -Outputs readable Markdown execution flow tables tracing the request across UI, API, Service, and DB layers. +Outputs five readable Markdown execution flow tables by default, tracing the request across UI, API, Service, and DB layers. Queries use dense embeddings by default (and may download the configured model); use `--top-k`, `--embeddings auto`, or `--embeddings off` to override this behavior. ### 5. Trace Exact Call Paths ```bash @@ -131,19 +138,25 @@ To evaluate codebase localization performance against industry baselines, TLDRGr ## 🤖 Works with any coding agent -TLDRGraph is driven **by** your agent, not the other way around. It never needs -to launch one, so there are no per-tool flags, auth or headless quirks to get -wrong — any agent that can read a file, read source, and run a shell command can -drive it. +TLDRGraph automatically launches a supported agent CLI when possible. Inside an +existing coding-agent session, or when no supported CLI is authenticated, it +falls back to a portable file handoff that any agent can drive. Every tool gets the **same two artifacts and no more**: one body of instructions and one `tldrgraph-init` command, byte-identical everywhere. | Artifact | Where | | --- | --- | -| **Instructions** | `AGENTS.md` — the cross-tool standard, read by Claude Code, Cursor, Antigravity, opencode, Codex, Gemini CLI, Zed and Copilot | +| **Instructions** | `AGENTS.md` — the cross-tool standard, read by Claude Code, Cursor, Codex, Antigravity, opencode, Gemini CLI, Zed and Copilot | | | `.clinerules/`, `.windsurf/rules/` — only for tools not known to read AGENTS.md | -| **Command** | `.claude/commands/`, `.cursor/commands/`, `.agents/skills/`, `.clinerules/workflows/`, `.windsurf/workflows/`, `.opencode/command/`, `.roo/commands/`, `.kilocode/workflows/`, `.goosehints/`, `.continue/prompts/` | +| **Command / skill** | `.claude/commands/`, `.cursor/commands/`, `.agents/skills/` (Codex), `.clinerules/workflows/`, `.windsurf/workflows/`, `.opencode/command/`, `.roo/commands/`, `.kilocode/workflows/`, `.goosehints/`, `.continue/prompts/` | + +Codex intentionally uses `.agents/skills/tldrgraph-init/SKILL.md`, not a +`.codex/commands/` mirror. Codex does not load repository commands from +`.codex/commands`; its supported repository-local workflow location is +`.agents/skills`. Open `/skills` and select `tldrgraph-init`, or invoke it as +`$tldrgraph-init`. TLDRGraph writes the same workflow body there that it writes +for Claude Code and Cursor. Tools with a marker directory are installed only when the repo shows them in use; `tldrgraph install --all-agents` writes them all. Adding a tool is one row @@ -155,18 +168,12 @@ No tool gets special treatment. Earlier versions shipped a Claude-only skill fil worded differently and each a different length; they contradicted each other within a release. `tldrgraph install` deletes those on sight. -### Letting TLDRGraph launch an agent itself - -Off by default, and opt-in per run: - -```bash -tldrgraph init --yes --agent-cli -``` +### Agent execution controls -This shells out to `claude`, `cursor-agent` or `gemini` if one is on `PATH`. It -is genuinely useful in a plain terminal with no agent attached, but it is not the -default: agent CLIs differ per tool, block with no output while they think, and -some IDEs ship no working CLI at all. +The `tldrgraph-init` workflow is the supported entry point. It can use the +agent's native session or a portable handoff, and preserves the graph while +providing the next action if agent work is unavailable. Run the workflow from +your coding agent; do not invoke `tldrgraph init` directly. ## 📁 Artifacts & Output Formats diff --git a/tests/conftest.py b/tests/conftest.py index a7550f9..fd14613 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -312,6 +312,7 @@ def env_no_llm(monkeypatch): # Port 9 (discard) on loopback: refused instantly, never leaves the host. monkeypatch.setenv("OLLAMA_HOST", "http://127.0.0.1:9") monkeypatch.setenv("TLDRGRAPH_NO_AGENT", "1") + monkeypatch.setenv("TLDRGRAPH_EMBEDDINGS", "off") return True diff --git a/tests/test_agent_loop.py b/tests/test_agent_loop.py index ab1b7c6..076e6ca 100644 --- a/tests/test_agent_loop.py +++ b/tests/test_agent_loop.py @@ -587,7 +587,7 @@ def test_dead_code_has_no_delete_capability(run): # Installer # --------------------------------------------------------------------------- # -def test_install_writes_claude_cursor_and_antigravity(tmp_path): +def test_install_writes_claude_cursor_and_codex(tmp_path): written = installer_module.install_agent_rules(str(tmp_path)) # One body of instructions and one command, per tool. No tool gets bespoke @@ -598,7 +598,7 @@ def test_install_writes_claude_cursor_and_antigravity(tmp_path): "AGENTS.md (instructions, all agents)": "AGENTS.md", "Claude Code (command)": ".claude/commands/tldrgraph-init.md", "Cursor (command)": ".cursor/commands/tldrgraph-init.md", - "Antigravity (command)": ".agents/skills/tldrgraph-init/SKILL.md", + "Codex (command)": ".agents/skills/tldrgraph-init/SKILL.md", } assert set(written) == set(expected) for key, rel in expected.items(): @@ -696,6 +696,15 @@ def test_contract_documents_the_full_response_schema(tmp_path): assert token in text, token +def test_contract_documents_persistent_full_campaign_approval(tmp_path): + written = installer_module.install_agent_rules(str(tmp_path)) + text = Path(written["contract"]).read_text(encoding="utf-8") + assert "enrichment_approval.json" in text + assert "--batch 200" in text and "--limit 200" in text + assert "without asking" in text + assert "Never add" in text and "--embeddings off" in text + + def test_install_command_reports_the_gitignore_problem(tmp_path): (tmp_path / ".gitignore").write_text("node_modules\n.agents\n", encoding="utf-8") result = CliRunner().invoke(cli, ["install", "--path", str(tmp_path)]) @@ -714,8 +723,10 @@ def test_install_command_lists_what_it_wrote(tmp_path): result = CliRunner().invoke(cli, ["install", "--path", str(tmp_path)]) assert result.exit_code == 0 for key in ("contract", "gitignore", "AGENTS.md", - "Claude Code (command)", "Cursor (command)", "Antigravity (command)"): + "Claude Code (command)", "Cursor (command)", "Codex (command)"): assert key in result.output + assert "Codex: /skills" in result.output + assert "$tldrgraph-init" in result.output # --------------------------------------------------------------------------- # @@ -738,11 +749,100 @@ def test_scan_still_accepts_rebuild(loop_repo): def test_query_still_accepts_top_k_and_path(loop_repo): result = CliRunner().invoke( - cli, ["query", "pension calculation", "--top-k", "2", "--path", str(loop_repo)] + cli, ["query", "pension calculation", "--top-k", "2", "--path", str(loop_repo), + "--embeddings", "off"] ) assert result.exit_code == 0, result.output +def test_query_defaults_to_five_results_and_embeddings_on(monkeypatch, tmp_path): + captured = {} + + class FakeLoader: + def __init__(self, path, embeddings): + captured["embeddings"] = embeddings + self.vector_store = object() + + def load_or_extract(self, enrich_llm): + assert enrich_llm is False + return object() + + class FakeEngine: + def __init__(self, graph, vector_store, root_dir): + pass + + def query_flow(self, query_text, top_k): + captured["top_k"] = top_k + return [] + + monkeypatch.delenv("TLDRGRAPH_EMBEDDINGS", raising=False) + monkeypatch.setattr(cli_module, "GraphLoader", FakeLoader) + monkeypatch.setattr(cli_module, "FlowEngine", FakeEngine) + + result = CliRunner().invoke( + cli, ["query", "pension calculation", "--path", str(tmp_path)] + ) + + assert result.exit_code == 0, result.output + assert captured == {"embeddings": "on", "top_k": 5} + + captured.clear() + result = CliRunner().invoke( + cli, ["query", "pension calculation", "--path", str(tmp_path), + "--top-k", "2", "--embeddings", "off"] + ) + + assert result.exit_code == 0, result.output + assert captured == {"embeddings": "off", "top_k": 2} + + +def test_query_help_displays_top_k_default(): + result = CliRunner().invoke(cli, ["query", "--help"]) + assert result.exit_code == 0, result.output + assert "--top-k" in result.output + assert "default: 5" in result.output + + +@pytest.mark.parametrize( + "environment,args,expected", + [ + ("off", [], "off"), + ("off", ["--embeddings", "auto"], "auto"), + ("auto", ["--embeddings", "on"], "on"), + ], +) +def test_query_embedding_overrides_preserve_precedence( + monkeypatch, tmp_path, environment, args, expected +): + captured = {} + + class FakeLoader: + def __init__(self, path, embeddings): + captured["embeddings"] = embeddings + self.vector_store = object() + + def load_or_extract(self, enrich_llm): + return object() + + class FakeEngine: + def __init__(self, graph, vector_store, root_dir): + pass + + def query_flow(self, query_text, top_k): + return [] + + monkeypatch.setenv("TLDRGRAPH_EMBEDDINGS", environment) + monkeypatch.setattr(cli_module, "GraphLoader", FakeLoader) + monkeypatch.setattr(cli_module, "FlowEngine", FakeEngine) + + result = CliRunner().invoke( + cli, ["query", "pension calculation", "--path", str(tmp_path), *args] + ) + + assert result.exit_code == 0, result.output + assert captured["embeddings"] == expected + + def test_state_filenames_are_the_documented_ones(): assert REQUEST_FILENAME == "enrichment_request.yaml" assert RESPONSE_FILENAME == "enrichment_response.yaml" diff --git a/tests/test_auto_agent.py b/tests/test_auto_agent.py index 8cb69d9..1af4038 100644 --- a/tests/test_auto_agent.py +++ b/tests/test_auto_agent.py @@ -16,8 +16,14 @@ import yaml from click.testing import CliRunner -from tldrgraph import agent_commands, agent_runner, installer as installer_module, paths +from tldrgraph import agent_commands, agent_runner, cli_pipeline, installer as installer_module, paths from tldrgraph.cli import cli +from tldrgraph.cli_pipeline import resolve_init_embeddings +from tldrgraph.init_policy import ( + APPROVAL_FILENAME, + enrichment_approval_is_active, + remember_full_enrichment_approval, +) from tldrgraph.propose_layers import ( NEEDS_LAYERS, auto_configure_layers, @@ -405,7 +411,7 @@ def test_init_applies_the_agents_enrichment_and_reaches_done(cli_repo): ]), encoding="utf-8", ) - res = runner.invoke(cli, ["init", str(cli_repo), "--yes"]) + res = runner.invoke(cli, ["init", str(cli_repo)]) assert res.exit_code == 0, res.output if "status: done" in res.output: break @@ -418,6 +424,51 @@ def test_init_applies_the_agents_enrichment_and_reaches_done(cli_repo): for n in snapshot["nodes"] if n.get("layer_id") != "shared" ) + assert not (state / APPROVAL_FILENAME).exists() + + +def test_full_approval_survives_manual_batches_without_reconfirmation(cli_repo): + _answer_layers(cli_repo) + runner = CliRunner() + first = runner.invoke(cli, ["init", str(cli_repo), "--yes", "--batch", "1"]) + assert "status: needs_enrichment" in first.output + + state = cli_repo / ".tldrgraph" + request = yaml.safe_load((state / "enrichment_request.yaml").read_text(encoding="utf-8")) + (state / "enrichment_response.yaml").write_text( + yaml.dump([{"id": request["nodes"][0]["id"], "intent": "Source-backed intent."}]), + encoding="utf-8", + ) + + continued = runner.invoke(cli, ["init", str(cli_repo), "--batch", "1"]) + assert continued.exit_code == 0, continued.output + assert "status: needs_confirmation" not in continued.output + assert "status: needs_enrichment" in continued.output or "status: done" in continued.output + + +def test_limited_approval_does_not_authorize_the_remaining_campaign(cli_repo): + _answer_layers(cli_repo) + runner = CliRunner() + first = runner.invoke(cli, ["init", str(cli_repo), "--yes", "--limit", "1"]) + assert "status: needs_enrichment" in first.output + assert not (cli_repo / ".tldrgraph" / APPROVAL_FILENAME).exists() + + request = yaml.safe_load( + (cli_repo / ".tldrgraph" / "enrichment_request.yaml").read_text(encoding="utf-8") + ) + (cli_repo / ".tldrgraph" / "enrichment_response.yaml").write_text( + yaml.dump([{"id": request["nodes"][0]["id"], "intent": "One approved node."}]), + encoding="utf-8", + ) + resumed = runner.invoke(cli, ["init", str(cli_repo)]) + assert "status: needs_confirmation" in resumed.output + + +def test_approval_does_not_cover_new_candidate_ids(tmp_path): + approved = [{"id": "a"}, {"id": "b"}] + remember_full_enrichment_approval(str(tmp_path), approved) + assert enrichment_approval_is_active(str(tmp_path), [{"id": "b"}]) + assert not enrichment_approval_is_active(str(tmp_path), [{"id": "b"}, {"id": "new"}]) def test_an_applied_response_is_not_applied_twice(cli_repo): @@ -452,18 +503,68 @@ def test_limit_caps_the_first_pass(cli_repo): assert len(request["nodes"]) == 2 -def test_agent_cli_is_opt_in_not_the_default(monkeypatch, cli_repo, agent_allowed): - """ - Shelling out is off unless asked for. It hung a real user's terminal for ten - minutes with no output, and it does not generalise across agent tools. - """ +def test_agent_cli_is_automatic_by_default(monkeypatch, cli_repo, agent_allowed): calls = [] monkeypatch.setattr(agent_runner, "find_agent_cli", lambda **kw: calls.append(1) or fake_agent()) + monkeypatch.setattr(agent_runner, "run_agent", lambda *a, **k: _fake_answer(a[1])) _answer_layers(cli_repo) - CliRunner().invoke(cli, ["init", str(cli_repo), "--yes"]) - assert calls == [], "init must not look for an agent CLI without --agent-cli" + res = CliRunner().invoke(cli, ["init", str(cli_repo), "--yes"]) + assert res.exit_code == 0, res.output + assert calls, "init must auto-detect an agent CLI by default" + assert "status: done" in res.output + + +def test_no_agent_cli_forces_the_manual_handoff(monkeypatch, cli_repo, agent_allowed): + calls = [] + monkeypatch.setattr(agent_runner, "find_agent_cli", + lambda **kw: calls.append(1) or fake_agent()) + _answer_layers(cli_repo) + res = CliRunner().invoke(cli, ["init", str(cli_repo), "--yes", "--no-agent-cli"]) + assert res.exit_code == 0, res.output + assert calls == [] + assert "status: needs_enrichment" in res.output + + +def test_init_defaults_to_two_hundred_node_batches(): + help_result = CliRunner().invoke(cli, ["init", "--help"]) + assert help_result.exit_code == 0 + assert "default:" in help_result.output and "200" in help_result.output + + +def test_init_defaults_embeddings_on_but_honours_environment(monkeypatch): + monkeypatch.delenv("TLDRGRAPH_EMBEDDINGS", raising=False) + assert resolve_init_embeddings(None) == "on" + monkeypatch.setenv("TLDRGRAPH_EMBEDDINGS", "off") + assert resolve_init_embeddings(None) == "off" + assert resolve_init_embeddings("auto") == "auto" + + +def test_interactive_init_asks_once_then_finishes(monkeypatch, cli_repo, agent_allowed): + _answer_layers(cli_repo) + _stub_agent_cli(monkeypatch) + monkeypatch.setattr(cli_pipeline, "stdin_is_interactive", lambda: True) + res = CliRunner().invoke(cli, ["init", str(cli_repo)], input="\n") + assert res.exit_code == 0, res.output + assert res.output.count("Enrich now?") == 1 + assert "status: done" in res.output + + +def test_automatic_agent_keeps_json_output_parseable(monkeypatch, cli_repo, agent_allowed): + _stub_agent_cli(monkeypatch) + res = CliRunner().invoke(cli, ["init", str(cli_repo), "--yes", "--json"]) + assert res.exit_code == 0, res.output + assert json.loads(res.stdout)["status"] == "done" + + +def test_embedding_failure_is_resumable(monkeypatch, cli_repo, agent_allowed): + _stub_agent_cli(monkeypatch) + monkeypatch.setattr(cli_pipeline, "_embedding_failure", lambda loader: "model unavailable") + res = CliRunner().invoke(cli, ["init", str(cli_repo), "--yes"]) + assert res.exit_code == 0, res.output + assert "status: needs_embeddings" in res.output + assert "model unavailable" in res.output def test_agent_cli_runs_the_whole_loop_when_asked(monkeypatch, cli_repo, agent_allowed): @@ -599,6 +700,10 @@ def test_install_writes_the_gitignore_and_the_one_command(tmp_path): assert cmd.is_file() body = cmd.read_text(encoding="utf-8") assert "tldrgraph init" in body + assert "--batch 200" in body and "--limit 200" in body + assert "Never add `--limit`" in body and "`--embeddings off` unless" in body + assert "without asking the user again" in body + assert "/skills" in body and "$tldrgraph-init" in body # Every branch of the state machine must be documented in the command. for status in ("needs_layers", "needs_confirmation", "needs_enrichment"): assert status in body, status @@ -656,6 +761,22 @@ def test_every_agent_gets_an_identical_command(tmp_path): assert len(bodies) == 1, "command files have drifted apart" +def test_codex_skill_matches_the_claude_command(tmp_path): + """Codex gets the same workflow through its supported repo-local skill path.""" + agent_commands.install_agent_commands(str(tmp_path)) + + claude = tmp_path / ".claude" / "commands" / "tldrgraph-init.md" + codex = tmp_path / ".agents" / "skills" / "tldrgraph-init" / "SKILL.md" + assert codex.is_file() + assert codex.read_text(encoding="utf-8") == claude.read_text(encoding="utf-8") + + +def test_codex_uses_supported_repo_skill_not_a_dead_dot_codex_command(tmp_path): + agent_commands.install_agent_commands(str(tmp_path)) + assert (tmp_path / ".agents/skills/tldrgraph-init/SKILL.md").is_file() + assert not (tmp_path / ".codex/commands/tldrgraph-init.md").exists() + + def test_no_tool_gets_a_bespoke_extra_artifact(tmp_path): """ Claude used to get a skill AND a CLAUDE.md section AND a command, while diff --git a/tests/test_embeddings.py b/tests/test_embeddings.py index e824c1a..5da089a 100644 --- a/tests/test_embeddings.py +++ b/tests/test_embeddings.py @@ -7,10 +7,10 @@ 1. **Nothing in this file may touch the network or download a model.** Every dense-path test is gated on the model already being cached locally and skips cleanly otherwise, and the gate itself is pure filesystem inspection. The - TF-IDF path -- which is what CI and the default install actually run -- is - covered unconditionally. + TF-IDF path -- which CI selects explicitly to stay offline -- is covered + unconditionally. -2. **TF-IDF must remain byte-for-byte the default.** Bridge resolution in +2. **TF-IDF remains the offline fallback.** Bridge resolution in ``graph_loader`` / ``cli`` resolves agent-supplied ``calls`` names through ``search()`` against a 0.35 floor. That is exact-identifier retrieval, TF-IDF is good at it, and a dense backend silently taking over would wreck it. @@ -115,10 +115,10 @@ def hybrid_store(index_path): # The default is TF-IDF, and it is honest about it # --------------------------------------------------------------------------- # -def test_default_backend_is_tfidf(index_path, monkeypatch): +def test_default_policy_uses_cached_embeddings_only(index_path, monkeypatch): monkeypatch.delenv(vs.EMBEDDINGS_ENV_VAR, raising=False) store = LocalVectorStore(index_path) - assert store.policy == vs.POLICY_OFF + assert store.policy == vs.POLICY_AUTO assert store.backend == BACKEND_TFIDF @@ -128,7 +128,7 @@ def test_off_policy_never_imports_fastembed(index_path, monkeypatch): reach for fastembed at all, so an install without it behaves identically. """ monkeypatch.delenv(vs.EMBEDDINGS_ENV_VAR, raising=False) - store = LocalVectorStore(index_path) + store = LocalVectorStore(index_path, embeddings="off") store.add_documents([dict(d) for d in DOCS]) assert store.embedder is None @@ -138,8 +138,8 @@ def test_off_policy_never_imports_fastembed(index_path, monkeypatch): @pytest.mark.parametrize("value,expected", [ - (None, vs.POLICY_OFF), - ("", vs.POLICY_OFF), + (None, vs.POLICY_AUTO), + ("", vs.POLICY_AUTO), ("off", vs.POLICY_OFF), ("0", vs.POLICY_OFF), ("false", vs.POLICY_OFF), @@ -148,7 +148,7 @@ def test_off_policy_never_imports_fastembed(index_path, monkeypatch): ("on", vs.POLICY_ON), ("1", vs.POLICY_ON), ("true", vs.POLICY_ON), - ("nonsense", vs.POLICY_OFF), + ("nonsense", vs.POLICY_AUTO), ]) def test_policy_parsing(value, expected, monkeypatch): monkeypatch.delenv(vs.EMBEDDINGS_ENV_VAR, raising=False) @@ -174,7 +174,6 @@ def test_module_docstring_does_not_overclaim(): doc = vs.__doc__ or "" assert "TF-IDF" in doc lowered = doc.lower() - assert "optional" in lowered or "opt-in" in lowered assert "falls back" in lowered or "fall back" in lowered or "fallback" in lowered @@ -374,7 +373,7 @@ def test_diagnostics_reports_the_backend_that_is_really_live(tfidf_store): def test_diagnostics_explains_why_dense_is_off(index_path, monkeypatch): monkeypatch.delenv(vs.EMBEDDINGS_ENV_VAR, raising=False) - d = LocalVectorStore(index_path).diagnostics() + d = LocalVectorStore(index_path, embeddings="off").diagnostics() assert d["embedder_available"] is False assert d["embedder_reason"], "doctor must always be able to say WHY" diff --git a/tldrgraph/agent_commands.py b/tldrgraph/agent_commands.py index 1ce6808..c8cf7db 100644 --- a/tldrgraph/agent_commands.py +++ b/tldrgraph/agent_commands.py @@ -30,12 +30,12 @@ BLOCK_BEGIN = "" BLOCK_END = "" -#: The command name installed everywhere. Deliberately not bare `tldrgraph`: +#: The workflow name installed everywhere. Deliberately not bare `tldrgraph`: #: that would collide with the package's own CLI name in some shells. COMMAND_NAME = "tldrgraph-init" -#: The cross-tool instructions standard. Read by Claude Code, Cursor, Antigravity, -#: opencode, Codex, Gemini CLI, Zed and Copilot, so those tools need no file of their own. +#: The cross-tool instructions standard. Read by Claude Code, Cursor, Codex, +#: Antigravity, opencode, Gemini CLI, Zed and Copilot, so those tools need no file of their own. AGENTS_MD = "AGENTS.md" @@ -49,13 +49,14 @@ class AgentTarget: frontmatter: bool = True -#: Every tool TLDRGraph knows, with paths for commands and instructions. +#: Every tool TLDRGraph knows, with paths for commands, skills, and instructions. TARGETS: Tuple[AgentTarget, ...] = ( AgentTarget("Claude Code", command_path=f".claude/commands/{COMMAND_NAME}.md"), AgentTarget("Cursor", command_path=f".cursor/commands/{COMMAND_NAME}.md"), AgentTarget("opencode", command_path=f".opencode/command/{COMMAND_NAME}.md", marker=".opencode"), - AgentTarget("Antigravity", + # Codex's repository-local skill convention. Antigravity also reads this path. + AgentTarget("Codex", command_path=f".agents/skills/{COMMAND_NAME}/SKILL.md"), AgentTarget("Cline", command_path=f".clinerules/workflows/{COMMAND_NAME}.md", @@ -90,9 +91,7 @@ class AgentTarget: ) -# --------------------------------------------------------------------------- # -# The single source of content -# --------------------------------------------------------------------------- # +# The single source of content. #: Always-loaded context. Short on purpose: it competes with everything else in #: the agent's window, so it says what TLDRGraph is for and where the detail is. @@ -121,10 +120,10 @@ class AgentTarget: Those are read-only and never trigger enrichment. -**To build or continue the graph**, run `tldrgraph init`, do what the `NEXT ACTION` -block prints, and run it again -- repeat until `status: done`. It has no template -fallback: if this repository has no architecture yet, it will stop and ask you to -design one from the code. Do not skip reading the files. +**To build or refresh the graph**, run `tldrgraph init`. It automatically handles +layer design, extraction, source-aware enrichment in 200-node batches, and dense +embeddings when a supported agent CLI is available. If it prints a `NEXT ACTION` +fallback, follow that handoff without guessing from symbol names. Full workflow: `.claude/commands/{COMMAND_NAME}.md` (identical copies live in every other agent directory). Schema: `.tldrgraph/AGENT_CONTRACT.md`. @@ -138,21 +137,35 @@ class AgentTarget: - **Modularity & Re-exports**: Keep modules decoupled; preserve backwards compatibility with top-level package re-exports. """ -#: The full workflow. Every branch of the `init` state machine, spelled out. +#: The full `init` workflow. COMMAND_BODY = """# TLDRGraph: build this repository's architecture graph -One command, run repeatedly until it says DONE. `tldrgraph init` never guesses: -it stops and tells you exactly what it needs. +In Claude Code or Cursor, invoke `/tldrgraph-init`. In Codex CLI, type `/skills` +and select `tldrgraph-init`, or mention `$tldrgraph-init` directly. + +One command handles layer design, extraction, enrichment, and embeddings: ```bash tldrgraph init ``` -Read the `NEXT ACTION` block it prints, do what it says, then run `tldrgraph init` -again. Repeat until the output says `status: done`. There are only three things -it can ask for. +By default TLDRGraph detects `claude`, `cursor-agent`, or `gemini`, asks once before enrichment token spend, processes every +eligible node in batches of 200, and downloads/builds the local embedding model. +Use `--yes` for non-interactive approval, `--batch N` to override the batch size, +`--embeddings off|auto|on` to override embeddings, or `--no-agent-cli` for the +manual file handoff. + +After the user approves the full run, use exactly `tldrgraph init --yes`. The +approval is saved for the current candidate set, so later `tldrgraph init` calls +must continue without asking again. `--batch 200` means all nodes in 200-node +batches; `--limit 200` means stop after only 200 nodes. Never add `--limit` or +`--embeddings off` unless the user explicitly requests a partial or no-embedding run. + +If no supported agent is available or dense embeddings cannot be built, `init` +preserves the graph and prints a resumable status. It never guesses source intent +or architectural layers. -## 1. `status: needs_layers` +## `status: needs_layers` TLDRGraph ships **no layer templates** and will not invent an architecture. Design one from this repository. @@ -183,25 +196,7 @@ class AgentTarget: } ``` -4. Run `tldrgraph init` again. - -### What a layer set looks like - -Sketches from other codebases, to show the *shape* of an answer. They are not a -menu and none of them will fit this repository -- read the code and name what you -actually find. - -- A web app might split presentation from request handling from domain logic - from persistence, with background jobs and deployment config as their own tiers. -- A CLI tool might split the command surface from the processing engine from - local state, with adapters to outside systems separate again. -- A library might split its public API from the core implementation from its - data types, with backend adapters separate. -- A data pipeline might split ingestion from transformation from model training - from serving. - -The useful question is not "which of these is it?" but "where does responsibility -change hands in *this* code, and what would a new engineer need named?" +4. Run `tldrgraph init` again; it continues with enrichment and embeddings. ### Rules that hold for any answer @@ -214,17 +209,17 @@ class AgentTarget: - Derive rules from paths and symbol names you actually saw. A rule matching nothing is worse than no rule; a rule matching everything collapses the map. -## 2. `status: needs_confirmation` +## `status: needs_confirmation` The output shows how many nodes need enrichment and how many agent round-trips that implies. **Ask the user whether to proceed, and show them that estimate.** Do not decide for them. -- They agree: `tldrgraph init --yes` +- They agree: `tldrgraph init --yes` saves approval for the full campaign - Smaller first pass: `tldrgraph init --yes --limit 100` - They decline: stop. The graph is already built and queryable. -## 3. `status: needs_enrichment` +## `status: needs_enrichment` 1. Read `.tldrgraph/enrichment_request.yaml`. 2. **Open the source file of every node in it.** This is the entire point: an @@ -242,8 +237,14 @@ class AgentTarget: calls: [ApplicationsService, pension_cases] ``` -4. Run `tldrgraph init --yes` again. It applies the response and hands you the - next batch, until there is nothing left. +4. Run `tldrgraph init` again. Approval is already saved. If another + `needs_enrichment` batch appears, process it immediately and repeat this loop + without asking the user again. Continue until `status: done`. + +Inside an existing Codex/Claude/Cursor session, nested-agent protection may stop +the CLI from launching a second agent. In that case **you are the enrichment +agent**: process every 200-node batch yourself. A `needs_enrichment` status is a +continuation instruction, not a reason to stop or request confirmation. **Copy every `id` verbatim.** A constructed id matches nothing, is dropped, and gets reported back to you -- but the work is wasted. diff --git a/tldrgraph/agent_runner.py b/tldrgraph/agent_runner.py index 72bc979..b3f6a43 100644 --- a/tldrgraph/agent_runner.py +++ b/tldrgraph/agent_runner.py @@ -1,17 +1,15 @@ """ Host coding-agent runner for TLDRGraph. -**This is the secondary path.** TLDRGraph is normally driven *by* an agent -through the file handshake in ``tldrgraph init``, which works with every tool -because it only needs "read a file, read source, run a command". Shelling out is -opt-in (``--agent-cli``) and exists for a plain terminal with no agent attached. +TLDRGraph normally detects and launches a supported agent CLI from ``tldrgraph +init``. The file handshake remains the portable fallback when no CLI is usable +or when ``--no-agent-cli`` is selected. -It is not the default for good reasons, all observed in the field: every CLI has -different flags, auth and headless semantics; a blocking ``subprocess.run`` shows -the user nothing while the agent thinks; and some IDEs ship no usable CLI at all -(Antigravity's ``agy`` is a broken symlink on a stock install). +Agent CLIs still differ in flags, authentication, and headless behavior, and some +IDEs ship no usable CLI at all. Discovery and failures therefore always fall +back without discarding the graph. -Nesting is refused. When TLDRGraph is itself being run by a coding agent +Nesting is still refused. When TLDRGraph is itself being run by a coding agent (``CLAUDECODE``, ``CURSOR_AGENT``, ...), spawning a second agent burns tokens to duplicate context the host already has, so the host is told what to do instead. ``TLDRGRAPH_AGENT_NESTED=1`` overrides. @@ -23,9 +21,8 @@ TLDRGRAPH_AGENT_TIMEOUT= per-call timeout in seconds (default 600) TLDRGRAPH_AGENT_MODEL= model to pass to the agent CLI -Model choice only applies to the CLI shell-out. When your own agent drives -TLDRGraph through the file handshake -- the normal path -- the model is whatever -that session is already using, chosen in your IDE. +Model choice only applies to the CLI shell-out. When an existing host agent +drives the file-handshake fallback, the model is whatever that session uses. """ from __future__ import annotations diff --git a/tldrgraph/cli.py b/tldrgraph/cli.py index 56cf440..8470961 100644 --- a/tldrgraph/cli.py +++ b/tldrgraph/cli.py @@ -52,6 +52,7 @@ APPLIED_RESPONSE_FILENAME, STATUS_DONE, STATUS_NEEDS_CONFIRMATION, + STATUS_NEEDS_EMBEDDINGS, STATUS_NEEDS_ENRICHMENT, STATUS_NEEDS_LAYERS, apply_pending_enrichment_response, @@ -70,6 +71,7 @@ resolve_call_target, ) from .installer import ensure_gitignore, gitignore_warnings, install_agent_rules +from .init_policy import resolve_default_on_embeddings from .layer_config import config_path from .layers import get_registry, layer_id_of from .propose_layers import ( @@ -85,22 +87,22 @@ "--embeddings", "embeddings", type=click.Choice([vs_mod.POLICY_OFF, vs_mod.POLICY_AUTO, vs_mod.POLICY_ON]), default=None, - help="Retrieval backend policy. Defaults to $TLDRGRAPH_EMBEDDINGS, itself 'off'.", + help="Policy override. Init and query default to 'on'; other read commands default to cached-only 'auto'.", ) _init_options = [ click.argument("path", default=".", type=click.Path(exists=True)), click.option("--yes", "-y", "assume_yes", is_flag=True, - help="Proceed with enrichment without asking (agents: only after the user agrees)"), - click.option("--batch", "batch_size", default=25, show_default=True, + help="Approve the current full enrichment campaign without asking again"), + click.option("--batch", "batch_size", default=200, show_default=True, help="Nodes handed to the agent per round"), click.option("--limit", "max_nodes", default=0, show_default=True, - help="Cap on nodes to enrich this run. 0 enriches every candidate."), + help="Partial-run cap. 0 authorizes every current candidate."), click.option("--rebuild", is_flag=True, help="Re-extract and rebuild enrichment from scratch"), click.option("--relayer", is_flag=True, help="Discard the layer set and design it again"), - click.option("--agent-cli", is_flag=True, - help="Shell out to an agent CLI (claude/cursor-agent/gemini) instead of " - "handing off. Off by default: agent CLIs differ per tool and can hang."), + click.option("--agent-cli/--no-agent-cli", default=True, show_default=True, + help="Automatically use a supported agent CLI for layers and enrichment; " + "disable to use the file handoff workflow."), click.option("--agent-model", default=None, help="Model for --agent-cli (e.g. opus, sonnet, gemini-2.5-pro). Defaults " "to $TLDRGRAPH_AGENT_MODEL. Ignored on the handshake path, where your " @@ -142,7 +144,7 @@ def cli(): @cli.command() @_with_init_options def init(path, assume_yes, batch_size, max_nodes, rebuild, relayer, agent_cli, agent_model, as_json, embeddings): - """Build this repository's graph: layers, extraction, and enrichment, in one command.""" + """Build layers, extract, enrich, and embed this repository in one command.""" init_pipeline(path, assume_yes, batch_size, max_nodes, rebuild, relayer, agent_cli, agent_model, embeddings, as_json) @@ -178,12 +180,12 @@ def visualizer_cmd(path, serve, port, open_browser): @cli.command() @click.argument("query_text") -@click.option("--top-k", default=3, help="Number of flow candidates to return") +@click.option("--top-k", default=5, show_default=True, help="Number of flow candidates to return") @click.option("--path", default=".", help="Repository root path") @embeddings_option def query(query_text, top_k, path, embeddings): """Hybrid search + trace end-to-end multi-layer execution flows. Read-only: never enriches.""" - loader = GraphLoader(path, embeddings=embeddings) + loader = GraphLoader(path, embeddings=resolve_default_on_embeddings(embeddings)) graph = loader.load_or_extract(enrich_llm=False) engine = FlowEngine(graph, loader.vector_store, root_dir=path) results = engine.query_flow(query_text, top_k=top_k) @@ -228,7 +230,7 @@ def layers(path): @cli.command("queue-enrichment") @click.option("--path", default=".", help="Repository root path") -@click.option("--limit", default=50, show_default=True, +@click.option("--limit", default=200, show_default=True, help="Maximum nodes to queue in this batch. 0 queues every remaining candidate.") @click.option("--requeue", is_flag=True, help="Also re-queue ids handed out earlier but never applied (abandoned batches).") @@ -325,9 +327,9 @@ def doctor(path, embeddings, as_json): @cli.command() @click.option("--path", default=".", help="Repository root path") @click.option("--all-agents", is_flag=True, - help="Write the /tldrgraph-init command for every agent tool TLDRGraph knows.") + help="Write the tldrgraph-init workflow for every agent tool TLDRGraph knows.") def install(path, all_agents): - """Install TLDRGraph agent rules for Claude Code, Cursor and Antigravity.""" + """Install TLDRGraph rules and workflows for Codex and other coding agents.""" gitignore = ensure_gitignore(path) res = install_agent_rules(path, all_agents=all_agents) click.echo("✅ TLDRGraph agent skills & rules installed successfully:") @@ -336,7 +338,8 @@ def install(path, all_agents): continue click.echo(f" • {k}: {v}") click.echo(f" • gitignore: {gitignore['path']} ({gitignore['status']})") - click.echo("\n💡 Your agent can now run /tldrgraph-init (or just `tldrgraph init`) to build the whole graph.") + click.echo("\n💡 Claude/Cursor: /tldrgraph-init · Codex: /skills → tldrgraph-init or $tldrgraph-init") + click.echo(" Any agent can also run `tldrgraph init` directly.") for warning in gitignore_warnings(path): click.echo(f"⚠️ {warning}") diff --git a/tldrgraph/cli_agent_loop.py b/tldrgraph/cli_agent_loop.py index 22303cf..04858ef 100644 --- a/tldrgraph/cli_agent_loop.py +++ b/tldrgraph/cli_agent_loop.py @@ -117,7 +117,7 @@ def run_agent_enrichment( path: str, loader: GraphLoader, agent: Any, - batch_size: int = 25, + batch_size: int = 200, max_nodes: int = 0, model: Optional[str] = None, ) -> Dict[str, Any]: diff --git a/tldrgraph/cli_pipeline.py b/tldrgraph/cli_pipeline.py index 7feee19..d4e2e2b 100644 --- a/tldrgraph/cli_pipeline.py +++ b/tldrgraph/cli_pipeline.py @@ -33,6 +33,14 @@ from .installer import ensure_gitignore, install_agent_rules from .layer_config import config_path from .layers import get_registry +from .init_policy import ( + clear_enrichment_approval, + embedding_failure as _embedding_failure, + embedding_summary as _embedding_summary, + enrichment_approval_is_active, + remember_full_enrichment_approval, + resolve_init_embeddings, +) from .propose_layers import ( RESPONSE_FILENAME as PROPOSE_RESPONSE_FILENAME, apply_proposed_layers, @@ -45,9 +53,9 @@ STATUS_NEEDS_LAYERS = "needs_layers" STATUS_NEEDS_CONFIRMATION = "needs_confirmation" STATUS_NEEDS_ENRICHMENT = "needs_enrichment" +STATUS_NEEDS_EMBEDDINGS = "needs_embeddings" APPLIED_RESPONSE_FILENAME = "enrichment_response.applied.yaml" - @contextlib.contextmanager def stdout_to_stderr_if(active: bool): if not active: @@ -124,7 +132,7 @@ def _check_confirmation(candidates: List[Dict[str, Any]], total: int, enriched: "", "ASK THE USER whether to proceed, showing them that estimate. Then:", "", - " they agree → tldrgraph init --yes", + " they agree → tldrgraph init --yes (approval persists until done)", " smaller first pass → tldrgraph init --yes --limit 100", " they decline → stop here; the graph is already usable", ], progress=progress, as_json=as_json) @@ -148,14 +156,24 @@ def _run_agent_cli_enrichment( if not as_json: click.echo(f"\n🤖 Enriching via {agent.display}...") - totals = run_agent_enrichment(path, loader, agent, batch_size=batch_size, max_nodes=max_nodes, model=agent_model) + with stdout_to_stderr_if(as_json): + totals = run_agent_enrichment( + path, loader, agent, batch_size=batch_size, max_nodes=max_nodes, model=agent_model + ) rem = len(enrichment_candidates(loader, compute_degrees(loader.graph))) - status = STATUS_DONE if not rem else STATUS_NEEDS_ENRICHMENT - emit_status(status, "enrichment", [ + if not rem: + clear_enrichment_approval(path) + embedding_error = None if rem else _embedding_failure(loader) + status = STATUS_NEEDS_ENRICHMENT if rem else ( + STATUS_NEEDS_EMBEDDINGS if embedding_error else STATUS_DONE + ) + retry = ["Run `tldrgraph init --yes` to continue."] if rem or embedding_error else [] + emit_status(status, "embeddings" if embedding_error else "enrichment", [ f"Enriched {totals['applied']} node(s) in {totals['batches']} batch(es); {totals['bridges']} bridge edge(s).", f"{rem} still un-enriched." if rem else "Nothing left to enrich.", - ] + (["Run `tldrgraph init --yes` to continue."] if rem else []), - progress={**progress, "remaining": rem}, as_json=as_json) + f"Dense embeddings could not be completed: {embedding_error}" if embedding_error else _embedding_summary(loader), + ] + retry, + progress={**progress, "remaining": rem, "embedding_backend": loader.vector_store.backend}, as_json=as_json) return status @@ -181,21 +199,36 @@ def _emit_manual_enrichment_handoff( " symbol name is worse than none -- it poisons semantic search.", f" 3. Write {os.path.join(STATE_DIR, RESPONSE_FILENAME)} (YAML list of", " {id, intent, input_fields, output_fields, calls}). Copy each id verbatim.", - " 4. Run: tldrgraph init", + " 4. Run: tldrgraph init (approval is remembered; do not ask again)", ], progress=progress, as_json=as_json) return STATUS_NEEDS_ENRICHMENT -def _emit_enrichment_done(total: int, enriched: int, excluded: int, registry: Any, as_json: bool) -> str: - emit_status(STATUS_DONE, "enrichment", [ +def _emit_enrichment_done(loader: GraphLoader, total: int, enriched: int, excluded: int, registry: Any, as_json: bool) -> str: + embedding_error = _embedding_failure(loader) + status = STATUS_NEEDS_EMBEDDINGS if embedding_error else STATUS_DONE + lines = [ f"{total} nodes across {len(registry)} layers. {enriched} enriched from source; {excluded} not eligible (utility bucket and prose nodes).", "", + ] + if embedding_error: + lines.extend([ + f"Dense embeddings could not be completed: {embedding_error}", + "Run `tldrgraph init --yes` again after fixing model access.", + ]) + else: + lines.extend([ + _embedding_summary(loader), + "", ' tldrgraph query ""', ' tldrgraph trace "" ""', " tldrgraph layers", " tldrgraph ui --serve", - ], progress={"total_nodes": total, "enriched": enriched, "remaining": 0}, as_json=as_json) - return STATUS_DONE + ]) + emit_status(status, "embeddings" if embedding_error else "enrichment", lines, + progress={"total_nodes": total, "enriched": enriched, "remaining": 0, + "embedding_backend": loader.vector_store.backend}, as_json=as_json) + return status def _handle_enrichment_step( @@ -219,7 +252,8 @@ def _handle_enrichment_step( excluded = total - enriched - len(candidates) if not candidates: - return _emit_enrichment_done(total, enriched, excluded, registry, as_json) + clear_enrichment_approval(path) + return _emit_enrichment_done(loader, total, enriched, excluded, registry, as_json) planned = min(len(candidates), max_nodes) if max_nodes else len(candidates) rounds = (planned + batch_size - 1) // batch_size @@ -231,12 +265,19 @@ def _handle_enrichment_step( "planned_this_run": planned, "batch_size": batch_size, "agent_rounds": rounds, + "approval_persisted": enrichment_approval_is_active(path, candidates), } - if not assume_yes: + authorized = assume_yes or enrichment_approval_is_active(path, candidates) + if not authorized: conf_status = _check_confirmation(candidates, total, enriched, excluded, rounds, batch_size, progress, as_json) if conf_status: return conf_status + authorized = True + + if authorized and not max_nodes: + remember_full_enrichment_approval(path, candidates) + progress["approval_persisted"] = True if agent_cli: res = _run_agent_cli_enrichment(path, loader, batch_size, max_nodes, agent_model, progress, as_json) @@ -318,6 +359,9 @@ def init_pipeline( as_json: bool, ) -> str: root = os.path.abspath(path) + embeddings = resolve_init_embeddings(embeddings) + if rebuild or relayer: + clear_enrichment_approval(path) if not as_json: click.echo(f"🔄 [TLDRGraph] {root}") diff --git a/tldrgraph/dense_embedder.py b/tldrgraph/dense_embedder.py index 55fa019..8c9af77 100644 --- a/tldrgraph/dense_embedder.py +++ b/tldrgraph/dense_embedder.py @@ -25,14 +25,14 @@ def resolve_policy(requested: Optional[str] = None) -> str: """Resolves policy from explicit argument or environment.""" raw = (requested if requested is not None else os.environ.get(EMBEDDINGS_ENV_VAR, "")).strip().lower() if not raw: - return POLICY_OFF + return POLICY_AUTO if raw in (POLICY_OFF, POLICY_AUTO, POLICY_ON): return raw if raw in _TRUTHY: return POLICY_ON if raw in _FALSEY: return POLICY_OFF - return POLICY_OFF + return POLICY_AUTO def default_model_cache_dir() -> str: diff --git a/tldrgraph/init_policy.py b/tldrgraph/init_policy.py new file mode 100644 index 0000000..24c10c5 --- /dev/null +++ b/tldrgraph/init_policy.py @@ -0,0 +1,87 @@ +"""Persistent init authorization and embedding policy helpers.""" + +from __future__ import annotations + +from datetime import datetime, timezone +import json +import os +from typing import Any, Dict, List, Optional + +from .dense_embedder import EMBEDDINGS_ENV_VAR, POLICY_ON, resolve_policy + +APPROVAL_FILENAME = "enrichment_approval.json" +APPROVAL_SCHEMA = "tldrgraph/enrichment-approval@1" + + +def resolve_default_on_embeddings(requested: Optional[str]) -> str: + """Enable downloads unless a CLI argument or environment override says otherwise.""" + if requested is None and not os.environ.get(EMBEDDINGS_ENV_VAR, "").strip(): + return POLICY_ON + return resolve_policy(requested) + + +def resolve_init_embeddings(requested: Optional[str]) -> str: + """Init downloads/builds embeddings unless explicitly overridden.""" + return resolve_default_on_embeddings(requested) + + +def embedding_failure(loader: Any) -> Optional[str]: + diag = loader.vector_store.diagnostics() + required_but_missing = ( + loader.graph.number_of_nodes() + and diag["policy"] == POLICY_ON + and diag["backend"] != "hybrid" + ) + if required_but_missing: + return str(diag.get("embedder_reason") or "dense embedding backend unavailable") + return None + + +def embedding_summary(loader: Any) -> str: + backend = loader.vector_store.diagnostics()["backend"] + if backend == "hybrid": + return "Dense embedding index is ready." + return "TF-IDF index is ready; dense embeddings are disabled or cached-only." + + +def approval_path(root: str) -> str: + return os.path.join(os.path.abspath(root), ".tldrgraph", APPROVAL_FILENAME) + + +def remember_full_enrichment_approval(root: str, candidates: List[Dict[str, Any]]) -> str: + """Authorize exactly the current campaign's candidate IDs until exhausted.""" + path = approval_path(root) + os.makedirs(os.path.dirname(path), exist_ok=True) + payload = { + "schema": APPROVAL_SCHEMA, + "approved_at": datetime.now(timezone.utc).isoformat(), + "candidate_ids": sorted(str(item["id"]) for item in candidates), + } + temp_path = f"{path}.tmp" + with open(temp_path, "w", encoding="utf-8") as handle: + json.dump(payload, handle, indent=2) + os.replace(temp_path, path) + return path + + +def enrichment_approval_is_active(root: str, candidates: List[Dict[str, Any]]) -> bool: + """Accept only a non-empty candidate set contained in the approved campaign.""" + try: + with open(approval_path(root), "r", encoding="utf-8") as handle: + payload = json.load(handle) + except (OSError, ValueError, TypeError): + return False + if not isinstance(payload, dict): + return False + approved = payload.get("candidate_ids") + if payload.get("schema") != APPROVAL_SCHEMA or not isinstance(approved, list): + return False + current_ids = {str(item["id"]) for item in candidates} + return bool(current_ids) and current_ids.issubset({str(item) for item in approved}) + + +def clear_enrichment_approval(root: str) -> None: + try: + os.remove(approval_path(root)) + except FileNotFoundError: + pass diff --git a/tldrgraph/installer_contract.py b/tldrgraph/installer_contract.py index 2938bf8..5ad481a 100644 --- a/tldrgraph/installer_contract.py +++ b/tldrgraph/installer_contract.py @@ -32,22 +32,26 @@ def generate_layers_prose(registry: Optional[LayerRegistry] = None) -> str: return "\n".join(lines) -_LOOP = """One command does everything -- layers, extraction, enrichment -- and is resumable: +_LOOP = """One command does everything -- layers, extraction, enrichment, and embeddings: ```bash -tldrgraph init # prints a NEXT ACTION block whenever it needs you -# do exactly what that block says (design layers, or read source and write intents) -tldrgraph init --yes # run it again; repeat until it prints status: done +tldrgraph init # interactive: asks once before enrichment token spend +tldrgraph init --yes # approve every current candidate until the campaign is done ``` -`init` never guesses. It has no template to fall back on, so if this repository has no -architecture yet it stops and asks you to design one from the code. When it needs -enrichment it hands you a batch, and you OPEN THE SOURCE FILES before writing anything. +`init` automatically detects a supported agent CLI, uses 200-node enrichment batches, +and downloads/builds dense embeddings. It never guesses: when no agent is usable it +preserves the graph and prints a manual layer or enrichment handoff. + +Full approval is persisted across continuation runs. In a nested coding-agent session, +the host agent must read, answer, and apply every 200-node batch without asking again. +`--batch 200` means all nodes in chunks; `--limit 200` means only 200 total. Never add +`--limit` or `--embeddings off` unless the user explicitly requests it. The underlying steps stay available for scripting: ```bash -tldrgraph queue-enrichment --limit 50 # writes .tldrgraph/enrichment_request.yaml +tldrgraph queue-enrichment --limit 200 # writes .tldrgraph/enrichment_request.yaml tldrgraph apply-enrichment # merges intents + bridge edges into the graph ``` @@ -63,7 +67,9 @@ def generate_layers_prose(registry: Optional[LayerRegistry] = None) -> str: (`ApplicationsService`), file names (`calc.ts`), IDs, and table names (`pension_cases`) match with 100% confidence; fallback vector search handles related terms with a 0.35 score floor. - **Write the response to a different file than the request.** The request is regenerated - on every run.""" + on every run. +- **Continue after approval until `status: done`.** A `needs_enrichment` batch is work to + process, not a reason to ask again. Do not add `--limit` or `--embeddings off`.""" _RESPONSE_SCHEMA = """```yaml - id: "" @@ -109,6 +115,7 @@ def generate_layers_prose(registry: Optional[LayerRegistry] = None) -> str: | --- | --- | --- | | `.tldrgraph/enrichment_request.json` | `queue-enrichment` | you | | `.tldrgraph/enrichment_response.json` | **you** | `apply-enrichment` | +| `.tldrgraph/enrichment_approval.json` | `init --yes` | continuation runs | | `.tldrgraph/pending_enrichment.json` | *(legacy)* | `apply-enrichment`, only if no response file exists | ## Response schema diff --git a/tldrgraph/vector_store.py b/tldrgraph/vector_store.py index 2122f82..b626b3d 100644 --- a/tldrgraph/vector_store.py +++ b/tldrgraph/vector_store.py @@ -1,5 +1,5 @@ """ -Local Vector Store for TLDRGraph: TF-IDF (default) and FastEmbed hybrid retrieval (optional opt-in, falls back to TF-IDF, 100% offline, $0 token cost). +Local Vector Store for TLDRGraph: FastEmbed hybrid retrieval when its model is cached, with a 100% offline TF-IDF fallback. """ from __future__ import annotations diff --git a/tldrgraph/visualizer/bpmn_phrasebook.py b/tldrgraph/visualizer/bpmn_phrasebook.py index c1f7076..b6ff5a1 100644 --- a/tldrgraph/visualizer/bpmn_phrasebook.py +++ b/tldrgraph/visualizer/bpmn_phrasebook.py @@ -254,25 +254,25 @@ "say": "If the existing rules cannot be read", "when": "OSError", }, - "tldrgraph/agent_commands.py:372:gateway": { + "tldrgraph/agent_commands.py:376:gateway": { "say": "Does the repo already have an AGENTS file?", "when": "os.path.isfile(agents_md)", "yes": "Update it", "no": "Skip it", }, - "tldrgraph/agent_commands.py:376:error": { + "tldrgraph/agent_commands.py:380:error": { "say": "If the AGENTS file cannot be written", "when": "OSError", }, - "tldrgraph/agent_commands.py:384:loop": { + "tldrgraph/agent_commands.py:388:loop": { "say": "For each coding agent this repo uses", "when": "active_targets(root, all_agents=all_agents)", }, - "tldrgraph/agent_commands.py:385:gateway": { + "tldrgraph/agent_commands.py:389:gateway": { "say": "Does this agent support slash commands?", "when": "target.command_path", "yes": "Install the command", "no": "No commands", }, - "tldrgraph/agent_commands.py:389:gateway": { + "tldrgraph/agent_commands.py:393:gateway": { "say": "Does this agent read an instructions file?", "when": "target.instructions_path", "yes": "Write the instructions", "no": "None needed",