diff --git a/.gitleaksignore b/.gitleaksignore index e6add874d..486f4453e 100644 --- a/.gitleaksignore +++ b/.gitleaksignore @@ -2,11 +2,22 @@ # policy names for intentional findings). One fingerprint per line: # commit:file:rule:startline # -# False positive: generic-api-key matching the shell variable inits -# `CLAUDE_TOKEN=""` / `CLAUDE_SHIM=0` in the AGENTS.md pointer-line resolver on -# the retired-conventions feature branches ("token" is a parsing token, not a -# credential; the flagged value is an empty string). The hygiene lane fetches -# every branch and scans --all, so this finding turns every PR red until the -# branch merges or rewrites; remove this line when that lands differently. +# False positive: generic-api-key firing on the AGENTS.md pointer-line resolver +# on the retired-conventions feature branches. TOKEN there is the parsing sense, +# never a credential: the `AGENTS_TOKEN=""` / `CLAUDE_TOKEN=""` inits supply the +# rule's keyword context, and the reported secret is the following line, +# `CLAUDE_SHIM=0`, a flag set to zero. +# +# The hygiene lane fetches every branch and scans --all, so this finding turns +# every PR red until the branch merges or rewrites. Verified against CI's pinned +# gitleaks 8.30.1, so the next reader need not retry them: renaming the flag +# still fires, and so does separating it from the TOKEN lines by a blank line. +# An inline `gitleaks:allow` marker does clear the working file, and the +# branches carry one, but it cannot clear a finding already recorded against an +# earlier commit, because the scan walks history and that blob is immutable. +# Hence fingerprints. +# +# Remove each line when the commit it names is no longer reachable (its branch +# squash-merged or deleted); the entry is dead config from that point on. f7aec1549dec60407a24a7d8ea720696b7b099af:plugins/claude-config/lib/resolve-convention-home.sh:generic-api-key:248 708739fb2ba1b0196e93f9c07d3293bddfeed2f0:plugins/plugin-quality/lib/resolve-convention-home.sh:generic-api-key:249 diff --git a/docs/conventions/instruction-exception-register/README.md b/docs/conventions/instruction-exception-register/README.md new file mode 100644 index 000000000..d88d3ffad --- /dev/null +++ b/docs/conventions/instruction-exception-register/README.md @@ -0,0 +1,79 @@ +# Instruction exception register — what a trimming pass may not delete + +Owner doc for the classes of standing instruction that survive an instruction-audit trim on +consequence grounds, whatever a length, redundancy, or model-era check says about them. + +The register exists because subtraction guidance has no natural floor. The current generation of +prompting guidance is explicitly subtractive: strip prescriptive scaffolding, let the model use +judgment, delete rules written for older models. That guidance is sound and this repo follows it, +but it is stated for behavioral rules and carries a carve-out its own wording leaves undefined, +"avoid making them overconstrained, **except in highly important areas**". Without a written +answer to which areas those are, a trimming pass has no principled stopping point. + +## The classes are Gate 0's, adopted by reference + +The consequence classes this register protects are exactly the six hard-deny classes in +[`plugins/instruction-placement/context/routing-rubric.md`](../../../plugins/instruction-placement/context/routing-rubric.md), +Gate 0: `irreversible-action`, `secret-handling`, `data-integrity`, `external-publication`, +`legal-compliance`, `agent-authority`. + +**This document does not restate that table, and no other surface may fork it.** One concern keeps +one adjudication chain: the class list has a single owner, and a second enumeration under a second +name is the drift this repo has already paid for elsewhere. + +What this register adds is the **operation**. Gate 0 governs *relocation*: whether a rule may be +demoted out of an always-loaded surface into a path-scoped or skill destination. Deletion is a +different operation with a strictly worse failure mode, because a demoted rule that fails to fire is +recoverable by re-promoting it while a deleted rule leaves nothing to re-promote. So: + +> A candidate matching any Gate 0 class is **not deletable** by an instruction-audit trim. It may be +> compressed in place, given a rationale, or converted to a deterministic mechanism such as a hook. +> It is never removed on redundancy, brevity, or "the model already does this" grounds. + +## Non-exhaustive, and tighten-only + +Two properties are load-bearing and neither is decoration: + +- **Non-exhaustive.** The classes are the recognized floor, never the complete set of things worth + keeping. **Omission from this register is not licence to delete.** A rule outside every class is + judged on its own merits by the auditing skill's normal criteria, exactly as it was before this + document existed; it does not inherit a deletion warrant from its absence here. +- **Tighten-only.** A consuming repo or a downstream skill may add protected classes. Nothing that + reads this register may use it to *weaken* a protection, and no argument, including an operator + asking in the moment, removes a Gate 0 class from the protected set. That is Gate 0's own rule and + it travels with the classes. + +**Recognition is by consequence, not by phrasing**, also inherited from Gate 0. Ask what breaks when +the instruction is absent at the moment it was needed, not how the sentence is worded. A one-line +"never force-push a shared branch" is a protected rail; a paragraph preferring one git subcommand +over another is style. + +## Who consumes this + +| Consumer | How it uses the register | +|---|---| +| `claude-config:audit-instructions` | Deletion-class criteria (I1, I4, I5) hold back a candidate matching a protected class and report the hold rather than proposing the cut | +| `claude-config:unhobble` | The bare-baseline experiment may strip a protected rule during the run, since the strip is reversible and branch-local, but Phase 4 restores it regardless of whether the ledger logged a stumble against it — a protected rule is never left deleted on the evidence of "no stumble was observed" | +| `instruction-placement:*` | Unchanged. It owns the classes and the relocation verdict; this register is the deletion counterpart and defers to it on class membership | + +A consumer that reads this register names it in its own criteria text. A register nothing consumes +changes no behavior, which is the failure mode this table exists to prevent. + +## What this is not + +- Not a second opinion on relocation. Gate 0 decides that, and a candidate held here may still be + legitimately relocated. +- Not a suppression record. A finding an operator has judged and accepted is the + [finding-suppression](../finding-suppression/README.md) convention's axis. +- Not a licence to keep everything. The subtractive posture stands for every candidate outside the + protected classes, and compression in place remains the right answer for a protected rule that has + genuinely grown bloated. + +## Provenance + +Adopted 2026-09-01 from the context-engineering corpus integration, decision Q2. That decision +contract is contract tier and was pruned before merge; it survives as the `
` paste in its +pull request, and the evidence behind it is in +[`docs/specs/context-engineering-corpus-knowledge.md`](../../specs/context-engineering-corpus-knowledge.md). The carve-out this register answers is quoted from a vendor-voice source and carries +that source's status: the wording is first-party, the definition of "highly important areas" is +this repo's own and is not claimed to be upstream doctrine. diff --git a/docs/conventions/upstream-drift/CHANGELOG.md b/docs/conventions/upstream-drift/CHANGELOG.md index bc189f59a..a6cc9b17e 100644 --- a/docs/conventions/upstream-drift/CHANGELOG.md +++ b/docs/conventions/upstream-drift/CHANGELOG.md @@ -4,6 +4,27 @@ Notable changes to the upstream-drift contract (SemVer). Changing a required par name, or an enforceability verdict is a major bump; additive guidance is a minor bump; docs-only clarification is a patch. +## 1.6.3 — 2026-09-01 + +Recorded near-miss evidence ADJACENT to the content-hashing deferral; docs-only, the deferral's +verdict does not change and its trigger has NOT fired. + +The context-engineering-integration corpus pass (signed off 2026-09-01; its decision contract is +contract tier and was pruned before merge, surviving as the `
` paste in its pull request, +with the durable evidence in +[`docs/specs/context-engineering-corpus-knowledge.md`](../../specs/context-engineering-corpus-knowledge.md)) +found the live +anthropic.com/research/building-effective-agents page silently revised after publication: the +page still says "Published Dec 19, 2024" while its body now names models and products that +post-date that date, so any citation of "the December 2024 post" at that URL quotes post-hoc +edits. This is the failure class a stored content hash would flag — but the deferral's recheck +trigger ("a stale stamp causes a real defect a stored hash would have flagged, or a fleet audit +completes without re-fetching every stamped claim") has not fired: no committed stamp on that +page exists, so no stale stamp caused a defect in a tracked record. Recorded here as adjacent +evidence for the eventual designed issue the deferral itself prescribes ("a hash store becomes +its own designed issue, not an inline addition here"); that issue is filed as a tracker item by +the integration effort. The deferral stays as written. + ## 1.6.2 — 2026-08-31 Recorded firing of the no-adoption-gate decision's recheck trigger; docs-only under this contract's diff --git a/docs/specs/context-engineering-corpus-knowledge.md b/docs/specs/context-engineering-corpus-knowledge.md new file mode 100644 index 000000000..3a781a553 --- /dev/null +++ b/docs/specs/context-engineering-corpus-knowledge.md @@ -0,0 +1,976 @@ +# Context-engineering corpus knowledge + +The durable record of the two-article context-engineering corpus this repo ingested on +2026-08-31 and 2026-09-01: what the sources say, in their own words where the wording is +load-bearing; the facts that exist only in their figures; the upstream facts the run settled +against current documentation; the custody findings about the sources themselves; and an honest +accounting of what was verified and what was not. + +The ingestion produced byte-verified digest slices, a fresh unbiased sweep, and reconciliation +tables in the session's memory tier, which is gitignored and does not survive the container. +This file is the graduation of that work: it inlines the substance rather than pointing at +artifacts a future reader cannot open. Where an in-session artifact is named below, it is named +only to record how a fact was produced, never as a path to follow. + +Companion documents: the signed-off decision contract this corpus fed, which was a contract-tier +slice and is therefore pruned before merge under the topic-docs convention, so it survives as the +`
` paste in its pull request; and the earlier `context-engineering-claude-5` plan, built +on P1 alone and never engaging P2, whose slice main pruned in the topic-docs v3.0.0 adoption. +Sibling durable outputs of the same corpus family: `docs/FINDING-YOUR-UNKNOWNS.md` and the +neighboring `context-engineering-*.md` specs listed at the end of this section. + +## Contents + +- [1. What the corpus is](#1-what-the-corpus-is) +- [2. P1: the new rules of context engineering](#2-p1-the-new-rules-of-context-engineering) +- [3. P2: effective context engineering for AI agents](#3-p2-effective-context-engineering-for-ai-agents) +- [4. Figure-borne facts](#4-figure-borne-facts) +- [5. Settled upstream facts](#5-settled-upstream-facts) +- [6. Custody findings (CF-1 to CF-7)](#6-custody-findings-cf-1-to-cf-7) +- [7. Coverage accounting and residue](#7-coverage-accounting-and-residue) + +## 1. What the corpus is + +### Primary sources + +**P1. "The new rules of context engineering for Claude 5 models"** +Thariq (Thariq Shihipar, `@trq212`), member of technical staff at Anthropic, published as a +personal X Article on 2026-07-24. + (status form: +). +Blog twin, the same content on a first-party surface, carrying the 80% figure in its page +description: . +Venue matters: this is a personal post by an Anthropic employee, not an Anthropic docs +property. Five digest units over a 119-line markdown channel; four body figures plus a cover, +recoverable only from the fxtwitter JSON channel. + +**P2. "Effective context engineering for AI agents"** +Anthropic Applied AI team: Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, and Jeremy Hadfield, +with contributions from Rafi Ayub, Hannah Moran, Cal Rueb, and Connor Jennings; support from +Molly Vorwerck, Stuart Ritchie, and Maggie Vo. Published 2025-09-29 (recovered from the +rendered metadata in the retained HTML snapshot; the markdown channel carries no date). +. +Six digest units over a 75-line markdown channel; two content figures that survive only as +image URLs in the HTML. P1 links this post as its definition of "context engineering"; P2 +predates it by ten months. + +### The nine linked first-party pages + +| # | URL | Role | +|---|---|---| +| T1 | | X-native surface of the Fable field guide, 2026-07-03; the same essay as T2, one content node | +| T2 | | "A Field Guide to Fable: Finding Your Unknowns", 2026-07-06 (modified 2026-08-03); the designated deep dive on prompting the Claude 5 generation, and the piece P1 hands off to | +| T3 | | "A harness for every task: dynamic workflows", 2026-06-02 (Shihipar, Bidasaria); the mechanism behind P1's rubrics and verifier-agents claim, and the target of P1's tree-of-files link | +| T4 | | "Writing effective tools for agents", 2025-09-11 (Ken Aizawa); the tool-design guidance P2's anatomy section leans on | +| T5 | | "Building effective agents", nominally 2024-12-19; P2's workflow-versus-agent definition source, and a silently revised page (see CF-1) | +| T6 | | "How we built our multi-agent research system", 2025-06-13; the evidence base for sub-agent architecture, including the token-economics numbers | +| T7 | | Context-management product announcement, Sonnet 4.5 launch window; the product surface behind P2's compaction and memory-tool claims | +| T8 | | Memory cookbook, 2025-05-22 (Alex Notov); the hands-on cookbook P2 closes with, now struck as a model-list authority (see CF-5) | +| T9 | | Prompt-engineering overview plus its "Prompting best practices" page; the docs P2 positions itself against, and the carrier of a documented generational reversal (see CF-6) | + +### Corpus timeline + +| Date | Item | +|---|---| +| 2024-12-19 (nominal) | T5 "Building effective agents"; the live page has been silently revised since | +| 2025-05-22 | T8 memory cookbook | +| 2025-06-13 | T6 multi-agent research system | +| 2025-09-11 | T4 writing effective tools for agents | +| 2025-09-29 | **P2** "Effective context engineering for AI agents" | +| Sonnet 4.5 launch window | T7 context-management announcement | +| 2026-06-02 | T3 dynamic workflows | +| 2026-07-03 and 2026-07-06 | T1 and T2, two surfaces of the Fable field guide | +| 2026-07-24 | **P1** "The new rules of context engineering for Claude 5 models" | +| undated, living | T9 prompt-engineering overview and best-practices pages | + +## 2. P1: the new rules of context engineering + +Quotes below are byte-exact from the P1 markdown channel as verified in-session by two +independent arms plus a scripted fence gate. Curly quotes, doubled spaces, and comma placement +inside backticks are reproduced from the source, not typos introduced here. + +### 2.1 The framing: prompt versus assembled context + +Context engineering is defined by enumerating the surfaces that assemble it, and separated from +prompting by a generality premise: a prompt is written for one request, context is amortized +across many and must be written without knowing the request. + +``` +But when you send a message to Claude, the prompt is only a small part of the context it gets. Much of your context is assembled from your system prompt, Skills, CLAUDE.md files, memory, and other sources. We call this context engineering, and it makes a big impact on the results you generate when using Claude Code or in building your own agents. +``` + +``` +Unlike a prompt, context is used generally across many requests, so it cannot be as specific. How do you build these general prompts and guidance for Claude, especially when you don’t know what a user’s prompt might be? +``` + +The headline empirical claim, and the only number in the article's body text: + +``` +This can be surprisingly difficult as Claude’s own capabilities evolve. Most recently, we noticed a large jump in the way we prompt the newest generation of Claude models. We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations. +``` + +Read the scope literally: no measurable loss on that team's coding evaluations, for Claude +5-generation models. Section 5 records what official surfaces do and do not corroborate. + +The tooling handoff, stated twice in the article: + +``` +Here’s what we’ve learned about prompting this new class of models, and how you can utilize it to update your context engineering. We’ve put these best practices in `claude doctor`, use the command /doctor in Claude Code to rightsize your skills, and CLAUDE.md files. +``` + +### 2.2 Unhobbling: over-constraint as a joint product of surfaces + +The diagnosis names three surfaces at once, and the evidence offered is transcript reading of +Anthropic's own internal use. + +``` +Overall, we found that we were over-constraining Claude Code, both through our system prompt and in our CLAUDE.md files and skills. +``` + +``` +For example, when we read transcripts of our own internal usage of Claude Code, we see several conflicting messages in a single request like “leave documentation as appropriate,” or “DO NOT add comments” as our system prompt, skills, and user requests clash with each other. +``` + +The cost is not a wrong answer, it is deliberation spent reconciling: + +``` +Generally, Claude can interpret the user’s intent to get to the right answer, but Claude must think more carefully about these overlapping and conflicting messages before deciding what to do. +``` + +``` +And while these constraints were once needed to avoid worst case scenarios, we have since found we can delete many of them and let the model use surrounding context and judgement instead. +``` + +CLAUDE.md's role narrows because other surfaces now exist: + +``` +Additionally, Claude Code now has many more tools. Claude used to rely on CLAUDE.md as a source of memory, information, and guidance. Now we have memory, artifacts, and skills, which Claude can use to create new ways of loading and sharing context across sessions. +``` + +The load-bearing thesis the article never states, flagged in the fresh pass as a true gap: its +logic only coheres if constraining the *interface* (an enum, a one-line behavioral rule) is good +while constraining the *behavior space* (worked examples, categorical prohibitions) is bad. An +enum is a harder constraint than an example, so "less constraint" is not the actual principle. + +### 2.3 The six then to now reversals + +The section premise: + +``` +There were a number of previous context engineering best practices that had become myths. +``` + +The six pairs, as rendered in the strikethrough table figure: Give Claude Rules to Give Claude +Judgement; Give Claude Examples to Design Interfaces; Put it all upfront to Use Progressive +Disclosure; Repeat Yourself to Simple Tool Descriptions; Memory in Claude.MDs to Auto-memory; +Simple Specs to Rich References. + +**1. Rules to judgement.** Early Claude Code shipped strong, sometimes-wrong rules to avoid +worst cases such as deleting files. + +``` +When we first rolled out Claude Code, we needed to be sure that Claude avoided worst case scenarios, such as deleting files. This meant we would give particularly strong guidance that might not always be true, +``` + +``` +Still, without these guardrails for older models, the comments Claude wrote would be incorrect in many cases and we had to accept this tradeoff. But newer models have better judgement and can handle these decisions well without explicit rules. +``` + +The worked pair, both quoted from Claude Code's system prompt. Old: + +``` +In code: default to writing no comments. Never write multi-paragraph docstrings or multi-line comment blocks — one short line max. Don't create planning, decision, or analysis documents unless the user asks for them — work from conversation context, not intermediate files. +``` + +New: + +``` +Write code that reads like the surrounding code: match its comment density, naming, and idiom. +``` + +**2. Examples to interface design.** + +``` +The number one rule for tool usage was to give Claude examples on how to use them. With our newest models, we’ve found that giving examples actually constrains them to a certain exploration space. +``` + +``` +Instead of using examples, think more about the design of your tools, scripts and files- what parameters does Claude have and how can they be more expressive? +``` + +``` +For example, in the Todo tool example, just listing status as an enumeration between pending, in_progress, and completed, hints to Claude about how to use it. The instruction on keeping one item in_progress helps define our requested behavior. +``` + +**3. Upfront to progressive disclosure.** + +``` +Since then, Claude Code has gotten very competent at using progressive disclosure- loading the right context at the right times. For example, we moved verification and code review into their own skills that Claude Code could selectively call. +``` + +``` +But progressive disclosure is not just for skills, we also use it for tools. Some of our tools are ‘deferred loading,’ which means the agent must search for their full definitions using ToolSearch before using them. This allows us to have more tools (such as our Task tools) that don’t take up context until they’re needed. +``` + +``` +The same can be applied to your own CLAUDE.md and Skill.md files. A common myth is that you want to make these a central repository for every known practice that you *might* run into, because Claude would not find it otherwise. Instead, consider having a tree of files that can be loaded at the right time. +``` + +**4. Repetition to simple tool descriptions.** + +``` +Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start. This meant our system prompt would sometimes have references to tools in the main system prompt as well as instructions in the tool description. +``` + +``` +We found we could delete these repeat examples and put instructions on how to use tools in the tool descriptions rather than the system prompt. +``` + +Note what "simple tool descriptions" actually means in practice: consolidation into a single +richer description surface, not a smaller total. The after panel of the TodoWrite figure still +carries a behavioral rule. + +**5. CLAUDE.md memory to auto-memory.** + +``` +We used to encourage users to save things to Claude’s memory, by using the # hotkey to write to their CLAUDE.md automatically. Instead, Claude now automatically saves memories that are relevant to the work and to you. +``` + +The source says "Instead", which supersedes the encouragement rather than announcing a removal. +Upstream has since removed the hotkey outright; see section 5. + +**6. Simple specs to rich references.** + +``` +But we’ve found that Claude can handle increasingly more complicated references. Instead of simple markdown files, Claude can reference HTML artifacts created by our new artifacts feature. +``` + +``` +You may also give Claude references in the form of code. A spec may also be a detailed test suite, or a function in a different codebase that Claude might port. +``` + +``` +Rubrics are another form of references. Rubrics allow Claude to try and verify your taste in a particular field (e.g. what does a good API design look like) by using dynamic workflows and spinning up verifier agents with those rubrics. +``` + +### 2.4 Per-surface placement guidance + +The article's placement section walks four surfaces. In the source each heading runs into its +body across a line break, so the quoted lines below literally begin with two asterisks. + +**System prompt.** + +``` +**A system prompt is heavily tied to the product context. It tells Claude what product it’s operating in and what it’s doing. For Claude Code, you will likely never modify this, but if you are building your own agent harness, this is where you should spend a lot of time. +``` + +**CLAUDE.md.** + +``` +**Keep your CLAUDE.md lightweight and briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase. For example, you may organize your code to keep types in one monolithic file and nowhere else. Avoid stating ‘the obvious’ things Claude should know by looking at your file system or your repo. +``` + +``` +Use progressive disclosure for more details, for example if you have several unique instructions on how to verify your work, create a verification skill and reference it from your CLAUDE.md. +``` + +"Lightweight" and "spend most of the tokens" pull against each other on the page. The resolution +the reconciliation settled on is composition, not weight: derivable content out, tribal +knowledge in. + +**Skills.** + +``` +**Think of skills as lightweight guides to let Claude find information when needed. Avoid making them overconstrained, except in highly important areas. +``` + +``` +For long skills, try and use progressive disclosure as much as possible- divide it into many files and split them out. +``` + +``` +It’s best when skills encode particular opinions, knowledge, or best practices that are particular to you, your team, or product. +``` + +The carve-out, "except in highly important areas", is undefined in the source and is the +calibration knob everything downstream turns on: it is what separates a legacy worst-case +guardrail (delete) from a genuine team convention (keep). + +**References.** + +``` +**You can @ mention files to include them as references. References allow Claude to refer to in-depth information about the current plan. +``` + +``` +This might be in specs files, mockups, or even entire codebases. Generally you should prefer files that are in code as it provides clear, high-fidelity instructions to Claude in a language it knows very well. For example, a HTML mockup of a design will generally produce better results than a description of the design or a screenshot. +``` + +### 2.5 The close + +``` +Across your system prompt, skills, and CLAUDE.md files, you may need to simplify just like we did. +``` + +``` +We rolled out a new command called `claude doctor,` which will help you do this automatically as well. +``` + +``` +For more details on prompting more advanced models specifically, check out our Fable field guide. +``` + +The comma inside the backticks is in the source. The "Fable field guide" anchor links to T2. + +### 2.6 Unstated preconditions the article inherits + +Recorded because any guidance derived from P1 inherits them silently. + +- "Match the surrounding code" presumes a coherent surrounding idiom. In a legacy or multi-team + codebase the rule points at noise. +- "Design your interfaces" presumes you own the interfaces. A reader consuming third-party MCP + servers or vendored tools cannot discharge it. +- Code-form references (test-suite specs, HTML mockups) presume authoring capacity a reader may + not have, leaving prose as the only spec they can produce. +- "No measurable loss" presumes an eval harness. Most readers have none, so the article's own + evidence standard is unavailable to them, and it offers no failure signal and no rollback + criterion for a deletion that goes too far. +- Security is absent. Deleting guardrails routes decisions through "surrounding context and + judgement", which makes malicious surrounding context an input to judgment, and the one named + worst case (file deletion) is never re-secured. +- Shared-surface governance is absent. The article's "you" is singular, while CLAUDE.md and + skills are team artifacts, and conflicting teammate preferences are exactly the conflict class + the article opens with. + +## 3. P2: effective context engineering for AI agents + +### 3.1 Definitions + +``` +Context refers to the set of tokens included when sampling from a large-language model (LLM). +``` + +``` +The engineering problem at hand is optimizing the utility of those tokens against the inherent constraints of LLMs in order to consistently achieve a desired outcome. +``` + +``` +Effectively wrangling LLMs often requires thinking in context — in other words: considering the holistic state available to the LLM at any given time and what potential behaviors that state might yield. +``` + +The dual definition that separates the two disciplines: + +``` +Prompt engineering refers to methods for writing and organizing LLM instructions for optimal outcomes (see our docs for an overview and useful prompt engineering strategies). Context engineering refers to the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts. +``` + +The enumerated context state, usable as a coverage checklist: + +``` +However, as we move towards engineering more capable agents that operate over multiple turns of inference and longer time horizons, we need strategies for managing the entire context state (system instructions, tools, Model Context Protocol (MCP), external data, message history, etc). +``` + +Why it is a loop rather than an authoring task: + +``` +An agent running in a loop generates more and more data that could be relevant for the next turn of inference, and this information must be cyclically refined. Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information. +``` + +``` +In contrast to the discrete task of writing a prompt, context engineering is iterative and the curation phase happens each time we decide what to pass to the model. +``` + +The coinage of the term is credited to Karpathy and the agent definition to Simon Willison, but +both credits exist only as link anchors rather than named attributions in the text (CF-3). + +### 3.2 The attention budget and context rot: the mechanism + +This is the post's load-bearing model and the reason every other technique in it exists. + +``` +Studies on needle-in-a-haystack style benchmarking have uncovered the concept of context rot: as the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases. +``` + +``` +While some models exhibit more gentle degradation than others, this characteristic emerges across all models. +``` + +``` +Context, therefore, must be treated as a finite resource with diminishing marginal returns. +``` + +``` +Like humans, who have limited working memory capacity, LLMs have an “attention budget” that they draw on when parsing large volumes of context. Every new token introduced depletes this budget by some amount, increasing the need to carefully curate the tokens available to the LLM. +``` + +The stated causes, in the post's order: architecture, training distribution, and the cost of the +usual mitigation. + +``` +This attention scarcity stems from architectural constraints of LLMs. LLMs are based on the transformer architecture, which enables every token to attend to every other token across the entire context. This results in n² pairwise relationships for n tokens. +``` + +``` +Additionally, models develop their attention patterns from training data distributions where shorter sequences are typically more common than longer ones. This means models have less experience with, and fewer specialized parameters for, context-wide dependencies. +``` + +``` +Techniques like position encoding interpolation allow models to handle longer sequences by adapting them to the originally trained smaller context, though with some degradation in token position understanding. +``` + +The shape of the resulting failure, which is the operationally important part: + +``` +These factors create a performance gradient rather than a hard cliff: models remain highly capable at longer contexts but may show reduced precision for information retrieval and long-range reasoning compared to their performance on shorter contexts. +``` + +A gradient means every token costs some recall precision well below the window limit, and the +failure is silent: missed instructions and forgotten constraints, not an overflow error. + +One dissent is recorded from the fresh pass and left unresolved: the n² paragraph describes the +mechanism by which every pair *is* attended, so the causal work in this chain is actually done +by the softer training-distribution and position-encoding claims, and dense attention is +assumed rather than argued. Cite the section as a mechanism sketch, not as a proof. + +### 3.3 The guiding principle and altitude calibration + +``` +Given that LLMs are constrained by a finite attention budget, good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome. +``` + +Restated in the conclusion as the portable takeaway: + +``` +the guiding principle remains the same: find the smallest set of high-signal tokens that maximize the likelihood of your desired outcome. +``` + +Altitude, defined by its two failure modes, is how that principle gets applied to a system +prompt: + +``` +System prompts should be extremely clear and use simple, direct language that presents ideas at the right altitude for the agent. The right altitude is the Goldilocks zone between two common failure modes. At one extreme, we see engineers hardcoding complex, brittle logic in their prompts to elicit exact agentic behavior. This approach creates fragility and increases maintenance complexity over time. At the other extreme, engineers sometimes provide vague, high-level guidance that fails to give the LLM concrete signals for desired outputs or falsely assumes shared context. The optimal altitude strikes a balance: specific enough to guide behavior effectively, yet flexible enough to provide the model with strong heuristics to guide behavior. +``` + +An altitude review therefore cuts in both directions: brittle if-else branching is a defect, and +so is mechanism-free vagueness. + +Structure advice, with the hedge that must travel with it: + +``` +We recommend organizing prompts into distinct sections (like , , ## Tool guidance, ## Output description, etc) and using techniques like XML tagging or Markdown headers to delineate these sections, although the exact formatting of prompts is likely becoming less important as models become more capable. +``` + +``` +Regardless of how you decide to structure your system prompt, you should be striving for the minimal set of information that fully outlines your expected behavior. (Note that minimal does not necessarily mean short; you still need to give the agent sufficient information up front to ensure it adheres to the desired behavior.) +``` + +``` +It’s best to start by testing a minimal prompt with the best model available to see how it performs on your task, and then add clear instructions and examples to improve performance based on failure modes found during initial testing. +``` + +Minimal is a signal-density target, not a length cap, and the iteration loop is minimal first, +then additions driven by observed failures rather than by speculation. + +### 3.4 Tools and examples + +``` +Because tools define the contract between agents and their information/action space, it’s extremely important that tools promote efficiency, both by returning information that is token efficient and by encouraging efficient agent behaviors. +``` + +``` +One of the most common failure modes we see is bloated tool sets that cover too much functionality or lead to ambiguous decision points about which tool to use. If a human engineer can’t definitively say which tool should be used in a given situation, an AI agent can’t be expected to do better. +``` + +The few-shot paragraph, which is the exact claim P1 later reverses for Claude 5 models: + +``` +Providing examples, otherwise known as few-shot prompting, is a well known best practice that we continue to strongly advise. However, teams will often stuff a laundry list of edge cases into a prompt in an attempt to articulate every possible rule the LLM should follow for a particular task. We do not recommend this. Instead, we recommend working to curate a set of diverse, canonical examples that effectively portray the expected behavior of the agent. For an LLM, examples are the “pictures” worth a thousand words. +``` + +Closing guidance across all components: + +``` +Our overall guidance across the different components of context (system prompts, tools, examples, message history, etc) is to be thoughtful and keep your context informative, yet tight. +``` + +### 3.5 Just-in-time retrieval and metadata as signal + +``` +Since we wrote that post, we’ve gravitated towards a simple definition for agents: LLMs autonomously using tools in a loop. +``` + +``` +Rather than pre-processing all relevant data up front, agents built with the “just in time” approach maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime using tools. Anthropic’s agentic coding solution Claude Code uses this approach to perform complex data analysis over large databases. The model can write targeted queries, store results, and leverage Bash commands like head and tail to analyze large volumes of data without ever loading the full data objects into context. This approach mirrors human cognition: we generally don’t memorize entire corpuses of information, but rather introduce external organization and indexing systems like file systems, inboxes, and bookmarks to retrieve relevant information on demand. +``` + +The post says teams are *augmenting* embedding-based retrieval with just-in-time strategies, not +replacing it. Both the slice summary and its index shaded that into replacement and the +reconciliation flagged it, so do not cite this post as taking an anti-embeddings position. + +Metadata is itself a behavioral signal, which makes tree hygiene a form of context engineering: + +``` +Beyond storage efficiency, the metadata of these references provides a mechanism to efficiently refine behavior, whether explicitly provided or intuitive. To an agent operating in a file system, the presence of a file named test_utils.py in a tests folder implies a different purpose than a file with the same name located in src/core_logic/ Folder hierarchies, naming conventions, and timestamps all provide important signals that help both humans and agents understand how and when to utilize information. +``` + +``` +Letting agents navigate and retrieve data autonomously also enables progressive disclosure—in other words, allows agents to incrementally discover relevant context through exploration. Each interaction yields context that informs the next decision: file sizes suggest complexity; naming conventions hint at purpose; timestamps can be a proxy for relevance. Agents can assemble understanding layer by layer, maintaining only what's necessary in working memory and leveraging note-taking strategies for additional persistence. +``` + +The trade-off is stated plainly, and so is the hybrid default: + +``` +Of course, there's a trade-off: runtime exploration is slower than retrieving pre-computed data. Not only that, but opinionated and thoughtful engineering is required to ensure that an LLM has the right tools and heuristics for effectively navigating its information landscape. Without proper guidance, an agent can waste context by misusing tools, chasing dead-ends, or failing to identify key information. +``` + +``` +In certain settings, the most effective agents might employ a hybrid strategy, retrieving some data up front for speed, and pursuing further autonomous exploration at its discretion. The decision boundary for the ‘right’ level of autonomy depends on the task. Claude Code is an agent that employs this hybrid model: CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees. +``` + +``` +Given the rapid pace of progress in the field, "do the simplest thing that works" will likely remain our best advice for teams building agents on top of Claude. +``` + +### 3.6 The long-horizon triad + +``` +Long-horizon tasks require agents to maintain coherence, context, and goal-directed behavior over sequences of actions where the token count exceeds the LLM’s context window. +``` + +``` +To enable agents to work effectively across extended time horizons, we've developed a few techniques that address these context pollution constraints directly: compaction, structured note-taking, and multi-agent architectures. +``` + +Waiting for bigger windows is explicitly not the answer: the post argues that windows of all +sizes stay subject to context pollution and relevance concerns wherever the strongest +performance is wanted. Note that "context pollution" is used as a term of art and never defined. + +**Compaction.** + +``` +Compaction is the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary. Compaction typically serves as the first lever in context engineering to drive better long-term coherence. +``` + +``` +In Claude Code, for example, we implement this by passing the message history to the model to summarize and compress the most critical details. The model preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs or messages. The agent can then continue with this compressed context plus the five most recently accessed files. Users get continuity without worrying about context window limitations. +``` + +``` +For engineers implementing compaction systems, we recommend carefully tuning your prompt on complex agent traces. Start by maximizing recall to ensure your compaction prompt captures every relevant piece of information from the trace, then iterate to improve precision by eliminating superfluous content. +``` + +``` +An example of low-hanging superfluous content is clearing tool calls and results – once a tool has been called deep in the message history, why would the agent need to see the raw result again? One of the safest lightest touch forms of compaction is tool result clearing, most recently launched as a feature on the Claude Developer Platform. +``` + +The post's own caveat sits right next to its confidence, and any compaction guidance that drops +it inherits the technique without its central risk. Paraphrased from the source's two adjacent +sentences: compaction is described as distilling the window in a high-fidelity manner with +minimal performance degradation, and then, immediately after, as an art of selecting what to +keep versus what to discard, because overly aggressive compaction loses subtle but critical +context whose importance only becomes apparent later. + +**Structured note-taking.** + +``` +Structured note-taking, or agentic memory, is a technique where the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times. +This strategy provides persistent memory with minimal overhead. Like Claude Code creating a to-do list, or your custom agent maintaining a NOTES.md file, this simple pattern allows the agent to track progress across complex tasks, maintaining critical context and dependencies that would otherwise be lost across dozens of tool calls. +``` + +The evidence is Claude playing Pokémon, which matters because it is a non-coding domain and +because the structure was not imposed: + +``` +Claude playing Pokémon demonstrates how memory transforms agent capabilities in non-coding domains. The agent maintains precise tallies across thousands of game steps—tracking objectives like "for the last 1,234 steps I've been training my Pokémon in Route 1, Pikachu has gained 8 levels toward the target of 10." Without any prompting about memory structure, it develops maps of explored regions, remembers which key achievements it has unlocked, and maintains strategic notes of combat strategies that help it learn which attacks work best against different opponents. +``` + +The design test that follows is reset survival: the value appears when the agent reads its own +notes after a context reset and continues. + +``` +As part of our Sonnet 4.5 launch, we released a memory tool in public beta on the Claude Developer Platform that makes it easier to store and consult information outside the context window through a file-based system. This allows agents to build up knowledge bases over time, maintain project state across sessions, and reference previous work without keeping everything in context. +``` + +That sentence is time-scoped to late September 2025. Section 5 records the current status. + +**Sub-agents.** + +``` +Sub-agent architectures provide another way around context limitations. Rather than one agent attempting to maintain state across an entire project, specialized sub-agents can handle focused tasks with clean context windows. The main agent coordinates with a high-level plan while subagents perform deep technical work or use tools to find relevant information. Each subagent might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens). +``` + +``` +This approach achieves a clear separation of concerns—the detailed search context remains isolated within sub-agents, while the lead agent focuses on synthesizing and analyzing the results. This pattern, discussed in How we built our multi-agent research system, showed a substantial improvement over single-agent systems on complex research tasks. +``` + +The 1,000 to 2,000 token condensed return is the concrete fan-out contract in this corpus: wide +exploration allowed, narrow return required. What the post does not carry is the cost side. T6 +supplies it (roughly 4x chat token usage per agent, roughly 15x for multi-agent systems, with +token usage explaining most of the eval variance), and T3's megatoken cost figures corroborate. +Cite T6 whenever the fan-out contract is adopted. + +**Choosing between the three.** + +``` +The choice between these approaches depends on task characteristics. For example: +- Compaction maintains conversational flow for tasks requiring extensive back-and-forth; +- Note-taking excels for iterative development with clear milestones; +- Multi-agent architectures handle complex research and analysis where parallel exploration pays dividends. +``` + +### 3.7 Self-obsolescence + +The post hedges its own durability, and P1 is the operationalization of that hedge. + +``` +The techniques we've outlined will continue evolving as models improve. +``` + +``` +We're already seeing that smarter models require less prescriptive engineering, allowing agents to operate with more autonomy. +``` + +``` +But even as capabilities scale, treating context as a precious, finite resource will remain central to building reliable, effective agents. +``` + +The durable layer is therefore the principle (smallest set of high-signal tokens, context as a +finite resource); the technique layer is expected to relax, and did, one generation later. + +### 3.8 The generational reversal, stated plainly + +P2 says few-shot examples remain strongly advised, with a caution against edge-case laundry +lists. P1, ten months later and for the Claude 5 generation, says examples constrain the model +to a certain exploration space and should be replaced by interface design. Same publisher +lineage, deliberate reversal, and the clearest instance of the corpus's own advice-shelf-life +theme. Where the two conflict for Claude 5-generation targets, P1 governs on technique and P2 +remains the principles layer. Note also that the reversal is not total: P2's own objection is to +edge-case enumeration, which is closer to P1's position than the headline pairing suggests. + +## 4. Figure-borne facts + +These facts exist in no text anywhere: not in the markdown channel, not in the HTML, not in the +alt text. They are readable only by opening the figures. A text-only ingestion of this corpus +loses them, which is why CF-2 treats figure-only evidence as systemic rather than incidental. + +**The TodoWrite before-size (P1, figure 3).** The before panel labels the old TodoWrite tool +description as approximately 9,100 characters of when-to-use lists and worked examples. The +after panel shows a short description beginning "Create and update a task list for the current +session…", a status field rendered as the three enum values pending / in_progress / completed, +and the callout "only one task in_progress at a time". This is the article's only quantification +of what interface design replaces, and the body text carries no number at all. Always attribute +it to the figure. + +**The six-layer context stack (P1, figure 4).** Six stacked boxes, top to bottom: "Your prompt" +(highlighted), "References" (subtitled "@-mentioned files, specs, mockups, codebases, +artifacts"), "System prompt", "Claude.MDs", "Skills", "Memory". Two things follow. First, Memory +appears as a sixth surface that the body's per-surface section never covers, so the stack is a +wider taxonomy than the prose. Second, the figure visually subordinates the system prompt, which +the body calls the highest-investment surface for harness builders, a tension worth knowing +before reproducing the diagram. + +**The prompt-versus-context curation loop (P2, figure 1).** The left panel shows prompt +engineering for single-turn queries: a context window holding a system prompt and a user +message, run through the model to produce one assistant message. The right panel shows context +engineering for agents: a larger pool of possible context (docs, tools, a memory file, +comprehensive instructions, domain knowledge, message history) passes through an explicit +"Curation" step into the context window each turn, the model emits an assistant message or a +tool call, and the tool call's result flows back into the message history pool for the next +cycle. The figure is the visual counterpart of the cyclical-refinement and per-turn-curation +claims. The fresh pass read it as additionally asserting lossiness, since the candidate pool +holds visibly more items than the window, meaning most candidate context should not enter. + +**The three calibration prompts (P2, figure 2, "Calibrating the system prompt").** A horizontal +gradient bar, red at both ends and green in the centre, labeled "Too specific", "Just right", +"Too vague", with three prompt cards beneath. These are the post's only complete worked system +prompts, its centerpiece pedagogy, and they exist purely as pixels: the HTML carries only alt +text and a CDN URL. + +- *Too specific* is a numbered procedure for a bakery support agent: classify intent into a + five-value enum, branch per value, then an "exhaustive list" of escalation cases that trails + off into an ellipsis, a "5/7 of the following requirements" fraction over an elided list, a + numbered step with no text, and nested ellipses at three levels. The card argues by absurdity + that enumeration cannot close. +- *Just right* opens by naming the role and scope ("customer support agent for Claude's + Bakery"), states the tools available and the goal, then gives a four-step response framework + (identify the core issue, gather necessary context with tools, provide clear resolution, + confirm satisfaction) and six guidelines: prefer the simplest solution that fully addresses + the issue, check order status before suggesting next steps, call the human_assistance tool + when uncertain, call it for legal issues, health and allergy emergencies, and out-of-policy + financial adjustments, and acknowledge frustration or urgency with appropriate empathy. It is + the concrete image of "specific enough to guide behavior, flexible enough to leave + heuristics". +- *Too vague* is a single sentence: solve customer issues "in a manner consistent with the + principles and essence of the company brand", escalate to a human if needed. It is the + falsely-assumed-shared-context failure mode made concrete, and its own comma splice reads as + part of the caricature. + +A full verbatim transcription of all three cards was produced in-session by the fresh pass, +independently re-verified against the image by a second reader, with one systematic +normalization (the figure renders curly quotation marks, the transcription uses straight ones), +and merged into the P2 digest slice. That slice lived in the memory tier and is gone. Anyone +needing the exact wording again must re-transcribe from the figure on the live P2 page; the +summaries above are what survives here. + +## 5. Settled upstream facts + +Verified against current first-party documentation on 2026-09-01. Recency anchor for all of +them: Claude Code changelog v2.1.252, fetched that day. + +**The `#` memory hotkey is removed, not merely de-emphasized.** Changelog v2.0.70: "Removed # +shortcut for quick memory entry (tell Claude to edit your CLAUDE.md instead)". It was added in +v0.2.54 and appears in no current memory, commands, or interactive-mode page. P1's "Instead" +posture, written when the mechanism still existed, aged into outright removal within weeks, +which makes it the fastest-aging claim in the corpus. Auto memory is documented and on by +default. + +**`/doctor`'s documented behavior, per `commands.md`.** The in-session `/doctor` (alias +`/checkup`) is a full setup checkup that diagnoses issues and can fix them: installation health, +duplicate or leftover installs, PATH problems, settings files it cannot parse, unused skills, MCP servers +and plugins measured against their context cost, slow hooks, and a release-channel version +check. On CLAUDE.md it deduplicates local files against checked-in ones, trims checked-in files +by cutting content Claude could derive from the codebase, and migrates the always-loaded +remainder into skills and nested CLAUDE.md files that load on demand. It reports findings first +and asks for confirmation before changing anything. From the terminal, `claude doctor` prints +read-only installation diagnostics without starting a session. The full checkup arrived in +v2.1.205 and the CLAUDE.md trim check requires v2.1.206 or later. Two scope corrections to P1's +pointer: no official doc uses the word "rightsize", and no doc describes `/doctor` rewriting or +simplifying skill *content*. The skills check is unused-skill detection by context cost, so +"rightsize your skills" overstates what the documented checks do. + +**Memory tool and context editing, and what Claude Code exposes.** The memory tool +(`memory_20250818`, client-side) requires no beta header, and current docs state it is available +on all Claude 4 and later models. Context editing is still beta (header +`context-management-2025-06-27`), with strategies `clear_tool_uses_20250919` and +`clear_thinking_20251015`, documented as available on all supported Claude models. Claude Code +the harness exposes neither API mechanism natively: its documented analogues are its own +file-based auto memory and its own client-side compaction (`/compact`, auto-compact). No +code.claude.com page documents `memory_20250818` or `context_management` in the harness. When +citing supported models, use the platform.claude.com memory-tool page and the code.claude.com +context-window page; both of T8's contradictory model lists are stale (CF-5). + +**The 80% figure has no official carrier.** No Anthropic docs surface, engineering blog page, +claude.com blog page, or Claude Code changelog entry carries "removed over 80% of Claude Code's +system prompt with no measurable loss". It is vendor-staff voice on a personal channel, plus the +claude.com blog twin of the same article, which carries the figure in its page description. +Simon Willison independently relays the same figure from a Claude Code team conversation, and +all press coverage traces back to the 2026-07-24 article. Directional, non-numeric official +corroboration does exist: changelog v2.1.154, "The lean system prompt is now the default for all +models except Haiku, Sonnet, and Opus 4.7 and earlier". Annotate the claim as OPINION-tier, cite +both carriers, and pair it with v2.1.154 rather than presenting it as a measured result. + +## 6. Custody findings (CF-1 to CF-7) + +The corpus's findings about its own sources. These are properties of the source material, so +they stay true regardless of what this repo decides to build. + +**CF-1. Silent post-publication revision.** T5's live page is revised after publication with no +visible changelog: it now names Haiku 4.5, Sonnet 4.5, and the Agent SDK, none of which existed +at its nominal December 2024 date. Any citation of it must be snapshot-dated, never "the Dec +2024 post". + +**CF-2. Figure-only evidence is systemic, not incidental.** Across the corpus the quantitative +core lives in images: P1's approximately 9,100-character TodoWrite before-size and its six-layer +context stack; P2's three worked bakery prompts and its curation feedback loop; T3's entire +workflow API surface (`agent(prompt, {schema, model, isolation, agentType})`, `parallel`, +`pipeline`) plus its megatoken cost data; T4's held-out accuracy numbers (67.4 to 80.1 percent, +79.6 to 85.7 percent) and its error-design pattern; T5's visual grammar distinguishing +programmatic from LLM aggregation. A text-only ingestion of this corpus loses all of it. + +**CF-3. Provenance stripping.** P2 credits the coinage of "context engineering" to Karpathy and +its agent definition to Simon Willison only through link anchors. Text extraction drops the +anchors, which silently re-attributes both to Anthropic's voice. Any quotation pipeline reading +the markdown channel alone will misattribute. + +**CF-4. Link defects cluster.** The literal text "CLAUDE.md" autolinks to a dead +`http://claude.md` in both primaries, apparently the same CMS behavior. P2 additionally carries +a self-link mislabeled as the Sonnet 4.5 announcement, and scheme-inconsistent http links. None +were repaired in the snapshots; they are recorded as source artifacts. + +**CF-5. Cookbook and announcement defects.** T8 carries two contradictory supported-model lists +(five models in one place, two in another) and a log line inconsistent with its own stated +clearing threshold, so it is struck as a model-list authority. T7's evaluation claims (39 +percent, 29 percent, 84 percent) name no eval and decompose nothing, and its "beyond any fixed +limit" framing oversells the mechanism: the limit is untouched, the workable length is extended. + +**CF-6. The generational reversal is now in official reference docs.** T9's second page +documents stripping self-check guidance entirely for Opus 5, alongside four model-specific +delegate pages. The corpus's advice-shelf-life thesis is therefore not just a blog-versus-blog +phenomenon; it is instantiated in first-party reference documentation, which means "the docs +say" is itself model-generation-scoped. + +**CF-7. Venue error in the prior plan.** Recorded for the pattern, not for repair: the document +described here was pruned from main with its slice in the topic-docs v3.0.0 adoption, so the +error is no longer live anywhere. That plan's `design/official-corroboration.md` opened by calling P1 +"a post on Anthropic's Claude blog". It is a personal X Article by an Anthropic employee. The +consequence is bounded, since that file's per-claim verdicts already treat unconfirmed claims as +OPINION-tier, but the wording inflates the venue's authority. The finding was flagged for that +plan's owner and deliberately not edited from the corpus run. + +## 7. Coverage accounting and residue + +### How the corpus was verified + +Both primaries went through the same pipeline: an immutable snapshot of the original, a +heading-and-figure inventory, one digest agent per unit, then verification. Every quoted claim +sits in a column-0 fence asserting source-verbatim bytes, and two scripted gates +(`check-fences-exact.py`, `check-snippets.py`) check every fence against the snapshot. Both +slices passed those gates, replayed on 2026-08-31. Claim counts: P1 carries 37 verbatim claim +rows across five digests (5 + 5 + 15 + 9 + 3), P2 carries 59 across six (5 + 6 + 10 + 11 + 20 + +7). + +Each slice ran two verification arms with the production rationale withheld, judging on-disk +state only, with every audited file's sha256 recorded and matched against a pinned manifest. Arm +B was meant to be cross-vendor; no cross-vendor verifier was installed in the session, so it +degraded to a second same-vendor adversarial refuter, and the degradation is recorded in the +verdict header rather than passed off as independence. Treat the agreement of the two arms as +weaker evidence than a true cross-vendor pass. Arm findings drove correction rounds: P2's round +two discharged all seven distinct MAJOR findings from both arms and was re-verified clean; P1's +corrections were re-verified against a re-pinned manifest, including an independent replay of +the link-anchor derivation from the article's entityMap, which produced exactly the six anchor +and URL pairs the inventory claimed, with none missing and none invented. + +A separate fresh unbiased pass was then ordered, with ten workers barred from reading any +earlier layer, sweeping every source at paragraph grain with a four-lens critical apparatus +(concepts, assumptions, omissions, tensions). Reconciliation then adjudicated every fresh row +against the verified slices and the prior repo plans, one verdict per row. + +### The verdict counts + +P1: 130 fresh rows (90 concept, 13 assumption, 15 omission, 12 tension) yielded 83 +covered-by-both-layers, 17 covered-by-digest, 8 covered-by-plan, 10 thin, and 12 gaps. All 90 +concept rows were covered at least thinly, and every gap is a fresh-angle critique rather than +missing article content. Two conflicts surfaced: the venue error (CF-7, flagged) and an index +gloss calling the `#` hotkey "deprecated" where the source says only "Instead" (fixed). No +disagreement was found on any quoted byte, figure content, number, link target, or section +boundary. + +P2: 171 fresh rows (129 concept, 14 assumption, 16 omission, 12 tension) yielded 117 covered, 21 +thin, and 33 gaps, of which only three are content gaps: the bakery-prompt transcriptions and +the compaction caveat. The rest are critical apparatus, 18 of 26 critique rows, which is the +honest characterization of the slice layer: faithful on what the post says, largely silent on +its tensions and unstated preconditions. Three conflicts: the recoverable publication date +(fixed), the augment-versus-replace shading on just-in-time retrieval (recorded in 3.5), and one +interpretive disagreement about whether the why-it-matters section establishes a mechanism or +assumes one (left open, recorded in 3.2). The reconciliation also established by bounded grep +that the prior `context-engineering-claude-5` plan never cited, corroborated +against, or even linked P2: zero hits for the URL, "context rot", "attention budget", +"karpathy", or any anthropic.com URL. P2 was net-new to this repo's knowledge work. + +Nine tier-2 pages were swept at paragraph grain by the fresh pass, producing between 16 and 76 +concept rows each. Where their earlier structural notes and the fresh sweep disagree, the fresh +sweep governs on structure and content, with two exceptions retained as complements: the +structural notes' relevance-hook sections, which have no fresh equivalent, and T8's byline. + +### What "fully accounted for" means here + +Every paragraph of both primaries is covered, or explicitly classified thin or gap, with +row-level citations. Every figure in the corpus was viewed by a worker or the orchestrator, +described, and, for both primaries and T7, archived or resolution-logged. Every inline link in +both primaries was extracted from the entityMap or DOM with anchors verified. Every linked +first-party page was deep-swept with the four-lens apparatus. Prior repo work was engaged and +diffed in both directions, with conflicts adjudicated: two fixed, three flagged with named +owners. + +### The residue, stated honestly + +- **Referenced-external sources are cataloged, not digested.** None were fetched or verified, so + claims resting on them rest on the primaries' characterization of them: the Karpathy coinage + tweet (), the context-rot benchmark study + (), the human working-memory paper + (), the transformer paper + (), the position-interpolation paper + (), an attention explainer + (), Willison's + tools-in-a-loop definition (), the Pokémon + stream (), and the MCP intro + (). The context-rot study is the + one that matters most: it is the empirical basis for the mechanism in 3.2, and this corpus + never checked it. +- T3's ten use-case subsections were absorbed at row grain, but any videos or GIFs they contain + were not played. +- Reply threads and social context around the X article were never in scope. +- P2's three calibration prompts are carried verbatim in appendix A below, transcribed from the + figure image and independently re-verified against it by a second reader. +- Verification independence is degraded, not absent: both arms on both slices were same-vendor. + +## Appendix A. The three calibration prompts, verbatim + +P2's centerpiece pedagogy exists only as pixels in its figure 2 ("Calibrating the system +prompt"): the page's HTML carries the alt text and a CDN URL, nothing more. The transcription +below was taken from the figure image and then independently re-verified against that image by +a second reader, who confirmed it faithful with one systematic normalization: the figure renders +curly quotation marks, the transcription uses straight quotes. Layout, left to right: a +gradient bar of discrete cells, red at both ends and green in the centre, labeled "Too +specific", "Just right", "Too vague", with a marker dropping from each label to its card. + +These are figure pixels rather than page bytes, so no byte-exactness gate covers them and none +should be claimed for them. + +### Card 1, "Too specific" + +```text +You are a helpful assistant for Claude's Bakery. +You must respond to the name Claude. +For every user request you MUST FOLLOW THESE STEPS: + +1. Identify the user intent as one of the following: ["incident_resolution", "general_inquiry", "order_resubmission", "account_maintenance", "requires_escalation"] +2. + - if user intent is "incident_resolution", ask 3 followup questions to gather information, then always call the resolve tool + - if user intent is "general_inquiry", do not ask followup questions and answer in one shot + - if user intent ... + - ... +3. Here is an exhaustive list of cases that should be tagged as "requires_escalation": + - If the intent is incident_resolution but the user is in a different country + - If the user left a physical belonging in the store + - ... +4. Once you've ruled out escalation scenarios you should consider all the tools at your disposal. +5. If the user_request contains an order_id you should tag the user intent as "order_resubmission", unless the user meets 5/7 of the following requirements: + - User is asking for time update + - User is asking for location update + - ... +6. If the user wants to request a new order, but they already have another order in flight, you should follow these 5 steps of the resolution procedure: + - (1) Call check_order tool to see where the current order is + - ... +... +``` + +The caricature is structural: an "exhaustive" list that trails into an ellipsis, a fraction +over an elided set, a numbered step with no text, ellipses nested three deep. Enumeration +cannot close, which is the card's argument. + +### Card 2, "Just right" + +```text +You are a customer support agent for Claude's Bakery. +You specialize in assisting customers with their orders and basic questions about the bakery. Use the tools available to you to resolve the issue efficiently and professionally. + +You have access to order management systems, product catalogs, and store policies. Your goal is to resolve issues quickly when possible. Start by understanding the complete situation before proposing solutions, ask follow-up questions if you do not understand. + +Response Framework: +1. Identify the core issue - Look beyond surface complaints to understand what the customer actually needs +2. Gather necessary context - Use available tools to verify order details, check inventory, or review policies before responding +3. Provide clear resolution - Offer concrete next steps with realistic timelines +4. Confirm satisfaction - Ensure the customer understands the resolution and knows how to follow up if needed + +Guidelines: +- When multiple solutions exist, choose the simplest one that fully addresses the issue +- If a user mentions an order, check its status before suggesting next steps +- When uncertain, call the human_assistance tool +- For legal issues, health/allergy emergencies, or situations requiring financial adjustments beyond standard policies, call the human_assistance tool +- Acknowledge frustration or urgency in the user's tone and respond with appropriate empathy +``` + +Note what the middle card does not do: it names no intent enum, no branch table, and no +exhaustive case list. It states a role, the tools, a goal, an ordered framework, and the +escalation conditions, then leaves the judgment to the model. + +### Card 3, "Too vague" + +```text +You are a bakery assistant, you should attempt to solve customers issues in a manner consistent with the principles and essence of the company brand. Escalate to a human if needed. +``` + +The card's own comma splice and "customers issues" read as part of the caricature. "Principles +and essence of the company brand" is the falsely-assumed shared context the post warns about, +made concrete. diff --git a/docs/specs/context-engineering-critical-apparatus.md b/docs/specs/context-engineering-critical-apparatus.md new file mode 100644 index 000000000..9bad509fc --- /dev/null +++ b/docs/specs/context-engineering-critical-apparatus.md @@ -0,0 +1,343 @@ +# Critical apparatus: what the context-engineering corpus assumes, omits, and contradicts + +## 1. What this layer is + +The corpus's two primary sources (P1, the trq212 X article "The new rules of context +engineering for Claude 5 models", 2026-07-24; P2, the Anthropic engineering post "Effective +context engineering for AI agents", 2025-09-29) plus nine linked pages were absorbed twice. +The first pass produced faithful digests: what each source says. This layer is the second +pass, and it records what each source rests on, leaves out, and says against itself. It was +produced by a fresh unbiased paragraph-grain sweep whose workers were barred from reading the +existing digests, four lenses per source (concept inventory, implicit assumptions, omissions +and glosses, internal tensions), followed by bidirectional reconciliation: every fresh row +adjudicated against the prior digests and the then in-flight +`context-engineering-claude-5` plan (COVERED / THIN / GAP), plus a reverse check +for prior-layer content the fresh pass missed. The sweep and its reconciliations lived in the +session memory tier, which is gitignored and gone; the row IDs below (`P1-A01`, `P2-T08`, +`T6-T05`) are provenance labels carried into this doc, not links to anything readable. The +sweep produced 318 apparatus rows across ten sources; 142 are carried here, merged into the +entries below, and 176 were dropped as trivia or as concerns another repo doc already owns. + +Reconciliation's headline finding: the digests were faithful on what the sources say (115 of +129 P2 concept rows covered; all 90 P1 concept rows covered at least thinly) and nearly silent +on the critical apparatus (8 of 12 P2 tension rows and 10 of 14 P2 assumption rows were +outright GAPs). An interview or a repo check run only from the digests would under-challenge +the sources. That asymmetry is why this file exists. + +Use it as follows: before turning any corpus claim into repo doctrine, find the claim's row +here and discharge the guardrails in section 5. + +## 2. Per source + +### 2.1 P1, "The new rules of context engineering for Claude 5 models" (X article) + +Carried: 36 of 40 apparatus rows, as 19 entries. Dropped: the cover-art description, the +give-versus-let wording variance (`P1-T02`), the missing before/after CLAUDE.md artifact +(`P1-O08`, already answered by that plan's `design/official-corroboration.md` include and +exclude table), the abandoned rubrics thread (`P1-O14`), and img4's visual subordination of +the system prompt (`P1-T09`). + +| Rows | Claim | Why it matters for adoption | +|---|---|---| +| A01, A03, O02, O12 | The 80% deletion was validated on Anthropic's internal coding evals, and the reader is told to simplify the same way with no eval harness of their own and no recipe for building one. | "No measurable loss" is unfalsifiable for a reader who cannot measure; any repo check derived from the article inherits an evidence standard the repo has to supply itself. | +| A02, O03 | Every "Now" column is gated to the frontier Claude 5 generation, and the article concedes the old rules were correct for older models, yet offers no migration or version-gating technique for one CLAUDE.md serving a mixed fleet. | Guidance adopted repo-wide reaches whatever model a contributor is running; a generation-gated claim adopted ungated is actively wrong for part of the fleet. | +| A04, O07, O15 | Progressive disclosure assumes the retrieval step reliably fires, counts only the saved upfront tokens, and never states a window budget; a skill or file that is never loaded is an invisible failure. | This repo's own conventions load on demand, and that trigger does not fire inside subagents or after a compaction. Deferral moves cost, it does not delete it. | +| A05, O10 | Product surfaces are named by their state on one day, and `claude doctor` is twice recommended as a trimmer of instruction files with no description of its criteria or failure modes. | The corpus's fastest-aging claim (the `#` memory hotkey) went from "de-emphasized" in the article to removed in Claude Code v2.0.70 within weeks. Cite the changelog, not the article, for any product behavior. | +| A06 | "Match the surrounding code" presumes the surrounding code is coherent enough to be a signal. | In a legacy or multi-team tree the new rule points at noise where the old absolute rule at least gave a deterministic answer. A check that recommends replacing an absolute with a judgment call needs this precondition. | +| A08, O13 | Conflicting instructions are resolved by deleting one, never by establishing precedence between layers, and the article's own figure shows a stack whose ordering is never given a meaning. | Deletion and precedence are different remedies with different blast radii. The official layering rules (CLAUDE.md additive, skills and subagents and MCP override by name, hooks merge) answer this and the article does not. | +| A09, O06, T11 | Auto-memory is celebrated as a new always-on, model-curated context source in the same piece that diagnoses ambient context conflict, with no accuracy, staleness, inspection, or pruning story, and no hygiene entry in the closing playbook. | The article's own diagnosis applies to its own recommendation. Anything adopting auto-memory needs the audit surface the article omits. | +| A10 | Rich references ("a spec may be a detailed test suite", an HTML mockup over a description) presume the author has the skill and time to produce code-form artifacts. | For many contributors prose is the only spec they can write. A check demanding code-form references is undischargeable for them. | +| A12 | Clean layer separation is assumed: users do not touch the system prompt, builders own it. | The middle audience is real here: output styles, `--append-system-prompt`, agent definitions, and plugins all reach the system prompt. Neither column of the article's advice is addressed to them. | +| A13 | The article diagnoses exactly one failure direction, over-constraint, and never addresses under-specification. | Read as license, "delete, simplify, trust judgment" produces the vagueness failure. The replacement for a deleted rule should still encode its intent. | +| O01 | No detection story for judgment failure: no monitoring signal, no transcript-review recipe packaged as reader advice, no rollback criterion for reinstating a deleted rule. | This is the missing half of the deletion advice, and the direct input to the repo's deletion-evidence threshold question. | +| O04 | Security is entirely absent. Removing guardrails and routing decisions through "surrounding context and judgement" makes malicious surrounding context the judgment input; the one named worst case (deleting files) appears as historical motivation and is never re-secured. | Highest-stakes gap in the whole corpus. Filed as the G-SEC cluster; see section 5, guardrail 6. | +| O05 | CLAUDE.md and skills are shared team artifacts; the article addresses a singular "you" and never says who owns a gotcha or how teammates' conflicting preferences get reconciled. | Conflicting teammate preferences are precisely the conflict class the article opens with. A cross-surface conflict check can surface whose instruction wins but cannot decide it. | +| O11 | The choice is framed as prose rule versus model judgment; hooks, permission systems, and CI never appear, despite shipping in the same product. | The third option is often the right one. The official boundary is explicit: a rule in CLAUDE.md is a request, a PreToolUse hook is enforcement. | +| T01, T03 | The anti-example article persuades by example throughout, and its "simple tool descriptions" heading actually prescribes moving behavioral rules into descriptions; the after-figure still carries a rule. | The rules were relocated and compressed, not deleted. A repo check that scored description simplicity would penalize the article's own recommended output. | +| T04 | Examples are condemned for constraining the exploration space, then an enum is praised precisely because it "hints to Claude about how to use it". An enum is a harder constraint than an example. | The load-bearing distinction is never stated. See section 4. | +| T05, O09 | "Avoid making them overconstrained, except in highly important areas" re-admits the whole disease in six words, and "highly important" is never characterized. | Any reader can classify their pet rules as highly important and change nothing; equally, a trimming pass has no principled stopping point. This is the load-bearing calibration knob and it is undefined. | +| T07 | The article's own history reverses twice on intermediate files: the old system prompt forbade planning documents, plan mode then relied on markdown plan files, and the new advice enriches persistent references further. Presented as linear progress. | Direct evidence that a rule stated with confidence in one generation is reversed in the next. Treat the current column as dated, not as settled. | +| T08, T10 | "Keep your CLAUDE.md lightweight" sits beside "spend most of the tokens on gotchas", and "cannot be as specific" as a prompt sits beside a highly specific gotcha prescribed for CLAUDE.md. | The real rule is composition, not weight: derivable content out, tribal knowledge in; general in applicability, precise in wording. Quoting "lightweight" alone imports a rule the article does not give. | +| A07, A11, T06, T12 | Routed to the cross-source section: interface ownership (3.5), efficiency evidence versus correctness rhetoric (3.3), and the figure that retracts the body's evidence (3.4). | | + +### 2.2 P2, "Effective context engineering for AI agents" (Anthropic engineering post) + +Carried: 39 of 42 rows, as 18 entries. Dropped: the best-model-first budget assumption +(`P2-A06`), the measurable-outcome assumption (`P2-A08`), and the human-analogy-both-ways +observation (`P2-T09`). Note that this post was never engaged by the prior +`context-engineering-claude-5` plan at all (grep-verified during reconciliation: zero hits for +its title, URL, "context rot", or "attention budget"), so its apparatus is entirely net-new to +the repo. + +| Rows | Claim | Why it matters for adoption | +|---|---|---| +| A01 | Context rot is established by one third-party needle-in-a-haystack benchmark and then generalized to every model, every task type, and the future. No first-party measurement appears anywhere in the post. | The corpus's most-cited mechanism rests on the thinnest evidence in it. Cite it as directional, not as measured. | +| A02, A03, T07 | The n-squared attention paragraph reads as rigor, but n-squared is the mechanism by which every pair is attended; the actual causal claims (training-distribution skew, position encoding) are the soft ones next door, and the dense-attention premise is silently assumed. | The digests called this section mechanism rather than assertion, and the fresh pass disputes exactly that. Do not cite this paragraph as the documented mechanism for degradation. | +| A05, O05, O01, O16, T01 | The whole methodology presumes an evaluation loop, "high-signal" is the load-bearing adjective and is never measured, no threshold in the post is quantified (compaction fill fraction, "tight", example counts, toolset size), and "context pollution" is introduced and never defined. | Two engineers can follow this post and build opposite systems, both claiming compliance. Any repo check derived from it must supply the threshold the post withholds. | +| A07 | Metadata as signal and progressive disclosure presume a well-organized information environment: meaningful names, honest timestamps, sane hierarchies. | In messy trees the same mechanism feeds the agent misleading signals. The precondition is the thing to check before adopting the technique. | +| A09, O03, O07 | Persistent notes and file-based memory get no staleness model, no provenance or integrity check, and no security framing at all; the words injection and trust do not appear. | Notes re-entering context are the vector by which a one-time injection becomes permanent. G-SEC again, and the reason a notes-file convention needs an owner before it needs a template. | +| A10, O02, O11, T11 | Sub-agents are recommended with no cost or latency criterion, no method for briefing them, no way to detect a silently dropped detail, and directly after a section warning that agents misuse tools and chase dead ends. | The post's own linked multi-agent case reports roughly 15x the token use of chat. A fan-out norm with a return-size figure but no cost criterion is half a contract. | +| A11, T02 | The post repeatedly teaches a technique and predicts its obsolescence in the same breath ("likely becoming less important as models become more capable"), then insists the principle is permanent without arguing why the principle survives what the techniques do not. | Everything in this corpus has a shelf life the corpus itself announces. Date and scope every derived rule. | +| A12 | Every worked example is an Anthropic product and the closing advice is scoped to teams building on Claude; transferability is asserted only implicitly. | Fine for this repo, which is Claude-specific, but it bounds any claim of general agent-design authority. | +| A13, O14 | The centerpiece figure's "just right" prompt is asserted by color: no eval, no failure-mode comparison, no evidence the middle prompt beats the left one, and the altitude calibration is demonstrated only on a mid-stakes assistant domain. | The post's flagship pedagogy teaches an aesthetic, not a test. Do not import "just right" as a standard. | +| O06 | Just-in-time retrieval, compaction, and tool-result clearing all rewrite or reorder context, which invalidates prefix caches, a platform feature with a direct price. The post's economics never mention caching. | Token efficiency measured per turn can be a cost regression per cache lifetime. The cookbook page has the same hole (`T8-O7`). | +| O08, T04 | "Do the simplest thing that works" sits atop a second half that is a catalog of non-simple machinery, and the post gives no negative guidance anywhere: no when-not-to, no when an agent is the wrong tool. | Its own predecessor is known for exactly that guidance. Adopting the machinery without the trigger economics is how a repo accretes always-on cost. | +| O09 | The move from embedding retrieval to just-in-time retrieval is presented as field convergence with no benchmark and no note of where embeddings still win. Note the verb: the post says teams are augmenting retrieval systems, and the digests shaded that into replacement. | Downstream artifacts citing the digest would overstate the post's anti-embeddings position. Quote the verb. | +| O10 | Notes, compacted summary, and live context can disagree, and the post never says which wins or how they compose, though real systems run all three at once. | A stale note against a fresh summary is a concrete failure this repo would hit in the first week. | +| O12 (with `T4-O009`) | Tool schemas and MCP servers occupy context before the first turn, and neither post gives a tool-count threshold or a pruning method. | The standing cost of a toolset is the part a per-turn token analysis never sees. | +| O13, T05 | Few-shot examples are strongly advised while edge-case lists are condemned, the distinction ("diverse, canonical" versus "laundry list") is aesthetic rather than operational, and the post's own methodology (add examples for observed failure modes) is precisely how laundry lists accrete. There is no versioning or regression discipline to catch it. | See 3.1. This is the internal half of the corpus's sharpest cross-source reversal. | +| O15 (with `T5-A013`) | All long-horizon advice assumes an autonomous run: checkpoints, human review, and approval gates are absent, while the older linked post insists agents add most value where they integrate meaningful human oversight. | Two sources in one corpus, opposite defaults, no reconciliation. Pick the default deliberately and record why. | +| T03 | Exploration is championed, then conceded to be slower and error-prone and dependent on thoughtful engineering, and the resolution offered is hybrid, depending on the task. | The recommendation dissolves at the moment of decision, which is the moment a repo check has to act. | +| T06 | Degradation is argued as real and universal, then hedged as a gradient rather than a hard cliff with models remaining highly capable at long contexts. Both directions are kept. | Enough alarm to motivate the discipline, enough reassurance to protect the long-context claim. Quote whichever you like, which is the problem. | +| T10 | The post warns against hardcoded brittle scaffolds while its flagship example, Claude Code, drops CLAUDE.md files into context up front (the post's own adverb is "naively") and ships a fixed five-most-recent-files constant in its compaction. | Anthropic's exemplar deviates from the post's aesthetic and the deviation is flagged, never reconciled. This repo's CLAUDE.md is loaded the same way. | +| T12 | The term's coiner and the agent definition's source are credited only as unlabeled hyperlink anchors, so a text-only reader receives both framings in Anthropic's voice. | Provenance is stripped by any extraction. Re-attach it when quoting. | +| A04 | Routed to 3.5 (interface and harness ownership). | | + +### 2.3 The nine linked pages + +Carried: 67 of 236 rows. These pages were absorbed for mechanism, so most of their apparatus +is page-local trivia; what survives is the rows that bite a repo decision. The tier-2 +structural notes taken earlier had no assumptions, omissions, or tensions section at all, so +every row here is net-new relative to that layer. + +**Agents judging agents (T3 dynamic workflows, T4 writing tools, T6 multi-agent research).** + +| Rows | Claim | Why it matters for adoption | +|---|---|---| +| T3-T001 | The model is trusted to write its own harness in the same article whose "why" section documents that model's agentic laziness, self-preferential bias, and goal drift. Harness-writing is never explained as exempt. | If the failure modes motivate the harness, they apply to the harness author too. | +| T3-T004, T3-A003, T6-A05, T6-T06 | Self-preferential bias is defined as the model preferring its own findings, especially when judging against a rubric, and the mitigation offered (a separate context window) is assumed to neutralize a bias described as belonging to the model. Elsewhere agents rewrite their own tool descriptions, in a post warning that bad descriptions send agents down wrong paths. Judge alignment with human judgment is asserted, never shown. | Every verifier and rubric pattern in this repo rests on this assumption. Separate context is a mitigation, not a proof. | +| T3-T007 | Lossy summarization is the bug that workflows exist to fight (goal drift) and the safety mechanism of the quarantine pattern (pass the structured summary only). | The same mechanic cannot be both without a stated boundary. | +| T3-T002 | Dynamic beats static because static must handle all edge cases, and then the article recommends freezing dynamic workflows into files and shipping them as skills, which makes them static again. The patch offered is to prompt the model to treat the skill as a template. | This marketplace ships skills. The tension is ours, not theirs. | +| T3-A010, T3-O009 | A workflow shipped inside a skill is presented purely as a benefit (anyone who installs the skill runs the same workflow), with no review step and no statement of what the executing sandbox can reach. | G-SEC. Distribution-channel trust is assumed. | +| T3-A007, T6-A07, T6-O03 | Token count is treated as the only cost axis: a figure shows 1.1M-token runs without comment, and multi-agent economics are given as multiples (roughly 4x for agents, 15x for multi-agent) with no dollar or latency figure anywhere. | Fan-out advice with no cost model is how a repo ships an expensive default. | +| T6-T08 | Multi-agent is pitched as a vital way to scale performance in the same section that concedes it is uneconomical except for high-value parallelizable tasks and unfit for most coding. | That is a niche, not a scaling law. It bounds every fan-out recommendation derived from this page. | +| T6-T05 | The lead agent's compression of subagent findings is the architecture's benefit in the body and a game of telephone causing information loss in the appendix, which recommends bypassing the coordinator. | The corpus contains its own refutation of the relay it recommends. | +| T6-T02, T6-T03 | Parallelism is credited with up to 90% time cuts while execution is conceded to be synchronous, so one slow subagent blocks the system; and heuristics-not-rigid-rules sits beside hard numeric scaling rules embedded in the prompts. | Both are places where the stated principle and the shipped artifact disagree. | +| T4-O004 | The headline gains (67.4% to 80.1%, 79.6% to 85.7%) prove the optimization loop works and never say which principle produced them: naming, descriptions, response format, or consolidation. | The numbers license the loop, not the individual principles the page teaches. Do not cite them for a specific rule. | +| T4-T001 | Consolidate multiple operations into one tool, and give every tool a clear distinct purpose, with no stated boundary between healthy consolidation and purpose blurring. | The boundary is exactly what a tool-design check would need. | +| T4-T003 | Agent feedback is the improvement engine and is declared unreliable in the same section ("LLMs don't always say what they mean"); the mitigation is manual transcript reading. | The optimization signal and the thing you cannot trust are the same channel. | +| T4-T007 | Tokens are scarce in tool responses (a 25,000-token cap, concise by default) and freely spent on chain-of-thought and interleaved thinking to raise effective intelligence, with no accounting for the trade. | Token thrift is a per-surface policy in this corpus, never a budget. | + +**Human in the loop and workflow design (T2 field guide, T5 building effective agents).** + +| Rows | Claim | Why it matters for adoption | +|---|---|---| +| T2-A008, T2-A011 | The workflow assumes the operator can recognize what they want when shown candidates, and can judge the quality of Claude's teaching in exactly the domains where they by definition lack knowledge. | The field-guide audit reached the same finding independently: the playbook assumes an evaluation function it never tests. Convergence from two directions is the strongest signal in the corpus. | +| T2-A003 | Implementation notes are assumed complete and honest, and a quiz authored by the agent that made the change is assumed to cover the genuinely risky parts. | Self-report plus self-authored assessment is the weakest verification shape available, and it is the only merge gate the guide offers. | +| T2-A010 | On mid-run deviation the agent picks the conservative option and continues without checking in. | The rule is magnitude-blind and unbounded. This is the audit's one source-weaker verdict: the repo's replan threshold already names the article's failure mode. | +| T2-A002, T2-A006 | Every practice ends in a human reaction step, and the guide is written in the first person singular with teammates appearing only as post-hoc reviewers. | Synchronous single-operator assumptions do not survive contact with a fleet or a background session. | +| T2-O004 | Nothing verifies the code. Tests, CI, and review never appear; the only merge gate is a human passing a comprehension quiz. | Comprehension is not authorization. Any adoption needs the verification floor the guide omits. | +| T2-O005, T2-T005 | Implementation notes exist so the next attempt can learn, are declared temporary, and have no channel into the durable map of skills and context. | The practice discards the knowledge it was created to capture. Found independently by the audit as a homeless-ownership problem. | +| T2-T001 | The guide warns that over-specific prompts make the model follow instructions when a pivot would be better, and delivers that warning through highly prescriptive exemplars, with no rule for which details are safe to pin. | Same shape as `P1-T01`. The corpus repeatedly violates its own anti-specificity advice in its exemplars. | +| T2-T004 | Mechanical refactoring is delegated sight unseen before implementation, and after implementation nothing merges until the human answers perfectly on everything that happened, including that trusted work. | Two trust postures, no principle separating them. | +| T2-T007, T2-O003 | Two definitions of "unknown" are stitched together (a map-territory gap, and the operator's awareness states), and no criteria are given for skipping the practices on small or well-known work. | The repo's own framing already unified the two definitions, which is why the audit never noticed the stitch. Trigger economics is the deepest treatment in any layer, and it is the repo's, not the article's. | +| T5-T001 | Agents are "just LLMs using tools in a loop" and implementation is called straightforward, while the appendix reports more time was spent optimizing tools than the overall prompt. | The loop is simple; making it work is not. The simplicity claim undersells where the engineering lives. | +| T5-T002 | The post recommends using LLM APIs directly and warns that frameworks obscure the underlying prompts, and the current page opens that same section by listing four frameworks including Anthropic's own SDK. | The tension was sharpened by a silent revision, not by the original text. | +| T5-T003, T5-T004 | The workflow-versus-agent binary is undercut by the post's own flagship example, which embeds fixed gates inside an agent loop; and the evaluator-optimizer workflow, the one most prone to non-termination, is given no stopping condition while the agents section prescribes maximum iterations. | If you adopt the taxonomy, adopt it as a continuum with an explicit iteration cap. | +| T5-T005 | Agents are ideal for scaling in trusted environments precisely because humans are out of the loop, and the appendix says agents add most value where they integrate meaningful human oversight. | Unreconciled, and it is the decision a repo has to make first. | +| T5-O015, T5-T008 | The page reads "Published Dec 19, 2024" while quoting model names and an SDK that postdate publication, and the framework list was changed after publication with no inline changelog. | Quoting "the December 2024 post" from this URL quotes a silently updated text. This is live evidence for the repo's deferred content-hashing reopen trigger. | +| T5-A001, T5-O007 | The reader is assumed to control the whole stack, and prompt injection is out of scope even though the examples include tools that issue refunds and agents that operate computers. | Ownership precondition (3.5) and G-SEC. | + +**Platform and reference surfaces (T7 platform announcement, T8 cookbook, T9 prompting docs).** + +| Rows | Claim | Why it matters for adoption | +|---|---|---| +| T7-T002 | The announcement claims agents can handle workflows extending beyond any fixed limit, while the mechanism it describes clears content when approaching token limits. What is extended is turn count, not the limit. | A within-window pruning mechanism described in unbounded language. Do not import the phrasing. | +| T7-T001, T7-A005, T7-O001, T7-O002 | The 39% combined and 29% context-editing-alone figures never isolate memory alone, the baseline is never operationally defined, and the 84% token reduction is given without absolute counts, cost, or a statement that quality was held constant. | The two-part framing's implicit symmetry is unsupported by its own numbers. | +| T7-T004 | An unqualified superlative sits directly beside data-flavored claims sourced to an unnamed internal evaluation set, with nothing distinguishing marketing assertion from benchmarked assertion. | Same page, two evidence tiers, no visual or rhetorical separation. Tier them yourself. | +| T7-O004, T8-T1 | Model scoping is unclear or self-contradictory: the announcement names only one model, and the cookbook gives two incompatible supported-model lists for the same feature (five models in one section, two in two others). | Both cookbook lists are stale as of the corpus pass: the memory tool needs no beta header and spans all Claude 4 and later models, and Claude Code exposes neither the memory tool nor context editing natively. Do not cite the cookbook as a model-list source. | +| T8-T2 | A worked transcript logs 6,611 input tokens against a stated 5,000-token trigger and reports that no clearing was triggered, with no explanation. | The corpus's only end-to-end trace of the mechanism does not match its own configuration on a literal read. | +| T8-A008, T8-O1 | Prompt-injection mitigation is "instruct Claude to ignore instructions in memory", offered without the standard caveat about prompt-based defenses, and the entire security implementation is named but never shown while non-security code is fully reproduced. | G-SEC. The security half of the page is the half you cannot read. | +| T8-A007, T8-T3 | Every demo constant is a teaching value to be rescaled by qualitative pointers, and production trigger guidance appears twice in different forms (a 30k to 40k prose range and a hardcoded 35000). | Copy a constant from this page and you have copied a demo. | +| T9-T002 | The general instruction to have Claude self-check against test criteria is immediately reversed for one model generation: remove these instructions rather than rewriting them, because they now cause over-verification. | The cleanest example in the corpus of a technique inverting within one model generation. See guardrail 1. | +| T9-O001 | Neither prompting page mentions context engineering by that term or a close synonym anywhere. | The corpus's central vocabulary is absent from the vendor's own prompting reference. Treat the term as a house coinage, not documented doctrine. | +| T9-T004, T9-T005, T9-A002, T9-A007 | The overview presents the best-practices page as the living reference and that page immediately fans back out to four further per-model pages; prompting and model choice are framed as alternative remedies on one page and as coupled on the other; and phrases like "Claude's latest models" are used dozens of times and enumerated never. | Any pointer chain into the vendor prompting docs terminates at a hub, not an answer, and the answer that matters is per-model. | +| T9-T001 | "Be clear and direct" because the model will not infer beyond what you ask, alongside repeated claims that the newest models proactively infer intent and delegate without instruction. | The general principle is really a safety default for cases the proactive behavior misses, which the page never says. | + +## 3. Cross-source tensions + +These are the disagreements that survive merging, where two sources in one corpus say +different things and a repo decision has to pick. + +### 3.1 The few-shot reversal (P2 against P1) + +P2 (September 2025) strongly advises few-shot examples, asking for a diverse set of canonical +examples rather than an exhaustive edge-case list. P1 (July 2026) puts examples in the "Then" +column and interfaces in the "Now" column, on the grounds that examples constrain the +exploration space. The same corpus therefore advises and deprecates the same technique eleven +months apart, and neither source acknowledges the other on this point. + +Three facts constrain the resolution. First, P1's claim is generation-scoped, and P1 concedes +the old rules were right for the older models. Second, the vendor's own prompting +documentation still recommends examples, and the examples-harm claim appears on no official +surface; the corroboration pass found it unsupported. Third, P2's internal tension +(`P2-T05`, `P2-O13`) shows the advice was already unstable: the distinction between canonical +examples and a laundry list is aesthetic, and the post's own iterate-on-failures methodology +is exactly how laundry lists accrete. + +Resolve it per model generation, not by publication date. On the generation P1 addresses, +prefer an interface constraint (an enum, a schema, a typed return) over a worked example +wherever an interface can carry the same information, and keep examples for what no interface +can carry: format, tone, and judgment calls with no enumerable range. On older or mixed fleets +keep the canonical examples, because that is the regime P2 measured and P1 concedes. In both +cases the deciding evidence is your own eval, because neither article supplies one. + +### 3.2 Compaction: high fidelity against its own caveat + +P2 calls compaction high fidelity and claims minimal performance degradation, then admits one +paragraph later that a piece of context's importance often becomes apparent only later +(`P2-T08`, `P2-A14`). Fidelity cannot be judged at compaction time by a summarizer that does +not yet know what will matter, so the confidence claim and the epistemic admission cannot both +be fully true. The post also never addresses what repeated compaction does (summaries of +summaries), how to detect a bad compaction after the fact, or how to recover once the +discarded history is gone (`P2-O04`). The platform announcement overstates in the same +direction (`T7-T002`), and the cookbook's own trace does not match its configured trigger +(`T8-T2`). + +For adoption: carry the caveat with the technique, always. A compaction or context-editing +recommendation that cites the fidelity claim without the later-importance admission has +inherited half a source. Both sentences travel together or neither does. + +### 3.3 Efficiency evidence, correctness rhetoric + +The measured claims in this corpus are efficiency claims. P1's 80% is a token-reduction claim +scoped to Anthropic's coding evals with no measurable loss reported on those evals +(`P1-O02`), and its own supporting sentence says the model can interpret intent and reach the +right answer but must think more carefully when instructions conflict (`P1-T06`), which is a +reasoning-tax argument, not a wrongness argument. The rhetoric wrapped around it is a +correctness rhetoric: myths, unhobbling, guidance that is wrong. T7's percentages carry the +trappings of measurement with no methodology (`T7-O001`). T6's headline 90.2% never defines +its metric or denominator (`T6-O01`). T4's gains prove a loop, not a principle (`T4-O004`). + +For adoption: separate the two before deriving anything. An efficiency delta licenses "this is +cheaper", never "the old rule was wrong". The repo's evidence tiering already has a place for +the middle case: the 80% figure appears on no official surface, but the changelog's +lean-system-prompt default is directional corroboration, so it is an opinion-tier claim with a +directional annotation, and the magnitude stays vendor voice. + +### 3.4 The figure that retracts the body + +P1's body presents its conflicting-instruction example as observed, from reading transcripts of +internal usage. The figure rendering those same quotes carries a footnote: + +``` +Illustrative examples, not verbatim quotes from any real prompt, skill, or user request. +``` + +The article's only transcript evidence is retracted by its own figure, and a careful reader +cannot tell whether the opening diagnosis rests on data or on invention (`P1-T12`). The figure +layer is doing this work in both directions across the corpus: T3's only cost numbers (22 +agents, 1.1M tokens, 11m3s) exist solely inside an image, T4's headline percentages are chart +content, and P2's centerpiece worked prompts exist only as pixels. + +For adoption: never cite a body claim without checking the figure, and never cite a figure +number without checking whether the body states it. Where the two disagree, the corpus gives no +rule for which wins, so record both. + +### 3.5 The interface-ownership precondition + +Four sources independently assume the reader owns the surfaces the advice acts on: P1 tells you +to redesign tool parameters (`P1-A07`), P2 assumes you decide what enters context each turn and +can implement compaction (`P2-A04`), T4 assumes you own tool names, schemas, response formats, +and error text (`T4-A002`), and T5 assumes you control the whole stack down to the API calls +(`T5-A001`). None addresses the reader consuming third-party MCP servers, vendored tools, or +someone else's harness, and none offers a fallback. + +For adoption: this is the precondition that decides whether a derived check is dischargeable. A +check telling a contributor to redesign a third-party tool's parameters cannot be satisfied, and +should not be written. Filed as the G-PRECOND cluster with `P1-A06` (coherent surrounding idiom) +and `P1-A10` (capacity to author code-form references). + +## 4. The unstated thesis + +The fresh pass identified one proposition that P1 never states and without which its own +argument does not cohere: + +> Constraining the interface is good. Constraining the behavior space by example is bad. + +Read P1 without it and three of its moves are contradictions. It condemns examples for +constraining the exploration space, then praises an enum precisely because it hints at how the +model should use a tool, and an enum is a harder constraint than an example (`P1-T04`). It +titles a section "simple tool descriptions" and then prescribes moving behavioral rules into +those descriptions, whose after-state still carries a rule (`P1-T03`). It is an anti-example +article that persuades entirely by example, and its most memorable example is an example about +not using examples (`P1-T01`). Read it with the thesis and all three resolve: an enum is +interface, a consolidated description is interface, and a worked example is behavior space. The +same thesis reconciles the field guide's parallel violation (`T2-T001`, prescriptive exemplars +delivering anti-specificity advice), and it is the coherent version of P2's +canonical-versus-laundry-list distinction, which P2 leaves as taste. + +Two consequences follow, and both matter more than the thesis itself. + +First, the thesis inverts the corpus's own advice about deletion. If the goal is to constrain +the interface, the answer to a prose rule is usually to move it into a schema, an enum, a type, +a hook, or a permission, not to delete it. P1 frames the choice as prose rule versus model +judgment and never mentions the third option even though the product ships it (`P1-O11`), and +the official boundary is explicit that a rule in prose is a request while a PreToolUse hook is +enforcement. + +Second, the thesis is only available to a reader who owns the interface, which is 3.5. Where +you do not own it, the corpus's advice reduces to deleting the rule and hoping, which is +precisely the case its security omission (`P1-O04`) leaves unaddressed. + +Filed as the G-THESIS cluster. Its immediate use is as design rationale wherever a repo check +recommends replacing prose guidance with something else: the something else should be an +interface where one exists, and the rationale should say so. + +## 5. Adoption guardrails + +Apply these before turning any corpus claim into repo doctrine. Each is distilled from the rows +above and names them, so a contributor can check the reasoning rather than take the rule on +faith. + +1. **Name the generation.** Every technique in this corpus is model-generation-gated, and the + corpus contains one technique reversed within a generation (`T9-T002`) and one reversed + across generations (`P1-A02`, section 3.1). Record which model a claim was measured on and + which models your fleet runs. A generation-gated claim adopted ungated is wrong for part of + the fleet, and the source will tell you so if you read its concession. +2. **Ask what was measured, then bound the claim to it.** An efficiency delta licenses a cost + argument, never a correctness argument (3.3). Scope each claim to the eval it came from: + coding evals for P1, search and browse benchmarks for T6, one third-party needle benchmark + for `P2-A01`. Where no vendor surface carries the figure, it stays opinion-tier, with a + directional annotation if a changelog corroborates the direction. +3. **Do not adopt a deletion without a detection and a rollback.** The corpus recommends + removing guardrails and supplies no monitoring signal, no failure-detection recipe, and no + criterion for reinstating a rule (`P1-O01`, `P1-A13`). Consequential deletions need ledger + evidence in the repo's existing grammar; trivial ones do not. If you cannot say how you + would notice the deletion was wrong, you are not ready to make it. +4. **Discharge the ownership and environment preconditions.** Before writing a check, ask + whether the reader owns the interface (3.5), whether the surrounding code is coherent enough + to be a signal (`P1-A06`), whether the tree is organized enough for metadata to inform rather + than mislead (`P2-A07`), and whether the reader can author the artifact the advice presumes + (`P1-A10`). A check that cannot be discharged should not ship. +5. **Price the whole loop, not the upfront tokens.** Progressive disclosure adds retrieval turns + and can silently fail to fire (`P1-A04`, `P1-O07`); fan-out costs multiples, not deltas + (`T6-A07`, `T3-A007`); every context rewrite invalidates a prefix cache (`P2-O06`, `T8-O7`); + and the toolset itself has a standing cost (`P2-O12`). +6. **Never route an injection-sensitive or destructive action through judgment alone.** The + corpus's largest shared hole is security: guardrail deletion with malicious surrounding + context as the judgment input (`P1-O04`), persistent notes as an injection that survives + resets (`P2-A09`, `P2-O03`), a prompt-based injection defense offered without caveat + (`T8-A008`), and skill-distributed executable workflows assumed safe (`T3-A010`). Any + judgment-delegating rewrite in these areas needs an enforcement-layer backstop, which is the + third option `P1-O11` omits. Tracked as G-SEC. +7. **Prefer an interface to a rule, and precedence to deletion.** Section 4 for the first half; + `P1-A08` and `P1-O13` for the second. Where two surfaces conflict, the official layering + rules already say how they compose, and establishing precedence keeps the guardrail that + deletion discards. +8. **Read the figure and the body against each other.** The corpus's only transcript evidence is + retracted in a footnote (3.4), and several of its only numbers exist nowhere but inside + images. Cite neither layer alone. +9. **Date, scope, and pin the citation.** Pages in this corpus are revised silently (`T5-O015`, + `T5-T008`), features named in them are removed within weeks of publication, and both cookbook + model lists were stale by the time of the pass. Cite the changelog or the reference page for + product behavior, and cite the article only for its argument. +10. **When two corpus sources disagree, do not resolve by date.** The newer source wins only + where the difference is genuinely generation-gated and the source says so. Otherwise the + disagreement is an open question for your own eval, and the honest artifact records both + positions (3.1; `P2-O15` against `T5-A013`). + +A last item that is not a rule but a posture. The digest layer of this corpus is faithful and +the sources are useful; nothing here argues against adopting their techniques. It argues against +adopting them as gospel, because on the evidence assembled above the sources do not hold +themselves to that standard either. diff --git a/docs/specs/context-engineering-deletion-evidence-attribution.md b/docs/specs/context-engineering-deletion-evidence-attribution.md new file mode 100644 index 000000000..4eb267a52 --- /dev/null +++ b/docs/specs/context-engineering-deletion-evidence-attribution.md @@ -0,0 +1,93 @@ +# Deletion-evidence attribution: making the consequential tier clearable + +Design record for the mechanism decision Q1 depends on. Signed off 2026-09-01 as a two-tier +threshold: an editorial pass may delete trivial legacy guards, a consequential rule needs ledger +evidence. That second tier was deliberately unclearable at sign-off, because the evidence grammar +it names runs the other direction. This document closes that gap by specifying the mechanism, and +records why the obvious shortcut does not work. + +## The problem + +`claude-config:unhobble` ships a re-add gate: after a branch-local strip of the project's standing +instructions, an instruction is restored only on at least two ledger rows sharing one underlying +cause, and the restoring commit cites those rows. Undefended deletions stay deleted. + +That grammar is sound and it is the repo's only existing evidence discipline for instruction +removal, so Q1 reached for it. But it answers a different question: + +| | unhobble's shipped gate | what Q1's consequential tier needs | +|---|---|---| +| Question | should this rule come *back*? | may this rule be *removed*? | +| Default | stays deleted absent evidence | stays present absent evidence | +| Evidence | stumbles observed while it was absent | dispensability while it is present | +| Scope | everything stripped at once | one rule at a time | + +The last row is the mechanical blocker. The experiment strips the whole surface, so a stumble is +attributable to "the bare configuration", not to any individual rule. Nothing shipped attributes an +observed stumble to the specific instruction whose absence caused it, and a per-rule deletion +warrant cannot be derived from an aggregate absence. + +## The mechanism + +Three parts, all of them the deletion counterpart of an existing unhobble concept rather than a new +vocabulary. + +### 1. The observation window is per-rule and stated up front + +A consequential deletion candidate enters a **watch**, recorded before any removal: + +- the rule, quoted, and the surface it lives on +- the class of work the rule governs, stated as the situations where its absence would show +- the window: a count of qualifying sessions, not a wall-clock duration, since an idle week proves + nothing. Qualifying means a session that actually entered the rule's governed situation +- the disqualifier: what would end the watch immediately (any stumble attributable to the rule) + +A watch that never accumulates qualifying sessions expires unresolved. That is a real outcome and +it is reported as one; it is never read as evidence of dispensability, which is the asymmetry the +whole design turns on. + +### 2. Attribution is by governed situation, not by proximity + +A stumble counts against a deletion candidate when the session entered the situation the rule +governs and the outcome went wrong in the way the rule exists to prevent. Two guards: + +- **Same-cause aggregation**, inherited from unhobble: two stumbles count as one row when they + share an underlying cause. A single flaky session does not decide a rule's fate in either + direction. +- **Co-absence is not attribution.** When several rules were removed in one change, a stumble is + attributed only if exactly one removed rule governs the situation. If two do, the row attaches to + the group and the group's deletions are reverted together. Splitting a group's evidence between + its members is how a wrong deletion survives its own evidence. + +### 3. The warrant, and what it is not + +A consequential deletion is warranted when the watch closes with its qualifying-session count met +and zero attributed rows. The removing commit cites the watch record, exactly as unhobble's +restoring commit cites its ledger rows. + +Three things this explicitly is not: + +- **Not a proof of harmlessness.** It is bounded evidence over a stated window, and the window is + named in the commit so a later reader can judge its weight. +- **Not available for a protected class.** A rule matching the [instruction exception + register](../conventions/instruction-exception-register/README.md) is not deletable at all, + so it never enters a watch. Compression in place or hook conversion are its remedies. +- **Not required for the editorial tier.** Derivable content, restated obviousness, and stale + model-era scaffolding are deleted on the auditing skill's normal criteria without a watch. The + watch is the price of removing something that governs behavior, not of tidying. + +## Why not simply run the bare experiment per rule + +Considered and rejected: strip one rule, work, observe. It is the cleanest possible attribution and +it is unaffordable. The experiment's cost is a full branch-local strip plus a working period per +rule, and an instruction surface has tens of candidates. The watch above buys most of the +attribution for the marginal cost of recording a decision that was being made anyway, and it +degrades honestly (an expired watch resolves nothing) where a shortcut would degrade silently. + +## Adoption + +Not yet wired into a skill. The mechanism is recorded here so the sign-off's consequential tier has +a specification to point at; wiring it into `claude-config:unhobble` (as a deletion mode alongside +the re-add gate, sharing the ledger grammar) is tracked separately. Until that lands, the +consequential tier remains unclearable in practice, which is the honest state and is preferable to +a tier that clears itself on absent evidence. diff --git a/docs/specs/context-engineering-linked-sources.md b/docs/specs/context-engineering-linked-sources.md new file mode 100644 index 000000000..87e529f2c --- /dev/null +++ b/docs/specs/context-engineering-linked-sources.md @@ -0,0 +1,968 @@ +# Linked sources: the nine first-party pages behind the context-engineering corpus + +Durable reference for the nine first-party pages that the two primary context-engineering +articles link. Written so a future contributor can use their content without re-fetching and +re-reading all nine. + +The two primary articles are: + +- **P1**, "The new rules of context engineering for Claude 5 generation models", Thariq + Shihipar (@trq212), 2026-07-24, published as an X Article + (`https://x.com/trq212/article/2080710971228918066`) and mirrored on the Anthropic blog + (`https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models`). +- **P2**, "Effective context engineering for AI agents", Anthropic engineering blog, + 2025-09-29 (`https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents`). + +The nine pages are the corpus's tier-2 set, labelled T1 through T9. T1 and T2 are one content +node (the Fable field guide, published on both X and claude.com/blog), so this file carries +eight sections for nine tier-2 entries. T9 is itself a pair of docs pages, cited separately +inside its section. + +## How this file was assembled, and what it is not + +Two evidence layers stand behind every section: + +1. **Structural notes**, fetched 2026-08-31. Heading outlines, link inventories, figure + inventories, per-section gists, and a per-page "relevance hooks" pass tying each page to + corpus themes. The relevance framing survives only in that layer, and is folded into the + "Why it matters here" lines below. +2. **Fresh paragraph-grain sweeps**, fetched 2026-09-01. Raw-HTML or DOM-parsed reads with + every figure viewed directly, verbatim quotes recovered, and a four-lens critical apparatus + (assumptions, omissions, internal tensions) added per page. + +The two layers were reconciled page by page. The fresh layer is authoritative on structural +facts and numbers for every page except two: on T7 the two layers disagree over whether the +article embeds an image at all, unresolved; on T8 the fresh layer dropped the byline and +publish date, which are restored below from the structural layer. Both layers live in the +session memory tier, which is gitignored and does not survive container recycling. This file +is the committed record of their content. + +**This file is a source reference, not an analysis.** The assumption, omission, and tension +rows for these pages, and the adjudication of what each page does and does not establish, live +in `context-engineering-critical-apparatus.md` alongside this file. Do not duplicate those rows here; cite +that file instead. + +**Snapshot discipline.** Every page below is a live vendor URL that can change without a +visible revision note. Fetch dates are given per page and are the honest currency stamp. T5 and +T7 are both known to have been edited in place after publication. When citing any of these +pages in a repo instruction surface, carry the fetch date, and treat the page's own dateline as +untrustworthy where noted. + +--- + +## 1. T1 / T2. A field guide to Claude Fable 5: finding your unknowns + +One essay on two publishing surfaces. Linked from P1 twice, as the "written previously" +reference and as the "Fable field guide". + +**Citation, X surface (T1)** + +- Title: "A Field Guide to Fable: Finding Your Unknowns" +- URL: `https://x.com/trq212/status/2073100352921215386` +- Author: Thariq (@trq212), Claude Code, Anthropic +- Published: 2026-07-03. Fetched 2026-08-31 via the xtomd markdown converter plus the + fxtwitter JSON API (the API-level `article` object confirms it is an X Article, not a + note-tweet or a reply chain). + +**Citation, blog surface (T2)** + +- Title: "A field guide to Claude Fable 5: Finding your unknowns" +- URL: `https://claude.com/blog/a-field-guide-to-claude-fable-finding-your-unknowns` +- Author: Thariq Shihipar, member of technical staff, Anthropic +- Published: 2026-07-06. Category "Claude Code", reading time 5 min. Fetched 2026-08-31, and + re-swept 2026-09-01 from raw HTML. + +The two surfaces carry the same argument. The X version has native entityMap link provenance +and five images; the blog version has three content figures and a related-posts block. The +blog version's closing section cross-links P1 as the companion piece, so the two articles point +at each other. + +**What the page contains** + +Heading structure (blog surface): an unheaded opening that states the map/territory framing, +then H2 "Knowing your unknowns", H2 "Help Claude help you", H2 "Pre-implementation" over five +H3 techniques (Blind Spot Pass, Brainstorms and prototypes, Interviews, References, +Implementation Plans), H2 "During implementation" over H3 "Implementation notes", H2 "Post +implementation" over H3 "Pitches and explainers" and H3 "Quizzes", H2 "How this comes together: +launching Fable", and the closing H2 "Matching the Map and Territory". The published HTML also +carries an empty FAQ heading stub. + +Mechanisms taught: + +- **Map and territory.** The map is what you give Claude (prompts, skills, context); the + territory is the codebase and the real world. The gap between them is what the article calls + *unknowns*. Claim: Fable is the first model generation where output quality is bottlenecked + by the author's ability to clarify unknowns rather than by model capability. +- **The four-quadrant unknowns taxonomy.** Known knowns, known unknowns, unknown knowns, + unknown unknowns. Each pre-implementation technique is mapped to the quadrant it surfaces. + Boris and Jarred (Jarred Sumner) are named and LinkedIn-linked as practitioners with few + unknowns because they are deeply in sync with both codebase and model behavior. +- **The specificity dilemma.** Too-specific instructions make Claude follow orders past the + point a pivot would serve better; too-vague instructions make it fall back on generic best + practices. The offered fix is context about the human's own starting point, experience, and + thought process, not more procedural instruction. +- **Blind Spot Pass**, for unknown unknowns: ask Claude, using that literal phrase, to name + what you do not know you do not know, given who you are and your experience level. +- **Brainstorms and prototypes**, for unknown knowns (taste you would recognize but cannot + specify): prototype cheaply, for example a static HTML mock before wiring state or a backend; + open sessions with a brainstorm to avoid scoping too narrow or too wide; the worked example + asks for roughly ten interventions ordered cheapest to most ambitious, reacted to by + resonance, and for four wildly different directions rather than variations on one. +- **Interviews**: ask Claude to interview you one question at a time, prioritizing questions + whose answer would change the architecture. +- **References**: when something cannot be described in words, point Claude at real source + code, even in a different language, rather than at diagrams or screenshots, because source + carries richer structural detail. The worked example maps a Rust crate's semantics onto + TypeScript. +- **Implementation Plans**: ask for a plan that foregrounds the decisions most likely to change + (data models, type interfaces, UX flows) and buries mechanical refactoring, allocating the + reviewer's attention rather than the model's. +- **Implementation notes**: start a fresh session per plan carrying the planning artifacts, and + keep a running `implementation-notes.md` where Claude logs deviations, defaulting to the + conservative choice and continuing, so failures become learning for the next attempt. +- **Pitches and explainers**: bundle spec, prototype, and notes into one shareable document, + on the theory that reviewers start with the same unknowns the author did. +- **Quizzes**: reading a diff gives only shallow understanding of behavior that depends on + existing code paths, so have Claude generate a report plus a quiz over the change and treat + passing it as a personal merge gate. +- **Worked case study**: editing Fable's own launch video end to end in Claude Code, including + a transcription explainer to test feasibility, a Remotion caption prototype, and the + color-grading pivot where the author asked Claude to *teach* him the domain rather than + produce variants he could not evaluate. + +**Why it matters here** + +This is the first-party articulation of the over-specification failure mode that the repo's +unhobbling and instruction-audit work rests on, and its Interviews and Blind Spot Pass patterns +are the named upstream of interview-style and blindspot planning skills in this marketplace. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. Note additionally that this page has a dedicated repo +integration whose durable output is `docs/FINDING-YOUR-UNKNOWNS.md`, landed separately, which +carries the article's claims into this marketplace's own contract deltas; the audit slice that +produced it was contract tier and has been pruned. That integration and +the critical apparatus are complementary: the audit is repo-facing, the apparatus is +article-facing. + +--- + +## 2. T3. A harness for every task: dynamic workflows in Claude Code + +Linked from P1 twice, as the source for the rubrics and verifier-agents claim and for dynamic +workflows themselves. + +**Citation** + +- Title: "A harness for every task: dynamic workflows in Claude Code" (the `` tag adds + the site suffix "| Claude by Anthropic") +- URL: `https://claude.com/blog/a-harness-for-every-task-dynamic-workflows-in-claude-code` +- Authors: byline metadata credits Thariq Shihipar; the closing line credits "Thariq Shihipar + and Sid Bidasaria, members of technical staff at Anthropic working on Claude Code" +- Published: June 2, 2026. Category "Claude Code", reading time 5 min. Fetched 2026-08-31 and + re-swept 2026-09-01 from raw HTML, with all nine inline figures downloaded and viewed. No + revision caveat: the two passes agree on date, byline, headings, and links. + +**What the page contains** + +Heading structure: H2 "Example prompts", H2 "How dynamic workflows work", H2 "Why dynamic +workflows", H2 "Dynamic vs static workflows", H2 "Helpful patterns when using dynamic +workflows" over six H3 patterns, H2 "Use cases" over ten H3 recipes, H2 "When not to use +dynamic workflows", H2 "Tips for building dynamic workflows" over four H3 tips (Prompting; +Combine with `/goal` and `/loop`; Token usage budgets; Saving and sharing dynamic workflows), +and H2 "A new starting point for discovery". + +**The workflow API surface.** A dynamic workflow executes a JavaScript file with special +functions for spawning and coordinating subagents, plus standard JavaScript (JSON, Math, Array) +for data processing. The API itself appears only in Figure 1, never in prose. Three primitives: + +``` +agent(prompt, opts?): Promise<string | JsonSchema> + +const bugs = await agent("audit auth.ts", { + schema: BugList, // JSON Schema -> validated JSON output + model: "haiku", // opus | sonnet | haiku. Omit = inherit + isolation: "worktree", // "worktree" (checkout) or "remote" + agentType: "reviewer" // custom / built-in subagent +}) + +parallel([ fns ]) // Fan out, run at once. Barrier - waits for all. +const all = await parallel(files.map(f => () => agent(f))) + +pipeline(items, ...) // Each item streams through every stage. No barrier. +await pipeline(items, x => agent(draft(x)), d => agent(check(d))) +``` + +`prompt` is described as "the agent's only input, required". The workflow itself chooses each +agent's model and whether subagents run in their own worktree, so Claude controls both +intelligence level and isolation. Interrupted workflows resume where they left off. `pipeline` +is never exercised in the article's prose; its semantics exist only in that figure. + +**The three named failure modes** of long single-context work, verbatim: + +``` +Agentic laziness refers to when Claude stops before finishing a particularly complex, +multi-part task and declares the job done after partial progress, for example addressing +35 of the 50 items in a security review. + +Self-preferential bias refers to Claude's tendency to prefer its own results or findings, +especially when asked to verify or judge them against a rubric. + +Goal drift refers to the gradual loss of fidelity to the original objective across many +turns, especially after compaction. Each summarization step is lossy, and details like +edge-case requirements or "don't do X" constraints can get lost. +``` + +The remedy offered for all three is the same structural move: subagents with their own context +windows and focused, isolated goals. The four task classes where single-context work breaks +down are named as long-running, massively parallel, highly structured, and adversarial. + +**The six named patterns**, each quoted in full on the page and drawn as a 2x3 grid figure +titled "Six Workflow Patterns": + +1. **Classify-and-act.** A classifier agent decides the task type and routes to different + agents or behavior, or classifies at the end to determine output. +2. **Fan-out-and-synthesize.** Split into smaller steps, run an agent per step, then + synthesize. Stated conditions: many small steps, or steps that benefit from a clean context + window so they do not cross-contaminate. The synthesize step is explicitly a barrier that + waits for all fan-out agents and merges their structured outputs, matching `parallel`. +3. **Adversarial verification.** For each spawned agent, run a separate agent to adversarially + verify its output against a rubric or criteria. The figure shows one worker against three + verifiers, so a panel is in scope. +4. **Generate-and-filter.** Overgenerate ideas, then filter by rubric or verification, dedupe, + and return only the highest-quality tested ideas. +5. **Tournament.** Rather than dividing work, N agents attempt the same task by different + approaches, and a judging agent compares pairwise until a winner emerges. +6. **Loop until done.** For work of unknown size, loop spawning agents until a stop condition + (no new findings, no more errors in the logs) instead of a fixed number of passes. This is + the direct structural counter to agentic laziness. + +**The quarantine architecture for untrusted content** appears under the "Triaging at scale" use +case and is the page's prompt-injection defense: + +``` +A useful pattern for triage workflows is quarantine. This involves barring the agents that +read untrusted public content from taking high-privilege actions, which are instead done by +the agents in charge of acting on the information. +``` + +The accompanying figure draws it as privilege separation across a trust boundary: an untrusted +backlog feeds a dashed quarantine zone labelled "read-only tools, no privileges" holding one +reader agent per item plus a dedupe step; only a structured summary crosses into the trusted +zone, where "high-privilege tools live here" and an actor agent acts on summaries and never on +raw content, then either attempts a fix and opens a PR or escalates to a human. `/loop` runs it +continuously. + +**Other named mechanisms and numbers.** + +- Invocation: ask Claude for a workflow, or use the trigger word `ultracode`. +- Ten use cases: migrations and refactors (Bun's Zig-to-Rust rewrite is cited), deep research, + deep verification (claim extractor, then per-claim checkers, then optional source auditors + checking the checkers), sorting, memory and rule adherence, root-cause investigation, + triaging at scale, exploration and taste, evals, and model and intelligence routing. +- Sorting: single-prompt sorting of 1000+ rows degrades and will not fit context. The stated + method is a tournament, a pipeline of pairwise-comparison agents, or parallel bucket-ranking + then merge, with the parenthetical methodological claim that comparative judgment is more + reliable than absolute scoring. The deterministic loop holds the bracket, so only the running + order stays in context. Every judging node in the figure is labelled "fresh agent". +- Memory and rule adherence: one verifier agent per rule with a clean context each, plus a + skeptic-persona reviewer to cut false positives. The reverse direction mines sessions and + code-review comments for repeated corrections, clusters them with parallel agents, + adversarially verifies each candidate against the counterfactual "would this rule have + prevented a real mistake?", and distills survivors into `CLAUDE.md`. +- Restraint: workflows may use significantly more tokens; the self-check is "does it really + need more compute?", with the observation that most traditional coding tasks do not need a + panel of five reviewers, cross-linked to the multi-agent post's claim that parallelism and + specialization have to earn their coordination cost. +- Tips: token budgets are set in natural language ("use 10k tokens") and become a real cap; + press `s` in the workflow menu to save; check workflows into `~/.claude/workflows` or ship + them in a skill folder as `*.workflow.js` referenced from `SKILL.md`, prompting Claude to + treat a shared workflow as a template rather than a script to run verbatim. +- The article's only concrete cost data lives inside the workflow-menu figure and appears + nowhere in prose: `deep-research` at 22 agents / 1.1M tokens / 11m 3s, `review-changes` at + 14 agents / 482k tokens / 6m 12s, `find-flaky-tests` at 6 agents / 121k tokens / 1m 48s. + +**Why it matters here** + +This is the first-party source for adversarial verification, fan-out with an explicit barrier, +and orchestration restraint, all of which this marketplace's audit, review, and fan-out skills +implement; the quarantine architecture is the citable pattern for any skill that reads +untrusted content, and the rule-verifier recipe is a first-party workaround for `CLAUDE.md` +rules a model ignores. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. + +--- + +## 3. T4. Writing effective tools for agents, with agents + +Linked from P2 as the tool-design guidance its tool-anatomy section leans on. + +**Citation** + +- On-page H1, verbatim: `Writing effective tools for agents — with agents`. The `<title>` tag + differs, verbatim: `Writing effective tools for AI agents—using AI agents \ Anthropic`. Both + passes report this mismatch identically. Quote whichever form the citation needs, but note + that both carry a dash this repo's own prose style does not. +- URL: `https://www.anthropic.com/engineering/writing-tools-for-agents` +- Author: Ken Aizawa, with contributions credited by team in the Acknowledgements +- Published: Sep 11, 2025. Fetched 2026-08-31, re-swept 2026-09-01 from raw HTML with all eight + content figures viewed. The 2026-09-01 sweep supersedes the earlier one: the first pass ran + through a summarizing fetch channel that returned no body text for parts of "Looking ahead". + +**What the page contains** + +Heading structure: H2 "What is a tool?", H2 "How to write tools" over H3 "Building a +prototype", H3 "Running an evaluation" (with sub-steps "Generating evaluation tasks", "Running +the evaluation", "Analyzing results") and H3 "Collaborating with agents", H2 "Principles for +writing effective tools" over five H3 principles, H2 "Looking ahead", H2 "Acknowledgements". + +Framing: a tool is a contract between deterministic software and a non-deterministic agent, so +tool design cannot copy conventional API or SDK conventions. Everything in the post is +inference-time and tool-side, footnoted as being beyond training the underlying models. + +**The quantified results.** Both are held-out test-set bar charts, and the numbers exist only +in the figures: + +- Slack tools: human-written MCP server **67.4%** test-set accuracy, Claude-optimized MCP + server **80.1%**, a gain of 12.7 points. Figcaption: "Held-out test set performance of our + internal Slack tools". +- Asana tools: human-written **79.6%**, Claude-optimized **85.7%**, a gain of 6.1 points. + Figcaption: "Held-out test set performance of our internal Asana tools". + +The article states it relied on held-out test sets to avoid overfitting to its training +evaluations, and that the improvements went beyond what expert implementations achieved, +whether written by researchers or generated by Claude. No task counts, error bars, model +identity, or train/test split are given. + +**The `response_format` enum**, the article's flagship expressive-parameter example and its +only code block, verbatim: + +``` +enum ResponseFormat { + DETAILED = "detailed", + CONCISE = "concise" +} +``` + +The worked example is a Slack search call with `responseFormat: "detailed"` against +`responseFormat: "concise"` over the same 89 results. Detailed is **206 tokens** and carries +`thread_ts`, `channel_id`, and `user_id`; concise is **72 tokens** and carries only thread +content. The figcaption states the ratio directly: "In this example, we use ~1/3 of the tokens +with 'concise' tool responses." The stated reason detailed mode exists at all is that agents +sometimes need identifiers to chain calls, for example `search_user(name='jane')` feeding +`send_message(id=12345)`. The article suggests adding further formats "similar to GraphQL where +you can choose exactly which pieces of information you want to receive". + +**Other mechanisms and numbers.** + +- Prototype and connect: `claude mcp add <name> <command> [args...]` for Claude Code; Settings + > Developer or Settings > Extensions for Claude Desktop. +- Evaluation loop: build realistic multi-step tasks with verifiable outcomes, avoid verifiers + so strict they reject correct-but-differently-phrased answers, capture full transcripts, and + feed them back to Claude Code for agent-driven refinement. +- Choosing tools: consolidate overlapping operations rather than exposing many narrow tools; + the running example merges an availability check and event creation into one + `schedule_event`. +- Namespacing: group by shared prefix or suffix (`asana_search`, `jira_search`); the article + reports that prefix versus suffix choice measurably affected evaluation performance, so test + both. +- Meaningful context: return high-signal fields (`name`, `image_url`, `file_type`) over + low-level identifiers (`uuid`, `256px_image_url`, `mime_type`); resolving arbitrary + alphanumeric UUIDs to semantically meaningful language, or even a 0-indexed scheme, is + claimed to improve retrieval precision by reducing hallucinations. Response serialization + (XML, JSON, Markdown) is stated to affect evaluation performance with no one-size-fits-all + answer, because models perform better on formats resembling their training data. +- Token efficiency: pagination, range selection, filtering, and truncation with sensible + defaults. The one hard number: "For Claude Code, we restrict tool responses to 25,000 tokens + by default." Errors should be actionable and specific so the agent can self-correct; the + article contrasts an unhelpful error figure against a helpful one. +- Tool descriptions: called one of the most effective levers, to be written as if onboarding a + new employee, with Claude Sonnet 3.5's SWE-bench Verified results cited as partly + attributable to description refinement. +- Figure-internal numbers, from the hero terminal transcript: accuracy 17/20 (85.0%), average + task duration 15.77s, 1,219 lines written. + +**Why it matters here** + +This is the citable first-party basis for token-budgeted tool returns, consolidated tool sets, +and expressive parameters, all of which bear directly on how this marketplace's plugins define +MCP tools and shape tool output. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. + +--- + +## 4. T5. Building effective agents + +Linked from P2 as the source of its workflow-versus-agent definition. + +**Citation, and the revision caveat** + +- On-page H1: "Building effective agents". The `<title>` tag reads "Building Effective AI + Agents \ Anthropic". +- URL: `https://www.anthropic.com/research/building-effective-agents` +- Authors: the Acknowledgements read "Written by Erik S. and Barry Zhang." The original release + credited Erik Schluntz; the live page shortens the surname. +- Dateline: "Published Dec 19, 2024". **Do not cite it by that dateline.** The page is silently + revised in place. As fetched, it names Claude Haiku 4.5 and Claude Sonnet 4.5 and the Claude + Agent SDK, all of which postdate December 2024, and its framework list has been swapped: the + original named LangGraph and Amazon Bedrock's AI Agent framework, while the live page names + the Claude Agent SDK, Strands Agents SDK by AWS, Rivet, and Vellum. **Cite it by fetch date: + fetched 2026-09-01** (raw HTML, all eight figures viewed; an earlier structural pass ran + 2026-08-31 with matching results). +- The live page also carries an editorial update note, verbatim: + +``` +Note: Much of the tooling landscape described in this post has changed since December 2024. +For our current approach, see how we built Claude Managed Agents and the Managed Agents +documentation. +``` + +**What the page contains** + +Heading structure, confirmed from raw HTML including anchor ids: H2 "What are agents?", H2 +"When (and when not) to use agents", H2 "When and how to use frameworks", H2 "Building blocks, +workflows, and agents" over H3 "Building block: The augmented LLM", H3 "Workflow: Prompt +chaining", H3 "Workflow: Routing", H3 "Workflow: Parallelization", H3 "Workflow: +Orchestrator-workers", H3 "Workflow: Evaluator-optimizer" and H3 "Agents", then H2 "Combining +and customizing these patterns", H2 "Summary" over H3 "Acknowledgements", H2 "Appendix 1: +Agents in practice" over H3 "A. Customer support" and H3 "B. Coding agents", and H2 "Appendix +2: Prompt engineering your tools". "Agents" and "Acknowledgements" are H3s in the live markup +even though they read as top-level. + +**The definitional split**, which P2 borrows: workflows are "systems where LLMs and tools are +orchestrated through predefined code paths", agents are "systems where LLMs dynamically direct +their own processes and tool usage, maintaining control over how they accomplish tasks". + +**The workflow patterns**, each with its own diagram: + +- **The augmented LLM**, the base unit: an LLM with retrieval, tools, and memory, which the + model drives itself (writing its own search queries, picking tools, deciding what to retain). + Guidance: tailor the augmentations, and give the model an easy, well-documented interface. +- **Prompt chaining**: a fixed sequence of LLM calls, each processing the prior output, with + optional programmatic gates between steps. Trades latency for accuracy where a task + decomposes cleanly. +- **Routing**: classify an input, send it down one of several specialized paths, so each path + gets a targeted prompt. Includes routing easy queries to a cheaper model and hard ones to a + larger one. +- **Parallelization**, in two variants: *sectioning* (independent subtasks run concurrently, + then aggregated) and *voting* (the same task run several times, then combined or voted). +- **Orchestrator-workers**: "a central LLM dynamically breaks down tasks, delegates them to + worker LLMs, and synthesizes their results", distinguished from parallelization because the + subtasks are determined at run time from the input rather than fixed in advance. +- **Evaluator-optimizer**: one call generates, a second evaluates and gives feedback, looping + until satisfied. Works where an evaluation criterion is clear and iterative feedback + demonstrably helps. +- **Agents**: what emerges once the model reliably understands complex input, plans, uses + tools, and recovers from errors. The compressed definition, which P2's lineage traces back + to, is that agents are "typically just LLMs using tools based on environmental feedback in a + loop". + +The throughline is minimalism: find the simplest solution possible and increase complexity only +when needed, which "might mean not building agentic systems at all"; for many applications a +single optimized LLM call with retrieval and in-context examples is enough; agentic systems +trade latency and cost for task performance. The Summary states three design principles: +maintain simplicity, prioritize transparency by showing the agent's planning steps, and +carefully craft the agent-computer interface. + +<!-- spellchecker:off --> +**The ACI appendix (Appendix 2)** is the origin of the agent-computer interface term. Its +content: tools deserve as much prompt-engineering attention as the rest of the prompt, because +there are many ways to specify the same action (a diff format versus a full file rewrite) and +some are much harder for a model to produce correctly. Practical guidance is to give the model +enough tokens to think before it commits, keep formats close to what occurs naturally in +internet text, and strip formatting overhead such as making the model count lines. It tells you +to invest in the ACI with the rigor of a human-computer interface: put yourself in the model's +shoes, write parameter names and descriptions as if writing an excellent docstring, test in the +workbench, and apply Poka-yoke mistake-proofing. +<!-- spellchecker:on --> +The cited case is the SWE-bench build, where +the team spent more time optimizing tools than the overall prompt, and fixed recurring +relative-filepath mistakes by requiring absolute filepaths. Appendix 1 names customer support +and coding agents as the two domains where agents demonstrably pay off, because those tasks +combine conversation and action, clear success criteria, feedback loops, and human oversight. + +**Why it matters here** + +This is the definitional ancestor for workflows versus agents and for orchestrator-workers, the +vocabulary this repo's orchestration and planning skills use, and Appendix 2 is the origin of +the tool-interface discipline that T4 later expands. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`, which also carries the tension between the page's +anti-framework stance and the framework list the revision inserted. + +--- + +## 5. T6. How we built our multi-agent research system + +Linked from P2 as its sub-agent architecture evidence base. + +**Citation** + +- Title: "How we built our multi-agent research system" (the `<title>` adds "\ Anthropic") +- URL: `https://www.anthropic.com/engineering/multi-agent-research-system` +- Authors, from the Acknowledgements: Jeremy Hadfield, Barry Zhang, Kenneth Lien, Florian + Scholz, Jeremy Fox, and Daniel Ford. The heading is misspelled on the page, missing its + second `d`. +<!-- spellchecker:off --> + (Verbatim: "Acknowlegements".) +<!-- spellchecker:on --> +- Published: Jun 13, 2025. Fetched 2026-08-31 from raw HTML, re-swept 2026-09-01 with all three + figures viewed. The two passes cross-validate closely and contradict nowhere. + +**What the page contains** + +Heading structure as it exists in the DOM, which is not normalized: every main-body section is +an H3 (Benefits of a multi-agent system; Architecture overview for Research; Prompt engineering +and evaluations for research agents; Effective evaluation of agents; Production reliability and +engineering challenges; Conclusion; the misspelled Acknowledgements heading noted above), and +only the trailing Appendix is an H2. There is no H4 anywhere. + +**The token multipliers and the variance finding**, verbatim: + +``` +agents typically use about 4x more tokens than chat interactions, and multi-agent systems +use about 15x more tokens than chats +``` + +``` +three factors explained 95% of the performance variance in the BrowseComp evaluation (which +tests the ability of browsing agents to locate hard-to-find information). We found that token +usage by itself explains 80% of the variance, with the number of tool calls and the model +choice as the two other explanatory factors. +``` + +The multipliers gate the economics: multi-agent systems require tasks whose value is high +enough to pay for the increased performance. The variance finding is used to validate +distributing work across agents with separate context windows to add capacity for parallel +reasoning. Adjacent claim: upgrading to Claude Sonnet 4 is a larger performance gain than +doubling the token budget on Claude Sonnet 3.7. The headline eval win: a multi-agent system +with Claude Opus 4 as lead and Claude Sonnet 4 subagents outperformed single-agent Claude Opus +4 by 90.2% on an internal research eval. Neither the internal eval nor the BrowseComp +regression is described methodologically, and the 90.2% figure's metric and denominator are +never defined. + +Poor fits are named explicitly: domains requiring all agents to share context, or with many +inter-agent dependencies. Most coding tasks are the named counter-example, having fewer truly +parallelizable subtasks than research, with LLM agents not yet good at real-time delegation to +each other. Good fits are high-value tasks with heavy parallelization, information exceeding a +single context window, and many complex tool interfaces. + +**Architecture**: an orchestrator-worker pattern where a lead agent coordinates and delegates +to parallel subagents, contrasted explicitly with static RAG's fixed similarity-chunk +retrieval. The lead agent persists its plan to Memory because context beyond 200,000 tokens +gets truncated. The figures show the lead agent's tool set as search tools plus MCP tools plus +memory plus `run_subagent` plus `complete_task`, with a separate CitationAgent inserting +citations into the finished report. + +**Eight prompt-engineering principles**, all reproduced verbatim in the fresh sweep: think like +your agents (build step-by-step simulations in Console); teach the orchestrator to delegate +with full task specs (objective, output format, tool and source guidance, explicit boundaries, +with the semiconductor-shortage example showing what duplicated work looks like without them); +scale effort to query complexity with embedded rules (simple: 1 agent, 3-10 calls; comparisons: +2-4 subagents, 10-15 calls each; complex: 10+ subagents with divided responsibilities); treat +tool design and selection as critical, mitigating inconsistent MCP description quality with +explicit heuristics; let agents improve themselves, where a dedicated tool-testing agent's +rewritten description cut future task-completion time by 40%; start wide then narrow; guide the +thinking process using extended thinking as a controllable planning scratchpad and interleaved +thinking for subagents evaluating tool results; and use parallel tool calling, where 3-5 +parallel subagents plus 3+ parallel tool calls each cut research time by up to 90%. + +**The evaluation methodology**, which is the page's most transferable section: + +- Multi-agent systems break the "same input, same expected path" assumption, because valid + trajectories vary. Evaluate outcomes and reasonable process, not prescribed steps. +- Start evaluating immediately with small samples. About 20 queries representing real usage + patterns sufficed early, because early-stage prompt tweaks can swing success rates from 30% + to 80% and effect sizes that large are visible in a handful of cases. The named mistake is + waiting to build hundreds of test cases first. +- LLM-as-judge against a rubric covering factual accuracy, citation accuracy, completeness, + source quality, and tool efficiency. A single LLM call emitting a 0.0 to 1.0 score plus + pass/fail was found most consistent and scales to hundreds of outputs. +- Human evaluation still catches what automation misses. The worked case: testers noticed early + agents favored SEO content farms over authoritative sources such as academic PDFs, fixed by + adding source-quality heuristics. +- Emergent behavior: small lead-agent prompt changes ripple unpredictably into subagent + behavior, so the best prompts are collaboration frameworks (division of labor, + problem-solving approach, effort budgets) rather than rigid instructions. + +**Production engineering**: agents are stateful and errors compound, so the system uses durable +execution resumable from failure with retries and checkpoints rather than expensive restarts; +debugging needs full production tracing of decision patterns without inspecting conversation +contents, for privacy; deployment uses rainbow deployments (gradual traffic shift between +simultaneously running versions) because agents are almost continuously mid-process; and +synchronous subagent execution is named as a current bottleneck, with async execution flagged +as future work. + +**The Appendix** adds three tips, each a bolded lead-in rather than a heading: end-state +evaluation for agents that mutate persistent state, judging the final state rather than tracing +the process, with discrete checkpoints for complex workflows; long-horizon conversation +management, where agents summarize completed phases into external memory, spawn fresh subagents +with clean context while preserving continuity through handoffs, and retrieve stored plans +rather than losing work at the context limit; and subagent output to a filesystem to avoid the +"game of telephone", where subagents persist artifacts externally and pass back lightweight +references instead of routing everything through the lead agent's conversation history. + +**Why it matters here** + +This is the only source in the set with cost data an "is fan-out worth it" argument can cite, +and it carries the spec-every-spawn and effort-scaling lessons this marketplace's orchestration +skills encode, plus an evaluation methodology directly reusable for skill evals. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`, including the tension between the page's own compression claim +and its Appendix's game-of-telephone framing of the same relay step. + +--- + +## 6. T7. Managing context on the Claude Developer Platform + +Linked from P2 as the product surface for its compaction and memory-tool claims. + +**Citation, and the dating caveat** + +- Title: "Managing context on the Claude Developer Platform" (the `<title>` adds "| Claude by + Anthropic"). Dek: "Introducing context editing and the memory tool to help developers build + more effective agents that handle long-running tasks." +- URL as linked: `https://www.anthropic.com/news/context-management`, which 308-redirects to + `https://claude.com/blog/context-management`. Cite the resolved URL. +- Category "Product announcements", product "Claude Platform", reading time 5 min. No named + author. +- Published: September 29, 2025. The page's embedded JSON-LD carries `datePublished` "Sep 29, + 2025" and `dateModified` "Jun 21, 2026", so the page was edited roughly eight to nine months + after publication with no in-body revision note. Fetched 2026-08-31 and re-swept 2026-09-01. + +**Resolved, in favor of the later sweep.** The two passes disagreed over whether the article +embeds any image: the 2026-09-01 sweep reported one diagram, the 2026-08-31 pass reported a +targeted `<img>` and `<figure>` search returning zero matches and concluded the page was +text-only. Settled 2026-09-01 by direct inspection of the retained HTML and of the image itself, +which is the evidence neither pass had: the page carries one content diagram at +`cdn.prod.website-files.com/.../8ad2952bc0513750088cdfd309ee83ba0fd15438-1920x800.webp`, titled +"Before context editing" over "After context editing". It shows a context window as a strip of +alternating tool-use and tool-result blocks; in the "after" strip the earliest pairs are +compressed to a fraction of their width and the reclaimed span is filled by a single green +"Available Context" block. The earlier pass's negative result came from searching a byline-to- +related-posts slice that excludes the figure's position. Treat a bounded-slice absence search as +evidence about the slice, never about the page. + +**What the page contains** + +Heading structure, four H2s: "Context windows have limits, but real work doesn't"; "Building +long-running agents"; "Performance improvements with context management"; "Getting started". + +**Scope: this is API-only.** Everything described is a Claude Developer Platform primitive that +a calling application wires up. Context editing is an API parameter. The memory tool is +client-side: Claude emits tool calls, and the developer's own code hosts the file backend in +their infrastructure. Claude Code is not named anywhere on the page. Do not read this +announcement as evidence that any harness, Claude Code included, does context editing natively; +it substantiates only that the API-level building block exists. Availability at publication: +public beta, natively on the Claude Developer Platform and also through Amazon Bedrock and +Google Cloud Vertex AI. + +**The two mechanisms**, verbatim on scope and trigger: + +``` +Context editing automatically clears stale tool calls and results from within the context +window when approaching token limits. As your agent executes tasks and accumulates tool +results, context editing removes stale content while preserving the conversation flow, +effectively extending how long agents can run without manual intervention. +``` + +``` +The memory tool enables Claude to store and consult information outside the context window +through a file-based system. Claude can create, read, update, and delete files in a dedicated +memory directory stored in your infrastructure that persists across conversations. +``` + +What context editing clears, as literally stated, is tool calls and tool results only, never +user or assistant text turns. The trigger is "when approaching token limits", with no threshold +given. Claude Sonnet 4.5 is credited with built-in context awareness, tracking its own +remaining tokens. Three use cases pair the two mechanisms: coding (editing clears old file +reads and test results, memory keeps debugging insights and architectural decisions), research +(memory stores findings, editing drops old search results), and data processing (memory holds +intermediate results, editing clears raw data). + +**The three figures, with their eval caveats:** + +``` +On an internal evaluation set for agentic search, we tested how context management improves +agent performance on complex, multi-step tasks. The results demonstrate significant gains: +combining the memory tool with context editing improved performance by 39% over baseline. +Context editing alone delivered a 29% improvement. +``` + +``` +In a 100-turn web search evaluation, context editing enabled agents to complete workflows +that would otherwise fail due to context exhaustion - while reducing token consumption by 84%. +``` + +Caveats that must travel with those numbers: both evals are internal and named only +descriptively, with no linked benchmark, no methodology, no sample size, no metric definition, +and no stated model. The word "baseline" is never operationally defined. No memory-tool-alone +figure is given, so the reader cannot attribute the gap between 29% and 39% to memory. The 84% +figure carries no absolute token counts and no statement that output quality was held constant. +The relationship between the "internal evaluation set for agentic search" and the "100-turn web +search evaluation" is never stated. + +**Why it matters here** + +This is the primary source for the API-versus-harness boundary that this repo's context-budget +and memory guidance must not blur, and it is the citable anchor for tool-result clearing as the +lowest-risk context-reduction lever. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. + +--- + +## 7. T8. Memory and context management with Claude Sonnet 4.6 (cookbook) + +Linked from P2 as the hands-on cookbook it closes with. + +**Citation** + +- Title: "Memory & context management with Claude Sonnet 4.6", rendered on the page in title + case as "Memory & Context Management with Claude Sonnet 4.6 | Claude Cookbook". Normalize + before quoting it as the exact title. +- URL: `https://platform.claude.com/cookbook/tool-use-memory-cookbook`. The `.md` twin at that + path returns 404; this docs site does not serve one. +- Author: Alex Notov (@zealoushacker). Published May 22, 2025. Notebook source: + `anthropics/claude-cookbooks/blob/main/tool_use/memory_cookbook.ipynb`. Categories: Tools, + Agent Patterns. +- Fetched 2026-08-31, re-swept 2026-09-01. The byline and date above come from the earlier + pass; the fresh sweep dropped them. + +**What the page contains** + +This is a notebook, not a prose doc. Section outline: Introduction: Why Memory Matters; +Prerequisites & Setup; Quick Start Examples (Setup Code, Example 1 Basic Memory Usage, Example +2 Cross-Conversation Learning, Example 3 Context Clearing While Preserving Memory); How It +Works (Memory Tool Architecture, Thinking Management, Understanding the Demo Code, What Claude +Actually Learns, Why This Matters); Use Cases; Best Practices & Security (Memory Management, +Path Traversal, Memory Poisoning); Real-World Applications; Sample Code Files; Helper +Functions; Conclusion & Next Steps. + +The running scenario is one Code Review Assistant across three sessions: it diagnoses a race +condition in a threaded `WebScraper` and writes the pattern to `/memories/review.md`; a new +conversation reviews an unrelated async `AsyncAPIClient` and recognizes the same pattern by +reading memory first rather than re-deriving it; then a long multi-file session runs both +clearing strategies while the on-disk memory stays intact. + +**The tool primitives.** The memory tool is client-side. Claude emits tool calls against a +`/memories` path space and the calling application executes them against real storage. Six +commands, each with an example payload: + +``` +view {"command": "view", "path": "/memories"} +create {"command": "create", "path": "/memories/notes.md", "file_text": "..."} +str_replace {"command": "str_replace", "path": "...", "old_str": "...", "new_str": "..."} +insert {"command": "insert", "path": "...", "insert_line": 2, "insert_text": "..."} +delete {"command": "delete", "path": "/memories/old.txt"} +rename {"command": "rename", "old_path": "...", "new_path": "..."} +``` + +The implementation, including path validation, is deferred to a `memory_tool.py` the page never +reproduces. + +**The combined API call shape**, as given for thinking plus tool-use context management: + +``` +client.beta.messages.create( + betas=["context-management-2025-06-27"], + model="claude-sonnet-4-6", + tools=[{"type": "memory_20250818", "name": "memory"}], + thinking={"type": "enabled", "budget_tokens": 10000}, + context_management={"edits": [ + {"type": "clear_thinking_20251015", "keep": {"type": "thinking_turns", "value": 1}}, + {"type": "clear_tool_uses_20250919", + "trigger": {"type": "input_tokens", "value": 35000}, + "keep": {"type": "tool_uses", "value": 5}} + ]}, + max_tokens=2048) +``` + +The beta header `context-management-2025-06-27` is required for both the memory tool and +context editing. `clear_thinking_20251015` requires extended thinking enabled in the same call +and must be listed first when combined with tool-use clearing; its `trigger` field is optional, +so clearing can be driven by `keep` alone, and `"keep": "all"` preserves every thinking block +for maximum cache hits. Example 3's demo values (trigger 5,000, keep 2, `budget_tokens` 1024) +are deliberately small; the page's production recommendation elsewhere is 30,000 to 40,000 +input tokens. + +**The token-threshold trace** from Example 3, the page's only numeric evidence: Review 1 +(`data_processor_v1.py`) at 6,611 input tokens reports no clearing triggered; Review 2 +(`sql_query_builder.py`) at 7,923 input tokens clears one thinking turn, saving 166 tokens; +Review 3 (`web_scraper_v1.py`) at 9,052 input tokens clears two thinking turns, saving 265 +tokens. A post-hoc cell then walks the demo memory directory and shows `memories/review.md` +still present at 318 bytes, which is the point being demonstrated: context editing clears the +conversation's transient history, never the durable memory filesystem. Note that Review 1's +6,611 tokens already exceed the demo's own 5,000-token trigger while the transcript says no +clearing fired, so the reported behavior is self-contradictory on a literal read. + +**The memory-poisoning threat list.** Two named security threats. *Path traversal* is one +sentence, "Always validate paths to prevent directory traversal attacks", with implementation +deferred to `memory_tool.py`. *Memory poisoning* is labelled a Critical Risk: memory files are +read back into Claude's context on later turns, so anything written into one is a +prompt-injection vector. Four named mitigations: content sanitization (filter dangerous +patterns before storing), memory scope isolation (per user, per project), memory auditing (log +and scan all memory operations), and prompt engineering (instruct Claude to ignore instructions +found inside memory content). Memory hygiene adds a do list (store task-relevant patterns not +conversation history, clear directory structure, descriptive filenames, periodic cleanup) and a +don't list (no secrets or PII, no unbounded growth, nothing indiscriminate). + +**The stale supported-model lists.** The page gives two incompatible "Supported Models" lists +for what appears to be the same feature. Section 1 lists five: Opus 4.1 (`claude-opus-4-1`), +Opus 4 (`claude-opus-4`), Sonnet 4.6 (`claude-sonnet-4-6`), Sonnet 4 (`claude-sonnet-4`), and +Haiku 4.5 (`claude-haiku-4-5`). The Use Cases and Real-World Applications sections each repeat +a narrower two-model list, Opus 4.1 and Sonnet 4.6, with identical wording. The page never +explains whether the narrow list is a smaller capability subset or simply stale copy, and it +carries no "check the docs for the current list" caveat. Treat both lists as stale and verify +against current model docs before citing either. + +Closing production guidance: start with a single `/memories/patterns.md`, set context-editing +triggers at 30,000 to 40,000 tokens for production, isolate memory per project. The page states +plainly that memory and context management are in beta. + +**Why it matters here** + +This is the worked example of the build-it-yourself side of the API-versus-harness axis, and +its memory-poisoning threat list is the citable basis for the security caveats this repo owes +any durable-memory or note-writing guidance. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. + +--- + +## 8. T9. The prompt-engineering docs pair + +Linked from P2 as the prompt-engineering docs it positions itself against. This is two pages, +not one. The earlier structural pass covered only the first; both were swept 2026-09-01, and +the second contributes the overwhelming majority of the technique content. + +**Citation, page 1 (the router)** + +- Title: "Prompt engineering overview". Frontmatter description: "Learn when prompt engineering + is the right solution, and find Claude prompting techniques and interactive tutorials." +- URL as linked: + `https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview`, which + 301s then 307s to the canonical + `https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview`. Cite the + canonical form. +- No named author, no publication date (docs page). Fetched 2026-08-31 and 2026-09-01 through + the `.md` channel. + +**Citation, page 2 (the living reference)** + +- Title: "Prompting best practices" +- URL: + `https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices` +- No named author, no publication date. Fetched 2026-09-01 through the `.md` channel. + +**What page 1 contains** + +Four sections and no technique content at all: "Before prompt engineering" (assumes you already +have success criteria, an empirical test method, and a first-draft prompt; otherwise go build +evals first; links a Colab metaprompt notebook), "When to prompt engineer" (one sentence, +scoping the guide to criteria controllable through prompting and stating that switching models +can be the easier lever for latency or cost), "How to prompt engineer" (delegates everything to +page 2, which it calls "the living reference"), and "Prompt engineering tutorial" (two external +interactive tutorials). The classic ordered technique list has moved off this URL entirely. The +page never uses the term "context engineering", which is a verifiable absence worth recording +given that P2 cites this page. + +**What page 2 contains** + +The page names its audience models (Fable 5, Mythos 5, Opus 5, Opus 4.8, 4.7, 4.6, Sonnet 5, +Sonnet 4.6, Haiku 4.5) and organizes into model-specific guidance, techniques for all current +models, and migration considerations. Four model-specific sections (Fable 5, Sonnet 5, Opus 5, +Opus 4.8) are stubs that delegate to dedicated pages. + +General principles: be clear and direct (the "brilliant but new employee" frame, and the golden +rule that a colleague with minimal context who would be confused signals a prompt Claude will +also find confusing); add context by explaining why an instruction matters, not just the +instruction; use examples effectively, named as few-shot or multishot, with 3 to 5 recommended +and an `<example>` / `<examples>` tag convention; structure prompts with XML tags, consistent +and descriptive, nested where content has natural hierarchy; give Claude a role via the system +prompt; long-context prompting for 20k+ token inputs, putting longform data at the top above +the query, with a reported quality gain of up to 30% from query-at-end, `<document>` / +`<document_content>` / `<source>` wrapping, and a ground-in-quotes-first pattern; model +self-knowledge; communication style and verbosity; format control (say what to do rather than +what not to do, XML format-indicator tags, match prompt style to desired output style, and a +detailed anti-markdown block); LaTeX output; and document creation. + +Mechanism changes worth carrying: + +- **Prefill deprecation.** Prefill on the last assistant turn is unsupported starting with + Claude 4.6 models and Mythos Preview, returning a 400 error. Prefill on earlier turns and on + pre-4.6 models is unaffected. Five migration sub-sections cover the former use cases: output + formatting (move to Structured Outputs, or just ask, or tools plus enum for classification), + eliminating preambles, avoiding bad refusals, continuations, and context hydration and role + consistency (inject reminders in the user turn; for agentic systems hydrate via tools or + during compaction). +- **Adaptive thinking replaces `budget_tokens`.** `thinking: {type: "adaptive"}` on Claude 4.6+ + and Mythos Preview lets the model decide when and how much to think, driven by `effort` and + query complexity. A per-model default table follows: Opus 4.6 through 4.8 and Sonnet 4.6 + default off unless set; Opus 5 and Sonnet 5 default on, with Opus 5 disable-able only at + effort <= high; Fable 5 and Mythos 5 are always on. Manual `budget_tokens` extended thinking + is deprecated, still functional on Opus 4.6 and Sonnet 4.6 but returning a 400 error on + Claude 4.7+. A migration code sample spans nine languages. Manual `<thinking>` / `<answer>` + chain-of-thought is explicitly demoted to a fallback for when thinking is off. +- **The Opus 5 self-check reversal.** The page states as general good practice that you should + ask Claude to self-check its work against test criteria, then immediately reverses it in the + same paragraph: Opus 5 already self-verifies, and the old verification instructions should be + **removed rather than rewritten**, because keeping them causes over-verification with a real + token and latency cost. This is the sharpest instance of a pattern running through the whole + page, where general advice is reversed for specific model generations. A related note flags + Opus 4.5 as over-sensitive to the literal word "think" when thinking is disabled, + recommending synonyms such as consider, evaluate, or reason through. + +Agentic-systems content: context awareness is named as a capability present in Sonnet 5, Sonnet +4.6, Sonnet 4.5, and Haiku 4.5 (the list does not name Opus models), letting the model track +its remaining token budget, and the page recommends telling Claude explicitly that compaction +or external file saving is available in the harness or it may wrap up work prematurely near the +limit. Multi-window workflows get six numbered practices: a different first-window prompt that +sets up scaffolding versus later windows that iterate a todo list; a structured test file such +as `tests.json` with an explicit never-remove-or-edit-tests instruction; setup scripts such as +`init.sh` to avoid repeated rediscovery; preferring fresh context plus filesystem rediscovery +over compaction in some cases, with a prescriptive bootstrap; providing verification tools for +autonomous correctness checking; and an explicit "use your entire context budget, do not stop +early" instruction. State management recommends JSON for state data, freeform text for progress +notes, and git itself as a checkpoint log. Further sections cover balancing autonomy and safety +(confirm before destructive, hard-to-reverse, or externally visible actions, with an explicit +anti-shortcut clause against `--no-verify`), research methodology (define success criteria, +cross-source verification, structured hypothesis tracking with confidence levels), subagent +orchestration (latest models delegate proactively; Opus 4.6 has a strong predilection for +spawning subagents where a direct grep would be faster, and Opus 5 also delegates more readily, +so a damping prompt is given), and prompt chaining, notably short because adaptive thinking and +subagent orchestration now absorb most multistep reasoning. + +Coding-specific tips: tool usage needs explicit direction to act, since "can you suggest +changes" may only get suggestions, and aggressive urgency language now risks over-triggering; +parallel tool calling is on by default and steerable in both directions; overthinking and +excessive thoroughness on Opus 4.6; reduced file creation; overeagerness and overengineering on +Opus 4.5 and 4.6; avoiding test hardcoding and helper-script workarounds; minimizing +hallucinations by requiring files be read before answering; vision improvements with a +crop-tool recommendation; and frontend design, with a detailed aesthetics block that names +specific cliches to avoid. + +**Why it matters here** + +This pair is the current first-party technique reference, and its prefill deprecation, adaptive +thinking migration, and per-model reversals are exactly the kind of drift that this repo's +instruction surfaces must be re-checked against; the Opus 5 self-check reversal is a concrete +worked case of an instruction that should be deleted rather than rewritten. + +**Where the deep apparatus lives** + +`context-engineering-critical-apparatus.md`. diff --git a/docs/specs/context-engineering-vertical-decisions.md b/docs/specs/context-engineering-vertical-decisions.md new file mode 100644 index 000000000..071de35ae --- /dev/null +++ b/docs/specs/context-engineering-vertical-decisions.md @@ -0,0 +1,166 @@ +# Vertical decisions V2 to V7, resolved in-session + +The corpus work was organized into seven verticals. V1 (unhobbling) was resolved by the signed-off +answer set in the topic's decision contract, which is contract tier and therefore pruned before +merge, surviving as the `<details>` paste in its pull request; the one V1 answer whose disposition +had no other durable home is restated at the end of this document. The remaining six were +originally routed to "future interview rounds", which was a mistake: the evidence those rounds +would have consumed lived only in an untracked memory tier, so a future session would have +inherited the questions without the answers. They are resolved here instead, on the evidence +gathered while it was in hand. + +Each entry states the decision, its basis, and what actually changes. Several resolve to "no change, +and here is why", which is a real outcome for a corpus pass over a repo that already does most of +this. + +## V2. Progressive disclosure + +**Decision: no new mechanism; the corpus's rationale is adopted as citation, not as a rule.** + +The repo already implements the pattern the sources advocate. `AGENTS.md` carries a load-on-demand +rules table with path triggers, skills carry their own progressive disclosure through reference +subdirectories, and `context-budget:audit` and `docs-hygiene:audit-progressive-disclosure` already +own the measurement. What the corpus adds is a *why* worth citing when those surfaces are +questioned: the attention-budget and context-rot mechanism, and the guiding principle that the +target is the smallest set of high-signal tokens rather than the shortest document. Both are +recorded in [`context-engineering-corpus-knowledge.md`](context-engineering-corpus-knowledge.md) with their sources. + +One caveat the sources do not supply and this repo already knows: its own rules table documents that +the path trigger does not fire inside subagents and can lapse after a compaction. Progressive +disclosure is therefore a budget mechanism here, never a guarantee of delivery, and the corpus's +enthusiasm does not change that. + +**Changes: none to any skill.** The rationale lands in the graduated corpus docs. + +## V3. Interface design over examples + +**Decision: adopt for tool and parameter surfaces; explicitly do not adopt for teaching skills.** + +The newer source's claim is that worked examples constrain a capable model's exploration space, and +that expressive parameters (an enum, one behavioral constraint line) teach usage better than a +worked example does. The older source from the same publisher advises the opposite, strongly, for +few-shot prompting. That reversal is real and generational, not a contradiction to resolve by +picking a winner. See [`context-engineering-critical-apparatus.md`](context-engineering-critical-apparatus.md), cross-source tensions. + +Resolution for this repo: + +- **Tool and parameter surfaces**: prefer an expressive interface over a worked example. This is + already the house pattern (enums, one-line constraints in tool descriptions); no edit needed. +- **Teaching skills** (`education:*`, `songwriting:*`, the methodology skills): examples are the + content, not scaffolding around it. The reversal does not reach them, and a future trimming pass + citing the corpus at them is misapplying it. Recorded here so that misapplication has a written + answer. +- **Skill bodies generally**: a worked example that demonstrates a *format the skill requires* + stays. One that demonstrates *how to think about the task* is a trimming candidate on the normal + criteria, not because the corpus says so. + +**Changes: none.** The boundary is the deliverable. + +## V4. Placement: which surface an instruction lives on + +**Decision: the settled facts are recorded; no placement doctrine is minted.** + +The corpus's per-surface guidance (system prompt, CLAUDE.md, skills, references, memory) maps onto +machinery this repo already has in `instruction-placement:*`, whose Gate 0 and routing rubric are +more specific than the article's prose. Two upstream facts do change what the repo may say, and both +are now recorded in [`context-engineering-corpus-knowledge.md`](context-engineering-corpus-knowledge.md): + +- The `#` memory hotkey was **removed** (Claude Code changelog v2.0.70), not merely de-emphasized. + Any repo text implying it exists is wrong. +- The Developer Platform memory tool and context editing are **not exposed natively by Claude Code** + (its analogues are auto-memory and compaction). A placement doctrine citing "the memory tool" as + a harness surface would be false. + +The article figure's six-layer stack (prompt, references, system prompt, CLAUDE.md, skills, memory) +is carried in the corpus doc as figure-borne, with the note that the article's own body never +covers the memory layer. + +**Changes: none to `instruction-placement`.** The register from Q2 is the one placement-adjacent +artifact this integration adds, and it lands as a convention rather than as doctrine. + +## V5. Rich references over simple specs + +**Decision: adopted as practice already in force; one gap named and left open deliberately.** + +The claim is that a spec is better expressed as code, a test suite, an HTML artifact, or a rubric +than as prose. This repo's planning pipeline already does the strong form of this: PLAN.md briefs +carry acceptance criteria, the verification plugins carry rubrics, and the dynamic-workflows +capability (documented upstream, see [`context-engineering-linked-sources.md`](context-engineering-linked-sources.md)) supplies the +verifier-agent pattern the source points at. + +The gap: this repo has no convention for *mockups or visual references* as specs, because it ships +no UI. The source's strongest example (an HTML mockup beats a description or a screenshot) has no +consumer here. Named rather than invented. + +**Changes: none.** Adoption is a no-op because the practice predates the corpus. + +## V6. Long-horizon techniques + +**Decision: annotate the existing orchestration guidance; do not mint a notes-file convention.** + +- **Condensed returns**: `session-flow:orchestrate` already required compressed verdicts. It now + carries the upstream magnitude (roughly 1,000 to 2,000 tokens returned from an exploration that + may span tens of thousands) and the reconciled token-multiplier figures, since the repo's own + 3-10x line and the upstream ~15x measurement were in tension. Shipped in this change. +- **Structured note-taking**: the sources recommend a persistent notes file whose test is + reset-survival. This repo already implements the pattern under different names (checklists, + handoff artifacts, per-skill ledgers, the topic memory slice), and a new cross-cutting convention + would collide with the topic-docs redesign already locked and in flight. **Not adopted**, and the + reason is sequencing, not disagreement. +- **Compaction**: harness-owned. The corpus's tuning guidance (maximize recall first, then trim for + precision; clear tool results before summarizing) applies to anyone *building* an agent loop, not + to this repo's consumers. Recorded, not adopted. + +**Changes: the orchestrate annotation, shipped here.** + +## V7. Corpus meta and custody + +**Decision: graduate the knowledge now; keep the raw slices untracked; record the custody findings +as a convention-level input rather than a new convention.** + +The original answer deferred graduation to a later pass. That was wrong for the same reason the +whole deferral posture was wrong: the memory tier does not survive. Resolved as: + +- **Graduated** into `docs/specs/` through the knowledge-vault seam: the corpus knowledge base, + the critical apparatus, the linked sources, and this decisions record. These are the durable + artifacts. +- **Not graduated**: the byte-verified digest slices, pin manifests, verification verdicts, and + reconciliation tables. They are process evidence for a run that has already been adjudicated, and + committing 800K of them would trade a real cost for no reader's benefit. Their *conclusions* are + in the graduated docs; the fact that they existed and what they proved is recorded in + [`context-engineering-corpus-knowledge.md`](context-engineering-corpus-knowledge.md). +- **Custody findings** (silent page revision, figure-only evidence, provenance stripping through + text extraction, link defects, contradictory vendor lists, the documented generational reversal in + official docs) are recorded in the corpus doc, and the one with a mechanism behind it, the silent + revision, is filed against the `upstream-drift` convention's deferred content-hashing decision as + near-miss evidence rather than as a new convention. + +**Changes: the four graduated `docs/specs/context-engineering-*.md` documents, plus the +upstream-drift changelog entry shipped separately.** + +## V1 residue: re-testing instructions after a model upgrade + +**Decision: a documented trigger only; no new re-test mechanism.** + +Every other V1 answer landed somewhere durable — the deletion threshold became the exception +register and its attribution design, the conflict-coverage reopen became a tracker item, and the +`/doctor` and 80%-figure findings are recorded as settled facts and tier posture in +[`context-engineering-corpus-knowledge.md`](context-engineering-corpus-knowledge.md). One did not, +so it is restated here rather than left to the pruned contract alone. + +The corpus argues that instructions written against one model generation can become dead weight, +or actively wrong, against the next, and that a model upgrade is therefore an occasion to re-test +what a repo's instruction surfaces are still buying. The question was whether this repo needs a +new mechanism to act on that. + +It does not. `claude-config`'s `audit-pass` already ships the ritual: a re-run contract with a +lease and epoch, finding suppression that survives across runs, and a three-scope inventory. A +model upgrade is a reason to invoke it, not a reason to build a second thing beside it. What the +corpus adds is the documented trigger, and the caution that a re-run after an upgrade should +expect *removals* to be the finding, which is the opposite of the drift most audits look for. + +Cite `audit-pass`'s shipped reference files when pointing at that contract. The design note the +original answer named was already drifting from what shipped, and it has since been pruned along +with its slice, so it is not a citable surface. + +**Changes: none.** The mechanism exists; only the occasion to run it was unrecorded. diff --git a/plugins/claude-config/.claude-plugin/plugin.json b/plugins/claude-config/.claude-plugin/plugin.json index d4a7b99a2..d3a543fe8 100644 --- a/plugins/claude-config/.claude-plugin/plugin.json +++ b/plugins/claude-config/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "claude-config", - "version": "0.40.23", + "version": "0.40.24", "description": "Nine configuration-health skills (plus setup) for a repo's Claude Code configuration: audit (settings.json / .mcp.json / hooks / plugins / permissions drift), audit-automation-gaps (evidence-gated verdicts on automation gaps), audit-permission-grants (allow-rule / allowed-tools grants for auto-mode durability and portability), audit-permission-state (the permission rules actually in effect — every settings scope merged with per-rule provenance, what auto mode drops on entry, config written where nothing reads it, and which managed intents are enforced versus loosenable), draft-auto-mode-rules (interview and draft a paste-ready autoMode classifier block; prints only, never writes), audit-instructions (locally-owned instruction surfaces vs current model capability — proposes removals/rewrites of instructions the model no longer needs, and detects cross-surface instruction conflicts), audit-prompting-postures (the additive lane — posture guidance the prompting guide says a component's purpose needs but the component does not carry), audit-pass (one coordinated, ordered, resumable pass over a named target — three-scope inventory, run-time-derived exclusion set, stable finding identity, suppression memory, resume, one human gate — delegating every check to the plugin that owns it), and unhobble (the empirical bare-baseline experiment: reversibly strip a repo's standing instructions, log real stumbles against the current model, re-add only what evidence earns).", "author": { "name": "Melodic Software", diff --git a/plugins/claude-config/CHANGELOG.md b/plugins/claude-config/CHANGELOG.md index fc8a72edc..ee6cc9700 100644 --- a/plugins/claude-config/CHANGELOG.md +++ b/plugins/claude-config/CHANGELOG.md @@ -3,6 +3,32 @@ All notable changes to the `claude-config` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.40.24] + +### Changed + +- **`unhobble`: Phase 4 no longer leaves a protected rule deleted on silence.** The readd phase + ended with "everything the ledger did not defend stays deleted", which is sound for ordinary + instructions and wrong for a rail whose absence is unrecoverable: such a rail will not usually + announce itself inside one experiment window, so "no stumble was observed" is the weakest + evidence available against it. Phase 4 now restores a rule matching a protected class in the + marketplace's instruction exception register whether or not the ledger logged against it. The + strip itself stays permitted — it is reversible and branch-local, which is why the experiment may + run over a protected rail at all. Register holds are recorded separately from the defence tally, + so restoring one is not counted as a deletion the ledger defeated. +- **`audit-instructions`: I1, I4 and I5 gain a hold verdict for protected instruction classes.** + All three deletion-class criteria remediated to `delete` with no stated exception, which left the + "except in highly important areas" carve-out in the source guidance undefined at the point of + use — the audit could cut a security rail on the strength of "the model already does this", the + weakest evidence available against a rule whose absence is unrecoverable. Each criterion now + names the marketplace's instruction exception register, reports a matching candidate as a hold + with its class, and proposes compression in place. I1 carries the additional reason its own bar + cannot see the problem: it asks whether removal would change behavior today, and a protected + rail's removal changes behavior only on the occasion it was written for. The register is + non-exhaustive and tighten-only, so a candidate absent from it is still judged on the criterion's + normal terms and never deleted *because* it is absent. I5's hook conversion stays available: + converting a protected rule to a deterministic mechanism is a remediation, deleting it is not. + ## [0.40.23] ### Changed diff --git a/plugins/claude-config/skills/audit-instructions/reference/criteria.md b/plugins/claude-config/skills/audit-instructions/reference/criteria.md index 862bad16b..9a1549a4a 100644 --- a/plugins/claude-config/skills/audit-instructions/reference/criteria.md +++ b/plugins/claude-config/skills/audit-instructions/reference/criteria.md @@ -212,6 +212,11 @@ Tier `mechanical` · Authority `ANTHROPIC-DOCS` · Severity `warning` · Surface - **Detect:** a line whose removal would not change behavior — restates a default, a truism, or something the model already does correctly. - **Remediate:** cut it, or (if it enforces something) convert per I5. +- **Hold instead of delete** when the candidate matches a protected class in the + [instruction exception register](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/instruction-exception-register/README.md), + on I5's terms. This bar asks whether removal would change behavior *today*; a protected rail's + removal changes behavior only on the occasion it was written for, which this criterion cannot + observe. - **Source:** best-practices — "For each line, ask: *Would removing this cause Claude to make mistakes?* If not, cut it." @@ -288,6 +293,12 @@ Tier `mechanical` · Authority `ANTHROPIC-DOCS` · Severity `warning` · Surface - **Detect:** content the model can derive from the code, standard language conventions it already knows, inlined API docs that should be a link, or self-evident practices. - **Remediate:** delete; link to the source of truth instead of inlining it. +- **Hold instead of delete** when the candidate matches a protected class in the + [instruction exception register](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/instruction-exception-register/README.md) + (the Gate 0 consequence classes, adopted there by reference for the deletion operation). Report + the hold and its class; propose compression in place instead. The register is non-exhaustive, so + a candidate absent from it is judged on this criterion's normal terms, never deleted *because* it + is absent. - **Source:** best-practices include/exclude table — exclude "Anything Claude can figure out by reading code" and "Standard language conventions Claude already knows." @@ -299,6 +310,11 @@ Tier `mechanical` · Authority `ANTHROPIC-DOCS` · Severity `info` · Surfaces: zero exceptions. - **Remediate:** delete the already-followed rule; convert the must-always rule to a hook, which is deterministic where an instruction is only advisory. +- **Hold instead of delete** when the candidate matches a protected class in the + [instruction exception register](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/instruction-exception-register/README.md). + "The model already does this" is the weakest possible evidence against a rail whose absence is + unrecoverable, and the hook conversion stays available: converting a protected rule to a + deterministic mechanism is a remediation, deleting it is not. - **Source:** best-practices — "If Claude already does something correctly without the instruction, delete it or convert it to a hook." diff --git a/plugins/claude-config/skills/unhobble/SKILL.md b/plugins/claude-config/skills/unhobble/SKILL.md index 39d29e772..d0f997731 100644 --- a/plugins/claude-config/skills/unhobble/SKILL.md +++ b/plugins/claude-config/skills/unhobble/SKILL.md @@ -163,9 +163,20 @@ rows after real work is a licensed permanent deletion. the restoring commit or an adjacent comment. 3. For instructions being rewritten rather than restored verbatim, route the text-level judgment to `audit-instructions` (same plugin), which owns instruction-content-vs-doctrine analysis. -4. Everything the ledger did not defend stays deleted. Close the experiment: final manifest update - (`phase: closed`, surfaces restored vs retired counts), and merge or fold the experiment branch - per the repo's normal PR flow. +4. Everything the ledger did not defend stays deleted — **except a rule matching a protected class + in the [instruction exception + register](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/instruction-exception-register/README.md)**, + which is restored regardless of whether the ledger logged a stumble against it. The strip itself + is fine: it is reversible and branch-local, which is why the experiment may run over a protected + rail at all. What the register forbids is leaving one deleted on the evidence of silence. A rail + whose absence is unrecoverable will not usually announce itself inside one experiment window; + "no stumble was observed" is the weakest evidence available against it, and the register exists + because that inference is the one this phase would otherwise make. Restoring a protected rule + this way is not a failed deletion, so do not count it as a retained surface in the ledger's + defence tally; record it as a register hold with its class. +5. Close the experiment: final manifest update (`phase: closed`, surfaces restored vs retired + counts, register holds listed separately), and merge or fold the experiment branch per the + repo's normal PR flow. ## Cadence wiring (optional) diff --git a/plugins/instruction-placement/.claude-plugin/plugin.json b/plugins/instruction-placement/.claude-plugin/plugin.json index e33efbeb6..878456468 100644 --- a/plugins/instruction-placement/.claude-plugin/plugin.json +++ b/plugins/instruction-placement/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "instruction-placement", - "version": "0.11.15", + "version": "0.11.16", "description": "Routes agent-instruction content to the surface that loads it at the right moment. The audit skill sweeps a repository's instruction layer and its ordinary markdown for content whose scope is narrower than the surface carrying it \u2014 conventions keyed to one file type or one subtree sitting in an always-loaded CLAUDE.md or AGENTS.md \u2014 and for normative conventions stranded in documentation Claude never loads at all, then classifies each against a routing rubric and proposes a destination whose `paths:` glob is machine-validated before it is ever offered. Safety-class content (irreversible actions, secrets, data integrity, external publication, compliance, agent authority) is hard-denied from demotion and reported as held back rather than proposed, because demotion trades guaranteed presence for conditional presence and deferred surfaces are invisible inside subagents and absent after compaction until re-triggered. Every accepted move regenerates an always-loaded index of deferred surfaces, which is what keeps a demoted rule reachable from a subagent that never receives its injection. The audit is read-only and emits a diffable findings artifact; realignment is a separate skill gated per item with no blanket-approve path; a deterministic check skill gates that every rule glob still resolves and the index is current; and a setup skill verifies the one thing no other gate can see \u2014 that the index target is a file Claude Code will actually read, since it reads CLAUDE.md and not AGENTS.md.", "author": { "name": "Melodic Software", diff --git a/plugins/instruction-placement/CHANGELOG.md b/plugins/instruction-placement/CHANGELOG.md index e01aeddaa..9f9fbfaad 100644 --- a/plugins/instruction-placement/CHANGELOG.md +++ b/plugins/instruction-placement/CHANGELOG.md @@ -3,6 +3,18 @@ All notable changes to the `instruction-placement` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.11.16] + +### Changed + +- **`routing-rubric.md`: Gate 0 names its deletion counterpart.** The six hard-deny classes now + govern two operations across two owners — this rubric decides relocation, and the marketplace's + instruction exception register adopts the same classes by reference to decide deletion. Gate 0 + gained a note saying so, and saying which question routes where, so a consumer asking "may this + be deleted" does not read a relocation verdict as an answer. The class list is not re-enumerated + anywhere else; this rubric stays its sole owner, which is what keeps one concern on one + adjudication chain instead of two lists that drift apart. + ## [0.11.15] ### Changed diff --git a/plugins/instruction-placement/context/routing-rubric.md b/plugins/instruction-placement/context/routing-rubric.md index aaad7495b..d5a5c8c41 100644 --- a/plugins/instruction-placement/context/routing-rubric.md +++ b/plugins/instruction-placement/context/routing-rubric.md @@ -58,6 +58,14 @@ When a hard-deny candidate is genuinely bloating an always-loaded file, the hone compression in place — tighten the wording, cut what is derivable — not relocation. Say that rather than proposing nothing. +**The same classes bind the deletion operation, owned elsewhere.** These six classes are adopted by +reference (never re-enumerated) by the [instruction exception +register](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/instruction-exception-register/README.md), +which applies them to *deletion* by an instruction-audit trim, the operation this gate does not +govern. This rubric stays the sole owner of the class list and of the relocation verdict; a +consumer asking "may this be deleted" reads that register, and one asking "may this be moved" reads +this gate. + ## The decision ladder First match wins. Stop at the first rung that fits; do not shop for a better-sounding destination diff --git a/plugins/session-flow/.claude-plugin/plugin.json b/plugins/session-flow/.claude-plugin/plugin.json index 1ce87ca8f..f520ec3ab 100644 --- a/plugins/session-flow/.claude-plugin/plugin.json +++ b/plugins/session-flow/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "session-flow", - "version": "0.34.16", + "version": "0.34.17", "description": "Session-lifecycle toolkit of fourteen skills: workflow (navigate a staged dev workflow and suggest the next stage), handoff (write a save-point and resume prompt for /clear-and-resume), continue-in-background (delegate the task to a fresh background agent that continues it now \u2014 same save-point engine as handoff, delivered by launching a detached claude --bg session seeded with the resume prompt; launches only on explicit user request), keep-going (recover and continue after any interruption OR when live off-thread work looks stalled \u2014 inventory off-thread work, inspect its real output, act only on evidence, then continue; after a usage limit lifts it continues rather than summarizing-and-stalling), find-handoff (recover a lost handoff after /clear \u2014 when the resume prompt was written but never copied \u2014 via a read-only detection ladder: known-location glob of the handoffs dir, then a bounded, recency-ranked transcript scan for the handoff directive and dashed-rail markers, then a confirm-before-resume gate; surfaces only the resume prompt + metadata, never raw transcript content), clean-stop (get to a durable, linked stopping point before the machine may go away \u2014 sweep every repo/worktree for uncommitted, unpushed, or PR-less work, push it durable, put breadcrumbs in PR/issue bodies, then give a free-and-clear verdict), retro (structured end-of-session retrospective with transcript metrics and learning codification), running-retro (in-flight retrospective checkpoints that spawn a subagent to analyze the transcript so far and append classified findings to a cumulative running ledger \u2014 capture and route only, the live counterpart to retro; also owns a detached-observer substrate that can watch a session out-of-band and run the checkpoint autonomously after the session ends), orient (read-only session orientation \u2014 synthesize where we stand, what we are doing, and why, from durable + off-thread state the built-in /recap never sees: ledgers, handoffs, workflow checklists, running-retro ledgers, open PRs and work-items, and git), orchestrate (arm a session or worker with proactive-orchestration imperatives), reanchor (verify a session's working assumptions are still true against live reality \u2014 referenced PRs/issues/branches, base-branch drift, renamed/version-drifted surfaces, stale memory-tier files, and the goal a handoff records, compared across the chain so a re-derived goal reports as drift \u2014 before building on them), reconcile (retire finished off-thread work and reconcile this session's task ledger with reality \u2014 the prune-and-reconcile counterpart to keep-going's resume: inventory the work this session spawned, inspect its real state, retire the finished and close proven-done tasks, auto-settling the finished and gating any kill of still-running work; sibling sessions in the project are reported read-only), setup (check-centric verification of the observer's runtime prerequisites and configuration), and show-options (lay out which skills fit this moment as a ranked, nothing-hidden menu \u2014 a shortlist per bucket plus the complete remainder by name, resolved from the full installed catalog rather than the truncated in-context listing, so the human decides and no option is withheld for looking already-done).", "author": { "name": "Melodic Software", diff --git a/plugins/session-flow/CHANGELOG.md b/plugins/session-flow/CHANGELOG.md index de3a1a81f..9a6127113 100644 --- a/plugins/session-flow/CHANGELOG.md +++ b/plugins/session-flow/CHANGELOG.md @@ -1,5 +1,23 @@ # Changelog — session-flow plugin +## [0.34.17] + +### Changed + +- **`orchestrate`: the token-multiplier line reconciled against the upstream measurement it + contradicted.** Imperative 1 carried "3-10x the tokens" as a bare figure while Anthropic's + multi-agent research write-up measures agents at roughly 4x a chat interaction and multi-agent + systems at roughly 15x, with token usage alone explaining most of the performance variance it + regressed. A reader who checked the source found the skill quoting a lower number with no + citation. The range is now labelled as this plugin's own operating figure and as a floor rather + than a ceiling, with the upstream measurement cited beside it and the instruction to size + research-shaped fan-outs against the higher number. +- **`orchestrate`: "compressed return" gained a magnitude.** The tiered-delegation section required + compressed verdicts without saying what compressed means, which is an unfalsifiable instruction. + It now carries the upstream shape — a sub-agent may explore across tens of thousands of tokens + and return roughly 1,000 to 2,000 — with its citation, and the explicit caveat that this is a + target shape and not a budget to spend up to. + ## [0.34.16] ### Added diff --git a/plugins/session-flow/skills/orchestrate/SKILL.md b/plugins/session-flow/skills/orchestrate/SKILL.md index 8941fe4ab..4e4d50e79 100644 --- a/plugins/session-flow/skills/orchestrate/SKILL.md +++ b/plugins/session-flow/skills/orchestrate/SKILL.md @@ -42,9 +42,16 @@ told: tool-restricted specialist. Decompose by what CONTEXT each piece needs, not by head-count or work-type. Sequential or shared-context steps stay in one agent. Coding parallelizes less than research: never split one feature across agents. Multi-agent costs 3–10× the tokens (returns - cost context too), so spend it on value + parallelism, not convenience. "Would flood context" - is a measurement, not a hunch, when the instrument exists: with the `context-guard` plugin - installed, resolve this session's zone word per its reader contract before a fan-out decision + cost context too), so spend it on value + parallelism, not convenience. That range is this + plugin's own operating figure and it is the floor, not the ceiling: Anthropic's multi-agent + research write-up measures agents at roughly 4× a chat interaction's tokens and multi-agent + systems at roughly 15×, with token usage alone explaining most of the performance variance it + regressed ([multi-agent research + system](https://www.anthropic.com/engineering/multi-agent-research-system), fetched + 2026-09-01). Size the spend against the higher figure when the fan-out is research-shaped. + "Would flood context" is a measurement, not a hunch, when the instrument exists: with the + `context-guard` plugin installed, resolve this session's zone word per its reader contract + before a fan-out decision (the contract owns the snapshot path, staleness rule, and bands. Read them there; this imperative consumes only the word, no band values). Never estimate your own remaining window, that guess is the failure the seam replaces. A degraded or `unknown` zone shifts the balance @@ -153,7 +160,11 @@ verdict rather than its reasoning. an output format (imperative 2). In a multi-tier tree the output format IS the context-economy lever: name the identifiers, the verdict, and where the bulky payload was parked, so the tier above can act without re-reading the work. A return that narrates cannot be summarized after the fact, -it has already been paid for. +it has already been paid for. A useful magnitude for "compressed": a sub-agent may explore across +tens of thousands of tokens and still return roughly 1,000 to 2,000 ([Effective context engineering +for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents), +2025-09-29, fetched 2026-09-01). Treat it as the shape a return should aim for, never a budget to +spend up to. **Workers are ephemeral, and the deeper the tier the shorter the life.** A worker that finishes and stays alive keeps costing the tier above, notifications, status, re-acknowledgement, for zero