agent-harness

A reliability harness for AI coding agents (Claude Code, Cursor, and friends).

Prompting an agent isn't enough. A good prompt gets you a good attempt — but attempts drift, forget, skip the tests, and confidently hand you code that doesn't compile. What makes an agent reliable is the environment around it: the rules it reads first, the memory it keeps between sessions, the guardrails that fire whether or not it remembers to behave, and the loop that makes it verify its own work.

This repo is my setup for that — the patterns and the actual scripts, generic enough to drop into any project. It's opinionated on purpose.

Harness engineering: treat the agent as one component in a system you design, not a genie you phrase wishes to. Rules, feedback, and verification beat cleverer prompts every time.

---

The through-line: nothing ships unverified

However capable the model, I don't let it (or myself) ship on vibes. Everything is gated by a test that had to fail first — and the kind of test depends on what you're building:

• Deterministic code → TDD / BDD. Write the behavior spec first, watch it fail, make it pass, refactor. Tests assert on behavior through public interfaces, never on internals.
• LLM / AI behavior → eval-driven development. You can't unit-test a probabilistic output, so the eval is the test: define the cases, run them against the real prompt, parser and validation (not a simplified stand-in), and gate on correctness and latency/cost before anything ships. Model swaps and prompt changes are A/B'd on replayed production cases.

flowchart TD
    START["New behavior to build"] --> Q{"Deterministic<br/>or probabilistic?"}

    Q -->|"deterministic code"| TDD["TDD / BDD"]
    subgraph L1[" "]
        TDD --> RED["🔴 write failing spec<br/>(behavior, not internals)"]
        RED --> GREEN["🟢 minimal code to pass"]
        GREEN --> REF["🔵 refactor"]
        REF -->|"next behavior"| RED
    end

    Q -->|"LLM / AI behavior"| EDD["Eval-driven development"]
    subgraph L2[" "]
        EDD --> CASES["define eval cases"]
        CASES --> RUN["run vs REAL prompt +<br/>parser + validation"]
        RUN --> GATE{"correctness AND<br/>latency/cost pass?"}
        GATE -->|no| IMPROVE["change prompt / model /<br/>data — A/B on replay"]
        IMPROVE --> RUN
    end

    REF --> SHIP["✅ commit / ship"]
    GATE -->|yes| SHIP

Loading

The harness below is simply how this discipline gets enforced on an AI agent — so it can't skip the failing test, quietly weaken an assertion, or ship an LLM change that never saw an eval.

---

The five pillars

flowchart LR
    R["① Rules-first<br/>CLAUDE.md"] --> A(("AI agent"))
    M["② Memory<br/>(persists across sessions)"] <--> A
    A --> H["③ Guardrail hooks<br/>(fire deterministically)"]
    H -->|blocks / injects| A
    A --> W["④ Workflow cadence<br/>start → verify → wrap"]
    W --> A
    A --> S["⑤ Subagents<br/>(fan-out, clean context)"]
    S --> A
    H --> V{"Verified?"}
    V -->|no| A
    V -->|yes| OUT["commit / ship"]

Loading

Pillar	Problem it solves	In this repo
① Rules-first	The agent guesses your conventions and guesses wrong	CLAUDE.md.template
② Persistent memory	Every session starts from zero; hard-won context is lost	memory/
③ Guardrail hooks	"Please run the tests" is a suggestion, not a guarantee	hooks/
④ Workflow cadence	Agents dive into code before understanding "done"	CLAUDE.md.template
⑤ Subagents	One context window fills up and quality collapses	CLAUDE.md.template

---

① Rules-first

The agent reads project rules before it writes anything. Not style trivia — the load-bearing stuff: architecture boundaries, what "done" means, how to commit, when to ask. See CLAUDE.md.template.

② Persistent memory

A file-based memory that survives session boundaries: an index (MEMORY.md) plus one atomic fact per file, each with typed frontmatter (feedback / project / user / reference) and [[links]] between them. The agent recalls relevant facts at session start and consolidates new ones before context is compacted (see pillar ③). Spec + templates in memory/.

flowchart LR
    subgraph S1["Session N"]
        RECALL["Recall relevant<br/>facts at start"] --> WORK1["Do the work"]
        WORK1 --> LEARN["Learn something<br/>non-obvious"]
        LEARN --> CONSOLIDATE["Consolidate before<br/>context compaction"]
    end
    CONSOLIDATE -->|"writes atomic facts<br/>+ updates index"| STORE[("MEMORY.md<br/>+ fact files")]
    STORE -->|"recalled next time"| RECALL2["Recall at start"]
    subgraph S2["Session N+1"]
        RECALL2
    end

Loading

③ Guardrail hooks

Hooks run deterministically, regardless of what the agent decides to do — this is where reliability actually comes from.

• pre-commit-review.sh — intercepts git commit and injects a self-review checklist the agent must pass first. Catches speculative scope, missing types, tests that assert on internals.
• pre-compact-memory.sh — fires before the context window is compacted and tells the agent to persist anything worth keeping while it still has full context. Turns lossy compaction into deliberate note-taking.
• test-filter.sh — reroutes test runs so a pass prints a one-line summary and a failure prints the full trace. Keeps the context window for signal, not noise.

sequenceDiagram
    participant A as AI agent
    participant H as pre-commit hook
    participant G as git
    A->>H: attempts `git commit`
    H-->>A: injects self-review checklist<br/>(scope, types, behavior-level tests)
    A->>A: verify each item
    alt something fails
        A->>A: fix it — no commit yet
    else all pass
        A->>G: commit proceeds
    end

Loading

The point: the checklist fires because the agent typed git commit, not because it remembered to be careful. Deterministic, not discretionary.

④ Workflow cadence

A rhythm the agent follows without being asked: establish baseline → write the test/eval → make it pass → verify → wrap up. Start by checking git state and test health and confirming what "done" means; write the failing test (or eval) before the code; review before committing; consolidate memory before ending. Encoded in the template.

flowchart LR
    B["🧭 Baseline<br/>git + tests green?<br/>what is 'done'?"] --> T["🔴 Test / eval first"]
    T --> C["🟢 Implement"]
    C --> V["🔎 Self-review<br/>(pre-commit hook)"]
    V -->|fails| C
    V -->|passes| WR["📝 Wrap up<br/>consolidate memory"]

Loading

⑤ Subagents

Delegate independent work to fresh-context subagents with file-based handoff, so the main thread stays clean and each piece runs in parallel. The main agent orchestrates and keeps the conclusions, not the file dumps.

---

Install (Claude Code plugin)

This repo is a self-hosted Claude Code plugin + marketplace. Inside Claude Code:

/plugin marketplace add RubLo/agent-harness
/plugin install agent-harness@agent-harness

That wires up the guardrail hooks, the /start-task · /self-review · /wrap-up commands, and the eval-driven-development · tdd-bdd · project-memory skills.

Then drop the rules into any project:

cp plugins/agent-harness/CLAUDE.md.template /your/project/CLAUDE.md   # edit for your stack

Prefer to cherry-pick? Everything is plain files under plugins/agent-harness/ — copy just the hooks or the template.

Why it works

Reliability in agentic systems isn't a smarter model or a longer prompt — it's structure the model can't skip. Rules it reads first, memory it can't lose, hooks that fire without its cooperation, and a loop that forces verification. Same principle as good CI: don't trust, verify — automatically.

---

By Ruben Tugnolo — full-stack engineer working in applied AI.