Skip to content

Latest commit

 

History

21 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

harness

An agent-based development harness. Tool-agnostic core, compiled per target tool. Current compile target: Claude Code. Validated providers: Azure DevOps, GitHub.

Principles

  • One owner per definition. Everything lives in core/ as plain Markdown + YAML frontmatter, with no tool-specific syntax. The installer compiles the core into the target tool's format. Never edit compiled artifacts — edit the core and run harness update.
  • The work item is the context contract. An approved item must contain everything an agent needs to develop it without access to the conversation that created it.
  • State lives upstream. Gate transitions (harness:proposedharness:approvedharness:in-devharness:in-reviewharness:done) are labels/tags on the provider, never session-only state.
  • Tests before code. BDD by default (Gherkin scenarios from the item become failing executable specs before implementation); TDD where BDD doesn't apply.
  • A test the pipeline doesn't run does not exist. Every suite is declared in the test-suites section of harness.yaml and wired to a pipeline stage.

Layout

core/           agents, rules, hooks, templates, stacks, workflows  (single source of truth)
providers/      neutral verb interface + adapters (azure-devops first; github/gitlab stubs)
schema/         commented harness.yaml example (per-repo config)
installer/      harness CLI: init · install · update · doctor — plus its tests
docs/           project docs, including the original plan (pt-BR)

Compiled into a target repo, that becomes:

CLAUDE.md                        process, this repo's config, pointers
.claude/agents/                  one per core agent, + engineer-<stack> per stack
.claude/commands/                the slash commands
.claude/skills/harness-<rule>/   one rule per skill, matched on demand by description
.claude/harness/                 provider recipes, workflow, templates, hook scripts
.claude/settings.json            CO-OWNED: harness owns only its own hook entries
lefthook.yml                     created once, then the repo's

Install into a project repo

One-time setup (the harness command becomes global; the core stays in this checkout and is read at runtime):

pipx install ~/git/harness/installer                    # or: pip install …
echo 'export HARNESS_HOME="$HOME/git/harness"' >> ~/.zshrc   # how the CLI finds core/

Then, from inside any project repo:

harness init --provider github      # writes harness.yaml (records core_path), docs skeleton
harness provider-setup              # GitHub/GitLab only: creates the harness:*, type:* and
                                    # routing labels upstream (idempotent; --dry-run to preview)
harness install --tool claude-code  # compiles core → CLAUDE.md, .claude/…, lefthook.yml
harness doctor                      # config, provider auth, labels, drift checks

init defaults to --provider azure-devops; provider-setup is a no-op there (work item types and tags are native fields). On GitHub it is not optional — gh issue create --label fails on a label that does not exist yet, so nothing works before it.

init also fills commands: (lint/format/test_fast/build) by reading the repo: the committed lockfile picks the JS package manager and the package.json scripts, ruff / poetry / uv show up in pyproject.toml, [workspace] in Cargo.toml, the .sln for dotnet. What it cannot read falls back to the stack's default, and --stacks js,rust chains the two with &&. --no-detect skips the repo and uses defaults only. The guess is printed for review — these commands gate every commit and the CI-on-PR stage, and doctor fails while any of them is still a placeholder.

harness.yaml stays the source of truth for them afterwards: edit a command there, run harness update, and the copies are rewritten in lefthook.yml and in the pipeline files (azure-pipelines.yml, .gitlab-ci.yml, .github/workflows/*).

lefthook.yml is synced structurally — harness knows which entries it owns (pre-commit lint/format, pre-push fast-tests), so harness.yaml simply wins and the file converges no matter how it drifted. The rest of the file is untouched: extra hooks, parallel:, glob:, comments, and an entry you renamed (harness then leaves it alone). The pipeline files are arbitrary YAML, so they can only be matched by the value the last install recorded in the manifest — a hand-tuned pipeline keeps everything else it says, and npm test never eats the npm test:integration of a suite entry. Since that match can be lost, doctor fails when commands.lint or commands.test_fast runs in no stage; a stage that wraps one declares itself with a [harness:lint] marker. test-suites[].command propagates the same way — that coupling is what doctor checks when it asks whether a suite is wired to a stage. What no file carried, and what would have broken the host YAML, is reported instead of applied.

stacks: reconciles the same way. Add one by hand and update appends that stack's link to every chain and compiles its engineer-<stack> agent; remove one and the link comes out — matched on what the last install recorded for that stack, so the other stacks' hand-tuned commands are untouched — and the orphaned agent is deleted. Without that deletion the repo keeps an agent for a stack it no longer has. Only .claude/** and CLAUDE.md are ever pruned; lefthook.yml is created once and belongs to the repo. An unknown stack name fails before anything is written.

Core resolution order: --core flag → $HARNESS_HOMEcore_path in harness.yaml → script location (checkout runs only). Without HARNESS_HOME, pass --core ~/git/harness on the first init; after that it's recorded in the repo.

Updating: changes to core/ need nothing (read live) — just harness update in each repo to recompile. Changes to installer/harness_cli.py need pipx reinstall harness-cli.

Working on the harness itself

Tests for the CLI and the hook scripts live in installer/tests/:

pip install -e './installer[dev]'
pytest installer                    # 75 cases: the CLI's pure functions + the hooks
ruff check installer core
shellcheck core/hooks/*.sh

The same four run in CI (.github/workflows/ci.yml, on every PR and push to main) alongside the SAST, dependency and secret scans the pipeline rule demands of every repo, and at commit/push time via lefthook.yml. Neither file is compiled — this repo has no harness.yaml — so both are hand-written and owned here.

Two deliberate choices in them, so they don't get "fixed" later:

  • Ruff's rule set is pinned in [tool.ruff.lint] select, not inherited. The default set grows between releases (0.9 → 0.16 added seven rules this code trips over), and a lint gate whose verdict depends on when it ran is not a gate.
  • No ruff format. This codebase is hand-aligned; the formatter would flatten it. Adopting one is the owner's call, in its own commit.

Per-repo customization goes in .harness/overrides/ (rules appended after core rules; overrides win). harness update recompiles from the current core without touching harness.yaml or overrides.

The flow

demand → issue-writer → items tagged harness:proposed
       → [HUMAN GATE: approve upstream → harness:approved]
       → orchestrator → (architects if flagged) → test-engineer (failing BDD specs)
       → engineer-{stack} (make specs pass; unit TDD)
       → PR → code-reviewer ∥ security-reviewer (comment upstream, never edit)
       → merge → qa builds suites → pipeline validates them (secdevops wires stages)
       → harness:done

Rules are skills

Each file in core/rules/ compiles to .claude/skills/harness-<rule>/SKILL.md, with the rule's description: as the match key. A rule is on-demand expertise, so it is loaded when it is relevant rather than pasted into CLAUDE.md and paid for on every turn.

The agents that must obey a rule are also given its path in their compiled preamble, so a rule reaches a dispatched subagent whether or not skill matching applies in that context. One file, two ways in — never a second copy of the text.

Per-repo customization: .harness/overrides/rules/<rule>.md is appended to that rule's skill under a "Repo overrides (win over everything above)" heading. An override whose name matches no rule is still repo policy, so agents get pointed at it instead.

Enforcement layers

Three layers, weakest to strongest: rules (prose the agents follow), git hooks (lefthook — lint/format/secret-scan at commit, fast tests at push; applies to humans and agents alike), and Claude Code hooks (shell guards the agent cannot skip, compiled into .claude/settings.json, toggled in harness.yaml → hooks):

  • protect_compiled (PreToolUse) — blocks hand-edits to compiled artifacts (.claude/harness/**, .claude/agents/**, .claude/commands/**, .claude/skills/harness-*/**, CLAUDE.md) and raw .drawio files. On by default.
  • check_conventions (PostToolUse) — after a write, hands back the textually decidable rule violations: a TODO/FIXME with no work item id (rules/engineering.md) and a skipped or .only test in a test file (rules/qa.md). Both are review blockers today; this catches them at the keystroke instead of at the PR. On by default.
  • session_context (SessionStart) — injects the gate reminder and warns when compiled artifacts are behind the core checkout. On by default.
  • stop_test_gate (Stop) — refuses to let a session finish while commands.test_fast fails on a dirty working tree. Opt-in: it runs the tests at the end of every response, so enable it only where test_fast is genuinely fast.

.claude/settings.json is co-owned: harness update replaces only the entries pointing at .claude/harness/hooks/, and leaves the repo's permissions, env and its own hooks alone — the same structural sync lefthook.yml gets. For that reason it is not recorded as a compiled artifact, so a permission the repo adds is not reported as drift. doctor checks that every hook enabled in harness.yaml is actually wired there, since a bad hand-merge would otherwise remove a guard silently.

Roadmap

  • Cursor compiler (AGENTS.md + .cursor/rules) and generic agents-md target. Rules degrade to .cursor/rules/*.mdc there; the skill/subagent orchestration does not port.
  • Packaging the claude-code output as an installable plugin. Deliberately not done: the compiled agents and hooks are a function of each repo's harness.yaml, and a plugin is static — it would have to ship the config-independent subset and lose the rest.
  • GitLab adapter validated end to end (recipes stubbed in providers/gitlab.md).
  • Optional reviewer identity on GitHub (a machine account token) so gh pr review --approve can replace the [harness:approved-by:*] marker comments.
  • Two-way GitHub Projects v2 sync for provider.project (today items are only added to the board; gate state is never mirrored into a board column).
  • Stale-item notifications (items sitting in harness:proposed).

Versioning

One version for the whole repo: core/VERSION and the harness-cli package (harness_cli.__version__, which pyproject.toml reads dynamically) are bumped together. Bump both on any change to core/ or to the installer.

harness --version prints both and flags a mismatch — the case that actually bites is an installer change you did not pipx reinstall, since core/ is read live from the checkout but the CLI is not:

$ harness --version
harness-cli 0.7.2
core        0.7.2  (/Users/you/git/harness)

harness doctor warns on the same drift, and separately fails when a repo's compiled artifacts are behind the core (that one is fixed with harness update, not a reinstall). No silent auto-update, either way.

The lock is now a test (installer/tests/test_version.py), so a one-sided bump fails on the PR rather than at runtime in the repo that already installed it.

Cutting a release

Releases are cut by hand — CI never decides when:

# bump core/VERSION and harness_cli.__version__ together, commit, then:
git tag v0.8.0 && git push --tags

The tag re-runs the same gates and, only if they are green, builds the sdist and wheel and publishes a GitHub Release. Before publishing it refuses a tag whose name disagrees with core/VERSION, and a tag that is not an ancestor of main — otherwise a tag pushed to a scratch branch would produce an official release.

Not published to PyPI: harness-cli there is an unrelated package by another author, and renaming this one is a decision for its own commit. The wheel alone is half the product anyway (the CLI reads core/ live from a checkout), so the source archive GitHub attaches is part of the release, not a formality.

License

Copyright (C) 2026 Douglas Picolotto

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the license along with this program in LICENSE. If not, see https://www.gnu.org/licenses/.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages