Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ If you use [herdr](https://herdr.dev) (an agent multiplexer for the terminal), p

| Paper | Example | Summary |
|---|---|---|
| [Self-Refine](https://arxiv.org/abs/2303.17651) | [self_refine.py](./examples/papers/self_refine.py) | Generate, self-critique, refine until the critic approves. |
| [Self-Refine](https://arxiv.org/abs/2303.17651) | [self_refine.py](./examples/papers/self_refine.py) | One seat: generate, `reflect` (self-critique), refine until its own critique approves. |
| [Reflexion](https://arxiv.org/abs/2303.11366) | [reflexion.py](./examples/papers/reflexion.py) | Verbal reflections on test failures accumulate as episodic memory across retries. |
| [CRITIC](https://arxiv.org/abs/2305.11738) | [critic.py](./examples/papers/critic.py) | Critiques grounded in external tool runs drive each correction. |
| [Self-Debug](https://arxiv.org/abs/2304.05128) | [self_debugging.py](./examples/papers/self_debugging.py) | Explain your own code line by line (rubber duck), then fix. |
Expand Down
4 changes: 2 additions & 2 deletions examples/papers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ and need no clean worktree.

| Score | Paper | The loop on h5i |
|---|---|---|
| [`self_refine.py`](self_refine.py) | Self-Refine ([arXiv:2303.17651](https://arxiv.org/abs/2303.17651)) | generate → critic seat (same model) reviews → revise, until approved. |
| [`self_refine.py`](self_refine.py) | Self-Refine ([arXiv:2303.17651](https://arxiv.org/abs/2303.17651)) | one seat: generate → `reflect` (self-feedback) → revise, until its own critique approves. |
| [`reflexion.py`](reflexion.py) | Reflexion ([arXiv:2303.11366](https://arxiv.org/abs/2303.11366)) | `verify` as the evaluator; verbal reflections accumulate in an episodic buffer delivered as the revise review. |
| [`critic.py`](critic.py) | CRITIC ([arXiv:2305.11738](https://arxiv.org/abs/2305.11738)) | a toolbox of `verify` commands grounds each critique; correct → re-verify. |
| [`self_debugging.py`](self_debugging.py) | Self-Debug ([arXiv:2304.05128](https://arxiv.org/abs/2304.05128)) | rubber-duck: explain your own code line by line before revising against the failure. |
Expand Down Expand Up @@ -98,7 +98,7 @@ and need no clean worktree.
The scores share scaffolding and build on each other:

1. `self_refine` → `reflexion` → `critic` — one worker, three feedback sources
(a critic seat, its own reflections, external tools).
(its own `reflect` critique, verifier-informed reflections, external tools).
2. `self_consistency` → `agent_forest` → `multiagent_debate` — from voting to
debating, same parallel-`ask` skeleton.
3. `chateval` and `mav_bon` — two ways to judge sealed candidates beyond
Expand Down
24 changes: 12 additions & 12 deletions examples/papers/self_refine.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@

One model plays two roles with two prompts: it generates a candidate,
critiques its own output, then refines against that critique — no external
signal, no training. h5i forbids a seat from reviewing its own artifact
(reviews are provenance-tracked turns), so the critic role is a second seat
pinned to the *same* model: the paper's "self" is the model, not the
session. The loop stops when the critic approves (the ``APPROVE`` first-line
convention) or the round budget runs out.
signal, no training. That maps to a single seat: ``reflect`` is the FEEDBACK
step, a first-class self-feedback turn recorded as a reflection (never as a
peer review — reviews stay peer-to-peer in h5i), and ``revise`` is REFINE.
The loop stops when the agent's own critique approves (the ``APPROVE``
first-line convention) or the round budget runs out.

python examples/papers/self_refine.py ["<task>"] # default: implement quicksort with pytest
"""
Expand All @@ -24,21 +24,21 @@
async def main(task: str) -> None:
async with Conductor(".", "self-refine-demo", launcher="resident", isolation="supervised") as c:
author = await c.hire("author", runtime="claude", model="claude-haiku-4-5")
critic = await c.hire("critic", runtime="claude", model="claude-haiku-4-5")

# Generate: one independent attempt, then seal the round (review is
# a sealed-phase turn).
# Generate: one independent attempt, then seal the round.
artifact = await author.work(task, expect_independent=True)
await c.freeze()

# Feedback → refine, until the critic approves or the budget is spent.
# Feedback → refine, until the author's own critique approves or the
# budget is spent. Reflections create no influence edge — the refined
# candidate stays stamped independent.
refinements = 0
for _ in range(MAX_ROUNDS):
review = await critic.review(artifact)
if review.approved:
feedback = await author.reflect(artifact)
if feedback.approved:
break
refinements += 1
artifact = await author.revise(artifact, review)
artifact = await author.revise(artifact, feedback)

print(f"refined {refinements} time(s); final candidate {artifact.id}")

Expand Down
25 changes: 25 additions & 0 deletions src/h5i/orchestra/_conductor.py
Original file line number Diff line number Diff line change
Expand Up @@ -643,6 +643,31 @@ async def ask(
last_value=value,
)

async def reflect(self, artifact: Artifact) -> Review:
"""Critique your OWN submission — the self-feedback turn (Self-Refine's
FEEDBACK step).

Recorded as a ``reflection_submitted`` event, deliberately distinct
from peer review: it never counts as review/quorum evidence, and it
creates no cross-agent influence edge, so a revision addressing it
stays stamped independent. The returned :class:`Review` has
``reviewer == target == <agent>`` and composes with :meth:`revise`
unchanged (``review.approved`` uses the same APPROVE first-line
convention). Reflecting on a teammate's artifact fails closed — that
is a :meth:`review`.
"""
raw = await self._conductor._turn_request(
"agent.reflect",
{
"agent": self.id,
"env_id": self.env_id,
"artifact": artifact.to_payload(),
},
self.id,
self.env_id,
)
return Review.from_raw(raw)

async def review(self, artifact: Artifact) -> Review:
"""Review a teammate's artifact (scoped read grant → posted review)."""
raw = await self._conductor._turn_request(
Expand Down
2 changes: 1 addition & 1 deletion src/h5i/orchestra/_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ class TurnContext:
run_id: str
agent_id: str
env_id: str
kind: str # "work" | "review" | "revise" | "ask"
kind: str # "work" | "review" | "revise" | "ask" | "reflect"
target: str | None
instruction: str
repo_workdir: str
Expand Down
36 changes: 36 additions & 0 deletions tests/test_conductor.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
Artifact,
AskParseError,
ProtocolError,
Review,
TurnContext,
Verdict,
)
Expand Down Expand Up @@ -99,6 +100,41 @@ async def test_work_round_trip_preserves_unknown_fields():
await c.close()


async def test_reflect_returns_self_review_that_composes_with_revise():
mock = MockOrchestra()
mock.on("agent.hire", lambda p: {"agent_id": p["name"], "env_id": f"env/{p['name']}/x"})
mock.on("agent.work", lambda p: artifact_raw(p["agent"]))
reflections = iter(["Needs edge-case tests.", "APPROVE — ready."])
mock.on("agent.reflect", lambda p: {
"reviewer": p["agent"], "target": p["agent"], "round": 1,
"body": next(reflections), "referenced_artifacts": [p["artifact"]["id"]],
})
mock.on("agent.revise", lambda p: artifact_raw(p["agent"], id_="sha:2"))
c = await launch_conductor(mock)
try:
author = await c.hire("author", runtime="claude")
artifact = await author.work("implement quicksort")

feedback = await author.reflect(artifact)
(reflect,) = mock.calls_to("agent.reflect")
assert reflect["agent"] == "author"
assert reflect["env_id"] == "env/author/x"
assert reflect["artifact"]["id"] == artifact.id
assert isinstance(feedback, Review)
assert feedback.reviewer == feedback.target == "author"
assert not feedback.approved

# The self-review composes with revise unchanged.
revised = await author.revise(artifact, feedback)
(revise,) = mock.calls_to("agent.revise")
assert revise["review"]["reviewer"] == "author"
assert revised.id == "sha:2"

assert (await author.reflect(revised)).approved
finally:
await c.close()


async def test_verify_sealed_from_passes_id_and_parses_seal_fields():
mock = MockOrchestra()
mock.on("agent.hire", lambda p: {"agent_id": p["name"], "env_id": f"env/{p['name']}/x"})
Expand Down
Loading