Reproduce papers

README.md

@@ -76,21 +76,60 @@ With the default `launcher="resident"`, `h5i` automatically starts the agent ses
 
 ## 3. Examples
 
-For example, you can program:
-
-- ask Claude and Codex to implement the same task independently, have them review and improve each other's work, and select the smallest candidate that passes the tests;
-- let Claude Fable and Codex GPT-5.6 Sol iteratively refine a design, then hand the agreed design to Claude Opus for implementation; or
-- repeat a Fable-design/Sol-review loop ten times, ask Opus to implement the result, and invoke Sol to repair the implementation only when Fable rejects it.
-
-See [examples/](./examples/) for complete scores, including:
-
-- [arena_score.py](./examples/arena_score.py): independent arena ranking;
-- [ensemble_score.py](./examples/ensemble_score.py): mutual-review ensembles;
-- [debate_then_build.py](./examples/debate_then_build.py): architect-to-implementer pipelines;
-- [review_escalation.py](./examples/review_escalation.py): conditional review escalation;
-- [judge_panel_score.py](./examples/judge_panel_score.py): LLM judge panels;
-- [tournament.py](./examples/tournament.py): tournament brackets; and
-- [custom_control_flow.py](./examples/custom_control_flow.py): custom Python control flow.
+[examples/tutorial](./examples/tutorial/) provides basic multi-agent orchestration patterns:
+
+- [arena_score.py](./examples/tutorial/arena_score.py): independent arena ranking;
+- [ensemble_score.py](./examples/tutorial/ensemble_score.py): mutual-review ensembles;
+- [debate_then_build.py](./examples/tutorial/debate_then_build.py): architect-to-implementer pipelines;
+- [review_escalation.py](./examples/tutorial/review_escalation.py): conditional review escalation;
+- [judge_panel_score.py](./examples/tutorial/judge_panel_score.py): LLM judge panels;
+- [tournament.py](./examples/tutorial/tournament.py): tournament brackets; and
+- [custom_control_flow.py](./examples/tutorial/custom_control_flow.py): custom Python control flow.
+
+[examples/papers/](./examples/papers/) re-implements the core workflow of 40 published multi-agent papers:
+
+| Paper | Example | Summary |
+|---|---|---|
+| [Self-Refine](https://arxiv.org/abs/2303.17651) | [self_refine.py](./examples/papers/self_refine.py) | Generate, self-critique, refine until the critic approves. |
+| [Reflexion](https://arxiv.org/abs/2303.11366) | [reflexion.py](./examples/papers/reflexion.py) | Verbal reflections on test failures accumulate as episodic memory across retries. |
+| [CRITIC](https://arxiv.org/abs/2305.11738) | [critic.py](./examples/papers/critic.py) | Critiques grounded in external tool runs drive each correction. |
+| [Self-Debug](https://arxiv.org/abs/2304.05128) | [self_debugging.py](./examples/papers/self_debugging.py) | Explain your own code line by line (rubber duck), then fix. |
+| [Constitutional AI](https://arxiv.org/abs/2212.08073) | [constitutional_ai.py](./examples/papers/constitutional_ai.py) | Per-principle critiques against a written constitution fold into each revision. |
+| [Self-Consistency](https://arxiv.org/abs/2203.11171) | [self_consistency.py](./examples/papers/self_consistency.py) | N independent reasoning paths, majority vote over the final answers. |
+| [More Agents Is All You Need](https://arxiv.org/abs/2402.05120) | [agent_forest.py](./examples/papers/agent_forest.py) | Sampling-and-voting, with a scaling curve over ensemble size. |
+| [Universal Self-Consistency](https://arxiv.org/abs/2311.17311) | [universal_self_consistency.py](./examples/papers/universal_self_consistency.py) | A selector picks the free-form response most consistent with the sample population. |
+| [Multiagent Debate](https://arxiv.org/abs/2305.14325) | [multiagent_debate.py](./examples/papers/multiagent_debate.py) | Answer independently, read the others, update; majority vote at the end. |
+| [MAD: Divergent Thinking](https://arxiv.org/abs/2305.19118) | [mad_divergent.py](./examples/papers/mad_divergent.py) | An obligated-to-disagree negative side debates the affirmative under an adaptive judge. |
+| [ReConcile](https://arxiv.org/abs/2309.13007) | [reconcile.py](./examples/papers/reconcile.py) | A model-diverse round table converges by confidence-weighted vote. |
+| [Persuasive Debate](https://arxiv.org/abs/2402.06782) | [persuasive_debate.py](./examples/papers/persuasive_debate.py) | Debaters argue assigned sides; a transcript-only judge decides. |
+| [Negotiation Self-Play](https://arxiv.org/abs/2305.10142) | [negotiation.py](./examples/papers/negotiation.py) | Buyer/seller bargaining games improve via a critic's in-context feedback. |
+| [ChatEval](https://arxiv.org/abs/2308.07201) | [chateval.py](./examples/papers/chateval.py) | Persona-diverse judges debate one-by-one before scoring the candidates. |
+| [Multi-Agent Verification](https://arxiv.org/abs/2502.20379) | [mav_bon.py](./examples/papers/mav_bon.py) | Best-of-n candidates times m binary aspect verifiers; most approvals wins. |
+| [Chain-of-Verification](https://arxiv.org/abs/2309.11495) | [chain_of_verification.py](./examples/papers/chain_of_verification.py) | Draft, verify with questions answered by seats that never saw the draft, revise. |
+| [SelfCheckGPT](https://arxiv.org/abs/2303.08896) | [selfcheckgpt.py](./examples/papers/selfcheckgpt.py) | Per-sentence consistency against independent samples flags hallucinations. |
+| [PRD: Peer Rank & Discussion](https://arxiv.org/abs/2307.02762) | [prd_peer_rank.py](./examples/papers/prd_peer_rank.py) | Contestants judge all answer pairs; agreement-weighted ranking plus discussion. |
+| [LLM-Blender](https://arxiv.org/abs/2306.02561) | [llm_blender.py](./examples/papers/llm_blender.py) | Pairwise-rank candidates in both orders, then fuse the top-k. |
+| [Mixture-of-Agents](https://arxiv.org/abs/2406.04692) | [mixture_of_agents.py](./examples/papers/mixture_of_agents.py) | Layered proposers each fed the whole previous layer; an aggregator synthesizes. |
+| [Tree of Thoughts](https://arxiv.org/abs/2305.10601) | [tree_of_thoughts.py](./examples/papers/tree_of_thoughts.py) | Beam search over partial plans; only the best leaf pays for real work. |
+| [Graph of Thoughts](https://arxiv.org/abs/2308.09687) | [graph_of_thoughts.py](./examples/papers/graph_of_thoughts.py) | Generate/score/aggregate/refine thought transformations on an explicit DAG. |
+| [LATS](https://arxiv.org/abs/2310.04406) | [lats.py](./examples/papers/lats.py) | MCTS over real attempts, with test results as reward and reflections on failures. |
+| [Least-to-Most](https://arxiv.org/abs/2205.10625) | [least_to_most.py](./examples/papers/least_to_most.py) | Decompose easiest-first; solve in order with every prior answer in context. |
+| [Skeleton-of-Thought](https://arxiv.org/abs/2307.15337) | [skeleton_of_thought.py](./examples/papers/skeleton_of_thought.py) | Outline first, expand every point in parallel, assemble in order. |
+| [Meta-Prompting](https://arxiv.org/abs/2401.12954) | [meta_prompting.py](./examples/papers/meta_prompting.py) | A conductor invents expert personas on the fly and consults fresh seats. |
+| [Chain of Agents](https://arxiv.org/abs/2406.02818) | [chain_of_agents.py](./examples/papers/chain_of_agents.py) | Sequential workers pass a communication unit across chunks; a manager answers. |
+| [STORM](https://arxiv.org/abs/2402.14207) | [storm.py](./examples/papers/storm.py) | Perspectives, simulated writer-expert interviews, outline, then the article. |
+| [CAMEL](https://arxiv.org/abs/2303.17760) | [camel.py](./examples/papers/camel.py) | Inception-prompted user/assistant role play, one instruction at a time. |
+| [AgentCoder](https://arxiv.org/abs/2312.13010) | [agentcoder.py](./examples/papers/agentcoder.py) | Programmer and mutually blind test designer; a neutral executor loops failures back. |
+| [MapCoder](https://arxiv.org/abs/2405.11403) | [mapcoder.py](./examples/papers/mapcoder.py) | Exemplar recall, confidence-ranked plans, and plan-wise bounded debugging. |
+| [MetaGPT](https://arxiv.org/abs/2308.00352) | [metagpt.py](./examples/papers/metagpt.py) | Roles exchange structured documents (PRD, design, QA report), never free chat. |
+| [ChatDev](https://arxiv.org/abs/2307.07924) | [chatdev.py](./examples/papers/chatdev.py) | A chat chain: every waterfall phase is a two-role dialogue with a settled deliverable. |
+| [CodeT](https://arxiv.org/abs/2207.10397) | [codet.py](./examples/papers/codet.py) | Rank blind solutions by agreement with an independently generated test suite. |
+| [AlphaCodium](https://arxiv.org/abs/2401.08500) | [alphacodium.py](./examples/papers/alphacodium.py) | Problem reflection and AI-generated tests before coding; iterate until all green. |
+| [Agentless](https://arxiv.org/abs/2407.01489) | [agentless.py](./examples/papers/agentless.py) | A fixed localize, repair, validate pipeline — no agentic wandering. |
+| [Parsel](https://arxiv.org/abs/2212.10561) | [parsel.py](./examples/papers/parsel.py) | Decompose into a function graph; implement parts in parallel, compose, test. |
+| [Exchange-of-Thought](https://arxiv.org/abs/2312.01823) | [exchange_of_thought.py](./examples/papers/exchange_of_thought.py) | Four communication topologies (bus/star/ring/tree) as a who-sees-what function. |
+| [DyLAN](https://arxiv.org/abs/2310.02170) | [dylan.py](./examples/papers/dylan.py) | Rank each round's contributions and deactivate the weakest seat as you go. |
+| [AgentVerse](https://arxiv.org/abs/2308.10848) | [agentverse.py](./examples/papers/agentverse.py) | Recruit, collaborate, evaluate — and re-recruit a better team with fresh mid-run hires. |
 
 ## 4. License
 

examples/README.md

@@ -48,14 +48,14 @@ as an optional CLI argument and falls back to the same demo task,
 **`implement quicksort with pytest`** — so a bare invocation always works:
 
 ```bash
-python examples/ensemble_score.py                          # demo task
-python examples/arena_score.py       "implement quicksort with pytest"
-python examples/review_escalation.py "fix the flaky msg_integration test"
-python examples/pipeline_score.py
-python examples/judge_panel_score.py
-python examples/debate_then_build.py
-python examples/tournament.py
-python examples/custom_control_flow.py   # uses the default "attach" launcher:
+python examples/tutorial/ensemble_score.py                          # demo task
+python examples/tutorial/arena_score.py       "implement quicksort with pytest"
+python examples/tutorial/review_escalation.py "fix the flaky msg_integration test"
+python examples/tutorial/pipeline_score.py
+python examples/tutorial/judge_panel_score.py
+python examples/tutorial/debate_then_build.py
+python examples/tutorial/tournament.py
+python examples/tutorial/custom_control_flow.py   # uses the default "attach" launcher:
                                          # park resident sessions yourself first
 ```
 
@@ -82,20 +82,28 @@ in an `apply` pause at a durable human gate: the question is delivered over
 
 | Example | Pattern | When to reach for it |
 |---|---|---|
-| [`ensemble_score.py`](ensemble_score.py) | `ensemble` | Consensus: independent attempts, mutual review/revise, verify, verdict, gated apply. |
-| [`arena_score.py`](arena_score.py) | `arena` | Competition: best of N independent tries, ranked by neutral verification + smallest diff. |
-| [`pipeline_score.py`](pipeline_score.py) | `pipeline` | Assembly line: architect → implementer → hardener, each stage fed the last stage's artifact. |
-| [`judge_panel_score.py`](judge_panel_score.py) | `judge_panel` | Judgment beyond tests: LLM judges score sealed candidates against a rubric, citing recorded evidence. |
-| [`debate_then_build.py`](debate_then_build.py) | `debate` | Decide before building: argue a design question, then let the conclusion steer real work turns. |
+| [`ensemble_score.py`](tutorial/ensemble_score.py) | `ensemble` | Consensus: independent attempts, mutual review/revise, verify, verdict, gated apply. |
+| [`arena_score.py`](tutorial/arena_score.py) | `arena` | Competition: best of N independent tries, ranked by neutral verification + smallest diff. |
+| [`pipeline_score.py`](tutorial/pipeline_score.py) | `pipeline` | Assembly line: architect → implementer → hardener, each stage fed the last stage's artifact. |
+| [`judge_panel_score.py`](tutorial/judge_panel_score.py) | `judge_panel` | Judgment beyond tests: LLM judges score sealed candidates against a rubric, citing recorded evidence. |
+| [`debate_then_build.py`](tutorial/debate_then_build.py) | `debate` | Decide before building: argue a design question, then let the conclusion steer real work turns. |
 
 ## Composed control flow (the define-by-run payoff)
 
 | Example | Shows |
 |---|---|
-| [`custom_control_flow.py`](custom_control_flow.py) | `ask` data turns feeding `if`, journaled `step`/`scope` effects, dynamic `map_reduce` fan-out (`integrate` under the hood), `patched` mid-run migration, a custom verdict policy as a plain function. |
-| [`review_escalation.py`](review_escalation.py) | An escalation ladder: cheap model first, senior review loop, senior takeover with the junior's artifact as material. Three workflow-node-types' worth of logic in one `for` and one `if`. |
-| [`tournament.py`](tournament.py) | Multi-run orchestration: a bracket of arena matches, semifinals in parallel — Conductors are just objects, so composing *runs* is composing function calls. |
+| [`custom_control_flow.py`](tutorial/custom_control_flow.py) | `ask` data turns feeding `if`, journaled `step`/`scope` effects, dynamic `map_reduce` fan-out (`integrate` under the hood), `patched` mid-run migration, a custom verdict policy as a plain function. |
+| [`review_escalation.py`](tutorial/review_escalation.py) | An escalation ladder: cheap model first, senior review loop, senior takeover with the junior's artifact as material. Three workflow-node-types' worth of logic in one `for` and one `if`. |
+| [`tournament.py`](tutorial/tournament.py) | Multi-run orchestration: a bracket of arena matches, semifinals in parallel — Conductors are just objects, so composing *runs* is composing function calls. |
 
 None of the pattern functions are privileged — each is ~40 lines of the same
 public SDK these examples use (`src/h5i/orchestra/patterns.py`). When a
 pattern almost fits, copy it into your score and edit it.
+
+## Paper scores
+
+[`papers/`](papers/README.md) holds reference implementations of forty
+published multi-agent workflows — Self-Refine, Reflexion, Tree of Thoughts,
+Mixture-of-Agents, MetaGPT, ChatDev, LATS, AgentVerse, and more — each
+paper's core loop expressed as an ordinary score over the same public SDK. Same prerequisites
+as above; see that README for the map from paper to primitives.