experiments: add unified runner and upstream Harbor reproduction pilots

[adithya-s-k]

Reproduction experiments now share a remote runner for their original entry points, with stage receipts, budget reservations, duplicate-attempt protection and Harbor exports. The work stays under reproductions/; the production CLI and Tasksmith are unchanged.

The first five pilots are joined by native runs of CLI-Gym, SWE-Flow/Trace, TerminalWorld, SWE-gen, SWE-Next, R2E-Gym, R2E, SEC-bench/SecVerifier, DataArc and SCALER, plus harden-v0 and SWE-rebench V2 released components. Results distinguish successful workflow execution, component replay, upstream failures and task quality. Several methods remain blocked or incomplete; this is an experimental checkpoint, not completion of the 100-per-generator target.

Results and scope

• 28 distinct Harbor environments: 25 coding/terminal and three SCALER reasoning instances. CLI-Gym's two instructions share one environment and are counted once.
• 21 pass reference/no-op execution contrast, including two explicitly separate first-batch repairs. Zero are training-approved; the first batch has two successful offline blind Sonnet samples.
• TerminalWorld passes its native reference, no-op and three partial trials, then passes independent Harbor execution after rebuilding the final export. Its native task still permits internet.
• Preserve concrete failures: CLI-Gym rewards empty parsed results; DataArc's reference lacks the tested API; SWE-Flow's fourth reference has a syntax error; SWE-Next/R2E-Gym fail test execution; SEC-bench's published base lacks the SecVerifier runtime.
• SWE-rebench V2's original annotations and golden evaluator pass on one published sample. RepoLaunch, SWE-bench-Live, SWE-Dev and SWE-Mutation runtime pilots remain outstanding. SWE-rebench Harbor replay packaging remains pending.
• Immutable private HF receipts retain task definitions, original/repaired outputs, historical source uploads, traces, settlements and teardown. The expanded snapshot has 3,980 hash-verified files; its uploaded archive passed a download/checksum round trip.

Validation

• Nine reproduction controller/export tests pass, including concurrent reservation/attempt protection and native reward-scale checks.
• Repository Ruff lint and format checks pass; all 51 new shell recipes pass bash -n.
• Full local suite: 716 passed, four skipped, one failed live GitHub test. test_e2e_public_trl sampled two current PRs that were both filtered (test_only_diff, too_many_files), leaving no emitted task. Production pipeline code was not changed.
• All task execution and image construction ran remotely on Modal. Both workers are terminated. Campaign accounting is $22.43 of the new $500 allowance, with no open reservations; model/cloud estimates are not invoices.

Start with python reproductions/run.py list, then plan EXPERIMENT and run EXPERIMENT STAGE --worker NAME. reproductions/STATUS.md, RUNBOOK.md and per-experiment RESULTS.md document stage prerequisites, current pilot inputs, limitations and remaining work.