Skip to content

feat: own research generation recipes and Harbor review/repair - #102

Closed
adithya-s-k wants to merge 15 commits into
mainfrom
codex/owned-generation-pipelines
Closed

adithya-s-k wants to merge 15 commits into
mainfrom
codex/owned-generation-pipelines

Conversation

@adithya-s-k

@adithya-s-k adithya-s-k commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Research reproductions currently require external research checkouts and separate runners. This PR owns fourteen generation recipes inside Repo2RLEnv, exposes them through the existing CLI, and emits standalone Harbor tasks using Modal/Daytona execution. Pipeline code, prompts and adaptations live here; ordinary SDKs and Harbor remain dependencies. The original reproduction draft #101 remains unchanged.

Family Implemented recipes
Repository mutation and reconstruction SWE-smith, SWE-Flow, R2E
PR and commit history SWE-gen, SWE-Next, R2E-Gym
Terminal synthesis and evolution SETA Seed2Synth, SETA Evol, TMax, Endless Terminals, DataArc
Workflow and environment reconstruction TerminalWorld, CLI-Gym
Reasoning synthesis SCALER, using released generator/reference families and native answer equivalence; adaptive training and new-family generation are outside this adaptation.

SEC-bench is deferred. Each implemented recipe has a configuration example, RFC, pipeline walkthrough, prompts and source/license provenance. Target repositories remain input data; upstream research implementations are not installed, imported or cloned at runtime.

The CLI provides recipe discovery, Rich/JSON progress, immutable Harbor bundles, metered calls, campaign reservations, explicit resume and reusable remote workers. A generation-independent repo2rlenv quality run component accepts a Harbor task and existing rollout evidence, or runs controls, semantic probes and a blind solver before bounded repairs. OpenAI and Anthropic reviewer/repair/solver routes are independent. Original tasks stay intact; changed revisions invalidate execution evidence. Known wrong-solution probes cannot be discarded, and a failed deterministic check overrides model approval. Invalid patch proposals get one metered correction; unresolved evidence and uncertain provider effects stop further acceptance. quality show reads results without paid calls.

The documentation includes pipeline diagrams, stage inputs/outputs, implementation maps and sixteen generated prompt/reference pages with a CI drift check. Start with the recipe overview, quality results, and review/repair component guide. RFCs 0011–0027 describe scope, contracts and attribution.

Current evidence:

  • 291 exported Harbor bundles; all fourteen recipes have at least twenty. All original content hashes remain intact.
  • Two assistant-reviewed quality pilots covered 70 original tasks and 246 Harbor trials, selecting 50 practical task slots including seven revisions. These targeted samples do not establish population yield; the older strict admission contract is unchanged.
  • Live component canaries exercised GPT-6 Astra review, Sonnet 4.6 review/solving, Opus 4.6 repair and Modal-hosted Harbor. A deliberately broken coding canary was repaired in one revision and passed fresh baseline/reference, wrong/valid semantic controls and a Sonnet rollout. Real R2E calibration exposed missing laziness coverage, a defective alternative probe and generated repair errors; unresolved attempts were withheld. A final GPT-6 Astra review/repair run started from the earlier laziness revision with retained probes, fixed the public contract and alternative, and passed fresh baseline 0, reference 1, wrong probe 0, valid alternative 1 and Sonnet 1. Its trace review returned usable. This staged result is not a one-shot yield measurement; detailed outcomes and limitations are recorded in the component guide. These canaries do not add to the 50 pilot selections.
  • 869 tests passed, one opt-in GitLab test skipped. External/live suites were excluded from this local run; target task code and Docker executed only in Modal. The optional Harbor suite is included in CI.
  • Ruff, formatting, all sixteen prompt-reference checks and the documentation build pass. GitHub CI also passed lint, Python 3.12/3.13/3.14 tests, optional owned/Harbor contracts and package builds for the implementation commit. Existing draft-RFC link warnings remain.
  • Both cloud adapters passed earlier execution probes. This component's live canaries exercised Modal; Daytona is wired through the shared adapter but was not rerun here.

This remains a draft. No merge, release or new 100-task campaign is included.

@adithya-s-k adithya-s-k changed the title feat: integrate owned research generation recipes feat: own research generation recipes and Harbor review/repair Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant