Emit OpenEnv environments as a second target alongside Harbor

[thegovind]

Summary

Repo2RLEnv can already produce Harbor task directories. This PR makes it able to produce the environment that serves them.

repo2rlenv generate --repo pallets/click --pipeline pr_runtime --out ./tasks
repo2rlenv export --format openenv ./tasks --out ./click-env    # <- new
docker build -t click-env ./click-env
docker run --rm -p 8000:8000 -v /var/run/docker.sock:/var/run/docker.sock click-env

The tagline is "turn any repository into an RL environment". Until now the pipeline ended at a directory of files that something else had to run. It now ends at a running environment speaking OpenEnv's reset() / step() / state API — deployable as a Docker image or a Hugging Face Space.

The task data is not touched. export copies the task directories byte for byte and adds a serving layer around them, so the same dataset still runs under harbor run and under OpenEnv's own generic harbor_env. This is packaging, not conversion.

---

Where this sits

flowchart LR
    R["Repo2RLEnv<br/>synthesis pipelines"]
    T["Harbor task directory<br/>task.toml · instruction.md<br/>environment/ · tests/ · solution/"]

    H["harbor run<br/><i>batch evaluation</i>"]
    X["repo2rlenv export --format openenv<br/><b>NEW</b> — deployable env package"]
    HE["OpenEnv harbor_env<br/><i>generic Harbor runtime</i>"]

    TR["Trainer / RL loop<br/>reset · step · state"]

    R -->|emits| T
    T --> H
    T -->|"wraps, does not convert"| X
    T --> HE
    X --> TR
    HE --> TR

    style X fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style T fill:#e3f2fd,stroke:#1565c0

Loading

Two ways to reach OpenEnv, and they are complementary:

	What it is	Use it when
repo2rlenv export --format openenv	We emit a deployable environment package around your dataset	You want a standalone image or an HF Space, with no OpenEnv checkout involved
OpenEnv's harbor_env	OpenEnv's own generic runtime for any Harbor task dir	You already work in an OpenEnv checkout, or mix our tasks with other Harbor producers

Both forward the reward from the task's own tests/test.sh. Neither rewrites the format.

---

What the export produces

flowchart TB
    subgraph PKG["./click-env  — the emitted package"]
        direction TB
        Y["openenv.yaml<br/><i>OpenEnv manifest</i>"]
        D["Dockerfile<br/><i>builds the server image</i>"]
        RM["README.md<br/><i>HF Space card, sdk: docker</i>"]
        P["pyproject.toml"]
        S["server/app.py<br/><b>2 lines</b> over build_app()"]
        TK["tasks/&lt;id&gt;/...<br/><i>copied verbatim</i>"]
    end

    LIB["repo2rlenv.openenv<br/><b>the runtime — versioned + tested in this library</b>"]

    S -->|imports| LIB

    style S fill:#fff9c4,stroke:#f9a825
    style TK fill:#e3f2fd,stroke:#1565c0
    style LIB fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px

Loading

The package is deliberately thin config, not generated logic. server/app.py is:

from repo2rlenv.openenv import build_app

app = build_app()

The runtime lives in repo2rlenv.openenv, so it is linted, tested and upgraded like the rest of the library rather than frozen into whatever someone exported six months ago.

---

An episode

sequenceDiagram
    participant TR as Trainer <br/>(orchestration)
    participant E as Repo2RLEnvEnvironment
    participant C as Task container
    participant V as tests/test.sh

    TR->>E: reset(task_id=...)
    E->>C: pull pushed image, or build environment/Dockerfile
    E-->>TR: Observation(instruction=instruction.md)

    loop agent works
        TR->>E: step(exec / read / write)
        E->>C: run in /workspace
        C-->>E: output
    end

    TR->>E: step(evaluate)
    E->>C: clear /logs/verifier, stage tests/ -> /tests
    E->>V: bash /tests/test.sh
    V-->>C: reward.txt + reward-details.json
    E-->>TR: Observation(reward=0.83, done=True)

    Note over E,V: reward is FORWARDED, never recomputed
    Note over TR,E: evaluate + solve are orchestration-only,<br/>never in the agent's action space

Loading

Invariants held

• Rewards are forwarded, never synthesized. A verifier that writes no reward file yields reward=None plus an explicit error — not a 0.0. A fabricated zero is indistinguishable from a real failure and would quietly poison a training run.
• Agents cannot grade or solve. exec / read / write are the agent's action space; evaluate and solve are orchestration. An agent that could grade on demand could end its own episode; one that could solve could read out the answer.
• Agents cannot reach the answer. Every agent-supplied path is resolved through resolve_within() and confined to the working directory, so /tests and /solution are unreachable.
• A planted reward cannot survive. /logs/verifier is cleared immediately before the verifier runs, so a reward file written by the agent during exec never reaches the score.

---

Why not just reuse OpenEnv's harbor_env?

Fair question — and you still can, that path is documented and unchanged. Two reasons this exists as well:

1. It has to be installable. OpenEnv's env packages are not published — openenv-harbor-env is a 404 on PyPI, only openenv itself is published. An emitted image cannot pip install it, so binding to it would have coupled Repo2RLEnv releases to an unmerged OpenEnv PR. The emitted package depends only on openenv (verified against the published 0.4.1) plus this library.

2. It can do better on our own data. A generic Harbor runtime cannot read [metadata.repo2env]. Ours does:

flowchart TD
    A["reset(task_id)"] --> B{"reproducibility.mode"}
    B -->|"registry"| C["pull image_ref<br/><i>pushed by repo2rlenv push — no build</i>"]
    B -->|"inline_dockerfile<br/>local_only"| D{"environment/Dockerfile?"}
    D -->|yes| E["build it"]
    D -->|no| F["refuse with an explanation<br/><i>text-only pr_diff task —<br/>score against solution/patch.diff</i>"]

    style C fill:#c8e6c9,stroke:#2e7d32
    style F fill:#ffebee,stroke:#c62828

Loading

local_only and inline_dockerfile refs may exist on no machine but the one that generated the task, so only registry is treated as pullable. A pushed dataset therefore starts episodes with no build step at all.

---

Verification

I did not want to assert interop, so a task was built with Repo2RLEnv's own Harbor emitter — version = "1.0", [metadata.repo2env], environment/Dockerfile with WORKDIR /workspace, tests/test.sh writing /logs/verifier/reward.txt and ending exit 0 — and run through every runtime.

Runtime	No-op agent	Oracle (solution/solve.sh)
harbor run (Harbor 0.20.0)	0.0	1.0
repo2rlenv export --format openenv	0.0	1.0
OpenEnv harbor_env, docker mode	0.0	1.0

The decisive check was done on a built container, not in-process:

1. docker build on the emitted Dockerfile
2. run it with the host Docker socket mounted
3. drive it over the WebSocket API: reset → write_file → evaluate
4. the task went 0.0 → 1.0 through an agent edit, not the oracle

reward-details.json arrived intact as observation.info["reward_details"] (F2P/P2P counts, resolved, parse_status), and the working directory resolved to /workspace from the image.

Tests

• 33 new tests in tests/test_openenv_export.py
• Full suite 739 passed, 15 skipped, on Python 3.12, 3.13 and 3.14
• ruff check / ruff format --check clean; mkdocs build clean

---

Keeping openenv optional

The default install must stay lean, and export itself must not require the OpenEnv stack.

flowchart LR
    subgraph CORE["pip install repo2rlenv"]
        DS["openenv/dataset.py"]
        RW["openenv/reward.py"]
        EM["emitter/openenv.py"]
    end

    subgraph EXTRA["pip install 'repo2rlenv[openenv]'"]
        MO["models · client · app"]
        SB["sandbox"]
        EN["environment"]
    end

    EM -->|reads tasks via| DS
    MO -.->|"lazy, PEP 562 __getattr__"| CORE

    style CORE fill:#e8f5e9,stroke:#2e7d32
    style EXTRA fill:#fff9c4,stroke:#f9a825

Loading

repo2rlenv.openenv resolves its serving names lazily via a module-level __getattr__, so importing the package or the emitter does not import openenv. There is a test that asserts exactly this by subprocess:

assert "openenv" not in sys.modules   # after importing repo2rlenv.openenv + the emitter

Anything requiring the extra fails with an actionable message rather than a bare ModuleNotFoundError.

CI: the extra is deliberately not added to the dev group — that would bloat the lock and put openenv's dependency tree on the 3.13/3.14 matrix legs. Instead a dedicated job runs uv run --with openenv --with docker pytest tests/test_openenv_export.py, so the runtime invariants genuinely execute in CI instead of silently skipping.

---

Files

flowchart TB
    subgraph EMIT["src/repo2rlenv/emitter/"]
        H["harbor.py<br/><i>existing — emits tasks</i>"]
        O["openenv.py<br/><b>new — emits the environment</b>"]
    end

    subgraph RT["src/repo2rlenv/openenv/  — new runtime"]
        DS2["dataset.py<br/>read an emitted task + discovery"]
        RW2["reward.py<br/>/logs/verifier contract"]
        SB2["sandbox.py<br/>Docker sandbox"]
        EN2["environment.py<br/>reset · step · state"]
        CL["client.py<br/>trainer-facing"]
        AP["app.py<br/>FastAPI factory"]
    end

    CLI["cli.py<br/>repo2rlenv export"]

    CLI --> O
    O --> DS2
    EN2 --> DS2 & SB2
    SB2 --> RW2

    style O fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style RT fill:#f3e5f5,stroke:#6a1b9a

Loading

Area	Files	Lines
Runtime	src/repo2rlenv/openenv/*	~1,455
Emitter	src/repo2rlenv/emitter/openenv.py	338
CLI	src/repo2rlenv/cli.py	+95
Tests	tests/test_openenv_export.py	405
Docs	docs/reference/OPENENV.md, README.md, RELATED_WORK.md, mkdocs.yml	~233
Lockfile	uv.lock	+1,529 (uv locks all extras)

19 files, +4,099 / −4. Excluding uv.lock that is ~2,570 lines, roughly a third of which is tests.

Suggested review order

1. docs/reference/OPENENV.md — the whole design in prose, start here
2. src/repo2rlenv/openenv/environment.py — the state machine and the action-boundary invariants
3. src/repo2rlenv/emitter/openenv.py — what gets written and why it is thin
4. src/repo2rlenv/openenv/reward.py — the forward-never-synthesize rule
5. Everything else is mechanical

---

Docs

docs/reference/OPENENV.md previously said Repo2RLEnv datasets run on OpenEnv with "no export step". That is now half the story, so rather than leave it contradicting the new command I rewrote it into two honest paths — export a deployable environment and serve with harbor_env — with the shared point made explicitly: the task data is identical either way.

README.md and RELATED_WORK.md were updated to say we now emit OpenEnv rather than only being compatible with it.

---

Known caveats

• The emitted Dockerfile defaults to repo2rlenv[openenv]>=<installed version>, but the openenv extra only exists once a release ships it. Until then, pass --requirement (a git ref or a local wheel) — that is exactly how the container verification above was run. It self-corrects on the next release.
• A Hugging Face Space cannot run Docker-in-Docker. A Space built from export serves the API but cannot start task containers, so Repo2RLEnv task sets need a Docker-capable host. This is stated in the emitted Space card and in the docs.
• Text-only pr_diff tasks (emit_harbor_env=False) ship no image and no verifier. export counts them, warns, and notes in the Space card that they are scored against solution/patch.diff client-side rather than executed.
• uv.lock grows by 1,529 lines. That is uv locking the new extra's transitive tree; uv sync --group dev --frozen still resolves and the default install is unchanged at 62 packages. Verified on 3.12 / 3.13 / 3.14.

---

Related

Companion PR in OpenEnv adds envs/harbor_env, the generic Harbor runtime referenced above. The two are independent — this PR depends only on the published openenv package and does not require that one to land.

[adithya-s-k]

@thegovind thanks for this, and the generator side has been working well for us: every dataset we
validated OpenEnv#1036 against came out of Repo2RLEnv, including
AdithyaSK/data_agent_rl_environment_train and AdithyaSK/repo2rlenv-pr-diff.

One thing worth knowing, since it did not exist when you opened this PR. OpenEnv#1036 adds Harbor
support to the OpenEnv CLI itself, so a Harbor task dataset is already servable and already
deployable with no packaging step:

openenv harbor serve --dataset you/click-tasks --llm-url $VLLM
openenv harbor push  --repo-id you/click-env --dataset you/click-tasks --llm-url $VLLM

That covers both rows of your table, including the Space case. What you get through it is
multi-harness training: the same dataset runs under the agents Harbor already installs, on Harbor's
sandbox backends, and a rollout comes back with per turn token ids and logprobs, so the dataset is
usable for training rather than only evaluation.