feat(harbor): upload standard verifier output

[Abhijeet Prasad (AbhiPrasad)]

Supersedes #716

AI Summary

Braintrust already received the reward numbers for each Harbor trial. Now it also receives the evidence the verifier wrote, so you can see why a trial failed without leaving the UI.

<trial-dir>/verifier/
├── reward-details.json   already uploaded
├── ctrf.json             NEW  standard test-report JSON
├── test-stdout.txt       NEW  raw verifier log
└── test-stderr.txt       NEW  raw verifier log

Verifier evidence shows up in two places: a bounded inline preview and one complete normalized verifier-output.json attachment.

root (experiment row)
├── task
│   └── verification    ← verifier_output_summary   (inline preview)
│                         verifier_output           (full evidence, attached)
└── scorer: reward      ← same two fields

Attachment modes

Harbor runs are evaluation environments, where complete failure evidence is generally more useful than omitting raw logs. The default therefore captures all standard verifier output.

attachments=	ctrf.json	test-stdout/stderr.txt	task artifacts
none	—	—	—
structured	✅	—	—
all (default)	✅	✅	✅ (with artifact_include)

structured is the JSON-only tier for environments that should not upload raw verifier logs. Task artifacts remain opt-in even under all: they require an explicit artifact_include glob.

Structured redaction matches sensitive key names, while plain text has no keys:

# ctrf.json — the key is named ANTHROPIC_API_KEY, so the value is caught
{"env": {"ANTHROPIC_API_KEY": "[REDACTED]"}}

# test-stdout.txt — plain text is covered only by configured patterns
"+ export ANTHROPIC_API_KEY=sk-ant-REAL-SECRET"

If an eval environment contains sensitive values, configure redact_patterns or use attachments="structured". redact_patterns is the only redaction applied to raw verifier logs and is empty by default.

Size limits are gone

There used to be a 5 MB per-file cap and a 20 MB per-trial cap. The per-trial cap was one shared pot, spent in source order:

artifacts ──► trajectory images ──► verifier output ──► reward details
^ spends first                                         ^ often got nothing left

That meant a few screenshots could push out the score evidence, and the same trial with a slightly larger screenshot could upload different evidence. Attachments are intended for large payloads, so both attachment caps and their shared-budget plumbing are removed and every accepted file uploads whole.

trajectory.json keeps a separate limit (max_trajectory_bytes, 20 MB) because it is parsed in-process rather than treated as an attachment.

Configuration changes

• attachments="all" is now the default.
• attachments="verifier-details" is renamed to attachments="structured".
• max_attachment_bytes and max_total_attachment_bytes are removed and no longer impose an upload ceiling.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.