Harden immutable SWE-bench builds and evaluation by mattteufel-e2b · Pull Request #2 · e2b-dev/swe-bench

mattteufel-e2b · 2026-09-02T23:00:26Z

Summary

bind template construction and runtime selection to immutable source-image identities
preserve the canonical SWE-bench setup commit while applying test patches
tighten artifact parsing, resume behavior, offline test guards, packaging, and CI
document the standalone E2B artifact boundary and optional Harbor integration

33 offline tests on Python 3.10, 3.11, and 3.12
Ruff lint and format checks
sdist/wheel build and Twine validation on all supported Python versions
live immutable-template smoke: gold patch resolved; empty control did not resolve
live cleanup: zero remaining sandboxes for the built template

mattteufel-e2b added 8 commits September 2, 2026 14:38

feat: import core reliability improvements

4e1ec86

fix: require exact artifact failure signatures

b2ba4a7

fix: validate artifact verdict inputs

033e73d

fix: make SWE-bench template identities immutable

bba6f73

fix: bind resumable runs to content identity

ce6a910

fix: preserve held-out test checkout commands

978d3a7

test: add reproducible SWE-bench release gates

84cee4e

docs: define the standalone immutable runner contract

27cb88a

cla-bot Bot added the cla-signed label Sep 2, 2026