Repository navigation
chore(infra)!: rename CROPWRIGHT_* env to OP_CROPWRIGHT_*, pin Docker bases by digest, trim build contexts, env.template checks - #314
Merged
Conversation
… names
BREAKING: CROPWRIGHT_IMAGE, _BIND_ADDRESS, _PORT, _INGEST_MAX_REQUEST_MB and
_DATASET_UPLOAD_MAX_MB are now OP_CROPWRIGHT_*. The deploy CLI and the
installer exit 2 naming the new variable when a .env still sets an old name;
the cropwright container entrypoint refuses one passed through by compose.
The remaining names (live-test URL/project, docs screenshot URL) are renamed
the same way. The unread OP_CROPWRIGHT_TAG and the container-name variables
only the deleted standalone compose used are dropped from the docs.
The docs check now requires an assignment line in env.template for every
compose ${VAR} (a prose mention no longer counts), and a new template check
rejects a non-OP_ setting nothing reads; the unread LOG_LEVEL is removed.
…me scripts set Every FROM is image:tag@sha256 (bump procedure in the release checklist). The evaluator and segmenter build their Python environment in a separate stage, the fake VLM test image runs as nobody, and the api image copies scripts/curation, scripts/datasets and the six shell libraries it runs in the container instead of all of scripts/ (the evaluator copies only scripts/curation). The root .dockerignore excludes artifacts_local, docs-site, frontend, .claude, .install and export_*. tests/test_dockerfile_hygiene.py checks the pins, the non-root user, the stages and that the copied scripts cover every import.
… workers One YAML anchor replaces five identical build blocks; docker compose config renders byte-identically for every file and profile combination the Makefile and CLI use.
# Conflicts: # frontend/CHANGELOG.md # frontend/CLAUDE.md # frontend/README.md
…n the 413 message
6 tasks
# Conflicts: # frontend/CHANGELOG.md
# Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chore/cleanup-infra: env rename, env.template checks, Dockerfile pins and runtime image content (PR 5c of 9 of the repo cleanup)
Plan:
docs/design/repo_cleanup_plan.mdsection 7 (F3 env part, F5), decisions D25, D31, D32.No
src/logic, nofrontend/src, no file splits. BREAKING: the Cropwright env names changed.Commits
feat(env)!CROPWRIGHT_{IMAGE,BIND_ADDRESS,PORT,INGEST_MAX_REQUEST_MB,DATASET_UPLOAD_MAX_MB}areOP_CROPWRIGHT_*in compose,env.template, the installer,frontend/docker-entrypoint.sh, tests and docs (live-test and docs-screenshot helper vars too).scripts/lib/retired_env.shholds the retired list:./openprocessor(every command except help/version) andsetup-openprocessor.sh(after the release is installed, before.envis written) exit 2 withCROPWRIGHT_PORT was renamed to OP_CROPWRIGHT_PORT: update <file>; compose passes the retired names through to the cropwright container whose entrypoint exits 2. Docs check: assignment line required inenv.template, newtemplatecheck,LOG_LEVEL(read by nothing) removed, docs-onlyOP_CROPWRIGHT_TAGremoved, container-name variables dropped fromfrontend/.env.exampleand READMEbuild(docker)refactor(compose)build:blocks indocker-compose.dev.ymldocs(changelog),fix(docs)frontend/CHANGELOG.md; the checker reads the retired list from the libFailing first
The entrypoint refusal test was written together with the entrypoint change (not seen failing alone).
After:
tests/installer,tests/compose, docs checks and the hygiene tests pass; full offline suite8250 passed, 19 skippedin 617 s (-n 6) with one failure that I fixed afterwards infix(docs)(
test_real_docs_match_the_code[compose]: the checker still read the retired list from the installer after I moved it to the lib;15 passed after).
pre-commit run --all-filesgreen (34 hooks).Plant-a-violation proofs
${PLANTED_VAR}indocker-compose.yml, and# set PLANTED_VAR somewhere(prose) inenv.templatedocker-compose.yml: ${PLANTED_VAR} is not in env.template(prose does not satisfy it); removed, 0 problems# PLANTED_SETTING=1inenv.templateenv.template: PLANTED_SETTING is set by no compose file, script or code; removed, 0 problemsFROM python:3.13-slim-trixie AS builderinDockerfileDockerfile: FROM python:3.13-slim-trixie is not pinned by digest; restoredCOPY scripts/datasets/Dockerfile does not COPY: ['scripts/datasets/_common.py', 'scripts/datasets/bench_set.py']; restoredD31 / D32 details
docker buildx imagetools inspect), writtenimage:tag@sha256:...(the tag stays inthe reference, which Dependabot also updates).
docker/hardened/tritonpins throughTRITON_TAG+ newTRITON_DIGEST.Bump procedure: new section in
docs/releases/RELEASE_CHECKLIST.md. hadolint 2.15.1: no new findings (info DL3066 as before).docker/evaluator: two stages (builder with build-essential/cmake/git; runtime with only the three shared libs, no git/curl;_code_sha()already returnsunknownwithout git).docker/segmenter: three stages (git clone, pip builder, runtime);runtime keeps gcc/g++ for
torch.compile, drops the/opt/sam3source copy and the pip cache.Dockerfile.fake-vlm:USER nobody..dockerignore:artifacts_local/ docs-site/ frontend/ .claude/ .install/ export_*/. Checked first: no root-contextDockerfile or compose build COPYs from them (frontend and docs-site build from their own contexts;
export/is untouched).scripts/set, derived by AST import closure oversrc export models config_templates scripts/curationplus thecommands
scripts/lib/model_setup.shruns in the container:scripts/curation/,scripts/datasets/(fetch_coco_subset, manifests),and six shell libs (
config.shand what it sources,download.sh). Evaluator:scripts/curation/only.tests/test_dockerfile_hygiene.pyrecomputes the closure, so a new import outside the set fails CI.Measured (built with
-p cleanup-infra, tagscleanup-infra-local/*:{before,after}, never the live project):build api)/app/scripts4 MB to 3 MB, rest is the newer pinned base)Smoke in the built images:
importofscripts.curation.worker,.worker.__main__,.vlm_worker,.auto_label_worker,.bakeoff.bakeoff_runner,scripts.datasets.fetch_coco_subset,src.main: all ok;python -m scripts.curation.worker --helpandscripts/curation/cluster_refresh_daemon.py --helpprint usage;source /app/scripts/lib/config.shanddownload.shwork as appuser;/app/scriptsis appuser-owned and writable. Evaluator: imports mlflow, onnxruntime, ultralytics, tritonclient, pycocotools and therunner,
--helpprints usage. Segmenter:sam3(withbpe_simple_vocab_16e6.txt.gzin site-packages),sam3_backend, runs as uid 1000,gcc present. The live write-path harness was not run (needs GPUs 0/2 and the test stack).
Compose rendering
docker compose -p cleanup-infra configbefore/after the anchor commit is byte-identical for: base, base + all profiles, dev,dev + all profiles, the Makefile
DEV_COMPOSE(dev + gpu-arbiter + all profiles) and base + docs. No dead services, profiles, volumes orbuild args found (every named volume and network is used;
OP_BUILD_SHAis consumed by all seven builds).Findings, not fixed here
docker-compose.yml + docker-compose.gpu-clustering.yml(make cluster-gpu) does not render on Compose 2.29.7 onmaineither:services.curation-auto-label-worker: can't set distinct values on 'mem_reservation' and 'deploy.resources.reservations.memory'.Needs its own issue (drop
mem_reservationfordeploy.resources.reservations.memoryor the reverse).${A:+${B:?}}evaluates the inner expression eagerly), so refusal at compose level is thecropwright entrypoint; a raw
docker compose upwith an old name starts the container which then exits 2 and restarts. The CLI and installer refuse before compose.frontend/src/lib/ingest/ingestRunController.svelte.ts:331and its test still printCROPWRIGHT_INGEST_MAX_REQUEST_MBin a user-facing message(outside this PR's scope); it should say
OP_CROPWRIGHT_INGEST_MAX_REQUEST_MB. Thewindow.__CROPWRIGHT_*__globals are unchanged.setup-openprocessor.shstays at its 3214-line baseline: the refusal logic is in the new lib and only the source plus call are added; the7-line comment on
choose_cropwright_bindwas condensed to 4 to make room.Conflicts to expect
PR 5a (#308) edits
frontend/.env.example,frontend/README.md,frontend/CLAUDE.mdandscripts/docs/check_docs_vs_code.py(drops thefrontend/docker-compose*.ymlcorpus entry; I addedfrontend/e2eandfrontend/scriptsroots next to it). I did not touch files 5a deletes(
frontend/docker-compose.yml,frontend/scripts/release/*). Resolve by keeping both sides. Not rebased onto 5a/PR 4.