From ccaa8cc6044495202df88fb17130436d53cf78ca Mon Sep 17 00:00:00 2001 From: davidamacey Date: Fri, 16 Oct 2026 00:59:21 -0400 Subject: [PATCH 01/17] docs(changelog): consolidate the 0.6.0 release section with highlights and upgrade notes --- CHANGELOG.md | 125 +++++++++++++++++++++++++++++++++++++++--- frontend/CHANGELOG.md | 54 ++++++++++++++---- 2 files changed, 160 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fd003ca0..53120e6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,56 +7,94 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Releases 0.2.0, 0.2.1 and 0.3.0 are published on GitHub; their entries below are the content of those tags. The entry headed `Pre-release 0.1.0` is the earliest private -history of this codebase and was never published. This release is `[0.5.0]`. +history of this codebase and was never published. This release is `[0.6.0]`. ## [Unreleased] -### Fixed -- Ingest no longer fails a JPEG that Pillow reads but libjpeg-turbo's header read refuses (`tjDecompressHeader3(): Could not determine subsampling level`, 2 of 100,000 Open Images files, `decode_failed`): that error alone sends the image to the Pillow decode; a damaged file still fails. -- Dispatcher-started bulk-ingest runs (poll, cron, webhook) never started: the tick ran in a worker thread and both the run and the poll called `asyncio.create_task` there (`ingest_dispatch_failed`, `no running event loop`). The tick now runs on the loop. A run whose spawn fails is marked `failed` instead of staying `queued` (which kept every later start, and the project's delete, answering 409), and `POST /ingest/bulk/{run_id}/cancel` finalizes an orphan (a run `queued` past 10 s with no task, or `interrupted` after a restart) as `cancelled`. -- `DELETE /projects/{p}` (and archive) answering `409 project_busy` right after an ingest finished (#249): the detection worker refreshed its in-flight count only every 15 s, so the last batch read as busy for up to that long after it had landed. The worker now rewrites a project's liveness within a second of its count changing, and the `project_busy` message names each running job and says when a retry is the answer. -- `make pe-build-trt` (`export/build_pe_trt.sh`, #277) no longer ships an FP32 engine silently: a failed or FP32-falling-back FP16 bake, a failed FP16 build or a failed uint8 input step now exits 4 with the cause. `ALLOW_FP32=1` opts in to an FP32 engine knowingly. +## [0.6.0] - 2026-10-16 + +### Highlights + +0.6.0 makes ingest much faster and adds a way to point the VLM at exactly the crops you choose. Images are decoded once and (for large photos) on the GPU, detector and embedding inputs travel to Triton as uint8, the embedding engine is FP16, the PE image encoder can spread across several GPUs, and ingest writes to OpenSearch are batched; on the pinned public COCO set one RTX A6000 went from 8.9 to 17.4 images/s through `/ingest/upload` on the FP16 engine alone, with the rest of the gains measured in `docs/PERFORMANCE.md`. The new **assist runs** (`/assist/runs`) send a crop, a selection, a cluster, a sub-cluster or a filter to the VLM, with cluster label proposals you apply explicitly. Bulk ingest runs in the background from a folder, a manifest or S3, with a preview first, and ingest can be detection-only. A project calls no VLM until you activate one. The repository itself was cleaned up (guards, dead code removed, large files split, one typed env module, one docs source of truth). + +### Upgrade notes / behaviour changes + +Read these before upgrading from 0.5.0. + +- **New projects start with the VLM off** (#282). A project calls no VLM until it activates one (`GET .../vlm/endpoints/active` answers `source: "off"`). Projects that exist already are pinned once, at the first start of this version, to what they had (`env` when `OP_VLM_URL` was set, else `off`), so their behaviour does not change. +- **`make` and `model` are no longer hard-coded VLM outputs.** The reply keys, `vlm_item_make` and `vlm_item_model` are gone (old values stay in old documents, unread). Declare such facts as prompt-pack `attributes`; assist runs store them in the item's `assist` object. +- **`POST /vlm/label_cluster/{cluster_id}` and `POST /vlm/verify_regions` are removed.** Use `POST /assist/runs` (a `cluster` target, or `verify` on a `crops` or `boxes` target), or `?cluster_id=` on `auto_label/start`. +- **`curation-evaluator` moved from the `curation` profile to its own `evaluator` profile**, so `--profile curation up -d` no longer reserves a GPU for it. Start it with `./openprocessor curation up curation-evaluator`. +- **`CROPWRIGHT_*` env names are now `OP_CROPWRIGHT_*`** (`_IMAGE`, `_BIND_ADDRESS`, `_PORT`, `_INGEST_MAX_REQUEST_MB`, `_DATASET_UPLOAD_MAX_MB`). A `.env` that still sets an old name is refused at startup (`./openprocessor` and the installer exit 2, the container exits). Rename the keys and run `./openprocessor restart`. +- **Region-profile fields `aspect_min` and `aspect_max` are retired** (and `OP_REGION_DETECTION_ASPECT_MIN` / `_MAX`). A stored profile that carries either is rejected by name, and skipped with a warning when the registry loads; re-save it without them. +- **Rebuild the detector and PE image encoder engines** (`./openprocessor models repair`): both now take uint8 input, and a model whose plan and `config.pbtxt` disagree does not reach READY. +- **GPU decode is on by default** for the `full` profile (`OP_INGEST_GPU_DECODE`; `minimal` and `standard` default off). It adds about 8.5 GB to the Triton card. +- **New projects use the Lucene vector engine** (`ef_construction` 128); existing projects keep their faiss fields (no migration; re-ingest to move one). +- **Tests default `OPENSEARCH_URL` to a refused local address**, so an offline test run never reaches a live OpenSearch; set `OPENSEARCH_URL` for the live suite. +- **Retired region statuses `pending` and `pending_verify`, `regions.by_human` (use `regions.by_human_drew`) and `scripts/curation/region_worker_main.py` are removed.** `scripts/curation/ingest_walker.py` is replaced by `scripts/curation/bulk_ingest.py` (`--folder` instead of the old flags). +- **`TRITON_WORKERS`, `MAX_BATCH_SIZE` and `SHM_SIZE` are gone** from `.env`, `env.template` and the profile JSON (nothing read them); `setup.sh` no longer writes `docker-compose.override.yml`. +- **New environment variables** (all optional, documented in `env.template`): `OP_ASSIST_JOBS_DIR`, `OP_BULK_INGEST_DIR`, `OP_API_LOOPBACK_URL`, `OP_CPU_POOL_THREADS`, `OP_INGEST_INFLIGHT_MB`, `OP_INGEST_GPU_DECODE`, `OP_INGEST_GPU_MIN_MEGAPIXELS`, `OP_INGEST_DISPATCH_S`, `OP_CLUSTER_REFRESH_INTERVAL_S`, `_GROWTH_THRESHOLD`, `_MAX_DEFERRAL_S`, `_RETRY_BACKOFF_S` and `_AUTO_LABEL`, `OP_S3_ALLOWED`, `OP_S3_ENDPOINT_URL`, `OP_S3_REGION`, `OP_S3_PATH_STYLE`, `OP_S3_FETCH_WORKERS`, `OP_S3_LIST_WORKERS`, `OP_S3_INFLIGHT_MB`, `OP_S3_MAX_OBJECT_MB`, `OP_SOURCE_ROOT_HOST`, `OP_VLM_POLL_S`, `OP_VLM_WAIT_S`, plus the renamed `OP_CROPWRIGHT_*` set. `OP_HNSW_EF_CONSTRUCTION` now documents the new default of 128. +- **Compose changes**: the `evaluator` profile above; Triton gets `CUDA_DEVICE_ORDER=PCI_BUS_ID` (so `TRITON_GPU_ID=0,2` means the lowest listed id first), `CUPY_CACHE_DIR` and glibc malloc tuning for the API; Triton runs with `--log-verbose=0`; the API's memory reservation uses the `deploy.resources.reservations` form; an optional git-ignored `docker-compose.local.yml` is loaded after the stock files; the `gateway-wrap` message names `OP_CROPWRIGHT_BIND_ADDRESS`. Image defaults now point at `0.6.0`. ### Added + - `scripts/docs/check_docs_vs_code.py` gains an `envnames` check: a non-`OP_` variable a doc names in a code span, on a line about env or variables, must be read by a compose file, script or code (#287). + - Example prompt pack `examples/prompt_packs/general_objects.json`: the generic item prompts plus three domain-neutral item attributes (`brand`, `colour`, `material`) for COCO-style objects; validated by the pack validation in a test. + - `GET /clusters` cards serve `subcluster_ids` (the sorted sub-cluster ids, at most 200; `n_subclusters` stays the exact count). + - The item wire serves the accuracy audit state (#166): `audit_sample`, `audit_batch_id`, `audit_sampled_at`, `audit_label_name`, `audit_label_source`, `audit_outcome` and `audit_human_name` (null for an item never drawn), so `GET /audit/queue` items can show the machine label they were drawn with. + - Cropwright **Assist…** (cluster page, one crop, one box, Matching items): a dialog to pick what the VLM sees (selected, all, a representative sample), the actions (classify, attributes, verify, segment) and the VLM and prompt pack, with the server's dry run; the cluster and sub-cluster label proposal card with the agreement, an explicit dry-run-first **Apply** and the `scope_changed` refusal; dashed "VLM suggests" attribute chips; and a **Region verification** switch (`region_verify`) on the VLM scope panel. See the guide [Selective VLM and segmenter assist](docs-site/docs/guides/selective-assist.mdx). - Pack attributes in whole-project VLM runs: `vlm_attributes=true` on `POST /pipeline/auto_label` and `/start` makes the VLM stage use the pack's combined `assist` call and store the item attributes in each item's `assist` slot. The region worker's combined call asks for the pack's item attributes on the items it classifies and writes them to `assist.attributes` too. + - Selectable segmenter verification: the VLM policy takes `region_verify` (`vlm`, the default, or `none`) for the region worker, and an assist run with the `segment` action re-segments chosen crops, a cluster or a filter, with `segment.verify` (`vlm` or `none`) for those items only. The run waits in the new status `waiting_region` and reports `result.segment`. Refusals: `409 region_profile_required`, `no_segmenter_leg`, `region_stage_paused`, `verify_endpoint_mismatch`. With the worker's VLM requested but unreachable, an item is written as `detection_failed` (`vlm_unavailable`). + - Assist runs on boxes: a `boxes` target (chosen `{crop_id, box_id}` pairs) takes `verify` and `attributes`. The `verify` action replaces `POST /vlm/verify_regions` (one verdict per box, only the named boxes, never a human- or import-owned one); box attributes the pack declares with `applies_to: box` are stored under the item's `assist.boxes[]`, apart from the box. + - Cluster and sub-cluster label proposals from an assist run: `result.proposal` (top class, agreement, vote distribution, new-class proposals by name, attribute summary) with an agreement gate (at least 3 answers, `min_agreement`), `scope_now` on every read (membership overlap against the members the run resolved, so a re-refine shows as a changed scope), `GET /assist/runs` for a cluster or sub-cluster, and `POST /assist/runs/{run_id}/apply`: the explicit, dry-run-first action that labels the scope's current members as a human would, skips locked and validated ones and refuses a changed scope (`409 scope_changed`). Assist class suggestions leave the item in its cluster. + - Assist runs, `POST /assist/runs` (dry run by default), `GET /assist/runs/{run_id}` and `POST /assist/runs/{run_id}/cancel`: one request shape sends a crop, a selection, a cluster, a sub-cluster or an item filter to the VLM, whole or as a seeded, size-scaled representative sample stratified by sub-cluster, with `classify` and `attributes` actions. Class answers go through the guarded VLM class writer (a locked, validated or holdout item keeps its class); every answer, a locked item's included, is stored in the item's new `assist` object (shown on the item wire), apart from the labels. One run per project (`409 assist_busy`); `OP_ASSIST_JOBS_DIR`. + - Prompt packs declare `attributes` (up to 16 free-form facts about an item or a box, with an optional closed vocabulary) and carry the `assist` (classify plus item attributes) and `attributes` (attributes alone) prompts. The labeler gains `assist_batch` and `attributes_batch`, and `POST /prompt_packs/test` accepts `call: assist` and `attributes`. Pack validation checks the attribute list and the prompts it needs (`pack_attribute_invalid`, `pack_attribute_duplicate`). The `vehicle_wheel` example pack declares `make`, `model`, `body_style` and a wheel `rim_material`. + - `GET /crops?cluster_subid=` and the item filter's `cluster_subid` select one AHC sub-cluster on the server, so a sub-cluster larger than a page can be listed whole. + - Detection-only ingest (#279): ingest policy `embedding.mode: "none"` stores boxes with no vector of any kind (the whole-frame vector included) and `clustering.automatic: false` keeps ingest and the embed scope from placing items in clusters, the `curation-cluster-refresh` daemon from refreshing the project and the auto-label worker's idle retrain from running. Embed and cluster later on demand. Cropwright's ingest-policy page offers both. + - The cluster-refresh daemon reads `OP_CLUSTER_REFRESH_INTERVAL_S`, `_GROWTH_THRESHOLD`, `_MAX_DEFERRAL_S`, `_RETRY_BACKOFF_S` and `_AUTO_LABEL` (#292); a command-line flag overrides each. No compose override file is needed. + - Pipeline modes guide: a recipe, run on public data, for "ingest without the VLM, cluster, name clusters by hand, then run the VLM for one cluster, filter or batch" (`vlm=` per run). - `./openprocessor` and `make` load an optional, git-ignored `docker-compose.local.yml` after the stock compose files: a second read-only source root, a changed `shm_size` or a mounted credentials file no longer need hand-built `docker compose` commands (#283). + - `./openprocessor curation up [service...]` and `make curation-up SERVICES="..."` start a subset of the curation services; a name that is not a curation service is refused (#285). + - `capture_docs_screenshots.py` requires `--public-projects` (`OP_CROPWRIGHT_PUBLIC_PROJECTS`), refuses a `--project` outside it, and refuses the pages that list every project (project list, combine, copy-settings, clone-from, model sharing) while the backend holds any project outside it (#284). + - `scripts/docs/check_docs_vs_code.py` checks `./openprocessor` commands and flags, `make` targets, `` file names and `NAME=` lines in fenced blocks against the code; the version test scans the frontend docs; `docs.yml` also runs on `docs/**` and type-checks the site; a weekly `docs-links.yml` checks external links (#287, #293). + - `scripts/codegen/check_cited_paths.py` (pre-commit): a comment or docstring that cites a `docs/`, `scripts/` or similar file cites one that exists. Citations of design notes that left the tree in the frontend sit in a shrink-only baseline (#289). + - Bulk-ingest preflight (#278): `POST /curation/projects/{project}/ingest/bulk/preview`, `scripts/curation/bulk_ingest.py --preview` and a Preview button on the Cropwright Ingest page report, without ingesting anything, a folder's, manifest's, S3 prefix's or registered source's file count, total @@ -65,16 +103,19 @@ history of this codebase and was never published. This release is `[0.5.0]`. uses only figures measured in `docs/PERFORMANCE.md`, served (with the section each came from) in `GET .../ingest/bulk/config` under `preview`. The walk is capped (200,000 files, 60 seconds) and answers `partial: true` with the reason, stops when the client disconnects, and refuses an unreadable root. + - `docs/design/engineering_standards.md`: the portable standards and lessons from the repo cleanup (principles, the lint, type and test config that worked, every guard with its baseline format, `--tighten` workflow and plant-a-violation proof, an ordered port checklist for a sister repo, pitfalls, before/after numbers). `CONTRIBUTING.md` points to it. + - `scripts/bench/cluster_bench.py` (#218): times the production cluster pass at a chosen pool size (`--sizes`, `--methods`, `--reps`) on real public COCO embeddings in a throw-away project and records method, wall time and counts; a pass that did not cluster the whole pool is refused. `docs/PERFORMANCE.md` gains a summary table of the optimization milestone's results, the k-NN native-memory formula checked against the stored 100k data (#56) and the go / no-go of the write ceiling against 100 images/s (#230). + - Ingest from S3 and S3-compatible stores (#264): `--s3` and `--source-id` on `scripts/curation/bulk_ingest.py`, an `s3` or `source_id` start on `POST /curation/projects/{project}/ingest/bulk`, and registered ingest sources under @@ -85,6 +126,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. already stored. S3 is off until `OP_S3_ALLOWED` is set; credentials come only from the `api` container's environment, a mounted file or a role. A dispatcher in the API acts on due triggers (`OP_INGEST_DISPATCH_S`). Guide: Ingest from S3 and watched sources. + - `scripts/bench/ingest_transport_bench.py` and the "Ingest transports (#263)" measurement in `docs/PERFORMANCE.md`: HTTP/1.1 multipart against `grpc.aio` streaming and unary, transport only, on public images. A gRPC ingest channel saves under 1.2 percent of the pipeline's API CPU per image and the @@ -110,18 +152,22 @@ history of this codebase and was never published. This release is `[0.5.0]`. - A checksum-pinned public benchmark set: `fetch_coco_subset.py --bench-set N` draws a seeded, license-filtered, all-class COCO 2017 set and pins per-image SHA-256 and bytes (`scripts/datasets/manifests/coco_bench_2000.json`); `--verify-only` re-hashes a download offline (#45). + - Ingest stage timers for the work that ran untimed: `hash`, `persist`, `letterbox`, `triton_request`, `whole_frame_preprocess`, `blur`, `doc_build`, `bulk_serialize` and `admission_wait` join the `op_pipeline_stage_seconds` stages, and `resize` is renamed `pe_preprocess` (it is the PE crop preprocessing). On 12-24 MP photos the timers name 93 percent of the API CPU, on 640 x 480 frames 54 percent; the table and the native profile of the rest are in `docs/PERFORMANCE.md` (#210). + - `OP_CPU_POOL_THREADS` (default 8) and `OP_INGEST_INFLIGHT_MB` (default 1024): the threads per API worker for pixel work on large images, and the megabytes of decoded pixels a worker admits into ingest at once (#241). + - `scripts/bench/baseline_suite.py` and `suite_report.py`: one entry point that records the environment, ingest through `/ingest/upload` and `/ingest/batch`, Triton statistics, perf_analyzer points, endpoints and VLM throughput as one JSON; the v0.5.0 numbers are in `docs/PERFORMANCE.md` and `docs/benchmarks/` (#45, #40). + - Wave 0 full-pipeline throughput probes: `scripts/bench/ingest_sweep_suite.py` (client concurrency and batch sweep, Triton instance-count, queue-delay and preferred-batch arms applied and restored through the model store, per-endpoint stage cost for faces, OCR and the combined analyze route, @@ -130,6 +176,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. `ingest_sweep_report.py` (the tables). `scripts/datasets/fetch_hires_set.py` pins a 500-image set of 12 to 24 MP public photos (`manifests/commons_hires_500.json`). Results and the binding resource per stage are in `docs/PERFORMANCE.md` ("Wave 0: full-pipeline throughput") (#45, #210, #230). + - `scripts/bench/parity.py` with `capture` and `compare` subcommands: the accuracy gate every v0.6.0 change passes. `capture` ingests the pinned COCO set into a fresh project and stores the per-stage outputs (detector boxes, crop rectangles, PE crop and whole-frame vectors, cluster @@ -139,11 +186,13 @@ history of this codebase and was never published. This release is `[0.5.0]`. neighbour overlap, cluster agreement and purity against the COCO ground truth, label agreement), applies the thresholds of `scripts/bench/parity_gates.json` and exits non-zero on failure. The FP32 against FP16 PE engine results are in `docs/PERFORMANCE.md` (#208). + - Measured INT8 evaluation of the PE image encoder on Ampere (RTX A6000 and RTX 3080 Ti), recorded in `docs/PERFORMANCE.md` and `docs/benchmarks/int8_pe_eval.json`. Result: explicit Q/DQ INT8 is 1.2-1.4x faster than FP16 but fails the parity gates (crop cosine median 0.94-0.99 against 0.999, neighbour overlap 0.52-0.82 against 0.95), and weight-only INT8 passes the gates with no speed or size gain. INT8 is not offered; FP16 stays the only PE precision and no code changed (#213). + - Measured system shared memory between the API and Triton for the detector and PE image batches, recorded in `docs/PERFORMANCE.md` and `docs/benchmarks/shared_memory_transport.json`. Result: it removes about 6 ms of client CPU per frame when the transport is measured alone, but ingest API CPU @@ -163,6 +212,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. repeated card, mixed compute capabilities, or a card without room; the installer takes `--gpu-plan triton=0+2` and refuses the same cases. `scripts/bench/pe_gpu_parity.py` compares embeddings of the same crops per card (#228). + - GPU decode measured and planned (`docs/design/gpu_decode_plan.md`, numbers in `docs/PERFORMANCE.md`): on 12-24 MP photos the CPU JPEG decode caps ingest at about 26 photos/s on this host; GPU decode cuts the decode's host cost 3.7x and a GPU decode cut API CPU per photo by @@ -171,6 +221,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. references while AP and kNN precision are unchanged, so it waits for an owner decision. New probe tools `scripts/bench/gpu_decode_bench.py` and `scripts/bench/gpu_decode_parity.py` with tested kernels and helpers (#217). No product code changed. + - GPU decode for photos, opt-in with `OP_INGEST_GPU_DECODE` and gated by `OP_INGEST_GPU_MIN_MEGAPIXELS` (3.0): the `gpu_image_prep` Triton model decodes each JPEG once on the GPU (nvJPEG with GPU Huffman decoding), builds the detector letterbox and the PE squares with fused @@ -210,19 +261,25 @@ history of this codebase and was never published. This release is `[0.5.0]`. `.env` and names the real restart command. `setup.sh` no longer writes `docker-compose.override.yml`, and `TRITON_WORKERS`, `MAX_BATCH_SIZE` and `SHM_SIZE` (read by nothing) are gone from `.env`, `env.template`, the profile JSON and the docs (#281). + - `curation-evaluator` moved from the `curation` profile to its own `evaluator` profile, so `--profile curation up -d` no longer reserves a GPU for it. Start it with `./openprocessor curation up curation-evaluator` or `docker compose run --rm curation-evaluator` (#285). + - In a checkout, `./openprocessor` with no `COMPOSE_PROJECT_NAME` in `.env` uses the documented default project `openprocessor` and says so, instead of exiting 3. An install directory still refuses (#291). + - `bulk_ingest.py` defaults `--api-base` to `localhost:8000` inside the container and `localhost:4603` on the host (#288). + - `docs-site/.nvmrc` follows the Node version of `docs-site/Dockerfile` (#289). + - Repo cleanup, PR 9a (typing): mypy now runs with `disallow_untyped_defs`, `disallow_incomplete_defs` and `warn_return_any` over `src scripts docker export` (`tests/` is outside its scope), and the nine-error-code relax override for the ML modules is gone. 328 errors were fixed with real types (route return annotations that match each `response_model`, `Protocol`s for the FAISS index and the ingest outcome, typed mixin attributes); the OpenAPI contracts are unchanged. + - Repo cleanup, PR 6 (backend split by concern): every `src/` and `scripts/` file of 600 lines or more is split into cohesive modules (routers, `src/main.py` into `src/app/`, the Triton and PE clients, curation, projects, training and config-store services, `src/config/curation.py`, three scripts), @@ -230,6 +287,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. contracts and behavior are unchanged. Twenty-three functions over complexity 15 are decomposed behind characterization tests and leave the complexity baseline. `duplicate_detection/__init__.py` no longer holds logic, and the two exporters share `export_models` and `export_writers`. + - **BREAKING**: the Cropwright variables are now `OP_CROPWRIGHT_*`: `CROPWRIGHT_IMAGE`, `CROPWRIGHT_BIND_ADDRESS`, `CROPWRIGHT_PORT`, `CROPWRIGHT_INGEST_MAX_REQUEST_MB` and `CROPWRIGHT_DATASET_UPLOAD_MAX_MB` became `OP_CROPWRIGHT_IMAGE`, `OP_CROPWRIGHT_BIND_ADDRESS`, @@ -238,6 +296,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. way). A `.env` that still sets an old name is refused: `./openprocessor` and `setup-openprocessor.sh` exit 2 and print the new name, and the cropwright container exits at start. Rename the keys in `.env` and run `./openprocessor restart`. + - Every Dockerfile base image is pinned by digest (`image:tag@sha256:...`); the bump procedure is in `docs/releases/RELEASE_CHECKLIST.md`. The evaluator and segmenter images build their Python environment in a separate stage (no compilers or pip cache in the evaluator runtime; the @@ -245,10 +304,12 @@ history of this codebase and was never published. This release is `[0.5.0]`. api image copies only the `scripts/` it runs (`scripts/curation`, `scripts/datasets`, six shell libraries) instead of the whole tree. The root `.dockerignore` also excludes `artifacts_local/`, `docs-site/`, `frontend/`, `.claude/`, `.install/` and `export_*/`. + - `check_docs_vs_code.py` requires an assignment line in `env.template` for every compose `${VAR}` (a prose mention no longer counts) and gains a `template` check: a non-`OP_` setting that no compose file, script or code reads is rejected. The unread `LOG_LEVEL` and the unread `OP_CROPWRIGHT_TAG` documentation are removed. + - Repo cleanup, PR 5b (duplicates): one `now_iso()` (`src/utils/timeutil.py`, UTC, microsecond precision) replaces 21 private timestamp helpers with three different bodies; the class-history and edit-history entries used `timespec='seconds'` and now carry microseconds like every other @@ -261,6 +322,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. `curation_umap.py` moved under `src/routers/curation/` (`train/`, `images.py`, `umap.py`), `src/services/clustering.py` to `src/services/visual_search/clustering.py`; routes and the OpenAPI contract are unchanged. `requirements-dev.txt` is gone (no consumer). + - Environment variables are read through one typed module, `src/config/env.py` (`env_str`/`env_int`/`env_float`/`env_bool`/`env_path`), and ruff `TID251` bans `os.environ`/`os.getenv` in `src/`. Behavior change: a variable that is set but malformed or @@ -287,9 +349,11 @@ history of this codebase and was never published. This release is `[0.5.0]`. `docs/benchmarks/v060_wave0_*.json` keep their names), and the review-id file and test names under `tests/curation/` lost their suffixes. Comments and docstrings no longer carry wave, round or work-package ids (`scripts/codegen/check_process_ids.py` now enforces that with an empty baseline). + - The region field that keeps the status an item had before a region requeue is now `region_status_before_requeue` (was `region_status_legacy`). Values already stored under the old name are not read; they only recorded a past requeue. + - Repo guards, the first step of the repo cleanup (`docs/design/repo_cleanup_plan.md`). Hooks: pre-commit-hooks 6, gitleaks 8.30, bandit 1.9, shellcheck 0.11, hadolint 2.15 and ruff 0.17 (the hook id is now `ruff-check`; Dockerfile health checks use the JSON form hadolint wants). @@ -310,7 +374,6 @@ history of this codebase and was never published. This release is `[0.5.0]`. `--root`, no progress file: the run resumes from the server). The parallel directory walk moved to `src/services/curation/bulk_ingest/walk.py`. - - The detector engines (`trt`, `trt_end2end`, YOLO26 and dual-head exports) take uint8 pixels and do the `/255` in the TensorRT graph, so a detector request carries a quarter of the bytes of the FP32 tensor it replaces (4.9 MB to 1.2 MB per image). `export/uint8_input.py` puts the cast and divide in @@ -319,6 +382,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. detector engine (`./openprocessor models repair`); a model whose plan and config disagree on the input type does not reach READY, so the installer rebuilds it. Promoted trained models (ORT, `output0`) and the bake-off engines keep float32 input (#214). + - The PE image encoder takes uint8 pixels and applies `(x / 255 - 0.5) / 0.5` in the TensorRT graph (`build_pe_trt.sh` and `build_pe_ort_fallback.sh` run `export/uint8_input.py` after the FP16 bake), so a PE request carries a quarter of the bytes of the FP32 tensor it replaces (10.0 MB to 2.5 MB @@ -329,12 +393,14 @@ history of this codebase and was never published. This release is `[0.5.0]`. 49 percent (0.118 to 0.062 CPU-s), and images/s rise 10 to 12 percent (19.1 to 21.0 upload); the gates pass against a rebuilt FP32 control, and rebuilding the detector engine alone already moves detections past the box gate of the v0.5.0 reference (`docs/PERFORMANCE.md`) (#214). + - `ocr_pipeline` ships two Python instances instead of one, and the installer profile now sets its count (`ocr_pipeline_instances`: minimal 1, standard 2, full 4). The Python BLS model runs one request per instance, so OCR was capped at about 21 images/s; Wave 0 measured 42 with two instances and 80 with four. A warm instance costs about 256 MiB of GPU memory (measured on a 12 GB card); `GPU_MIB_PER_INSTANCE` in the model config and `python_backend_budget_mib` per profile bound the count, and `tests/test_triton_instance_budget.py` enforces it (#240). + - Response serialization no longer uses FastAPI's deprecated `ORJSONResponse` (about 15 routers, the error handlers and four routes that built it directly). Routes with a response model serialize through Pydantic straight to bytes; the curation routes that return a @@ -343,6 +409,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. about 4 ms instead of about 69 ms and the acceptance log no longer repeats the deprecation warning. Response bytes are unchanged; the one contract change is that `POST .../datasets/imports/{import_id}/undo` now documents its response schema (#153). + - Ingest on 12-24 MP photos is no longer bound by one event-loop thread per worker. Decode, the detector letterbox, PE and whole-frame preprocessing, blur scoring and crop encoding of a large image run on a bounded per-worker thread pool (work under 4 MiB of pixels stays in place: a pool hop cost @@ -350,6 +417,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. decoded megabytes fit under `OP_INGEST_INFLIGHT_MB`. On the 500-photo set at 8 images per request throughput is 21.5, 27.3, 23.6 and 24.1 images/s at 8, 16, 32 and 64 clients against 5.2, 8.7, 10.8 and 4.9, and no longer falls past 32 clients (#241). + - API CPU per 640 x 480 frame is down 29 to 33 percent (0.170-0.192 to 0.113-0.136 CPU-s; the 40 percent target was not met) with no change to any stored value: vectors reach OpenSearch through `ndarray.tolist()` and an orjson serializer (34 to 1 ms per image), PE normalization is a lookup table @@ -359,8 +427,10 @@ history of this codebase and was never published. This release is `[0.5.0]`. checks each distinct index once, and the API container keeps freed buffers instead of trimming them (`MALLOC_TRIM_THRESHOLD_`, `MALLOC_TOP_PAD_`, `MALLOC_MMAP_THRESHOLD_` in `docker-compose.yml`, up to about 128 MB resident per worker) (#211). + - Triton runs with `--log-verbose=0` instead of logging an allocation and a free for every tensor of every request (#211). + - The PE image encoder engine is 17 to 25 percent faster (181 to 225 images/s through Triton at batch 16, same output: crop cosine median 0.99996, every `precision` parity gate passes). `export/pe_rope_layout.py`, the last step of `export/export_pe_image_encoder.py`, permutes the query and key channels of every @@ -410,6 +480,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. that used to stop at about 26 images/s now sustains 100 (and 175 to 187 at 200), with 62 percent less API CPU per image at 100 and k-NN p95 under the same writes down from 449 to 28 ms at 25. The upsert moved to `src/clients/occ_upsert.py` and returns an outcome per document (#248). + - New projects build their vector fields on the Lucene engine with `ef_construction` 128 (was faiss 512): the items index is 48 percent smaller on the product stack, filtered recall stays near 1.0, the cosine score is unchanged, and the review-default overlap@10 against faiss is 0.992 (gate 0.95) with cluster @@ -450,6 +521,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. the status and body the old `HTTPException` carried. No route, status code or response body changed. An auto-label job queued before the upgrade names the old function path and fails to resolve it: requeue it. + - The frontend's oversize files are split by concern: `api.ts` into per-domain `api_.ts` modules, the review, train, export, clusters, cluster-detail and classes pages into a controller plus components, `types.ts`, the slot-config parser, the large vitest files, `e2e/conftest.py` @@ -459,13 +531,17 @@ history of this codebase and was never published. This release is `[0.5.0]`. ### Removed - `POST /vlm/verify_regions`: use `POST /assist/runs` with action `verify` on a `crops` or `boxes` target. + - The hard-coded `make` and `model` VLM outputs: the reply keys, `VlmClassPrediction.make/model`, the combined reply's `make/model`, and the `vlm_item_make` / `vlm_item_model` item fields (never on the wire). A prompt pack declares such facts as `attributes` instead; they are stored in the item's `assist` object by assist runs. Existing values in `vlm_item_make` / `vlm_item_model` stay in old documents and are not read. + - `POST /vlm/label_cluster/{cluster_id}`: use `POST /assist/runs` with a `cluster` target (and `sample.mode: "all"`), or `?cluster_id=` on `auto_label/start` for the whole-cluster job. Cropwright's cluster-page "Run VLM" and the dashboard's "Run VLM on cluster" dialog are replaced by **Assist…** on the cluster page, which uses the assist run. + - The region-profile fields `aspect_min` and `aspect_max` (and `OcrRegion.is_region_shaped`, which was their only reader and had no caller): from the profile dataclass, the profile editor schema, the license plate example, the docs and the OpenAPI contract. A stored profile that still carries either key is rejected by name and skipped with a logged warning when the registry loads, so re-save it without them. `OP_REGION_DETECTION_ASPECT_MIN` and `_MAX` no longer exist. + - Dead code, found by the new vulture guard (`scripts/codegen/check_vulture.py`) and the plan's list: `src/services/cluster_maintenance.py`, the unused `ImageService`, the unused `region_lean` module (and the `paddleocr` requirement only it imported), the unused @@ -474,30 +550,46 @@ history of this codebase and was never published. This release is `[0.5.0]`. and other helpers only a test called, and the `src.utils` package re-exports (import from the submodule). Reviewed dynamic-dispatch cases are in `scripts/codegen/vulture_whitelist.py` with a reason each. + - The `scripts/curation/region_worker_main.py` compatibility entry point: start the detection worker with `python -m scripts.curation.worker` (the compose file already does). + - `regions.by_human` in `GET /curation/projects/{project}/stats/dataset`: it was the old name of `regions.by_human_drew`. + - The retired region statuses `pending` and `pending_verify`: queries, the worker and the queue gauges now match only the `RegionStatus` values, which is all any writer has emitted since the rename. + - The `check-no-literal-region-fields` hook and `scripts/codegen/check_no_literal_region_fields.py`: the naming-leak scan and `tests/test_no_legacy_region_scalars.py` cover the same ground. ### Fixed +- Ingest no longer fails a JPEG that Pillow reads but libjpeg-turbo's header read refuses (`tjDecompressHeader3(): Could not determine subsampling level`, 2 of 100,000 Open Images files, `decode_failed`): that error alone sends the image to the Pillow decode; a damaged file still fails. + +- Dispatcher-started bulk-ingest runs (poll, cron, webhook) never started: the tick ran in a worker thread and both the run and the poll called `asyncio.create_task` there (`ingest_dispatch_failed`, `no running event loop`). The tick now runs on the loop. A run whose spawn fails is marked `failed` instead of staying `queued` (which kept every later start, and the project's delete, answering 409), and `POST /ingest/bulk/{run_id}/cancel` finalizes an orphan (a run `queued` past 10 s with no task, or `interrupted` after a restart) as `cancelled`. + +- `DELETE /projects/{p}` (and archive) answering `409 project_busy` right after an ingest finished (#249): the detection worker refreshed its in-flight count only every 15 s, so the last batch read as busy for up to that long after it had landed. The worker now rewrites a project's liveness within a second of its count changing, and the `project_busy` message names each running job and says when a retry is the answer. + +- `make pe-build-trt` (`export/build_pe_trt.sh`, #277) no longer ships an FP32 engine silently: a failed or FP32-falling-back FP16 bake, a failed FP16 build or a failed uint8 input step now exits 4 with the cause. `ALLOW_FP32=1` opts in to an FP32 engine knowingly. + - `POST /reprocess` with a filter target and `limit` / `sample` capped only the image scopes; the `region` and `vlm` scopes ran on every match. They now act on the capped, seeded selection, and the dry run counts it. + - CI stability: the offline suite no longer builds its app-lifespan OpenSearch client against the in-cluster hostname, which resolved slowly on the runner (10 to 15 s per lookup, several per startup) and made integration test setup hit the 400 s timeout, doubling the suite's run time. Tests now default `OPENSEARCH_URL` to a refused loopback port. The stubbed e2e harness gains a default 404 for `GET /vlm/endpoints/active`. - `PUT /ingest/policy` dropped the `clustering` block: saving `clustering.automatic: false` answered 200 with `true` and stored `true`, so detection-only clustering could never be switched off. + - The auto-label VLM stage no longer reports `completed` with `predicted: 0` when it could label nothing: it adds `failed` and `unreadable` counts, and fails the job when every selected crop failed (an unreachable VLM, or source images the worker cannot read). + - `docs-site` `npm run typecheck` passes: `baseUrl` (deprecated, TS5101) is dropped for `paths` alone, and the missing `@types/react-dom` is added (#293). + - Monitoring alerts (#290): the Grafana rules **Triton Model Not Ready** (`nv_model_ready_state`), **API High Error Rate** (`http_requests_total`) and **OpenSearch Cluster Unhealthy** (`opensearch_cluster_status`) queried series nothing exports and could never fire; the first and last are @@ -509,10 +601,13 @@ history of this codebase and was never published. This release is `[0.5.0]`. it), and the two contact points are renamed `unconfigured-default` and `unconfigured-critical` because they post to a localhost address nothing listens on: nobody is notified until you configure a receiver (`monitoring/README.md`, `docs-site/docs/operations/monitoring.mdx`). + - A bulk-ingest folder walk that was stopped (a cancelled run, a closed walk) kept listing the rest of the tree and waited for a stuck directory listing before returning; it now stops at once. + - Starting a bulk ingest on a folder the API cannot read is refused (`bulk_ingest_invalid_source`, "not readable") instead of completing with zero images discovered. + - The auto-label, cluster-refresh and VLM workers no longer log `Unclosed client session` on shutdown: each opened a second OpenSearch client for its project registry that nothing closed. `script_project_registry` now takes the worker's own client, which the worker already closes. @@ -521,13 +616,16 @@ history of this codebase and was never published. This release is `[0.5.0]`. raises `IndexError` and fails the whole batch. Only the affected image is refused, with the reason `face_landmarks_length_mismatch`; `detect_faces`, `recognize_faces` and the single-image face paths report the same reason. + - App shutdown (#319): each lifespan shutdown step (background tasks, pools, executors, Triton and OpenSearch clients) now runs independently; a failing step is logged as `shutdown_step_failed` with its step name and no longer skips the steps after it. + - `make cluster-gpu` renders again on Compose 2.29.7 (#315): the auto-label worker states its 2 GB memory floor under `deploy.resources.reservations`, which the GPU overlay extends, instead of `mem_reservation`, which Compose rejects next to it. A new test renders every compose file combination the Makefile and CLI use. + - The `vlm use` interrupt test passes when pytest is started in the background (#304): the child inherited an ignored SIGINT, which bash cannot trap; the test now restores the default. @@ -545,25 +643,31 @@ history of this codebase and was never published. This release is `[0.5.0]`. a cluster run raise instead of returning `0`. A crashed background job is logged at shutdown and the ingest dispatch tick runs off the event loop. About thirty best-effort OpenSearch cleanup handlers now catch only OpenSearch errors. + - A job state file that exists but cannot be read counts as a running job in every start guard (`FileJob.is_live`, and the auto-label guard), so a second job is not started beside one it cannot see (#161). + - The cluster-refresh daemon keeps its per-project growth state (items at the last retrain, the job in flight, the previous poll's count) in the project's settings document, so a restart neither retrains a project that was just retrained nor starts a retrain in the middle of an ingest (#235). An audit-gate `409` from `auto_promote` no longer stops the retrain from being queued (#280). + - An auto-label job still queued after `queue_max_age_seconds` (one hour) fails with "auto_label worker not running" and its trigger is withdrawn; the status serves `queue_age_seconds` and `queue_max_age_seconds` (#236). + - An unreachable VLM endpoint no longer leaves the VLM worker spinning with a full queue and nothing reported. `POST /curation/projects/{project}/vlm/label_batch` answers `503 vlm_unreachable` when no crop got an answer (nothing is written, no crop is marked failed); the worker releases its in-flight crops, backs off (poll interval, doubling to 60 s; the same for an unreachable API) and resumes by itself when the endpoint answers. The stall reason is logged and served in the worker's heartbeat, so the compose healthcheck output reads `ok: ... stalled: VLM endpoint unreachable: ...` while the worker stays healthy. + - The VLM worker skips a project that has no class to label as, so its crops are never claimed in flight and no longer fill the queue and starve the other projects; it logs the change once per project and picks the project up on the next poll after its first class. + - An image that does not decode reports `not a decodable image` (or Pillow's own error for a damaged file) instead of `No module named 'pi_heif'` from the Ultralytics `Image.open` patch (#297). @@ -583,6 +687,7 @@ history of this codebase and was never published. This release is `[0.5.0]`. channel multiplier, so `bake_fp16_onnx` keeps those four convs in FP32. The engine is 75 MB against 147 MB, 1.5x (batch 8) to 2.1x (batch 32) faster per image, cosine similarity to the FP32 engine at least 0.9998 on 200 COCO images (#224). + - The `curation-vlm-worker` no longer polls every project each cycle and takes a 409 `vlm_not_configured` from the ones with no VLM (#207, about 2.7 idle cores at 18 projects). Each cycle it asks the registry and each project's activation which projects have an active VLM, @@ -594,10 +699,12 @@ history of this codebase and was never published. This release is `[0.5.0]`. per embedding, ingest 17.4 against 8.9 images/s on the 2,000-image COCO set, cosine similarity to FP32 0.9997 mean). Before, the FP16 build failed at ONNX parse and the installer silently shipped the FP32 engine. + - The installer no longer reports a precision fallback as a clean success: the FP32 retry warns with the model, the FP16 failure reason and the consequence, is recorded in `.install/precision.tsv`, marks the model group `degraded` (a re-run retries FP16) and shows under "Degraded precision" in `./openprocessor models status` and in the install summary. + - `frontend/scripts/capture_docs_screenshots.py` no longer aborts after the first image when `--out` is outside `frontend/` (it printed the path with `relative_to`, which raised `ValueError`). @@ -607,8 +714,10 @@ history of this codebase and was never published. This release is `[0.5.0]`. expecting TYPE_UINT8". `export/check_input_dtype.py` compares the built input datatype to `config.pbtxt` and fails the install on a mismatch; the build stamp records `input=uint8`, so an engine built before this fix is rebuilt on the next `models install`. + - `./openprocessor models install|repair` in a checkout without `.install/state.json` no longer exits silently: it logs that every model group will be built and goes on (#270). + - The committed `yolov11_small_trt_end2end` config declares the FP16 build's `det_scores` datatype, so a default export leaves the tree clean; a test compares each committed exporter-written config with its renderer. The installer no longer builds the unused standard `yolov11_small_trt` engine, and that @@ -1022,7 +1131,6 @@ optional Cropwright 0.1.0 UI that ships staged in the deploy bundle. - Full gate on the release commit: 6,943 tests passed (12 skipped), `pre-commit run --all-files` and `generate_contracts.py --check` clean. - Measured on the public COCO set (2,000-image subset, one run): 13.39 images/s, 0 failed, 1.37 crops per image (`docs/PERFORMANCE.md`). - ### Added - Every compose service rotates its logs (`json-file`, 10 MB x 5) and has a `mem_reservation` (OpenSearch reserves its heap size). - Installer warns when `vm.max_map_count` is below 262144, with the fix. @@ -1251,7 +1359,6 @@ optional Cropwright 0.1.0 UI that ships staged in the deploy bundle. - Auto-label's `vlm` stage reports `skipped` with an explicit `reason` when it selects nothing or `run_vlm` is off (#116). - ReDoc no longer loads its footer logo from `cdn.redoc.ly`; a test scans the vendored bundles for external fetches (#117). - - The config-store poll no longer logs a 404 WARNING per request for projects that never wrote `meta:config_revision`: optional documents (revision counter, activations, curation settings) are read through one shared reader that treats a miss as absent without a transport error, and an absent revision is remembered for one poll interval (`OP_CONFIG_POLL_S`). - `resource_links` no longer lists `triton_metrics` and `dcgm_metrics` (and `OP_TRITON_METRICS_*` / `OP_DCGM_*` are gone): they are Prometheus scrape endpoints bound to localhost, not browser UIs; Grafana and Prometheus show their data. - API `/metrics` now aggregates every uvicorn worker: compose sets `PROMETHEUS_MULTIPROC_DIR` to a tmpfs on `yolo-api`, so the `op_*` and request metrics behind the ingest and API dashboards reflect the whole API instead of one worker. diff --git a/frontend/CHANGELOG.md b/frontend/CHANGELOG.md index 1427a1be..083616ee 100644 --- a/frontend/CHANGELOG.md +++ b/frontend/CHANGELOG.md @@ -7,10 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.6.0] - 2026-10-16 + +### Highlights + +Cropwright 0.6.0 adds **Assist…** (send a crop, a box, a selection, a cluster or a sub-cluster to the VLM, review the proposed label and apply it explicitly), a bulk-ingest panel with Preview, a sources panel for S3 buckets and watched folders, detection-only ingest settings, and a tidier, split-by-concern codebase with one accessible dialog component. + +### Upgrade notes / behaviour changes + +- The container reads `OP_CROPWRIGHT_INGEST_MAX_REQUEST_MB` and `OP_CROPWRIGHT_DATASET_UPLOAD_MAX_MB` (were `CROPWRIGHT_*`); a container started with an old name set exits with the new name in the message. `OP_CROPWRIGHT_IMAGE`, `_BIND_ADDRESS` and `_PORT` are the compose-side names. +- The "Run VLM on cluster" dialog and the cluster page's "Run VLM" are replaced by **Assist…**, which needs a backend with `/assist/runs` (0.6.0). +- `scripts/capture_docs_screenshots.py` requires `--public-projects`. + ### Added - The sub-cluster tabs on the cluster page list the ids the server serves on the cluster card (`subcluster_ids`), so a sub-cluster whose members are not on the loaded page still has its tab. The assist dialog has an **Agreement needed** field (`min_agreement`) for a cluster classify run: the server's default shown, the contract's 0.5 to 1.0 range. + - The audit queue shows, on each drawn crop, the machine label and its source the crop carried when it was drawn (`audit_label_name`, `audit_label_source`, now on the item wire, #166). + - **Assist…** on the cluster page (the cluster or the open sub-cluster, or the selected crops), on one crop's details panel, on one box in `/review`, and on Matching items (a filter): a dialog that posts one assist run (`POST {prefix}/assist/runs`). It offers what the VLM sees (selected, all members, a seeded representative sample with @@ -19,15 +33,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 run and shows its counts (members sent, estimated VLM calls, strata, what was left out, the endpoint and pack pinned) or its refusal verbatim; Start run stays disabled on a refusal. The run is followed to its end, including `waiting_region` for a segment run, with the served result counts; Cancel run stops it. + - The newest label proposal for the viewed cluster or sub-cluster is a folded line above the grid and opens a proposal card: the proposed class, the agreement and gate, the votes, new-class names (with Add class…), attribute suggestions marked "not labels", a stale banner from the served `scope_now`, and **Apply {class} to {N} members**, the explicit button, with the server's dry-run count first. The backend's `scope_changed` and `class_not_in_registry` refusals are shown as served, and the override is offered only after the refusal. + - Attribute chips (`assist.attributes`, per box `assist.boxes`) on the crop card, the details panel and `/review`, dashed and labelled "VLM suggests"; "VLM said: {class}" when the VLM's answer differs from the item's class. + - The VLM scope panel on Settings has a **Region verification** choice (`region_verify`: verify with the VLM, or accept without a VLM). + - Wire types `types_assist.ts`, `api_assist.ts` and the contract test `assistContract.test.ts`; the item wire's `assist` object; `cluster_subid` on the item filter and the reprocess filter; `assist` and `attributes` in the pack test calls. @@ -38,6 +56,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 manifest holds and shows the served file count, bytes, extensions, image-size sample, skipped entries and the estimated time and index size with the measured figures behind them. A capped scan is marked "partial" (every figure a lower bound); pressing the button again while it scans cancels it. + - `/ingest` has a "Bulk ingest (folder or manifest)" panel, shown with the server-path ingest it sits beside: start a run over a folder or a manifest the backend has mounted, follow served progress (counts, speed, time left, seconds per stage), cancel, resume, and read the failed images with their reasons. Everything shown and every @@ -54,6 +73,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The sub-cluster tab asks the server for the sub-cluster (`GET /crops?cluster_subid=`) instead of filtering the loaded page, so a sub-cluster larger than a page is shown whole. The tab list is the sub-clusters seen while the whole cluster is shown. + - The dashboard's quick action links to the clusters (where Assist runs) instead of opening a "Run VLM on cluster" dialog. @@ -66,6 +86,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 start while a retired `CROPWRIGHT_*` name is still set. The browser-side `window.__CROPWRIGHT_*__` globals are unchanged. `.env.example` and the README drop the container-name variables only the removed standalone compose used. + - The oversize frontend files are split by concern, with no behavior change. `lib/api.ts` is the HTTP core and the calls live in `lib/api_.ts` modules (`api_errors`, `api_projects`, `api_review`, `api_clusters`, `api_train` and the rest); `lib/types.ts` keeps the core entities and the rest are `types_.ts`. The @@ -105,30 +126,51 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 and the route tests under `src/routes/p/[project]/` are named by what they test (for example `reviewDismissedPanel`, `clusterPurityDisplay`, `servedTrainingCohorts`) instead of by the pass that wrote them. Same 4,575 tests; no rendered change. + - `scripts/capture_docs_screenshots.py` gains states for the sources panel and the bulk-ingest panel (idle, running, failures), the ingest policy page, a project with its VLM off, the audit page, a cluster page, the dataset imports list and the prompt-pack, region-profile and open-vocabulary lists. `scripts/docs_screenshot_fixtures.py` gains `bulk` and `vlm-off` to set up throwaway `cwlife-*` projects for them. +### Removed + +- Standalone-repository leftovers: the release scripts (`scripts/release.sh`, `scripts/release/`, + `scripts/release-smoke.sh`), the frontend's own compose files, pre-commit config, diagram and GIF scripts, + and its own `SECURITY.md` and `CONTRIBUTING.md`. The Cropwright image ships through the repository's + `scripts/release/build_and_publish.sh`; the README and docs point at the root files. + +- `runVlmOnCluster`, `pollAutoLabelJob` and `getAutoLabelJobStatus`, the dashboard's "Run VLM on cluster" modal and the + cluster page's "Run VLM" button: the backend deleted its per-cluster label route; an assist run replaces it. + ### Fixed - At narrow widths the top navigation wraps onto a second row, so Export, Models, Train, Bake-off and Settings are reachable without scrolling a clipped strip. + - The dashboard "Run VLM Labeling" action becomes a link to the Models settings ("activate a VLM first") when the backend serves the project's VLM source as `off`, instead of a ready primary button. + - Model titles on the Models page wrap instead of truncating. + - Ingest policy cost preview marks its figures as belonging to the previous policy while a new preview is computed (a large project takes seconds), instead of showing stale numbers as current. + - A project with no classes says "No classes yet" on the Classes page and "No classes in this project yet" on Export instead of "No classes match this filter" and "ingest some data first". + - Removed the internal "Plan §B4" reference from the test-holdout texts. + - At 430 px the Settings page selects (long option text) and the Review sort button no longer widen the page, and served filter fields wrap instead of overflowing when "More filters" is open. + - Review, multi-box tabs: after Tab moved to the next box, the response of the previous accept or reject re-seeded the controller and snapped the selection back to box 1, so a fast `R`, `Tab`, `Y` accepted the wrong box. A re-seed of the same crop now keeps the selection (clamped to the boxes that remain). + - Infinite-scroll grids (`createPager`) stop requesting pages after three in a row add nothing. A server whose `total` outruns what it serves (it still counts a crop just moved out of `/clusters/[id]`) made the scroll sentinel fetch pages without end. + - Stubbed e2e suite: a key pressed right after a write's response met the app before it had recorded the undo entry ("Nothing to undo", then a 45 s `expect_response` timeout), and a boot chunk fetched with `socket hang up` was not retried. The tests now wait for the app's own signal (the success toast) and the harness retries both spellings of a closed keep-alive socket. + - Stubbed e2e suite: the project-switch test judged "requests to the previous project" by an index into the stub's answered list, which counted requests sent before the switch; it now listens to the requests the page sends. The harness gains `E2E_CPU_THROTTLE` to run the suite as a slow runner would, and its docstring states the stability @@ -136,22 +178,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `/clusters` semantic search no longer throws `each_key_duplicate` (and re-requests `/search/text` in a loop) when the backend serves one crop id more than once: the first, best-ranked hit per id is kept. + - At 430 px the ignore-reason menu on `/clusters/[id]` opens inside the viewport, the `/train` past-runs table keeps a readable Name column (the table scrolls sideways instead), and the `/clusters` error state shows no count. + - A per-box accept or reject key (`y`/`r`) pressed while the previous box write is still in flight is applied in order instead of being silently dropped; this was the cause of the intermittent `test_per_box_accept_and_reject_keys_patch_immediately_without_removing_from_queue` CI timeout. -### Removed - -- Standalone-repository leftovers: the release scripts (`scripts/release.sh`, `scripts/release/`, - `scripts/release-smoke.sh`), the frontend's own compose files, pre-commit config, diagram and GIF scripts, - and its own `SECURITY.md` and `CONTRIBUTING.md`. The Cropwright image ships through the repository's - `scripts/release/build_and_publish.sh`; the README and docs point at the root files. - -- `runVlmOnCluster`, `pollAutoLabelJob` and `getAutoLabelJobStatus`, the dashboard's "Run VLM on cluster" modal and the - cluster page's "Run VLM" button: the backend deleted its per-cluster label route; an assist run replaces it. - ## [0.5.0] - 2026-10-09 Cropwright joins the OpenProcessor repository as `frontend/` and from here on is versioned with From b7fd93c84c53c0f35d752fe77ed60ff26d5f0797 Mon Sep 17 00:00:00 2001 From: davidamacey Date: Fri, 16 Oct 2026 01:05:21 -0400 Subject: [PATCH 02/17] chore(release): bump version to 0.6.0 --- VERSION | 2 +- docker-compose.yml | 20 +++++++++---------- .../docs/getting-started/quick-start.mdx | 2 +- docs-site/package-lock.json | 4 ++-- docs-site/package.json | 2 +- frontend/Dockerfile | 2 +- frontend/package-lock.json | 4 ++-- frontend/package.json | 2 +- openprocessor | 2 +- pyproject.toml | 2 +- setup-openprocessor.sh | 2 +- tests/installer/shims/docker | 2 +- 12 files changed, 23 insertions(+), 23 deletions(-) diff --git a/VERSION b/VERSION index 8f0916f7..a918a2aa 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.5.0 +0.6.0 diff --git a/docker-compose.yml b/docker-compose.yml index 394cf9de..d1616a4b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -64,7 +64,7 @@ x-gateway-wrap: &gateway_wrap | services: triton-server: - image: ${OP_TRITON_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-triton:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_TRITON_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-triton:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-triton logging: *default-logging restart: always @@ -139,7 +139,7 @@ services: - triton_net api: - image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-api logging: *default-logging restart: always @@ -331,7 +331,7 @@ services: cropwright: profiles: ['cropwright'] # OP_CROPWRIGHT_IMAGE (a full reference, e.g. a digest pin) wins over the OP_IMAGE_TAG default. - image: ${OP_CROPWRIGHT_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/cropwright:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_CROPWRIGHT_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/cropwright:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-cropwright # Soft scheduling floor for the static-file nginx, not a cap. mem_reservation: 64m @@ -399,7 +399,7 @@ services: # =================================================================== curation-detection-worker: - image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-detection-worker logging: *default-logging mem_reservation: 1g @@ -465,7 +465,7 @@ services: - triton_net curation-vlm-worker: - image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-vlm-worker logging: *default-logging mem_reservation: 512m @@ -504,7 +504,7 @@ services: - triton_net curation-auto-label-worker: - image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-auto-label-worker logging: *default-logging # deploy form, not mem_reservation: Compose rejects the two together, and @@ -573,7 +573,7 @@ services: - triton_net curation-cluster-refresh: - image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-cluster-refresh logging: *default-logging mem_reservation: 1g @@ -610,7 +610,7 @@ services: # profile, so `--profile curation up -d` does not reserve a GPU for it. # `docker compose run --rm curation-evaluator`. curation-evaluator: - image: ${OP_EVALUATOR_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-evaluator:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_EVALUATOR_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-evaluator:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-evaluator logging: *default-logging profiles: @@ -652,7 +652,7 @@ services: # OP_SEGMENTER_URL=http://segmenter:8000 \ # docker compose --profile curation --profile segmenter up -d segmenter: - image: ${OP_SEGMENTER_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-segmenter:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_SEGMENTER_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-segmenter:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-segmenter logging: *default-logging profiles: @@ -721,7 +721,7 @@ services: # =================================================================== curation-trainer: - image: ${OP_TRAINER_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-trainer:${OP_IMAGE_TAG:-0.5.0}} + image: ${OP_TRAINER_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-trainer:${OP_IMAGE_TAG:-0.6.0}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-trainer logging: *default-logging profiles: diff --git a/docs-site/docs/getting-started/quick-start.mdx b/docs-site/docs/getting-started/quick-start.mdx index fdd80c66..00e99424 100644 --- a/docs-site/docs/getting-started/quick-start.mdx +++ b/docs-site/docs/getting-started/quick-start.mdx @@ -62,7 +62,7 @@ both `triton` and `opensearch` report `"ok": true`: ```json { "status": "ready", - "version": "0.5.0", + "version": "0.6.0", "api_version": "v1", "services": { "triton": {"ok": true, "detail": "is_server_live OK", ...}, diff --git a/docs-site/package-lock.json b/docs-site/package-lock.json index 12d24b07..b8e0e981 100644 --- a/docs-site/package-lock.json +++ b/docs-site/package-lock.json @@ -1,13 +1,13 @@ { "name": "openprocessor-docs", - "version": "0.5.0", + "version": "0.6.0", "lockfileVersion": 3, "requires": true, "license": "AGPL-3.0-or-later", "packages": { "": { "name": "openprocessor-docs", - "version": "0.5.0", + "version": "0.6.0", "license": "AGPL-3.0-or-later", "dependencies": { "@docusaurus/core": "3.10.2", diff --git a/docs-site/package.json b/docs-site/package.json index 71c71931..1caf53c9 100644 --- a/docs-site/package.json +++ b/docs-site/package.json @@ -1,6 +1,6 @@ { "name": "openprocessor-docs", - "version": "0.5.0", + "version": "0.6.0", "private": true, "license": "AGPL-3.0-or-later", "scripts": { diff --git a/frontend/Dockerfile b/frontend/Dockerfile index e10199f7..f316dbbc 100644 --- a/frontend/Dockerfile +++ b/frontend/Dockerfile @@ -15,7 +15,7 @@ RUN npm run build # rebuild never silently changes what's shipped; Dependabot bumps this. FROM nginxinc/nginx-unprivileged:1.31.2-alpine3.23@sha256:6320020c7da8714feab524e02c08c5a1958675c4e68700e93a2fd8970b065786 -ARG VERSION=0.5.0 +ARG VERSION=0.6.0 ARG REVISION=unknown LABEL org.opencontainers.image.title="Cropwright" \ org.opencontainers.image.description="From raw images to a trained detector, without labeling one box at a time. Labeling frontend for OpenProcessor." \ diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 92084a28..fca124bd 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -1,12 +1,12 @@ { "name": "cropwright", - "version": "0.5.0", + "version": "0.6.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "cropwright", - "version": "0.5.0", + "version": "0.6.0", "license": "AGPL-3.0-only", "dependencies": { "svelte-dnd-action": "^0.9.80" diff --git a/frontend/package.json b/frontend/package.json index 77d151ce..72e9619c 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "cropwright", "private": true, - "version": "0.5.0", + "version": "0.6.0", "description": "From raw images to a trained detector, without labeling one box at a time. Labeling frontend for OpenProcessor.", "license": "AGPL-3.0-only", "author": "OpenProcessor Contributors", diff --git a/openprocessor b/openprocessor index 67967f94..184e9db9 100755 --- a/openprocessor +++ b/openprocessor @@ -450,7 +450,7 @@ main() { cmd_uninstall "$@" ;; version) - echo "openprocessor CLI ${OP_CLI_VERSION:-0.5.0}" + echo "openprocessor CLI ${OP_CLI_VERSION:-0.6.0}" ;; help|--help|-h) load_cli_core_lib diff --git a/pyproject.toml b/pyproject.toml index 2dadd242..e0829f57 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,7 +3,7 @@ [project] name = "openprocessor" -version = "0.5.0" +version = "0.6.0" description = "Self-hosted AI-powered visual processing engine with object detection, face recognition, CLIP embeddings, OCR, and vector search" readme = "README.md" requires-python = ">=3.12" diff --git a/setup-openprocessor.sh b/setup-openprocessor.sh index 9f98bc08..f30e22a6 100755 --- a/setup-openprocessor.sh +++ b/setup-openprocessor.sh @@ -74,7 +74,7 @@ OP_GH_DEFAULT_REF="${OP_GH_DEFAULT_REF:-main}" OP_IMAGE_NAMESPACE="${OP_IMAGE_NAMESPACE:-davidamacey}" OP_DOCS_URL="${OP_DOCS_URL:-}" -SCRIPT_VERSION="0.5.0" +SCRIPT_VERSION="0.6.0" EXIT_USAGE=2 EXIT_COLLISION=3 diff --git a/tests/installer/shims/docker b/tests/installer/shims/docker index d12739f3..f28cec64 100755 --- a/tests/installer/shims/docker +++ b/tests/installer/shims/docker @@ -125,7 +125,7 @@ cmd_compose() { if [[ " ${args[*]:-} " == *" --images "* ]]; then local k v repo tag repo="$(env_val "$envfile" OP_IMAGE_REPO)"; repo="${repo:-davidamacey}" - tag="$(env_val "$envfile" OP_IMAGE_TAG)"; tag="${tag:-0.5.0}" + tag="$(env_val "$envfile" OP_IMAGE_TAG)"; tag="${tag:-0.6.0}" local profiles wanted profiles=",$(env_val "$envfile" COMPOSE_PROFILES)," wanted="OP_API_IMAGE:openprocessor OP_TRITON_IMAGE:openprocessor-triton" From 81d904fe0261d402827d841f450da421a14969f8 Mon Sep 17 00:00:00 2001 From: davidamacey Date: Fri, 16 Oct 2026 01:11:08 -0400 Subject: [PATCH 03/17] docs(release): add the 0.6.0 release notes and refresh status docs for 0.6.0 --- CLAUDE.md | 2 +- README.md | 15 ++ docs-site/docs/about/vision-and-goals.mdx | 11 +- docs-site/docs/api-reference/curation.mdx | 2 +- .../docs/getting-started/introduction.mdx | 2 +- docs-site/src/data/roadmap.json | 19 ++- docs/CURATION.md | 2 +- docs/PERFORMANCE.md | 2 +- docs/README.md | 2 +- docs/VISION_AND_GOALS.md | 30 ++-- docs/design/README.md | 8 +- docs/releases/RELEASE_CHECKLIST.md | 83 +++++------ docs/releases/v0.6.0.md | 141 ++++++++++++++++++ 13 files changed, 243 insertions(+), 76 deletions(-) create mode 100644 docs/releases/v0.6.0.md diff --git a/CLAUDE.md b/CLAUDE.md index 67ca09ae..324146a2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -7,7 +7,7 @@ where things live, and the commands and rules that matter. Before making design decisions or judgment calls, read [`docs/VISION_AND_GOALS.md`](docs/VISION_AND_GOALS.md): what OpenProcessor is -for, the v0.5.0 feature scope, and the standards the code is held to (no dead +for, the v0.6.0 feature scope, and the standards the code is held to (no dead code, no compatibility shims, fail-closed isolation, class identity by name never index, review depth matched to real risk). diff --git a/README.md b/README.md index 86c0d759..7257207b 100644 --- a/README.md +++ b/README.md @@ -53,12 +53,27 @@ README is the overview, the install and a first result. - **Dataset import, combine, export, train, promote.** YOLO, COCO and OpenProcessor-export layouts; merge up to eight projects; bake-off model comparison; promotion back into Triton. +- **Selective VLM assist.** `POST /assist/runs` sends a crop, a box, a + selection, a cluster, a sub-cluster or a filter to the VLM, whole or as a + representative sample, and proposes a cluster label you apply explicitly + (see [Selective VLM and segmenter assist](docs-site/docs/guides/selective-assist.mdx)). + A project calls no VLM until it activates one. +- **Bulk ingest.** A folder, a manifest or an S3 bucket runs as a background + job with a preview first; ingest can also be detection-only. - **Class identity by name.** Import, combine, export, train, promote and model sharing map classes by name, never by index. - **The lock rule.** A human-set or import-validated label or box, and anything in the frozen test holdout, is never overwritten by an automated writer. +**What's new in 0.6.0.** Faster ingest (images decoded once, uint8 inputs to +Triton, an FP16 embedding engine, GPU decode for large photos, multi-GPU +embedding, batched writes; numbers in [`docs/PERFORMANCE.md`](docs/PERFORMANCE.md)), +assist runs, bulk and S3 ingest, and a cleaned-up codebase. Upgrading from 0.5.0: +read the upgrade notes in [`CHANGELOG.md`](CHANGELOG.md) (new projects start +with the VLM off, `CROPWRIGHT_*` env names are now `OP_CROPWRIGHT_*`, the +evaluator has its own `evaluator` profile). + The whole curation surface is in [`contracts/openapi/curation.json`](contracts/openapi/curation.json); the guides are under [`docs-site/docs/guides/`](docs-site/docs/guides/). diff --git a/docs-site/docs/about/vision-and-goals.mdx b/docs-site/docs/about/vision-and-goals.mdx index 0b544f57..8e5d440f 100644 --- a/docs-site/docs/about/vision-and-goals.mdx +++ b/docs-site/docs/about/vision-and-goals.mdx @@ -21,7 +21,7 @@ The pipeline, end to end: ingest, detection, region stage (one or many boxes per embedding and clustering, human and AI review, import of labeled datasets, export, training, bake-off, and promotion back into serving. -## What v0.5.0 delivers +## What v0.6.0 delivers - Isolated projects, each with its own indexes, directories and class registry. - Editable prompt packs and region profiles, versioned and revertible. @@ -31,6 +31,8 @@ training, bake-off, and promotion back into serving. - Dataset import (YOLO, COCO, OpenProcessor exports) and project combine, with classes mapped by name. - A one-line installer. +- Assist runs: send a crop, a selection, a cluster or a filter to the VLM and apply the proposed label explicitly. +- Bulk ingest from a folder, a manifest or S3, with a preview, and measured ingest speed-ups (decode once, uint8 inputs, FP16 embeddings, GPU decode, multi-GPU embedding). - Documentation for every feature, and a public, reproducible cars-and-wheels example. ## Standards the code is held to @@ -44,10 +46,11 @@ training, bake-off, and promotion back into serving. operation. - Work is merged in waves, each with its own gate. -## Not in v0.5.0 +## Not in v0.6.0 -Deep GPU and Triton pipeline optimization (issue #40), benchmarks, a technical white -paper and more domain showcases are planned after the release. +The full zero-copy GPU pipeline (issue #40), the 1M-image scale step, request-driven +processing plans, faces and OCR on resident crops, a technical white paper and more +domain showcases are planned for v0.7.0 and after. ## More detail diff --git a/docs-site/docs/api-reference/curation.mdx b/docs-site/docs/api-reference/curation.mdx index 01128997..252fbfba 100644 --- a/docs-site/docs/api-reference/curation.mdx +++ b/docs-site/docs/api-reference/curation.mdx @@ -6,7 +6,7 @@ title: Curation routes # Curation routes :::info[Experimental] -The curation subsystem is experimental in v0.5.0. It is real and tested, +The curation subsystem is experimental in v0.6.0. It is real and tested, but backward compatibility across releases is not guaranteed until it graduates. It ships opt-in behind the `curation` compose profile. ::: diff --git a/docs-site/docs/getting-started/introduction.mdx b/docs-site/docs/getting-started/introduction.mdx index eda74468..00bf7678 100644 --- a/docs-site/docs/getting-started/introduction.mdx +++ b/docs-site/docs/getting-started/introduction.mdx @@ -54,7 +54,7 @@ The curation subsystem is **domain-agnostic**: nothing about "what an item or a region of interest is" is hardcoded. It is **bring-your-own-model**: you supply the item detector, and optionally a VLM, a region detector and training data. It ships **opt-in** behind the `curation` compose profile and, -like the rest of the API, has **no authentication**. v0.5.0 keeps no +like the rest of the API, has **no authentication**. v0.6.0 keeps no compatibility shims: a renamed environment variable is rejected at startup, not translated. diff --git a/docs-site/src/data/roadmap.json b/docs-site/src/data/roadmap.json index 69de2e01..96c89f60 100644 --- a/docs-site/src/data/roadmap.json +++ b/docs-site/src/data/roadmap.json @@ -2,6 +2,20 @@ "repo": "https://github.com/davidamacey/OpenProcessor", "note": "Hand-maintained, not derived from an issue tracker. 'Shipped' means it is on main today; 'Planned' items are design work that has not landed and may change after measurement. Update this file when scope actually changes; never describe work that isn't planned.", "releases": [ + { + "version": "v0.6.0", + "headline": "Faster ingest and assist runs: selective VLM labeling, bulk ingest, measured GPU pipeline steps", + "stage": "shipped", + "summary": "The measured first half of the Triton throughput plan (decode once, uint8 inputs, FP16 PE image encoder, GPU decode, multi-GPU embedding, coalesced writes) plus assist runs, bulk and S3 ingest, detection-only ingest and a repository cleanup. Numbers are in docs/PERFORMANCE.md.", + "items": [ + "Assist runs (/assist/runs): send a crop, box, selection, cluster, sub-cluster or filter to the VLM; cluster label proposals applied explicitly", + "Prompt-pack attributes replace the hard-coded make and model; a project calls no VLM until one is activated", + "Bulk ingest of a folder, manifest or S3 with a preview; detection-only ingest", + "Ingest speed-ups: decode once, uint8 inputs, FP16 PE image encoder, GPU decode, data-parallel PE encoding, coalesced writes, Lucene vector engine for new projects", + "Breaking: CROPWRIGHT_* env names are OP_CROPWRIGHT_*; /vlm/label_cluster and /vlm/verify_regions are replaced by /assist/runs; the evaluator has its own compose profile", + "Repository cleanup: guards, dead code removed, large files split, one typed env module, one docs source of truth" + ] + }, { "version": "v0.5.0", "headline": "The monorepo release: backend and Cropwright in one repository, label confirmation, one-port gateway", @@ -19,12 +33,11 @@ ] }, { - "version": "Next", + "version": "v0.7.0", "headline": "Triton throughput: keep images on the GPU", "stage": "next", - "summary": "A measured, wave-by-wave optimization plan for the inference and ingest pipeline. The goal is to decode each image once on the GPU and keep it resident across the models that need it, with every change behind a flag, a parity gate against today's outputs, and a p50/p99 benchmark gate. None of this has shipped.", + "summary": "The rest of the measured optimization plan: keep each decoded image resident on the GPU across the models that need it, with every change behind a flag, a parity gate against today's outputs and a p50/p99 benchmark gate. The 1M-image scale step is a goal here, not a claim. None of this has shipped.", "items": [ - "CPU-side fixes first: decode once, stop blocking the event loop, shorter Triton queue delays", "Smaller model outputs (end2end secondary detector, in-graph OCR argmax) and a GPU-resident OCR BLS", "GPU JPEG decode (DALI or nvJPEG, chosen by bake-off) and static Triton ensembles for /detect and /embed", "A GPU-resident curation ingest pipeline (BLS) that cuts crops and embeddings from the full-resolution frame", diff --git a/docs/CURATION.md b/docs/CURATION.md index c4babf45..04d19f03 100644 --- a/docs/CURATION.md +++ b/docs/CURATION.md @@ -5,7 +5,7 @@ This page is a map. The user guides live in the documentation site page that explains it. Headings that code, compose files and scripts cite are kept, so those references still resolve. -> **Status: experimental for v0.5.0.** The subsystem is working and tested, but its +> **Status: experimental for v0.6.0.** The subsystem is working and tested, but its > API can still change between releases. It ships opt-in, behind Docker Compose > profiles. diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index a30016ba..3e38da0e 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -2,7 +2,7 @@ Complete guide for optimizing FastAPI and Triton Inference Server performance. -> **Status (v0.5.0):** this guide covers the API-layer tuning, the measured results of the GPU and +> **Status (v0.6.0):** this guide covers the API-layer tuning, the measured results of the GPU and > Triton pipeline optimization milestone (summary table below; plan in > [`design/triton_pipeline_optimization_plan.md`](design/triton_pipeline_optimization_plan.md)), > and the benchmark protocol. The scale ladder (10k, 100k, 1M images) is not in this guide yet. diff --git a/docs/README.md b/docs/README.md index 9fdf94a9..d36f7580 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ Build the site with `make docs-build`. | Document | What it covers | |---|---| -| [VISION_AND_GOALS.md](VISION_AND_GOALS.md) | What the project is for, the v0.5.0 scope and the standards the code is held to (canonical) | +| [VISION_AND_GOALS.md](VISION_AND_GOALS.md) | What the project is for, the v0.6.0 scope and the standards the code is held to (canonical) | | [ARCHITECTURE.md](ARCHITECTURE.md) | Services, project isolation, data model, the multi-box region cascade, config store, workers | | [opensearch_schema_design.md](opensearch_schema_design.md) | Global and per-project index schemas, `region_boxes`, clustering | | [design/curation_api_contract.md](design/curation_api_contract.md) | Route table and wire models for `/curation` | diff --git a/docs/VISION_AND_GOALS.md b/docs/VISION_AND_GOALS.md index 6d91a21d..44784937 100644 --- a/docs/VISION_AND_GOALS.md +++ b/docs/VISION_AND_GOALS.md @@ -1,7 +1,7 @@ # OpenProcessor: Vision and Goals This document is the canonical statement of what OpenProcessor is for, what -the v0.5.0 release must deliver, and the standards the codebase is held to. +the v0.6.0 release delivers, and the standards the codebase is held to. It exists so that anyone — human or AI agent — working in this repository can orient quickly: `CLAUDE.md` and `README.md` both link here rather than restating it. @@ -28,10 +28,10 @@ Every stage is designed to be reused across domains by *configuring* it (detection profiles, prompt packs, class registries) rather than by forking the code per domain. -**Explicit owner scope decision:** the full v0.5.0 feature set below was +**Explicit owner scope decision:** the full v0.5.0 and v0.6.0 feature set below was deliberately *not* trimmed to ship faster. Nothing here is a stretch goal. -## What "done" means for v0.5.0 +## What "done" means for v0.6.0 - **Multiple isolated projects.** Each project is its own dataset workspace — its own indexes, its own directories, its own class @@ -104,20 +104,20 @@ release. touching data integrity, isolation, or security — an independent review before it lands, not batched into one release-day merge. -## What's explicitly not in v0.5.0, and why +## What's explicitly not in v0.6.0, and why -These are real, tracked commitments, not abandoned ideas — they are -sequenced deliberately rather than included now: +These are real, tracked commitments, not abandoned ideas (milestone v0.7.0): -- **Deep GPU / Triton inference optimization** (issue #40). Batching - strategy, GPU utilization, and a zero-copy pipeline (decode once, keep - data resident on the GPU across models) are real, planned work — but - they should be tuned against the *final* workload shape. Full-image - SAM3 detection has shipped in 0.4.1, so the call pattern is now known and - optimization work (plan: `docs/design/triton_pipeline_optimization_plan.md`) - follows the release. -- Benchmarks, a technical white paper, and additional domain showcases - beyond the car→wheel example are explicitly post-release deliverables. +- **The full zero-copy GPU pipeline** (issue #40). v0.6.0 ships the measured steps + (decode once, uint8 inputs to Triton, an FP16 PE image encoder, GPU decode for large + photos, data-parallel PE encoding, coalesced ingest writes); keeping the decoded + frame resident on the GPU across every model is v0.7.0 work. Plan: + `docs/design/triton_pipeline_optimization_plan.md`. +- **The 1M-image scale step.** The scale ladder is a goal, not a v0.6.0 claim. +- Request-driven processing plans (#67), faces and OCR on resident crops (#220), + smaller-GPU engine builds (#42) and slimmer images (#41). +- A technical white paper and additional domain showcases beyond the car-to-wheel + example. ## Where to look for more detail diff --git a/docs/design/README.md b/docs/design/README.md index 76892c65..e9f83232 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -18,12 +18,12 @@ in GitHub issues; the repository file is the canonical, versioned copy of each p |---|---|---|---| | [generic_detector_and_selective_embedding_plan.md](generic_detector_and_selective_embedding_plan.md) | implemented (W0 to W9; deferred: arbitrary-filter clustering, model-delete guard for a project's own detector, optional `backfill_embedding_state.py`) | #52 | Default detector, store every detection, per-project embedding policy | | [sam3_full_image_detection_plan.md](sam3_full_image_detection_plan.md) | implemented (waves 1 to 8; as-built deviations in section 16) | #30 (shares the gating layer with #46, region-stage gating #46 also shipped) | Open-vocabulary full-image detection with SAM 3 | -| [triton_pipeline_optimization_plan.md](triton_pipeline_optimization_plan.md) | open (Wave 0 tooling and the v0.5.0 set A baseline in PR #206; optimization waves not started; sequenced for v0.6.0 by `v060_wave_plan.md`) | #40 | Decode once, stay on GPU, Triton pipeline optimization with before/after numbers; design detail and accuracy gates | -| [v060_wave_plan.md](v060_wave_plan.md) | open (plan only) | #208 to #221, #207, #153 (umbrella #40) | v0.6.0 sequencing: FP16 PE and engine build stamp, API and idle CPU, cluster scheduling, INT8 evaluation, uint8 inputs, shared memory, GPU-resident ingest, clustering acceleration, downstream parity chain | +| [triton_pipeline_optimization_plan.md](triton_pipeline_optimization_plan.md) | partially done (shipped in 0.6.0: the baseline, decode once, uint8 inputs, FP16 PE engine, GPU decode, multi-GPU PE encoding, coalesced writes; the fully GPU-resident pipeline and the 1M scale step are v0.7.0, issue #40) | #40 | Decode once, stay on GPU, Triton pipeline optimization with before/after numbers; design detail and accuracy gates | +| [v060_wave_plan.md](v060_wave_plan.md) | implemented in 0.6.0 (waves executed; measured results in `docs/PERFORMANCE.md`; the unfinished items moved to milestone v0.7.0) | #208 to #221, #207, #153 (umbrella #40) | v0.6.0 sequencing: FP16 PE and engine build stamp, API and idle CPU, cluster scheduling, INT8 evaluation, uint8 inputs, shared memory, GPU-resident ingest, clustering acceleration, downstream parity chain | | [v060_serving_backend_trade_study.md](v060_serving_backend_trade_study.md) | open (study and experiment plan; nothing implemented) | WP-S1 to WP-S7 issues (umbrella #40, #67) | Roofline ceilings per stage on one and two A6000s, trade matrix of serving architectures (Triton hardened, bulk ingest runner, ortloom-serve, custom offline), recommendation, gated experiments toward a 1M-image batch | | [gpu_decode_plan.md](gpu_decode_plan.md) | implemented (built and on by default above 3 MP; build and A/B in section 13 of the doc) | #217 (WP-2.4; related #67, #219, #220) | GPU decode once and a GPU-resident ingest GPU decode: dataflow map, decode and resize costs per image size, contention with PE, decoder coverage, parity and AP effect, size gate, placement, batching, implementation items | -| [v060_vector_db_evaluation_plan.md](v060_vector_db_evaluation_plan.md) | open (plan only) | #221, #56 | Fair benchmark of OpenSearch k-NN (shipped and tuned) against dedicated vector stores, datastore needs beyond vectors, split-architecture costs, decision criteria | -| [ingest_surfaces_plan.md](ingest_surfaces_plan.md) | open (plan only; implementation is a later task) | #264, #263 | S3 and S3-compatible bulk source, webhook, schedule and poll-with-checkpoint triggers on one queue, Cropwright sources panel, and the measured decision on a gRPC ingest channel | +| [v060_vector_db_evaluation_plan.md](v060_vector_db_evaluation_plan.md) | implemented (tuned OpenSearch compared with Milvus, Qdrant, pgvector and LanceDB; results in `docs/PERFORMANCE.md`; Lucene engine adopted for new projects) | #221, #56 | Fair benchmark of OpenSearch k-NN (shipped and tuned) against dedicated vector stores, datastore needs beyond vectors, split-architecture costs, decision criteria | +| [ingest_surfaces_plan.md](ingest_surfaces_plan.md) | implemented in 0.6.0 (S3 and registered sources, bulk ingest runner, poll, cron and webhook triggers, Cropwright sources panel; transports measured and gRPC skipped) | #264, #263 | S3 and S3-compatible bulk source, webhook, schedule and poll-with-checkpoint triggers on one queue, Cropwright sources panel, and the measured decision on a gRPC ingest channel | | [release_acceptance_plan.md](release_acceptance_plan.md) | open (release scripts, `make release-verify` and API-level live tests exist; the full clean-install run is not scripted) | #54 (installer live test #44) | The scripted clean-install-to-uninstall run that gates each release | | [combine_preview_warnings_plan.md](combine_preview_warnings_plan.md) | implemented | #55 | `embedding_model_mismatch` and `region_profiles_differ` combine preview warnings | | [storage_sizing_and_ingest_baselines.md](storage_sizing_and_ingest_baselines.md) | partially done (measurements done; re-measured on the public set in PR #206; quantization benchmark open, planned in `v060_vector_db_evaluation_plan.md`) | #45, #53, #56 | Measured OpenSearch storage per vector, item and image; sizing formula; ingest and pipeline baseline to repeat | diff --git a/docs/releases/RELEASE_CHECKLIST.md b/docs/releases/RELEASE_CHECKLIST.md index a5ccfcf1..ca57065c 100644 --- a/docs/releases/RELEASE_CHECKLIST.md +++ b/docs/releases/RELEASE_CHECKLIST.md @@ -1,4 +1,4 @@ -# v0.5.0 release checklist +# v0.6.0 release checklist Every step is manual and owner-run. Run from a clean checkout of the release commit. Nothing here is automated by CI (images are built locally). @@ -10,7 +10,7 @@ line per key. The keys come from `scripts/lib/image_keys.sh`, the one table the script and the installer share. After `make release` there are 17 lines: - Six images built from this repository and pushed by step 4 - (`davidamacey/...:0.5.0`, plus `latest`): `api` (`openprocessor`), `triton` + (`davidamacey/...:0.6.0`, plus `latest`): `api` (`openprocessor`), `triton` (`openprocessor-triton`), `evaluator` (`openprocessor-evaluator`), `segmenter` (`openprocessor-segmenter`), `trainer` (`openprocessor-trainer`) and `cropwright` (`cropwright`, built from `frontend/Dockerfile` with `frontend/` as its build context). @@ -19,18 +19,15 @@ script and the installer share. After `make release` there are 17 lines: `alloy`, `node_exporter`, `dcgm_exporter`. Each tag must be pulled locally first (`docker pull `), because the script reads its RepoDigest. -The committed `images.lock` still holds the 0.4.1 digests (16 lines, no `cropwright` row) +The committed `images.lock` still holds the 0.5.0 digests (17 lines, including the `cropwright` row) until step 4 rewrites it entirely; do not hand-edit it, and do not bump it ahead of the -release. In lock mode the installer dies with "images.lock has no 'cropwright' entry" for -every tier, because it requires a row for each build key, so a lock-mode install of this -branch is impossible until step 4. That is expected: `make release-verify` builds its bundle -with `ALLOW_UNPINNED_LOCK=1` and installs by `--image-tag`, and the installer tests pass with the -committed lock as it is. The version-consistency test does not compare lock tags with `VERSION`; -the bundle builder checks only that every line is digest-pinned and not `latest`. +release. `make release-verify` builds its bundle with `ALLOW_UNPINNED_LOCK=1` and installs by +`--image-tag`. The version-consistency test does not compare lock tags with `VERSION`; the bundle +builder checks only that every line is digest-pinned and not `latest`. Cropwright is no longer released from its own repository or its own lock. There is no `cropwright.lock`, no `CW_RELEASE_DIR` staging and no `cropwright-release/` directory in the -tarball; do not run `frontend/scripts/release/*` for 0.5.0. The `cropwright` tier is a compose +tarball; do not run `frontend/scripts/release/*` for 0.6.0. The `cropwright` tier is a compose profile of the main project and takes its image from `images.lock`. ## Base image digests: the bump procedure @@ -54,12 +51,12 @@ To bump a base (a Dependabot docker PR does the same edit): ## Release decisions (#44, #63): all DECIDED -Carried forward from 0.4.1; nothing here changes for 0.5.0 unless noted. +Carried forward from 0.5.0; nothing here changes for 0.6.0 unless noted. -- **`:latest` tag: DECIDED.** v0.5.0 is the numbered release and `latest` is published - together with it, so `latest` equals `0.5.0` at release for all six images, including +- **`:latest` tag: DECIDED.** v0.6.0 is the numbered release and `latest` is published + together with it, so `latest` equals `0.6.0` at release for all six images, including `davidamacey/cropwright` (whose `latest` previously followed the standalone 0.1.x line). - `make release` pushes `:0.5.0` and `:latest` for each image, and refuses a non-X.Y.Z + `make release` pushes `:0.6.0` and `:latest` for each image, and refuses a non-X.Y.Z version, so `latest` can only ever point at a stable release. A backport or an old line is pushed by hand with `docker push` of the version tag only. The installer pins digests from `images.lock` and never uses `latest`. @@ -89,11 +86,10 @@ Carried forward from 0.4.1; nothing here changes for 0.5.0 unless noted. change it before exposing the stack beyond the LAN. Gateway mode refuses the default. - The API has no authentication (documented in `SECURITY.md`). - #44 (clean-host install from the published artifacts) stays open until run against the - published 0.5.0 assets. + published 0.6.0 assets. - The 430 px UI fix (#184) is covered by the stubbed e2e suite only. -- #119 and #61 live acceptance: record the results in `docs/releases/v0.5.0.md` (every - `TODO-LIVE` marker there must be replaced before step 10; `rg TODO-LIVE docs/releases` - must find nothing). +- Fill the placeholder blocks in `docs/releases/v0.6.0.md` (the 100k results and the verification + summary; `rg ' + +**Repository and operations** +- Guards in pre-commit (file size, layering, complexity, broad excepts, unused code, cited paths, + mypy with untyped defs refused), digest-pinned base images, environment variables read through one + typed module, and the large source files split by concern. +- The documentation site is the single source of truth for user and operator docs; a script checks + docs against the code (routes, `OP_*` variables, CLI commands, make targets, links). +- New monitoring alerts (Triton model not ready and others, #290), a git-ignored + `docker-compose.local.yml` overlay, `./openprocessor curation up [service...]`, and + `./openprocessor profile` that sets the GPU decode flag. + +## Upgrade notes / behaviour changes + +Read these before upgrading from 0.5.0. + +Read these before upgrading from 0.5.0. + +- **New projects start with the VLM off** (#282). A project calls no VLM until it activates one (`GET .../vlm/endpoints/active` answers `source: "off"`). Projects that exist already are pinned once, at the first start of this version, to what they had (`env` when `OP_VLM_URL` was set, else `off`), so their behaviour does not change. +- **`make` and `model` are no longer hard-coded VLM outputs.** The reply keys, `vlm_item_make` and `vlm_item_model` are gone (old values stay in old documents, unread). Declare such facts as prompt-pack `attributes`; assist runs store them in the item's `assist` object. +- **The VLM routes `label_cluster` (per cluster) and `verify_regions` are removed.** Use `POST /assist/runs` (a `cluster` target, or `verify` on a `crops` or `boxes` target), or `?cluster_id=` on `auto_label/start`. +- **`curation-evaluator` moved from the `curation` profile to its own `evaluator` profile**, so `--profile curation up -d` no longer reserves a GPU for it. Start it with `./openprocessor curation up curation-evaluator`. +- **`CROPWRIGHT_*` env names are now `OP_CROPWRIGHT_*`** (`_IMAGE`, `_BIND_ADDRESS`, `_PORT`, `_INGEST_MAX_REQUEST_MB`, `_DATASET_UPLOAD_MAX_MB`). A `.env` that still sets an old name is refused at startup (`./openprocessor` and the installer exit 2, the container exits). Rename the keys and run `./openprocessor restart`. +- **Region-profile fields `aspect_min` and `aspect_max` are retired** (and the matching aspect min and max region-detection variables). A stored profile that carries either is rejected by name, and skipped with a warning when the registry loads; re-save it without them. +- **Rebuild the detector and PE image encoder engines** (`./openprocessor models repair`): both now take uint8 input, and a model whose plan and `config.pbtxt` disagree does not reach READY. +- **GPU decode is on by default** for the `full` profile (`OP_INGEST_GPU_DECODE`; `minimal` and `standard` default off). It adds about 8.5 GB to the Triton card. +- **New projects use the Lucene vector engine** (`ef_construction` 128); existing projects keep their faiss fields (no migration; re-ingest to move one). +- **Tests default `OPENSEARCH_URL` to a refused local address**, so an offline test run never reaches a live OpenSearch; set `OPENSEARCH_URL` for the live suite. +- **Retired region statuses `pending` and `pending_verify`, `regions.by_human` (use `regions.by_human_drew`) and `scripts/curation/region_worker_main.py` are removed.** `scripts/curation/ingest_walker.py` is replaced by `scripts/curation/bulk_ingest.py` (`--folder` instead of the old flags). +- **`TRITON_WORKERS`, `MAX_BATCH_SIZE` and `SHM_SIZE` are gone** from `.env`, `env.template` and the profile JSON (nothing read them); `setup.sh` no longer writes `docker-compose.override.yml`. +- **New environment variables** (all optional, documented in `env.template`): `OP_ASSIST_JOBS_DIR`, `OP_BULK_INGEST_DIR`, `OP_API_LOOPBACK_URL`, `OP_CPU_POOL_THREADS`, `OP_INGEST_INFLIGHT_MB`, `OP_INGEST_GPU_DECODE`, `OP_INGEST_GPU_MIN_MEGAPIXELS`, `OP_INGEST_DISPATCH_S`, `OP_CLUSTER_REFRESH_INTERVAL_S`, `_GROWTH_THRESHOLD`, `_MAX_DEFERRAL_S`, `_RETRY_BACKOFF_S` and `_AUTO_LABEL`, `OP_S3_ALLOWED`, `OP_S3_ENDPOINT_URL`, `OP_S3_REGION`, `OP_S3_PATH_STYLE`, `OP_S3_FETCH_WORKERS`, `OP_S3_LIST_WORKERS`, `OP_S3_INFLIGHT_MB`, `OP_S3_MAX_OBJECT_MB`, `OP_SOURCE_ROOT_HOST`, `OP_VLM_POLL_S`, `OP_VLM_WAIT_S`, plus the renamed `OP_CROPWRIGHT_*` set. `OP_HNSW_EF_CONSTRUCTION` now documents the new default of 128. +- **Compose changes**: the `evaluator` profile above; Triton gets `CUDA_DEVICE_ORDER=PCI_BUS_ID` (so `TRITON_GPU_ID=0,2` means the lowest listed id first), `CUPY_CACHE_DIR` and glibc malloc tuning for the API; Triton runs with `--log-verbose=0`; the API's memory reservation uses the `deploy.resources.reservations` form; an optional git-ignored `docker-compose.local.yml` is loaded after the stock files; the `gateway-wrap` message names `OP_CROPWRIGHT_BIND_ADDRESS`. Image defaults now point at `0.6.0`. + +Upgrade with `./openprocessor upgrade` (or `./setup-openprocessor.sh --version v0.6.0`); it backs up +first and `--rollback` restores the newest backup. Then run `./openprocessor models repair` to rebuild +the uint8 detector and PE engines, and hard-refresh the browser after upgrading. Cropwright 0.6.0 +requires backend 0.6.0. + +## Requirements + +Unchanged from 0.5.0: Linux, Docker with Compose v2, an NVIDIA GPU (Ampere or newer) with driver 535+ +and the NVIDIA Container Toolkit; about 16 GB VRAM for `core` (6 to 8 GB cards use `--profile +minimal`); 16 GB RAM minimum; about 60 GB disk for `core`; `vm.max_map_count` of at least 262144. +The GPU decode adds about 8.5 GB to the Triton card, so it is on by default only for the `full` +profile. + +## Quick start + +```bash +curl -fsSL https://raw.githubusercontent.com/davidamacey/OpenProcessor/main/setup-openprocessor.sh | bash +``` + +Pin this release with `bash -s -- --version v0.6.0`; add `--dry-run` to see every command first. +Then open `http://localhost:4603/docs`. Full guide: `INSTALLATION.md`. + +## Known limitations + +- **The zero-copy GPU pipeline and the 1M-image scale step are v0.7.0.** 0.6.0 decodes once and + decodes large photos on the GPU, but does not keep a decoded frame resident across every model. + The v0.7.0 milestone: + - #40 Triton inference pipeline optimization (decode once, stay on GPU) + - #41 slim the container images + - #42 smaller-GPU (12 GB) TensorRT engine builds + - #67 request-driven processing plans + - #84 car-wheel teacher-student loop on public COCO + - #219 work crops at model size for the VLM and region consumers + - #220 faces and OCR on resident crops + - #231 ortloom-serve against Triton on the ingest-path models + - #268 stage 2 follow-ups (set B photos, minimal and standard profiles, two-card instance sweep) + - #286 iterative cluster-sampled VLM auto-labeling +- **The API has no authentication.** Do not expose it; see `SECURITY.md`. Prometheus and MLflow have + no authentication either, and Grafana keeps the default `admin` password on LAN-only installs. +- Existing projects keep their faiss vector fields; only new projects use the Lucene engine. +- #259: the embed reprocess pass stops at about 58 percent GPU and its crop vectors differ slightly + from ingest vectors (cosine minimum 0.9855). +- Open follow-ups: #101 (pin the VLM connection to the validated IP), #154 (the trainer still uses + deprecated MLflow registry stages), #155 (SvelteKit `config.alias` deprecation). +- The documentation screenshots may predate the newest UI changes. + +## Verification summary + + + +## Credits and attribution + +Third-party code and licenses: `ATTRIBUTION.md` and the docs licenses page +(`docs-site/docs/reference/licenses.mdx`). Thanks to the upstream projects this release builds on: +NVIDIA Triton and TensorRT, OpenSearch, Ultralytics YOLO, SCRFD and ArcFace, MobileCLIP, PaddleOCR +PP-OCRv5, SAM 3, vLLM and MLflow. Public docs, tests and benchmarks use license-checked public +datasets (COCO, Open Images). From 62200044075010d7af3c967c0750adb430437f94 Mon Sep 17 00:00:00 2001 From: davidamacey Date: Fri, 16 Oct 2026 01:13:40 -0400 Subject: [PATCH 04/17] docs(release): name only the current release in docs and rename the 0.5.0 baseline section --- docs-site/docs/about/vision-and-goals.mdx | 2 +- docs-site/src/data/roadmap.json | 16 ------- docs/PERFORMANCE.md | 48 +++++++++---------- docs/VISION_AND_GOALS.md | 6 +-- .../triton_pipeline_optimization_plan.md | 2 +- docs/design/v060_wave_plan.md | 2 +- frontend/e2e/stubbed/test_ingest.py | 2 +- frontend/src/lib/test/fixtures/bulkIngest.ts | 2 +- .../curation/bulk_ingest/preflight_rates.py | 2 +- 9 files changed, 33 insertions(+), 49 deletions(-) diff --git a/docs-site/docs/about/vision-and-goals.mdx b/docs-site/docs/about/vision-and-goals.mdx index 8e5d440f..9d4537c8 100644 --- a/docs-site/docs/about/vision-and-goals.mdx +++ b/docs-site/docs/about/vision-and-goals.mdx @@ -50,7 +50,7 @@ training, bake-off, and promotion back into serving. The full zero-copy GPU pipeline (issue #40), the 1M-image scale step, request-driven processing plans, faces and OCR on resident crops, a technical white paper and more -domain showcases are planned for v0.7.0 and after. +domain showcases are planned for the next release and after. ## More detail diff --git a/docs-site/src/data/roadmap.json b/docs-site/src/data/roadmap.json index 96c89f60..b57756fd 100644 --- a/docs-site/src/data/roadmap.json +++ b/docs-site/src/data/roadmap.json @@ -16,22 +16,6 @@ "Repository cleanup: guards, dead code removed, large files split, one typed env module, one docs source of truth" ] }, - { - "version": "v0.5.0", - "headline": "The monorepo release: backend and Cropwright in one repository, label confirmation, one-port gateway", - "stage": "shipped", - "summary": "The Triton-backed core API and the opt-in /curation active-learning subsystem, now released together with the Cropwright frontend (frontend/) from one repository and one docs site. Adds label confirmation (VLM scope policy, detector class kept beside VLM labels, accuracy audit, audit-gated auto-promote), the gateway, and a refreshed dependency set.", - "items": [ - "Backend and the Cropwright web UI in one repository, built and pinned together; one docs site", - "Label confirmation: per-project VLM scope policy, the detector's own class kept beside VLM labels, a detector_disagreements review tab, an accuracy audit with Wilson 95% intervals, and an audit gate on auto-promote", - "Optional detector hint for VLM prompt packs (detector_hint_min_confidence_pct, off by default; +7.8 points overall accuracy on a COCO oracle)", - "Gateway: Grafana, Prometheus, OpenSearch Dashboards and MLflow on sub-paths of the single Cropwright port (OP_GATEWAY_SUBPATHS)", - "Breaking: the compose service yolo-api is now api; the unread ocr_det_* region profile fields are removed", - "/curation: 266 method routes in the generated OpenAPI contract", - "Refreshed dependencies: FastAPI 0.143 and Pydantic 2.14, SvelteKit 3 and TypeScript 6, MLflow 3.17, PyTorch 2.14.1 segmenter base, Playwright 1.63", - "Opt-in compose profiles: curation, segmenter, vlm, training, monitoring, benchmark, cropwright" - ] - }, { "version": "v0.7.0", "headline": "Triton throughput: keep images on the GPU", diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 3e38da0e..d9dd097d 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -403,8 +403,8 @@ ingested at **13.39 images/s** (0 failed, 1.37 crops per image, batch p50 10.0 s 4,000-image mixed set that is half 12 to 20 MP JPEGs ingested at 4.90 images/s (about 15 for the small photos, about 2.5 for the large ones; decode and resize dominate). Those runs used the narrow vehicle detector; the full-vocabulary -policy rows above are computed, not measured. The v0.5.0 re-measurement on the pinned -2,000-image set is in "v0.5.0 baseline" below. +policy rows above are computed, not measured. The 0.5.0 re-measurement on the pinned +2,000-image set is in "0.5.0 baseline" below. ### Measured VLM labeling speed @@ -510,10 +510,10 @@ Triton compute_infer, A all: pe_image_encoder 1,590 s total for 33,388 inference A second baseline on a set of real high-resolution (about 20 MP) photos was measured privately; its numbers are not published. It showed the same pipeline is decode and I/O bound at that image size (the GPU was mostly idle), which is what the GPU decode and crop-at-model-size waves in the optimization plan target. Single run per row; a 3-round median is planned before the optimization work starts. -### v0.5.0 baseline (set A, public COCO, 2,000 images) +### 0.5.0 baseline (set A, public COCO, 2,000 images) The Wave 0 baseline of `docs/design/triton_pipeline_optimization_plan.md`, measured on the -published v0.5.0 images before any optimization (issues #45 and #40). Raw JSON: +published 0.5.0 images before any optimization (issues #45 and #40). Raw JSON: `docs/benchmarks/v050_baseline.json` (environment, ingest, Triton statistics, perf_analyzer points, endpoints, VLM) and `docs/benchmarks/v050_storage.json` (the storage run); idle-CPU readings in `docs/benchmarks/v050_idle_background.json`. (The VLM model label in the raw JSON metrics is replaced by `catalog-default`.) Every @@ -524,7 +524,7 @@ unless stated. **8.9 images/s** through `POST /curation/projects/{project}/ingest/upload` and **9.2 images/s** through `.../ingest/batch`, with the GPU 91 percent busy. The run-to-run spread is under 6 percent. 90 percent of the GPU time is -the PE image encoder (7.4 embeddings per image). That engine is FP32, and a fresh v0.5.0 +the PE image encoder (7.4 embeddings per image). That engine is FP32, and a fresh 0.5.0 install builds the same FP32 engine (see Caveats), so this is the true "before" for the default install. @@ -543,11 +543,11 @@ default install. Software under test: the images pinned by `images.lock` (API `openprocessor:0.5.0` `sha256:b0af838e...`, Triton `openprocessor-triton:0.5.0` `sha256:1ac56ac9...`, -OpenSearch `3.6.0`), installed with `setup-openprocessor.sh --version v0.5.0 --tiers +OpenSearch `3.6.0`), installed with `setup-openprocessor.sh --version 0.5.0 --tiers core,curation --local-only --bind 127.0.0.1 --skip-models` into an isolated compose project on its own ports, GPU 0 only. The VLM worker was stopped for the measured ingest runs. OpenSearch heap 8 GB, no replicas, one node. Models: TensorRT engines built on 2026-09-26 and copied -from an earlier v0.5.0 install on the same host. The PE image encoder engine is FP32 +from an earlier 0.5.0 install on the same host. The PE image encoder engine is FP32 (1.27 GB); every other engine is consistent with FP16 by file size. #### Method @@ -723,7 +723,7 @@ already seen is much faster (27 to 31 crops/s) because of vLLM's multimodal cach image) that build fails at parse time on both `Einsum` nodes: `IEinsumLayer must have all inputs of same type. Input 1 has type Half and input 0 has type Float` (`/visual/Einsum`: the FP16 graph rewrite converted its float `Constant` to half but left the explicit float32 `Cast` that - feeds the other input). The installer then retries from the FP32 ONNX, which builds. A fresh v0.5.0 + feeds the other input). The installer then retries from the FP32 ONNX, which builds. A fresh 0.5.0 install therefore ends up with the same FP32 engine as this baseline, so the baseline is the true "before" for the default install. The fix and its measured effect are in the next section. - OpenSearch data and the upload store live on a RAID array (Docker data root), not on NVMe. @@ -746,7 +746,7 @@ already seen is much faster (27 to 31 crops/s) because of vLLM's multimodal cach The first change of `docs/design/triton_pipeline_optimization_plan.md` after the baseline above: the PE image encoder is built as a real FP16 engine. Same pinned set, same harness -(`scripts/bench/baseline_suite.py --phases env,ingest`), same host and GPU 0, same published v0.5.0 +(`scripts/bench/baseline_suite.py --phases env,ingest`), same host and GPU 0, same published 0.5.0 images; the only difference is the engine behind `pe_image_encoder`. Raw JSON: `docs/benchmarks/v060_fp16_pe.json` (ingest, Triton statistics, GPU) and `docs/benchmarks/v060_fp16_pe_engine.json` (engine size, build time, parity). Medians with the @@ -759,7 +759,7 @@ stay equivalent to the FP32 engine (cosine similarity 0.9996 mean, 0.9976 minimu #### Root cause and fix -The v0.5.0 FP16 bake (`bake_fp16_onnx` in `export/trt_utils.py`) never produced a buildable graph on +The 0.5.0 FP16 bake (`bake_fp16_onnx` in `export/trt_utils.py`) never produced a buildable graph on TensorRT 11.1, and the installer hid it by retrying from FP32. Two defects, both found by building the baked ONNX with `trtexec`: @@ -783,7 +783,7 @@ Caveats). #### Engine and parity -| Item | FP32 (v0.5.0) | FP16 (this change) | +| Item | FP32 (0.5.0) | FP16 (this change) | |---|---:|---:| | Plan size | 1,272,018,292 B | 644,778,380 B (0.51x) | | `trtexec` engine build | 38 s | 81 s | @@ -847,7 +847,7 @@ The earlier private FP16 measurement of about 170 embeddings/s (5.9 ms) matches - Parity covers 200 public images on whole-frame preprocessing. Retrieval and clustering gates on labelled data (plan section 7) are open. - The fix bakes FP16 for every model that calls `bake_fp16_onnx`. On TensorRT 11.1 the MobileCLIP - image encoder's v0.5.0 bake also failed at parse (`ElementWiseOperation DIV must have same input + image encoder's 0.5.0 bake also failed at parse (`ElementWiseOperation DIV must have same input types`); with the reconcile pass it parses, but the build stopped at `Could not find any implementation for node .../reparam_conv/Conv + PWN(...)` (#224). The failing layers are the depthwise 7x7 stride-2 convs with a channel multiplier of 2 (the three @@ -1064,8 +1064,8 @@ reference: [parity_references.md](benchmarks/parity_references.md). #### FP16 against FP32 PE engine -Same isolated stack (the published v0.5.0 images, GPU 0, own ports, no VLM), only the file behind -`pe_image_encoder` changes. The FP32 plan is the one of the v0.5.0 baseline (1,272,018,292 B). The FP16 plan +Same isolated stack (the published 0.5.0 images, GPU 0, own ports, no VLM), only the file behind +`pe_image_encoder` changes. The FP32 plan is the one of the 0.5.0 baseline (1,272,018,292 B). The FP16 plan is built through the fixed installer path (`export/trt_utils.py` bake, then `trtexec` with the shapes of `scripts/lib/model_setup.sh`): 641,185,140 B, built in 95 s. Each configuration was captured twice, once with 4 and once with 1 upload request in flight (run 1 and run 2), and the FP32 plan was captured @@ -1132,7 +1132,7 @@ Reading the numbers: #### Caveats -- One host, one GPU shared with other projects, the published v0.5.0 images; the container image ids are +- One host, one GPU shared with other projects, the published 0.5.0 images; the container image ids are in each `capture.json`. The summaries of the four comparisons are in `docs/benchmarks/parity/`. - The VLM gate and the label agreement of a registry-labelled project are not exercised: no VLM is configured and the project has no class registry, so stored class names are empty. The label gates here @@ -1186,8 +1186,8 @@ parentheses. | GPU | one RTX A6000 (GPU 2, 300 W cap, 48 GB), driver 615.71.09, CUDA 13.4; foreign SM load on it 0.0 percent in every sample of all 198 measured runs | | Other GPUs | GPU 0 holds another project (16 GB resident, up to 95 percent utilization during one of my windows), GPU 1 another (7 GB): neither was used | | Host | 2 x Xeon E5-2680 v3, 48 logical cores, 504 GB RAM, load average 5 to 59 over the runs (other projects), whole-host CPU busy 10 to 77 percent (columns below) | -| Stack | published v0.5.0 images (`images.lock` digests), isolated compose project, own ports, loopback only, OpenSearch heap 8 GB, no replicas, 32 API workers, curation workers and VLM not running | -| PE engine | FP16, baked with the fixed `export/trt_utils.py` and built with `trtexec` as in Wave 1: plan 644,771,508 bytes; all other engines as installed by v0.5.0 | +| Stack | published 0.5.0 images (`images.lock` digests), isolated compose project, own ports, loopback only, OpenSearch heap 8 GB, no replicas, 32 API workers, curation workers and VLM not running | +| PE engine | FP16, baked with the fixed `export/trt_utils.py` and built with `trtexec` as in Wave 1: plan 644,771,508 bytes; all other engines as installed by 0.5.0 | | Set A | the pinned COCO set (2,000 images, 640 x 480 median 166 KB); sweeps use its first 1,000 images, the headline rows all 2,000 | | Set B | 500 public photos from the Wikimedia Commons featured-picture category, CC0, public domain, CC BY and CC BY-SA, 12.0 to 24.0 MP (median 17.0, mean 17.4), 5.2 GB (mean 10.4 MB), pinned in `scripts/datasets/manifests/commons_hires_500.json` (manifest sha256 `c013c1f9...`, seed 20261010, fetched with `scripts/datasets/fetch_hires_set.py`) | | Protocol | the Wave 1 protocol: fresh project per repetition, warm-up project first, 3 repetitions, policy `all`; clients are threads of one Python process; cluster training and the storage merge are skipped in these runs (`--skip-cluster --skip-settle`) so that the window is the ingest alone | @@ -1572,7 +1572,7 @@ those costs, fixes the ones that measure as worth fixing, and repeats the Wave 0 Medians with the minimum and maximum of 3 repetitions in parentheses. **Method.** The Wave 0 stack and protocol on GPU 2 of the same host (an isolated compose project, -loopback ports, the published v0.5.0 images with the branch's `src/` mounted over `/app/src`, +loopback ports, the published 0.5.0 images with the branch's `src/` mounted over `/app/src`, the FP16 PE plan of the Wave 0 reference, plan sha256 `b9a6f397...`, 32 uvicorn workers, a fresh OpenSearch for each series). Set A is the pinned COCO set (first 1,000 images, `/ingest/upload`, 32 images per request), set B the 500 photos of `commons_hires_500.json` (`/ingest/batch`, 8 images per @@ -1801,13 +1801,13 @@ Runtime backend (`output0`, used by the region detector) and the bake-off engine #### Method -One RTX A6000 (GPU 2), the published v0.5.0 images with the checkout's `src/` mounted over `/app/src`, +One RTX A6000 (GPU 2), the published 0.5.0 images with the checkout's `src/` mounted over `/app/src`, an isolated compose project on loopback, OpenSearch 8 GB heap, 32 API workers, a fresh stack per configuration, policy `all`. Set A: the pinned COCO set (2,000 images, `coco_bench_2000.json`), `scripts/bench/baseline_suite.py --phases env,ingest --reps 3 --skip-cluster --skip-settle` (32 images per request, 4 clients; upload and batch). Set B: the 500 pinned photos, `scripts/bench/ingest_sweep_suite.py --phases sweep --sweep-points 8x16 --modes batch`. Configurations: **base** is `origin/main` with the -detector plan of the v0.5.0 install (Sep 26 build) and an FP16 PE plan from `build_pe_trt.sh`; +detector plan of the 0.5.0 install (Sep 26 build) and an FP16 PE plan from `build_pe_trt.sh`; **detector uint8** is the first commit of this change (PE still FP32); **final** is both. The uint8 detector plan was built with the real exporter path (`export_models.export_trt_end2end`), the PE plan with `build_pe_trt.sh`, both on the TensorRT 11.1 of the Triton image. @@ -1911,7 +1911,7 @@ Three findings, in order of importance. crop rectangles are identical on 98.7 percent of items against the control and 99.1 to 99.2 percent between two runs of the same engine. 2. **Rebuilding the detector engine, with no input change, fails the gates against `ref-fp16`.** - The v0.5.0 detector plan (Sep 26) was built from an ONNX baked before the FP16 cast fix of the FP16 + The 0.5.0 detector plan (Sep 26) was built from an ONNX baked before the FP16 cast fix of the FP16 PE section; the plan rebuilt today from the same weights moves 6.5 percent of images beyond the box gate, 2 confident boxes go unmatched, and crop-vector agreement drops to 0.978 (1st percentile). This is independent of the input type (the "rebuilt FP32" row has no uint8 in it) and applies to every @@ -1957,7 +1957,7 @@ shipped**: no setting, no compose change, no client code. The implementation and commits on the local branch `perf/shared-memory-impl` (`4de40001`, `8e20afe5`), for reference only. **Method.** The isolated stack of the uint8 section (GPU 2, foreign load 0.0 percent in every sampled -run, published v0.5.0 images with the branch `src/` mounted, uint8 detector and PE plans, PE plan sha256 +run, published 0.5.0 images with the branch `src/` mounted, uint8 detector and PE plans, PE plan sha256 `a4feb7df...`, a fresh project per repetition, policy `all`), one extra tmpfs volume mounted as `/dev/shm` in both containers and `--allow-client-shm=true` on Triton. Wire and shared memory were the same code with the transport switch off or on, the API container recreated between launches, arms interleaved (wire, shm, shm, @@ -2731,7 +2731,7 @@ does the packing across concurrent requests; the fix is to have concurrent reque |---|---| | GPU | RTX A6000, GPU 2, 0.0 percent foreign SM share in every counted repetition; another project held GPU 0 at 100 percent and was not touched. SM clock 1,416-1,444 MHz (c16), 1,449-1,482 MHz (c4) | | Host | 48 logical cores shared with other projects; 1-minute load 5-9 on the c16 and c4 repetitions, 12-20 at c32, 14-25 on set B; host 12-15 percent busy (set A, c4 and c16) | -| Stack | project `xbatch`: Triton, API (the published v0.5.0 image with this checkout's `src/` mounted), OpenSearch 8 GB heap, 32 API workers, loopback ports; the uint8 FP16 PE plan of `ref-u8fp16` (sha256 `a4feb7df...`), `pe_image_encoder` dynamic batcher `preferred_batch_size: [4, 8, 16, 32]`, `max_queue_delay_microseconds: 15000` (unchanged) | +| Stack | project `xbatch`: Triton, API (the published 0.5.0 image with this checkout's `src/` mounted), OpenSearch 8 GB heap, 32 API workers, loopback ports; the uint8 FP16 PE plan of `ref-u8fp16` (sha256 `a4feb7df...`), `pe_image_encoder` dynamic batcher `preferred_batch_size: [4, 8, 16, 32]`, `max_queue_delay_microseconds: 15000` (unchanged) | | Load | set A: 2,000 COCO images, `POST /ingest/batch`, 32 images per request, policy `all`, 1 warm-up and 3 counted repetitions (`ingest_sweep_suite.py --phases env,sweep --sweep-points 32x16,32x4 --modes batch --skip-cluster --skip-settle`); set B: 500 photos, 8 per request, 16 clients; the embed bench is `scripts/bench/embed_missing_bench.py` (500 images ingested, then 3 rewrites of every crop and frame vector) | #### Measurement first: PE batch sizes Triton sees today (set A, unchanged ingest code) @@ -2973,7 +2973,7 @@ finish at 100,000 and HDBSCAN was not run there (see above), so only `ivf` belon Question: does decoding JPEGs on the GPU, and building each model's input there, speed up ingest? The full design, the dataflow map and the implementation plan are in [design/gpu_decode_plan.md](design/gpu_decode_plan.md); this section keeps the numbers. One RTX -A6000 (GPU 2; SM86 has no hardware JPEG decoder), the published v0.5.0 images, the `ref-u8fp16` +A6000 (GPU 2; SM86 has no hardware JPEG decoder), the published 0.5.0 images, the `ref-u8fp16` engines, an isolated compose project, only this work's processes on GPU 2 during every timed run. Tools: `scripts/bench/gpu_decode_bench.py`, `scripts/bench/gpu_decode_parity.py`. diff --git a/docs/VISION_AND_GOALS.md b/docs/VISION_AND_GOALS.md index 44784937..f09753e0 100644 --- a/docs/VISION_AND_GOALS.md +++ b/docs/VISION_AND_GOALS.md @@ -28,7 +28,7 @@ Every stage is designed to be reused across domains by *configuring* it (detection profiles, prompt packs, class registries) rather than by forking the code per domain. -**Explicit owner scope decision:** the full v0.5.0 and v0.6.0 feature set below was +**Explicit owner scope decision:** the full feature set below was deliberately *not* trimmed to ship faster. Nothing here is a stretch goal. ## What "done" means for v0.6.0 @@ -106,12 +106,12 @@ release. ## What's explicitly not in v0.6.0, and why -These are real, tracked commitments, not abandoned ideas (milestone v0.7.0): +These are real, tracked commitments, not abandoned ideas (the next milestone): - **The full zero-copy GPU pipeline** (issue #40). v0.6.0 ships the measured steps (decode once, uint8 inputs to Triton, an FP16 PE image encoder, GPU decode for large photos, data-parallel PE encoding, coalesced ingest writes); keeping the decoded - frame resident on the GPU across every model is v0.7.0 work. Plan: + frame resident on the GPU across every model is next-release work. Plan: `docs/design/triton_pipeline_optimization_plan.md`. - **The 1M-image scale step.** The scale ladder is a goal, not a v0.6.0 claim. - Request-driven processing plans (#67), faces and OCR on resident crops (#220), diff --git a/docs/design/triton_pipeline_optimization_plan.md b/docs/design/triton_pipeline_optimization_plan.md index ed9ad46c..7aefd8e4 100644 --- a/docs/design/triton_pipeline_optimization_plan.md +++ b/docs/design/triton_pipeline_optimization_plan.md @@ -306,7 +306,7 @@ wire per model, API and Triton CPU seconds per image, GPU samples with the forei split out per process, cluster-training wall time, settled OpenSearch bytes per image and per vector, perf_analyzer points at batch 1/8/16/32 for five TensorRT models, the single-image endpoints, and a VLM labeling run. Results and the method are in -`docs/PERFORMANCE.md` ("v0.5.0 baseline") and the raw JSON in `docs/benchmarks/`. Earlier +`docs/PERFORMANCE.md` ("0.5.0 baseline") and the raw JSON in `docs/benchmarks/`. Earlier tools: the manifest tool and the run harness are `scripts/bench/select_baseline_set.py` (`local` and `coco` modes, checksummed manifest, license sidecar) and `scripts/bench/run_baseline.py` (one command per diff --git a/docs/design/v060_wave_plan.md b/docs/design/v060_wave_plan.md index e8a99005..d3667d02 100644 --- a/docs/design/v060_wave_plan.md +++ b/docs/design/v060_wave_plan.md @@ -49,7 +49,7 @@ the `OP_*` name and adds it to `env.template` and `tests/test_env_surface.py`. ## 1. Measured baseline this plan answers to (v0.5.0, PR #206) -Source: `docs/PERFORMANCE.md` "v0.5.0 baseline" and parent plan section 11.1 on branch +Source: `docs/PERFORMANCE.md` "0.5.0 baseline" and parent plan section 11.1 on branch `perf/baseline-045` (PR #206, not yet merged); raw JSON `docs/benchmarks/v050_baseline.json`, `v050_storage.json`, `v050_idle_background.json` on that branch. One RTX A6000 (GPU 0), Xeon E5-2680 v3 48 threads, Triton 2.70.0, TensorRT `libnvinfer.so.11`, OpenSearch 3.6.0. diff --git a/frontend/e2e/stubbed/test_ingest.py b/frontend/e2e/stubbed/test_ingest.py index e77a022b..1f06ef62 100644 --- a/frontend/e2e/stubbed/test_ingest.py +++ b/frontend/e2e/stubbed/test_ingest.py @@ -51,7 +51,7 @@ "source_section": "Decode once and the GPU decode (#217)"}, ], "embeddings_per_image": {"value": 7.4, "unit": "embeddings per image", "measured_on": "COCO, policy all", - "source_section": "v0.5.0 baseline (set A, public COCO, 2,000 images)"}, + "source_section": "0.5.0 baseline (set A, public COCO, 2,000 images)"}, "opensearch_bytes_per_image": {"value": 65298, "unit": "bytes per image", "measured_on": "COCO, policy all", "source_section": "Storage per image and per vector"}, "source_document": "docs/PERFORMANCE.md", diff --git a/frontend/src/lib/test/fixtures/bulkIngest.ts b/frontend/src/lib/test/fixtures/bulkIngest.ts index 886e8ae2..ea934280 100644 --- a/frontend/src/lib/test/fixtures/bulkIngest.ts +++ b/frontend/src/lib/test/fixtures/bulkIngest.ts @@ -44,7 +44,7 @@ export function configFixture(over: Partial = {}): BulkIngestC embeddings_per_image: { value: 7.4, unit: 'embeddings per image', - source_section: 'v0.5.0 baseline (set A, public COCO, 2,000 images)', + source_section: '0.5.0 baseline (set A, public COCO, 2,000 images)', measured_on: 'COCO, policy all', }, opensearch_bytes_per_image: { diff --git a/src/services/curation/bulk_ingest/preflight_rates.py b/src/services/curation/bulk_ingest/preflight_rates.py index fa16b472..9b0ec5c1 100644 --- a/src/services/curation/bulk_ingest/preflight_rates.py +++ b/src/services/curation/bulk_ingest/preflight_rates.py @@ -16,7 +16,7 @@ DECODE_SECTION = 'Decode once and the GPU decode (#217)' STORAGE_SECTION = 'Storage per image and per vector' -BASELINE_SECTION = 'v0.5.0 baseline (set A, public COCO, 2,000 images)' +BASELINE_SECTION = '0.5.0 baseline (set A, public COCO, 2,000 images)' @dataclass(frozen=True) From e9571469f2c7ca2da651dbdd4e8df42f871bbf03 Mon Sep 17 00:00:00 2001 From: davidamacey Date: Thu, 15 Oct 2026 18:36:05 -0400 Subject: [PATCH 05/17] feat(bench): add scale_run driver for 100k ingest benchmarks --- scripts/bench/scale_run.py | 420 +++++++++++++++++++++++++++++++++++++ tests/test_scale_run.py | 84 ++++++++ 2 files changed, 504 insertions(+) create mode 100644 scripts/bench/scale_run.py create mode 100644 tests/test_scale_run.py diff --git a/scripts/bench/scale_run.py b/scripts/bench/scale_run.py new file mode 100644 index 00000000..b23b595d --- /dev/null +++ b/scripts/bench/scale_run.py @@ -0,0 +1,420 @@ +"""Scale benchmark driver: one bulk ingest with a forced cancel, a resume and resource sampling. + +Starts ``POST /curation/projects/{p}/ingest/bulk`` for a folder, polls progress every 15 s +(ingested / processed / img/s recent and cumulative), cancels once at a fraction of the TRUE +final total (the folder preview's file count, not the growing discovery count), resumes by +starting the same source again, and samples GPU, host, container and OpenSearch state every +30 s. It writes ``/