Skip to content

Add a Review tab for bulk annotation auditing as a chip grid - #1897

Open
mattdawkins wants to merge 26 commits into
mainfrom
dev/review-grid
Open

Add a Review tab for bulk annotation auditing as a chip grid#1897
mattdawkins wants to merge 26 commits into
mainfrom
dev/review-grid

Conversation

@mattdawkins

@mattdawkins mattdawkins commented Sep 9, 2026

Copy link
Copy Markdown
Member

Adds a top-level Review tab (desktop: right of Training; web: right of Models) that shows many annotations at once as a grid of cropped image chips, so a whole class, or everything carrying an attribute, can be audited across one or more datasets and wrong types corrected in place without opening each sequence in the viewer.

What it does

  • Datasets / Grid toggle at the top left; only one view is shown at a time. The Datasets view adds datasets through the platform picker (web) or project listing (desktop), shows load state and track counts, and can reload or drop a dataset. Multicamera parents expand into their cameras. Library (desktop) and the Data page (web) get a Review button for the current selection, and /review?datasetIds=a,b deep links work on both.
  • Grid view: a rows × columns grid (5 × 4 by default; settable, plus zoom in/out that keeps the shape) and a Context slider for the margin around each box (30% by default). Settings persist per browser. Arrow keys / PageUp / PageDown / Home / End page through the results.
  • Query by type at or above a confidence threshold ("Any type" included), or by attribute key / value over track and/or detection attributes. Results can be sorted by dataset + track id, confidence (either direction), or frame. The grid keeps its entries until Show is pressed again, so editing a type never reshuffles the page being worked on.
  • Chips are cropped client-side: image sequences load their frame directly, videos are decoded by a hidden <video> element seeking to the frame (the frame → time mapping is extracted from VideoAnnotator into videoSeek.ts and shared). A concurrency-limited queue renders the first box of every visible entry before any track's extra frames; track entries then cycle through up to 8 boxes sampled along the track with the object kept centred. Chips are rendered at the cell's resolution and aspect ratio.
  • Editing: each cell has a type field (with every seen type as suggestions) and a "mark correct" action; Page actions sets a type for, or accepts, everything on the page. Edits use the same reassignPairs / acceptPairAsCorrect semantics as the viewer, respecting the dataset's type hierarchy. Edits are batched and written with saveDetections on Save; unsaved edits are counted, Discard reloads, and navigating away with unsaved edits prompts.
  • Open in viewer: clicking a chip opens the annotation viewer on that dataset, seeks to the frame and selects the track. The viewer gains initialFrame / initialTrackId props, read from ?frame=&track= on both platforms.

Plumbing

  • Api.peekConfig? (optional): loadConfig without the platform's viewer bookkeeping (desktop recents, web browse location). The web dataset store now shares its config merge (mergeDatasetConfig) with it.
  • Shared code lives in dive-common/review/* (pure item building, chip cropping, frame sources, chip queue), dive-common/use/useReview.ts (service) and dive-common/components/Review/*; platform shells are thin (ReviewPage.vue on desktop, views/Review.vue on web).
  • ReviewCell / ReviewGrid are presentation-only with action / footer slots so other item sources can reuse them; this is the hook a follow-up PR uses to show video-search results (Add optional video search capabilities to DIVE [DRAFT] #1743) in the same panel.
  • Docs: docs/Review.md, linked from the user guide nav.

Testing

  • Unit tests for item building, sorting, vocabularies, chip regions, viewer deep links and the review service (load, multicam expansion, assign/accept, save, failed save, discard, errors).
  • npm run lint, npm run typecheck, npm test and vite build pass.
  • Desktop dev build exercised by hand: image-sequence, video and multicam projects loaded together, chips and track cycling rendered, zoom, a type edit saved (verified in the written result_*.json, then reverted), and chip click → viewer at the right frame with the track selected.
  • Not yet exercised against a running Girder server; the web side is covered by the shared code, typecheck and the build only.

Known limitations

  • Tiled large-image datasets load but their chips cannot be cropped yet (placeholder shown).
  • Web uses the dataset's default annotation set.
  • Type edits only; boxes, attributes and deletion still happen in the viewer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

mattdawkins and others added 3 commits September 9, 2026 01:23
@mattdawkins @claude
Add a Review page that audits annotations as a grid of chips
3d97480 
New top-level Review tab (desktop: after Training; web: after Models)
showing many annotations at once as cropped image chips, so a whole class
(or everything carrying an attribute) can be checked across several
datasets and wrong types corrected in place, without opening each
sequence in the viewer.

- Datasets view / Grid view toggle: pick datasets through the platform
  picker or listing (multicam parents expand into their cameras), then page
  through matching annotations in a rows x columns grid (5x4 by default,
  settable, with zoom in/out keeping the shape) and a context slider for
  the margin around each box (30% by default).
- Query by type above a confidence threshold, or by attribute key/value on
  tracks and/or detections; sort by dataset, confidence or frame. The grid
  keeps its entries until the query is re-run so edits never reshuffle it.
- Chips are cropped client-side from image sequences or by seeking a hidden
  video element (shared frame->time mapping extracted from VideoAnnotator
  into videoSeek.ts), through a concurrency-limited queue that renders the
  first box of every visible entry before any track's extra frames. Track
  entries then cycle through up to 8 boxes sampled along the track, with
  the object kept centred.
- Types are edited per cell (assign, or mark correct) or for a whole page,
  batched and saved through saveDetections; unsaved edits are counted and
  guarded on navigation.
- Clicking a chip opens the viewer on that dataset, seeking to the frame
  and selecting the track (new initialFrame / initialTrackId Viewer props
  read from the route query on both platforms).
- Api gains an optional peekConfig for reading dataset configs without the
  viewer bookkeeping (desktop recents, web browse location); the web
  dataset store now shares its config merge with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Crop review chips to the grid cell's aspect ratio
a60cc1a 
A square chip letterboxed into a wide cell wasted most of the cell, so the
crop region now extends the padded square around the box to the cell's
width/height ratio (coarsened to a tenth so window resizes rarely force a
re-render), keeping the object centred whatever the grid shape.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Share grid paging, zoom and controls between chip grids
03a2d44 
Move the review page's paging, zoom, chip-resolution and keyboard handling
into useReviewGrid, its grid-shape/context/pager row into
ReviewGridControls, and the persisted grid settings into gridSettings.ts,
so another view showing ReviewItems (e.g. video search results) gets the
same behaviour without copying the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins mattdawkins mentioned this pull request Sep 9, 2026
Draft
@mattdawkins @claude
Review grid: larger cell text, Results first, in-place geometry editi…
b874917 
…ng, overlays and sharper chips

The Results panel is now the first and default panel, with the Datasets
panel second; with no dataset added it says so and links across. Cell type
fields and captions scale up as the grid shows fewer entries, and the
threshold and context sliders and fields are larger.

Right clicking a cell (or its edit action) opens the frame it is showing
for editing in place: the box gains drag handles, and polygon vertices and
head/tail points can be dragged too; Enter/Apply keeps the edit, Esc/Cancel
drops it, and the chip is re-cropped around the new box. Polygons and
head/tail points are drawn over every chip. Opening the viewer from a cell
seeks to the frame the cell is showing and selects the track.

Paging quickly only loads the page landed on: queued work for pages passed
over is dropped at once and new loads wait for paging to settle. Chips are
rendered at the cell's resolution with high-quality resampling and stored
losslessly when upscaled, so small objects are as sharp as the source allows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins
mattdawkins marked this pull request as ready for review September 9, 2026 20:18
mattdawkins and others added 17 commits September 9, 2026 16:33
@mattdawkins @claude
Review grid: annotator-style editing handles, right-click confirm, fr…
44a7b6e 
…ame stepping and auto-save

Polygon vertices now move while dragged (the draft arrays are replaced
instead of index-assigned, which Vue 2 cannot see). Editing handles are the
circles the annotator draws, in the type's colour and red while dragged, and
a right click while editing locks the change in. A single click on a chip no
longer opens the viewer (double click or the action does), so a slip after
an edit does not raise the unsaved-changes prompt. Track cells gain arrows to
step through their sampled frames, which pauses the cycling until resumed.
When auto-save is enabled in the settings, review edits are saved after the
same delay the annotator uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: settle the cell text between the old and enlarged sizes
048385a 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: pause on edit, real-time cycling, overlaid boxes, wheel …
1394ec6 
…zoom, live queries and toolbar polish

Starting an edit pauses a track's cycling and it stays paused until resumed.
Cycling runs at the dataset's real-time rate: consecutive frames advance
every 1/fps seconds and sparser samples wait proportionally longer. The box
is drawn over the chip instead of into it, so an edit no longer re-crops the
chip and the view keeps its zoom. While editing, the mouse wheel zooms the
chip about the cursor (up to 16x) and dragging empty space pans it.

The type field's arrow closes its list on a second press, the hover actions
grow under the mouse, entries sort by confidence (highest first) by default,
and the tooltip over each chip is gone. Query changes apply as soon as they
settle, so the Show button is gone; datasets loading or leaving refresh the
grid on their own. A settings gear next to Save opens the annotator's
settings dialog for auto-save. All toolbar fields read at the same size as
the Results and Datasets buttons.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: keep editing handles a constant screen size while zoomed
ebb1cb6 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: one entry per track across stereo and multi-camera rigs,…
e7dbfbf 
… with add-box and delete actions

The cameras of a multicamera dataset now share a grid entry per track,
shown as one chip per camera side by side and labelled with the camera. The
chips show the same sampled frames on every side; where a camera has no
detection on a frame the track has elsewhere, that side is cropped at a
position interpolated from its own neighbouring boxes, shows no box, and
offers an add-box action that creates the detection there ready to be
adjusted. Type edits apply to the track in every camera and opening the
viewer opens the rig. The chip (image, overlay, editing) is split out of the
cell into ReviewChip so a cell can hold several. A red X next to the accept
action deletes the track, written on the next save.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: cameras of an entry cycle in step, with frame controls k…
78aef86 
…ept clear of the camera label

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review grid: keep multi-camera frame controls clear of the camera label
8cf98b9 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Merge branch 'main' into dev/review-grid
571a34c 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Add a shared DatasetPicker component
f182032 
Review, Query, Training, Scoring and Pipelines each grew their own way of
choosing datasets: an autocomplete, a searchable data table with a plus
button, a checkbox table with select all. This adds one component that
covers all of them (search field, list of the datasets on offer with an
add button per row, select all for whatever the search lists, and an
optional browse button for platforms without a listing) so the pages can
adopt it in follow-up changes. Nothing uses it yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Use DatasetPicker on the Training and Pipelines pages
055661a 
Both pages offered their datasets through their own data table with a
plus button (Pipelines also had its own search and select all); they now
share the picker, which keeps the fps column and Training's view button
through the picker's headers and row-actions slot. Training's staged
items are typed as the cache rows they always were.

The picker's search field is clearable, which yields null; filter on an
empty string in that case.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
DatasetPicker: remove all, per-row remove, and hold the list in place
6d850ed 
Selected rows now show a clickable check that removes them, and a
"Remove all (n)" button drops every listed selection, mirroring select
all. Adding or removing keeps the picker at the same place on screen:
the pages list the selection above it, so growing that list used to
push the picker down under the user's pointer.

Training and Pipelines section titles and descriptions lose their card
padding so they line up with the tables and the picker under them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Pipelines: list datasets on entry and put the selection under the picker
9e70358 
The available datasets no longer wait for a pipeline to be chosen; the
run button stays disabled until one is. The selected datasets and the
run button move below the picker, so adding to the selection no longer
pushes the list the user is working in down the page. That makes the
picker's scroll hold unnecessary, so it is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Pipelines: list every dataset before a pipeline is chosen
193622f 
getAvailableItems still returned nothing without a selected pipeline,
which left the picker empty on entry despite the section now showing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Training: put the selected datasets under the available list
80262f9 
Same order as the Pipelines page: configuration first (name, config
file, labels, the annotated-frames and fine-tuning options), then the
picker, then the selected datasets with the Train button. Interrupted
runs follow the selection instead of splitting the configuration from
the datasets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins
Merge branch 'dev/dataset-picker' into dev/review-grid
67a39e6
@mattdawkins @claude
Review: choose datasets with the shared DatasetPicker
f8470b1 
The Datasets view drops its autocomplete for the picker used by Training
and Pipelines: search, add or remove per row, select all and remove
all, with the web's folder browser behind a Browse button. The selected
datasets keep their status table underneath.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review: queue datasets picked on the Datasets view until Results opens
40ceabc 
Adding a dataset used to read its annotations at once, so picking many
on the Datasets view meant waiting for every one to load before doing
anything. Picked datasets are now queued and load together when the
Results view opens (or on reload). Datasets handed over by the library
still load immediately, as results are shown right away.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins mattdawkins mentioned this pull request Sep 10, 2026
Closed
mattdawkins and others added 5 commits September 10, 2026 15:32
@mattdawkins @claude
Review: wheel zoom and drag pan at any time, shared across an entry's…
f4f4eb8 
… cameras

Zooming a chip no longer requires edit mode: the wheel zooms about the
cursor and dragging the zoomed image pans it, with the action buttons
left alone. Multi-camera entries hold one view in the cell, so zooming
or panning one camera moves the others with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review: offer only types present at the current threshold
b47f3ab 
The type dropdown listed every type on any confidence pair of the
selected datasets, including low-confidence secondary classes that a
query at the current threshold would never match. It now lists the
types some track carries at that confidence or above, so every choice
has results; the cell type field keeps offering every known type.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins
Merge remote-tracking branch 'ssh/main' into dev/review-grid
fd60685 
# Conflicts:
#	client/dive-common/components/DatasetPicker.vue
#	client/dive-common/datasetPicker.spec.ts
#	client/dive-common/datasetPicker.ts
@mattdawkins @claude
Review: own type dropdown per entry, and middle-button panning
b7ee475 
The entry type field used a native datalist, whose arrow could open the
list but never close it. A small dropdown of our own replaces it: the
arrow toggles, typing filters, arrow keys and Enter pick, Escape
reverts, and the list floats out of the clipped cell.

The middle mouse button now pans a zoomed chip from anywhere on it, in
or out of edit mode, with the browser's autoscroll suppressed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
@mattdawkins @claude
Review: resume the session when coming back to the page
ea119cf 
Leaving the Review page hands its service, view and grid page to a
held session that the next visit takes back, so navigating away and
returning shows the same datasets and page, unsaved edits included.
Starting Review from the library with a different selection begins a
fresh session, unless the held one has unsaved edits, in which case it
is resumed and the new datasets are added. The route-leave prompt goes
away since nothing is lost; closing the app still warns.

The entry type field also commits a typed type on Enter directly
instead of relying on the blur that follows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BryonLewis BryonLewis Awaiting requested review from BryonLewis

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mattdawkins