Skip to content

video-mode: calibrate the render window from the cover, not the recorder endpoint (fixes the black flash) - #38

Merged
mmkal merged 5 commits into
mainfrom
rendered-black-flash
Aug 26, 2026
Merged

video-mode: calibrate the render window from the cover, not the recorder endpoint (fixes the black flash)#38
mmkal merged 5 commits into
mainfrom
rendered-black-flash

Conversation

@mmkal

@mmkal mmkal commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Rendered videos flashed black near the end — the teardown calibration cover leaking into the render. This PR fixes the calibration and turns the reproify spec into a passing regression guard.

Before (0.36–0.40s black flash right before the finalHold, deterministic):

black-flash-rendered.webm

After (same spec, fixed):

black-flash-fixed.webm

Mechanism

At teardown, settleVideoRecorder paints a fullscreen rgb(1, 2, 3) cover (near-black) and holds it — deliberately outside the render range — so the recorder's endpoint can serve as the wall→raw clock marker. That assumed Playwright's ≥1s final-frame padding stops at the close instant; it actually lands ~1s past it, so sourceOffset = rawDuration − recordingEndedAt came out ~1s too large, every translated coordinate sat ~1s too deep in the raw, and the derived source-range end ran into the cover: content → black flash → finalHold. Every video-mode-start-behaviors rendering carried a ~0.76s version; iterate/iterate#2523's demo video flashed 0.32s.

Fix

The cover itself becomes the marker — it's the one thing in the recording whose wall-clock time videoMode knows exactly and whose pixels are unmistakable:

  • settleVideoRecorder returns the moment it painted the cover (midpoint of DOM insert and the forcing screenshot).
  • detectCalibrationCoverStartMs finds the cover's first raw frame by color — a backwards scan of the file's trailing cover run, so an app that legitimately paints near-black mid-test can never match. Tolerance covers webm quantization and true-black padding.
  • sourceOffset = coverStartMs − coverPaintedAt (clamped ≥0); endpoint arithmetic stays as the fallback (no cover captured, decode failure, cover-only instant recordings).
  • The derived source-range end is capped one frame short of the cover. Annotation minimums still win — now including each hold's sourceFrameAt — so near-instant recordings whose only captured frames ARE cover still render their synthetic reveals.
  • Popup children get the same treatment: cover-calibrated childOffsetMs, and overlay windows (closeMs/enableToMs) capped at the cover's composite time.

Verification

  • The former test.fail repro now passes as a plain regression test (frame-level: zero near-black frames in the render).
  • Full suite: 152 passed, 0 failed.
  • ffmpeg blackdetect over every rendered artifact in test-results: all tail flashes gone (start-behaviors ~0.76s each → clean; captions spec clean). Remaining single-frame (40ms) detections predate this change: dark leading frames on two reveal specs, and popup fade-boundary frames in the composites — candidates for a follow-up, an order of magnitude below the reported flash.

Baseline pacing video per AGENTS.md (spec/todo-app.spec.ts on this branch):

todo-baseline-fixed.webm

🤖 Generated with Claude Code


Coding agent session: cb910f09-4383-4d33-a11c-f02251cea10d


Note

Medium Risk
Changes core video timeline calibration and trim bounds for all rendered outputs; fallback paths remain, but mis-detection could shorten ranges or skew popup overlays.

Overview
Fixes a black flash right before finalHold in rendered demo videos: wall→raw timeline calibration used the recorder close endpoint, but Playwright keeps padding the last screencast frame ~1s past close, so annotations and the source trim ran ~1s into the teardown rgb(1,2,3) calibration cover.

Calibration now pairs settleVideoRecorder’s returned paint time with detectCalibrationCoverStartMs (backward ffmpeg scan for the trailing cover). sourceOffset uses coverStartMs − coverPaintedAt when both exist; recorder-end math remains the fallback. The derived source range end is capped one margin frame before the cover (highlight mins still win, including each hold’s sourceFrameAt). Popup children store calibrationCoverPaintedAt, use the same offset logic, and cap overlay closeMs / enableToMs at the cover’s composite time.

Adds spec/video-mode-black-flash.spec.ts: ffmpeg frame scan asserts zero near-black frames on a light-page render (regression guard for the former repro).

Reviewed by Cursor Bugbot for commit 6ead100. Bugbot is set up for automated code reviews on this repo. Configure here.

mmkal and others added 3 commits August 26, 2026 16:28
The raw screencast's tail is black — Chromium's final frame at recording
stop is black and Playwright pads that last frame by >=1s into the saved
file — and the renderer's default source range runs to the raw file's
full duration, so the black plays in the output right before the
finalHold reveals the real final frame. test.fail spec pins the intended
behavior: a rendered video never shows frames the page never painted.

Deterministic: 3/3 local runs flash black at 1.6s for 0.36-0.40s. Also
visible in every video-mode-start-behaviors rendering (~0.76s each) and
in the wild (iterate/iterate#2523 demo video, 0.32s flash).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…der endpoint

The teardown calibration cover (rgb(1,2,3), settleVideoRecorder) was
leaking into renders as a black flash: the wall-to-raw offset derived
from the recorder ENDPOINT assumed Playwright's final-frame padding
stops at the close instant, but it lands ~1s past it, so every
translated coordinate sat ~1s too deep and the render range ran into
the cover.

The cover is now the marker instead: settleVideoRecorder stamps when it
painted it, detectCalibrationCoverStartMs finds its first raw frame by
color (backwards scan of the trailing run, so mid-test near-black pages
can never match), and that pair calibrates the offset — the endpoint
stays as fallback. The derived source-range end is capped a frame short
of the cover, with the annotation minimums (now including each hold's
sourceFrameAt) still winning on near-instant recordings that never
captured anything but cover. Popup children get the same treatment:
cover-calibrated offsets plus overlay windows capped at the cover.

The black-flash spec flips from test.fail to a passing regression
guard. Remaining known single-frame artifacts (dark leading frames on
two reveal specs, popup fade-boundary frames) predate this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mmkal mmkal changed the title Reproify: rendered videos flash black near the end video-mode: calibrate the render window from the cover, not the recorder endpoint (fixes the black flash) Aug 26, 2026
@pkg-pr-new

pkg-pr-new Bot commented Aug 26, 2026

Copy link
Copy Markdown

Open in StackBlitz

pnpm add https://pkg.pr.new/middlewright@38

commit: 6ead100

Comment thread src/plugins/video-mode.ts
Comment thread src/plugins/video-mode.ts Outdated
… short of the cover

Bugbot's two catches: the popup closeMs/enableToMs clamp landed ON the
first cover frame (ffmpeg between-windows include their end timestamp),
and the parent margin used the raw's average frame duration where the
detector's 25fps tick can be larger. One calibrationCoverMarginMs
helper now takes max(frameDurationMs, detection tick) on both paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mmkal

mmkal commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Note for the record: while verifying the review fixes I found a pre-existing load-flake family — the pixel-at-timestamp specs (uses a normal pointer tail after text cursor holds, reveals a stable single-line textarea fill, pans to an offscreen waitFor result…) each pass in isolation but a rotating member fails under full-suite parallel load, including on clean main (2/2 runs failed :2912 there under identical conditions). Not introduced by this branch; probably deserves its own task — under load the screencast captures fewer frames and these assertions sample within one frame of a boundary.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c4f2609. Configure here.

Comment thread src/plugins/video-mode.ts
An admitted cover start could previously sit inside the margin and
produce a zero-width cap when the raw runs faster than the detector's
25fps grid. Guard and cap now share calibrationCoverMarginMs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mmkal
mmkal merged commit a65cda0 into main Aug 26, 2026
3 checks passed
mmkal added a commit that referenced this pull request Aug 26, 2026
…s main CI) (#40)

Main CI has been red since #38 (`a65cda0`): `reveals an expanding
textarea one line at a time at its final geometry` fails 2/2 there with
frames 0-1 showing the *filled* text before the reveal — the render
window starts after the fill. It never reproduces locally (3/3 green),
which fits the cause:

#38 calibrates the render window from the calibration cover. On a slow
runner the screencast trails the paints it maps by a few frames, and the
cover-based offset inherits that lag — so a selector-driven trim start
(`trimStart: ["selector", ...]`) can land past the first fill. The
existing race guard only clamped a trim start back to a highlight when
it landed within **one frame** after it; CI's lag is bigger.

Nothing acts on the app before it's ready, so *any* trim start after a
recorded highlight is measurement error, and starting at the highlight
is always harmless (the video opens on the action instead of a beat
before it). The guard now clamps unconditionally:

```ts
// before: a start ≤ 1 frame after a highlight moved back to it; further = kept → opens post-action
// after:  a start after any highlight moves back to the earliest such highlight
```

No new spec: the failure is runner-timing-specific and this PR's CI run
is the verification. Unrelated: `uses a normal pointer tail after text
cursor holds` is a pre-existing local flake on main (1/3 fails with
`--repeat-each=3`), untouched here.

Blocks #39's CI (inherited the same red).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Session: `b7f6f792-6606-44be-9ec3-207eb762c4b6`

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes render trim boundaries for all selector-driven starts when
highlights precede the computed start; behavior is intentional but
affects every video-mode render path that sets `sourceRange.start`.
> 
> **Overview**
> Fixes rendered videos that could open **after** the first recorded
action when `trimStart` uses a selector or cover-based calibration lags
on slow CI runners.
> 
> During finalization, if `sourceRange.start` is set but sits past one
or more highlight timestamps, the render window start is pulled back to
the **earliest** such highlight. The previous guard only did this when
the gap was within **one frame**; multi-frame calibration/screencast lag
is now covered too.
> 
> Comments in `video-mode.ts` spell out the rationale: a trim start
after a highlight is treated as timing error, and clamping to the
highlight is safe because it keeps the opening frame on the action
instead of post-fill footage.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
c737189. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant