Skip to content

[Agent] Live View: show the last captured image when a camera can't connect #974

Closed

Description

@0xCyberFortress

Problem and evidence

User request (Discord #lightnvr, 2026-11-14): a camera on a weak Wi-Fi link (garage) connects only intermittently. While it is down, its Live View tile shows only the black error card ("Cannot connect to camera source" + retry countdown), so the operator sees nothing at all for that camera. LightNVR already has footage from the moments it was connected, but the tile never shows it.

Request: when a live tile cannot play, it should at least show the most recent image LightNVR captured from that camera, clearly marked as not live.

Current code: each live cell renders its own full-tile error overlay on an opaque rgba(0,0,0,0.8) background, with no image behind it:

  • web/js/components/preact/MSEVideoCell.tsx ({error && ( block, "Error overlay")
  • web/js/components/preact/HLSVideoCell.tsx ({error && ( block)
  • web/js/components/preact/WebRTCVideoCell.tsx ({error && ( block)

The data needed already exists, so no backend change is required:

  • GET /api/recordings?stream=<name>&sort=start_time&order=desc&limit=<n> lists the camera's newest recordings.
  • GET /api/recordings/thumbnail/<id>/<index> serves a cached JPEG of a recording: index 2 is about 1 s before the recording's end (its last frame), and index 0 is the first frame. Index >0 returns 400 when thumbnails_per_recording <= 1.

Expected behavior and scope

When a live tile is in its error state (any of the three cells: MSE, HLS, WebRTC), the area behind the existing error overlay shows a last known image of that camera instead of plain black:

  1. Same-session freeze frame (preferred, freshest). If the tile played video earlier in this page session, keep the last decoded frame the user saw (capture it from the <video> element when the error occurs, before the element is torn down or its source reset).
  2. Recording fallback. Otherwise, fetch the camera's most recent recording that has a non-zero end_time and show /api/recordings/thumbnail/<id>/2. If that fails (for example a 400 because thumbnails_per_recording <= 1, or a 404), try index 0. If no recording exists or all requests fail, keep today's black overlay.
  3. A small caption, for example "Last image · 10:41 PM", or "Last image · Nov 14, 10:41 PM" when not from today, using the user's locale and timezone, so the image is never mistaken for live video.
  4. The existing error text, retry button and auto-retry countdown stay exactly as they are, on top of the image. The overlay background becomes semi-transparent (for example rgba(0,0,0,0.55)) so the image is visible but the text stays readable.
  5. When the stream recovers and plays again, the fallback image disappears immediately.

Put the shared logic in one component or hook (for example web/js/components/preact/live/LastFrameFallback.tsx plus a useLastFrame(streamName, videoRef, hasError) hook) used by all three cells, not three copies.

Make it user-controllable: a per-browser Live View toggle, "Show last image when a camera is offline", default on, persisted the same way the other Live View display preferences are (see useLiveDisplayMode.ts and the Fit/Fill toggle). If a server-side setting is preferred instead, that's fine too.

Acceptance criteria

  • A tile that was playing and then loses its source shows its last decoded frame behind the error overlay, with the "Last image · " caption, where time is when that frame was captured.
  • A tile that has never played this session (page opened while the camera is down) shows the thumbnail of the newest completed recording (index 2, falling back to 0), with the recording's end_time as the caption time.
  • No recordings, or every thumbnail request fails: the tile looks exactly like today (black overlay, error text, retry). No console errors are thrown and no broken-image icon appears.
  • Error text, retry button, auto-retry countdown and tile status glyphs are unchanged in behavior and still clickable on top of the image.
  • When the stream recovers, the image and caption are removed and live video shows normally.
  • The toggle turns the feature off completely: no extra /api/recordings or thumbnail requests are made while it's off.
  • Works in all three live modes: MSE, HLS and WebRTC. It also works in the fullscreen/single-tile view.
  • Bounded network cost: at most one recordings-list request and one or two thumbnail requests per tile each time it enters the error state. Never on every retry tick or countdown second. The browser cache is fine; the thumbnail endpoint already caches on disk.
  • Capturing the freeze frame never throws. If the canvas would be tainted or the video has no decoded frame (videoWidth === 0), skip quietly to the recording fallback.
  • Respects existing auth: requests go through the same fetch helpers and credentials as the rest of Live View.
  • New strings (caption, toggle label) are added through the i18n system (web/js/i18n locales). Do not hard-code English.

Validation

  • Unit tests (jest, web/tests) for the hook/helper. Cover: picks the newest recording with end_time > 0; falls back from index 2 to 0 on 400/404; returns nothing when there are no recordings; does not refetch while already in error; disabled toggle → no fetch.
  • Component test: the error overlay renders the image and caption when a fallback URL exists, and renders exactly as before when not.
  • Manual: point a stream at an unreachable host (or block a camera's IP), load Live View → thumbnail and caption appear behind "Cannot connect to camera source". Restore the camera → live video returns and the image disappears. Repeat in MSE, HLS and WebRTC modes. No camera hardware needed beyond any RTSP source that can be stopped.
  • npm run lint, typecheck and the existing web test suite pass.

Constraints and rollback

  • Frontend-only; no API, schema or C changes. It uses existing endpoints.
  • No new dependencies.
  • Performance: one canvas drawImage at the moment of failure, not continuous frame grabbing while healthy. Grids of 16+ tiles must not get heavier while every camera is healthy.
  • Security: use only same-origin LightNVR endpoints already used by the recordings UI. No new unauthenticated paths.
  • Rollback: revert the PR, or users switch the toggle off.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions