Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions App/serval_app/lib/data/dashboard_socket.dart
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,31 @@ class DashboardSocket {
_connected.add(false);
}

/// Lets the socket go while nobody is looking at what comes down it, leaving this ready for a
/// later [connect] — which is what `reconnectNow` on the way back amounts to.
///
/// Every camera's JPEG arrives here about once a second for as long as this is open, and a phone
/// in somebody's pocket is paying for all of it in radio and in battery. Nothing on the other side
/// stops: those frames are encoded at `Ingest:SnapshotFps` regardless, to feed the vision pipeline
/// and `/snapshot.jpg`, so what this saves is egress and this client's own decoding.
///
/// The other half of what it saves is frames. Each arrival writes a notifier, which rebuilds a
/// tile, which asks the browser for an animation frame that a hidden page is never given — so an
/// open socket guarantees there is a frame outstanding at the moment the page goes away, which is
/// the state `frame_watchdog.dart` describes as unrecoverable from the inside.
///
/// Deliberately silent, where [disconnect] announces itself: `connected: false` is a claim that
/// the Server could not be reached, and this is the App choosing to stop listening. Announcing it
/// would also rebuild the whole wall on the way past, which is precisely the frame this is here
/// to not schedule.
void pause() {
if (_closed) return;
_teardown();
// Somebody coming back should not spend a wait earned before this, and the reconnect that
// follows a pause is always a person asking.
_backoff = _minBackoff;
}

/// Reconnects at once rather than waiting out the backoff.
///
/// For the App coming back from the background, where the backoff is exactly wrong: a phone away
Expand Down
19 changes: 19 additions & 0 deletions App/serval_app/lib/data/live_repository.dart
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,25 @@ class LiveServalRepository implements ServalRepository {
/// Not on [ServalRepository] either, matching [start] and [stop]: `SampleServalRepository` has no
/// sockets and no clock, and a no-op on the interface would be a member the widget tests have to
/// answer for. `_RepositoryStarter` already narrows before calling any of the three.
/// Nobody is looking. Let go of the traffic whose only purpose is to be looked at.
///
/// The wall socket alone, and the asymmetry with [resumeLive] is deliberate. `WS /api/events` is
/// the alerting path — an alert that arrived while a phone was in a pocket is the single most
/// important thing this App carries — and it is nearly free, a message per thing that happens
/// rather than a frame per camera per second. Dropping it to save nothing would be trading the
/// feature for the bill.
///
/// Nothing here is torn down beyond the socket: the registry, the feed and the arrangement are
/// all still true, and the wall must be able to paint the moment somebody comes back. That is
/// what separates this from [stop], which is about a session ending rather than a person looking
/// away.
///
/// Not on [ServalRepository], matching [start], [stop] and [resumeLive], for the same reason
/// given there.
void pauseLive() {
_dashboard.pause();
}

void resumeLive() {
_startListening();
_dashboard.reconnectNow();
Expand Down
73 changes: 60 additions & 13 deletions App/serval_app/lib/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -114,20 +114,31 @@ class _ServalMaterialAppState extends State<_ServalMaterialApp> {
void initState() {
super.initState();

// The other half of a tap landing where it was sent. Routing it is only half the job: a
// browser that has come back from the background unable to paint holds the screen it was on
// however correct the route underneath it is. See [watchFrames], including why the route it
// recovers to has to come from the router rather than from the address bar.
//
// Before the push wiring below, which reaches into it: a tap can arrive at any moment after
// that line, and it must not find a watchdog that does not know where it would reload to.
watchFrames(() => _router.routeInformationProvider.value.uri.toString());

// A tapped notification on a tab that is already open arrives as a message from the service
// worker rather than as a navigation — see `sw.js`, which prefers focusing this tab to opening
// a second copy of a live-video app. Routing it has to happen here because this is where the
// router lives; without it the tab would come to the front unchanged, which reads as the tap
// having done nothing at all.
//
// The route is set and *then* the frame pipeline is asked about, in that order. The message
// arrives whether or not the App can paint, so this is the one moment the App knows somebody is
// waiting on a screen it may be unable to draw — and by then `go` has already told the router
// where the reload should land.
//
// A no-op off the web and on a browser with no push, so there is no platform branch here.
PushClient.onNavigate(_router.go);

// The other half of a tap landing where it was sent. Routing it is only half the job: a
// browser that has come back from the background unable to paint holds the screen it was on
// however correct the route underneath it is. See [watchFrames], including why the route it
// recovers to has to come from the router rather than from the address bar.
watchFrames(() => _router.routeInformationProvider.value.uri.toString());
PushClient.onNavigate((route) {
_router.go(route);
probeFrames();
});
}

@override
Expand Down Expand Up @@ -186,6 +197,16 @@ class _RepositoryStarterState extends ConsumerState<_RepositoryStarter> {
/// — constructs nothing new and observes nothing.
AppLifecycleListener? _lifecycle;

/// The grace running between the App going away and the wall socket being let go.
Timer? _pause;

/// How long the App must be away before it stops listening to the wall.
///
/// The same figure `WebRtcView` uses to decide whether a resume was long enough to be worth
/// rebuilding a session for, and for the same reason: it is how long a glance at something else
/// lasts. Both are about the difference between looking away and going away.
static const _pauseAfterHidden = Duration(seconds: 10);

@override
void initState() {
super.initState();
Expand All @@ -194,23 +215,29 @@ class _RepositoryStarterState extends ConsumerState<_RepositoryStarter> {
_onAuthChanged();

if (ref.read(repositoryProvider) is LiveServalRepository) {
// `onShow`, and deliberately not `onResume`. Both fire on the way back, but `onResume` also
// fires when the window merely regains input focus — and a second monitor showing the wall
// while you work elsewhere is the case this must leave alone. `onShow` is the hidden→visible
// edge alone, which is what `document.visibilityState` means, and it buys that rule with no
// bookkeeping of our own.
_lifecycle = AppLifecycleListener(onShow: _onShown);
// `onShow`/`onHide`, and deliberately not `onResume`/`onPause`. All four fire on the way back
// and away, but the resume pair also fires when the window merely regains or loses input
// focus — and a second monitor showing the wall while you work elsewhere is the case this
// must leave alone. The show pair is the hidden→visible edge alone, which is what
// `document.visibilityState` means, and it buys that rule with no bookkeeping of our own.
_lifecycle = AppLifecycleListener(onShow: _onShown, onHide: _onHidden);
}
}

@override
void dispose() {
_pause?.cancel();
_lifecycle?.dispose();
_auth?.removeListener(_onAuthChanged);
super.dispose();
}

void _onShown() {
// Before the `_started` gate: a pause armed while signed in must not survive a sign-out into
// the next session, and cancelling one that is not there costs nothing.
_pause?.cancel();
_pause = null;

// Only for a session that is actually running. Coming back to a tab sitting on `/login` must
// not raise sockets, and `_started` is the same flag both edges below turn on.
if (!_started) return;
Expand All @@ -221,6 +248,26 @@ class _RepositoryStarterState extends ConsumerState<_RepositoryStarter> {
repository.resumeLive();
}

/// The App has gone away. Start the clock on letting the wall socket go.
///
/// Through a timer rather than at once because most of what this edge reports is somebody glancing
/// at something else. Dropping the socket for a five-second look and rebuilding it on the way back
/// would churn a connection and repaint a wall to save five seconds of frames — and the reconnect
/// is the expensive half of that trade, not the frames.
void _onHidden() {
if (!_started) return;

_pause?.cancel();
_pause = Timer(_pauseAfterHidden, () {
_pause = null;

final repository = ref.read(repositoryProvider);
if (repository is! LiveServalRepository) return;

repository.pauseLive();
});
}

void _onAuthChanged() {
final auth = _auth;
if (auth == null) return;
Expand Down
18 changes: 18 additions & 0 deletions App/serval_app/lib/platform/frame_watchdog.dart
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,14 @@ import 'frame_watchdog_stub.dart'
/// here. It reloads, which is the same thing the only available workaround does — closing the App
/// and opening it again — minus the person having to know that.
///
/// Two properties are what make it fire at all, and both are answers to the same trap: the thing
/// being measured cannot be the thing doing the measuring. **Nothing it waits on is a
/// `requestAnimationFrame`** — a latched pipeline is one whose animation frame never arrives, so a
/// deadline counted in those cannot end in the very state it names; the deadline is a timer, which
/// a wedged page still runs. And **it does not rely on catching the moment**: it asks on every edge
/// that could carry a wedge and on a slow heartbeat besides, because an App that is wedged stays
/// wedged whether or not anything saw it happen.
///
/// [route] is where to reload *to*, and it must be the router's own answer rather than the address
/// bar's. `Router` reports a navigation to the browser from a post-frame callback, so an App that
/// cannot paint never writes the new address — a tapped notification routed while wedged leaves
Expand All @@ -43,3 +51,13 @@ import 'frame_watchdog_stub.dart'
///
/// A no-op off the web, so there is no platform branch at the call site.
void watchFrames(String Function() route) => platform.watchFrames(route);

/// Asks the question now rather than waiting for an edge or a heartbeat.
///
/// For the caller who has just done something on somebody's behalf and knows they are watching for
/// the result. A tapped notification is the case that matters: it reaches the App through the
/// service worker's message, which is delivered whether or not the App can paint, so a tap landing
/// on a wedged App is indistinguishable from a tap that did nothing at all.
///
/// A no-op off the web, and a no-op before [watchFrames] has run.
void probeFrames() => platform.probeFrames();
59 changes: 59 additions & 0 deletions App/serval_app/lib/platform/frame_watchdog_decision.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/// What one liveness probe concluded about the frame pipeline.
enum FrameVerdict {
/// A frame was produced, or the App is not in a state this can judge. Nothing to do.
healthy,

/// The main thread was too busy for the answer to mean anything. Ask again.
retry,

/// The pipeline is latched shut, and only a reload clears it.
reload,
}

/// How much later than its deadline a wait may end before the thread counts as jammed rather than
/// merely idle.
///
/// Generous on purpose. The cost of calling a jammed thread wedged is reloading an App that was
/// about to paint; the cost of calling a wedged thread jammed is one more probe, a heartbeat later.
/// Those are not the same price.
const _jammed = 2;

/// Reads one probe's result.
///
/// Lives apart from the browser plumbing so it can be tested: the conditional import in
/// `frame_watchdog.dart` hands `flutter test` the stub, so nothing reachable from a widget test may
/// touch `dart:js_interop`.
///
/// [waited] is the wall clock that actually passed while waiting [deadline] for a frame, and the gap
/// between the two is why this is not a single comparison. The timer ending the wait runs on the
/// same thread as the frame it waits for, so it can only fire when the event loop is free — which
/// means a wait that ended *on time* carries a second fact beyond "nothing painted": the thread was
/// free to paint and did not. A wait that ended late says only that the thread was busy, which is
/// what a resume decoding every camera's snapshot at once looks like, and a busy App is not a
/// broken one.
///
/// Deliberately not a count of animation frames. A latched pipeline is one whose
/// `requestAnimationFrame` callback never arrives, so a deadline counted in those is a deadline
/// that cannot end in the very state it exists to name.
FrameVerdict judgeFrame({
required bool painted,
required Duration waited,
required Duration deadline,
required bool visible,
required int attemptsLeft,
}) {
if (painted) return FrameVerdict.healthy;

// A page that went back to the background mid-probe never owed anybody a frame. That is not the
// fault this recovers from, and reloading somebody's App on the strength of it would be worse
// than the fault. The next time the page is shown it is asked again.
if (!visible) return FrameVerdict.healthy;

if (waited > deadline * _jammed) {
// Out of probes on a thread that has been busy throughout. Still no evidence of a latch, so
// this leaves it alone rather than guessing; the heartbeat asks again.
return attemptsLeft > 0 ? FrameVerdict.retry : FrameVerdict.healthy;
}

return FrameVerdict.reload;
}
2 changes: 2 additions & 0 deletions App/serval_app/lib/platform/frame_watchdog_stub.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@
/// specifically — a `requestAnimationFrame` that a hidden page never receives — and a native
/// embedder drives its frames from a vsync signal that a backgrounded app is simply not sent.
void watchFrames(String Function() route) {}

void probeFrames() {}
Loading
Loading