Add matrixed image snapshot testing for WhereUI#101
Open
kyleve wants to merge 43 commits into
Open
Conversation
Create the reusable Shared/ snapshot-testing pair: SnapshotKit (generic, shippable; SwiftUI/Foundation/UIKit only) and SnapshotKitTesting (test-only; links swift-snapshot-testing + cashapp/AccessibilitySnapshot, re-exports both plus SnapshotKit). Wire the WhereUISnapshotTests bundle under Where/WhereUI/SnapshotTests with its own testScheme (kept out of the Stuff-iOS-Tests scheme), add a SnapshotKitTests bundle to Stuff-iOS-Tests, and add a dedicated 'snapshot' CI job (needs: format, checkout lfs: true) so image snapshots stay out of the main test job. Route **/__Snapshots__/**/*.png to Git LFS via a scoped root .gitattributes (existing app-icon PNGs stay plain blobs). A smoke snapshot test proves the pipeline end to end: it records, then passes against, an LFS-stored reference. Closes plan step: scaffold-modules. Co-authored-by: Cursor <cursoragent@cursor.com>
Build the generic appearance matrix: a Hashable SnapshotConfiguration over colorScheme, dynamicType, contrast, device Frame, and snapshotType (.standard/.accessibility); a combinations() Cartesian-product builder with per-axis default singletons; additive .componentDefaults/.screenDefaults presets; identifierParts that omit default axes (so only dark/ax5/contrast/accessibility/ iPad reach reference-image names); a uiTraitCollection bridge; and a snapshotTraits(_:) modifier (increased contrast via a UIKit trait override, since SwiftUI has no setter). Cover the combination counts and identifier omission in SnapshotKitTests. Closes plan step: matrix-config. Co-authored-by: Cursor <cursoragent@cursor.com>
A SnapshotCase (name + configurations + content) that is also a View, so it renders a labeled cutsheet of its variants; a SnapshotCaseBuilder result builder so providers declare cases without array boilerplate; the SnapshotProviding protocol; and a snapshotPreviews cutsheet that renders every case's non-accessibility variants for an Xcode #Preview (accessibility captures need the test-only VoiceOver parser, so they're filtered out). Covered by SnapshotKitTests. Closes plan step: provider-cutsheet. Co-authored-by: Cursor <cursoragent@cursor.com>
Add the capture pipeline: renderSnapshotImage + a Snapshotting<UIViewController, UIImage>.snapshotKitImage strategy that renders any view at any size on the fixed CI simulator. Ports the workarounds needed for deterministic, device-independent captures: safe-area-inset swizzling, CATransaction.performWithoutAnimation, animation quiescing, text-cursor hiding, waitForStableSize for SwiftUI hosting controllers, and an AccessibilitySnapshotViewController wrapper (cashapp AccessibilitySnapshotCore) for .accessibility captures. A development probe confirmed UIKit still returns a blank image for views taller than ~2000pt on iOS 26.2, so tile-and-stitch (SnapshotWrappingViewController + tileAndStitchImage) is retained and guarded by LargeViewCaptureTests (short = one tile, 3000pt = multiple tiles). Precision defaults: precision 0.999, perceptualPrecision 0.98. Closes plan step: render-pipeline. Co-authored-by: Cursor <cursoragent@cursor.com>
Runner (provider + inline overloads) maps each SnapshotConfiguration to a trait-configured, sized hosting controller and asserts through the pipeline strategy. Makes SnapshotCase.init / SnapshotProviding.snapshots MainActor so authors can build SwiftUI content directly. Co-authored-by: Cursor <cursoragent@cursor.com>
Wraps snapshot content in whereBroadwayRoot() so WhereUI SnapshotProviding authors get trait-accurate stylesheet tokens without repeating the root wrapper. Closes the whereui-adapter plan step. Co-authored-by: Cursor <cursoragent@cursor.com>
The render pipeline now suspends (async/await) instead of pumping the run loop synchronously: SwiftUI .task bodies are main-actor concurrency jobs on the non-reentrant main queue, so a nested RunLoop.run() could never interleave them and .task-driven content snapshotted as its placeholder. Suspending frees the main actor so content loads. The safe-area override is scoped to the captured root (and the capture scaffolding above it) only; views inside the hosted tree keep the native implementation so interior contributions — bars, safeAreaInset accessories, additionalSafeAreaInsets — still compose on top of the zeroed base. Before capture, a settle loop renders quarter-resolution pixel digests and waits until two consecutive frames are byte-identical (bounded by a min/max duration), so finite reveals and async loads finish while views that never quiesce can't hang the capture. Includes regression tests: AsyncContentCaptureTests proves a .task-driven view captures its loaded state; LargeViewCaptureTests covers tiled capture through the async path; SnapshotPixelProbe is the shared pixel-sampling helper. Co-authored-by: Cursor <cursoragent@cursor.com>
Views that animate forever (repeatForever pulses, TimelineView(.animation) loops) can never satisfy the pipeline's pixel-stability settle loop, so their captures land mid-phase. The new \.isCapturingSnapshot environment value lets such a view render a deterministic end-state of motion during capture. The contract lives on the property: read it only to freeze motion at a canonical phase — never to change layout, content, or behavior; the settle loop remains the fallback for views that don't opt in. The key is bridged from a UIKit trait (SnapshotCaptureTrait) rather than a plain @entry so it survives the pipeline's re-hosting: the capture pipeline sets traitOverrides on the captured content controller (removed after capture), and UIKit propagates it into the hosted SwiftUI tree in every capture path — plain, accessibility-wrapped, and the intrinsic-measurement probe. The preview cutsheet applies the same flag via the environment so previews mirror what tests capture. Co-authored-by: Cursor <cursoragent@cursor.com>
TypewriterText renders the full final text immediately during capture (the reveal's end state), the same path Reduce Motion takes — its sentence pauses (340ms) outlast the settle loop's stability window, so a capture could land mid-reveal. LaunchSplashView skips the repeatForever icon pulse and freezes the radar rings at a canonical phase: the TimelineView clock is paused and the phase pinned to a constant, because a paused TimelineView still reports the wall-clock pause time, which would leak run-dependent ring radii/opacities into the image. AppIconActivityIndicator (the recent-activity loading pulse) adopts the same pin as the splash pulse — the only other repeatForever in WhereUI. System ProgressView spinners are deliberately left alone: their motion isn't ours to freeze, and no snapshot fixture pins one mid-spin. SnapshotCaptureFlagTests proves a view reading the flag sees true through the real pipeline (probe view captured via renderSnapshotImage, pixel-probed green) — including across the WhereUI dynamic-framework boundary. Co-authored-by: Cursor <cursoragent@cursor.com>
The repo points core.hooksPath at .githooks (set by ./ide), so the standard hooks 'git lfs install' writes must be tracked there for clones to run LFS clean/smudge maintenance. Pure infrastructure, no behavior change to the tracked pre-commit formatting hook. Co-authored-by: Cursor <cursoragent@cursor.com>
Every top-level WhereUI screen declares its snapshot matrix via a SnapshotProviding conformance in ScreenSnapshots.swift (driving the #Preview cutsheet and the image tests off one declaration), and ScreenSnapshotTests is one assertSnapshots(of:) call per screen. WhereSnapshot gains the compact phoneLightDark/componentLightDark matrices for a screen's secondary states. PreviewSupport fixtures are pinned for determinism: a fixed referenceNow (midday Pacific, mid-sample-year) drives every model's now so today chrome doesn't churn daily; loadedModel() is pre-onboarded over in-memory preferences so RootView renders logged-in UI and the host's UserDefaults can't leak in; previewServices() gains a NoopDataIssueAlertScheduler so the launch sequence can't suspend on a real notification-permission prompt. Reference images (ScreenSnapshotTests) are recorded on iPhone 17 / iOS 26.2 and stored via Git LFS. Co-authored-by: Cursor <cursoragent@cursor.com>
WidgetSnapshots declares matrices for the widget entry views and lock-screen accessories over a fixed snapshot day; AppFlowSnapshots covers the launch splash (default + slow-launch caption, frozen via isCapturingSnapshot), onboarding, the logged-in root scene, and the DEBUG SwiftData inspector against a live seeded in-memory store. WidgetSnapshotTests and AppFlowSnapshotTests are one assertSnapshots call per surface; reference images recorded on iPhone 17 / iOS 26.2, stored via Git LFS. Co-authored-by: Cursor <cursoragent@cursor.com>
Records the four known-nondeterministic areas left after the async pipeline fix and isCapturingSnapshot adoption: the live-log debugLogViewer captures, the iPad ax5 sheet-offset shift, the unexplained in-memory-vs-on-disk comparison discrepancy, and root.LoggedIn rendering the empty state. Co-authored-by: Cursor <cursoragent@cursor.com>
The WhereUISnapshotTests suites now cover every screen, widget, and app-flow surface these hosted 'renders without crashing' tests exercised, so delete the nil-only hosting tests and keep the behavioral ones: - Delete ScreenHostingTests, WidgetViewsTests (hosting half), LaunchSplashViewTests, WhereUITests (rootViewBuilds), the OnboardingView render test, and the inspector render test. - Keep behavioral coverage: OnboardingModelTests, the inspector schema-drift guard, and the WidgetSnapshot ranking tests (moved to WidgetSnapshotRankingTests to stay 1:1 with WidgetSnapshot+Ranking); widget string checks move into StringsTests. - Remove TestHostSupport.waitForOneRunloop — its only callers were the deleted smoke tests, closing the flake-paradise TODO in Where/TODOs.md. Stuff-iOS-Tests passes (200 suites, 0 failures). Co-authored-by: Cursor <cursoragent@cursor.com>
- Root AGENTS.md: note that image snapshot bundles run via standalone schemes + the dedicated CI snapshot job (not Stuff-iOS-Tests), that __Snapshots__/ references live in Git LFS, and add SnapshotTests/ to the module directory skeleton. - Where/AGENTS.md: point the testing section at WhereUISnapshotTests and warn off re-adding hosting smoke tests for covered surfaces. - WhereUI AGENTS.md + README.md: document the SnapshotTests/ bundle, the SnapshotProviding conformances in Sources/Preview/, and how to re-record a reference (delete the PNG, run the scheme). - SnapshotKit / SnapshotKitTesting docs verified accurate; no changes. Docs-only change; ran ./sync-agents. Co-authored-by: Cursor <cursoragent@cursor.com>
Pin UIGraphicsImageRendererFormat.preferredRange = .standard on the tile, stitch, and settle-digest renderers, and round-trip the final capture through pngData()/UIImage(data:scale:) so the perceptual compare sees exactly the bytes flushed to disk. Wide-gamut in-memory captures were diffing against sRGB PNG references while the on-disk artifacts were pixel-identical — the in-memory-vs-on-disk deltaE flake. All 199 references re-recorded as 8-bit sRGB (previously 16-bit Display P3). Remaining verify failures (calendar scroll position, debugLogViewer live buffer) are addressed by later phases or already filed in Where/TODOs.md. Co-authored-by: Cursor <cursoragent@cursor.com>
Apply .transaction { disablesAnimations = true; animation = nil } to
the styled root in makeHostingController, so animated state changes in
hosted trees commit their end state instantly instead of being caught
mid-flight by the capture. The settle loop remains for .task-driven
async content.
Zero re-records: the full suite passes against the phase-1 references
except the two known-cause failures — calendar scroll position
(addressed by phase 4) and debugLogViewer's live log timestamps
(filed in Where/TODOs.md).
Co-authored-by: Cursor <cursoragent@cursor.com>
SnapshotCase (and the whereSnapshot factory) gain a SnapshotSettle axis: .settled (default) runs the async settle loop; .immediate — for content that is fully renderable after a layout pass — skips the digest-render loop entirely, doing one Task.yield plus a re-layout, in both resolveContentSize and the main capture path. The static widget cases and the appIcon screen case are marked .immediate (none have .task-driven content or TypewriterText). The settle loop now samples at 16ms (was 60ms) against an anchored 120ms quiet window: adjacent-frame comparison at the denser cadence would declare slow crossfades stable, so stability is judged against the first sample of the quiet window instead. minDuration rises to 250ms — the iOS 26 glass toolbar/tab bar adapts its material to content a few hundred ms after hosting, and a lower floor captured the pre-adaptation glass (primary/root regressions during tuning). Deviation from the plan's pixel-neutral goal: the two regionMap references are re-recorded. The new timing consistently captures the map before MapKit's place labels fade in (the old timing was a coin flip between mid-fade and loaded). Live-Map tile loading remains inherently nondeterministic — filed in Where/TODOs.md. Widget suite: 45.4s -> 29.9s. Full suite roughly neutral (the higher settle floor offsets the faster cadence on settled screens). Co-authored-by: Cursor <cursoragent@cursor.com>
The regionMap.Default_iPhone reference recorded during the phase-3 re-record caught a one-off state where MapKit's northern overlay tiles had not finished rendering; the two runs since both captured the same more-complete polygon (identical diff bbox), so adopt that capture as the reference. Live-Map nondeterminism remains filed in Where/TODOs.md. Co-authored-by: Cursor <cursoragent@cursor.com>
CalendarView reads \.isCapturingSnapshot and skips scrollToCurrentMonth under capture: the landing offset of a proxy scroll over a LazyVStack depends on how much lazy content has been measured at scroll time, so the captured scroll position was nondeterministic (the recurring calendar.WithData_iPhone / _iPad_ax5 failures). Snapshots now capture the deterministic top-of-year state — the re-recorded calendar.WithData_iPhone shows January at the top. All 10 calendar.* references were deleted and re-recorded; the two accessibility variants came back byte-identical (they already captured the unscrolled state), so 8 changed. Co-authored-by: Cursor <cursoragent@cursor.com>
Move the in-memory CILabDeltaE mismatch item to completed with its root cause (wide-gamut extended-range captures vs sRGB PNG references, fixed in the phase-1 sRGB pinning + PNG round-trip), annotate the iPad ax5 sheet-offset item as likely fixed by the phase-4 capture-time scroll skip, and extend the regionMap note to cover overlay-tile completeness as well as label fade-in. Co-authored-by: Cursor <cursoragent@cursor.com>
Add a Frame.fullContent(name:width:) preset (a new SizeStrategy case mapping onto the intrinsic content-measured pipeline) that renders a scroll view's whole content in one image — verified by a probe that a bare ScrollView measures to its content height under the unbounded sizeThatFits proposal, so no .fixedSize shim is needed. Content measurement now iterates to a fixed point so lazy stacks (whose first measurement is an estimate) materialize fully instead of cutting off; non-lazy content still measures identically on the first pass. Adds a probe-based ScrollView regression to LargeViewCaptureTests, a full-year calendar exemplar (CalendarYearGrid split out of CalendarView so the case wraps content, not NavigationStack chrome), and its new calendar.FullYear_fullHeight reference. Existing references unchanged.
Add the @MotionIsStatic property wrapper (Sources/Shared) — a DynamicProperty deriving one Bool from Reduce Motion OR snapshot capture — so a view with continuous/looping motion asks a single question: should it render its static end-state? Adopt it in TypewriterText, AppIconActivityIndicator, and LaunchSplashView (including RadarPingBackground's TimelineView pause, which drops its animated: parameter; the capture-only phase pin keeps reading \.isCapturingSnapshot, and the one-shot caption fade keeps honoring Reduce Motion alone). Behavior is identical, and the full snapshot suite passes pixel-neutral with zero re-records (only the known debugLogViewer nondeterminism failed, unchanged by this step). Records the rule in WhereUI's AGENTS.md; CLAUDE.md regenerated via ./sync-agents.
The debugLogViewer snapshot rendered the live shared WhereLog buffer, so wall-clock timestamps and run-dependent log lines leaked into the image — it failed every verify run. The case now renders a frozen fixture: a fresh production LogStore seeded through its @_spi(Testing) record seam with fixed lines (one per interesting level/category) whose timestamps pin around PreviewSupport.referenceNow. No production changes — the injectable seam (LogViewerConfiguration taking any store) already existed. Deletes and re-records both debugLogViewer references; the test passed two consecutive verify runs. Moves the TODOs.md item to completed.
The regionMap snapshot captured a live MapKit Map, whose tile/label loading is cache/network-dependent — no settle window could pin it. Under \.isCapturingSnapshot, RegionMapView now renders a deterministic stand-in of identical layout: the same region-outline polygons, drawn with the live fill/stroke styling in a Canvas over a flat substrate, framed by the same enclosing-region math the live camera uses (now shared via enclosingRegion(of:)). Only the tile substrate is substituted; the view's own overlays and legend still render for real. This bends the capture flag's never-change-content contract, so the contract doc on isCapturingSnapshot (and the SnapshotKit README) now carves out the exception explicitly: externally-loaded, non-deterministic content may substitute a deterministic placeholder of identical layout; everything else must render real content. Deletes and re-records both regionMap references; the test passed two consecutive verify runs. Marks the TODOs.md item resolved.
The WhereUISnapshotTests scheme's test action now pins the environment the LFS reference images were recorded on (SNAPSHOT_EXPECTED_SIMULATOR_RUNTIME_VERSION=26.2, SNAPSHOT_EXPECTED_SCREEN_SCALE=3) via a new optional testEnvironmentVariables parameter on the shared testScheme helper; other schemes keep an argument-less test action. Both assertSnapshots entry points guard those expectations against the live simulator (SIMULATOR_RUNTIME_VERSION + UIScreen.main.scale) before rendering anything: on a mismatch they record ONE clear issue naming expected vs actual and assert nothing, instead of failing every comparison with confusing pixel diffs. Runs without the expectation variables (direct renderSnapshotImage callers, non-scheme invocations) skip the guard. Guard proven to fire by forwarding a bogus expectation with TEST_RUNNER_SNAPSHOT_EXPECTED_SCREEN_SCALE=2 (every test failed fast with the mismatch message); full suite green with the real values. Also files the pre-existing root.LoggedIn glass-adaptation flake under the deferred snapshot-flakiness notes: it reproduced on an unmodified tree (1 in ~4 isolated AppFlowSnapshotTests runs), so it is not a pixel-neutrality regression of this change. Co-authored-by: Cursor <cursoragent@cursor.com>
Before rendering anything, assertSnapshots(of:) collects every full reference identifier (case name + configuration identifier, exactly as references are named on disk) across the provider's cases and refuses to assert when any collide, recording one issue that lists the colliding identifiers — a duplicate would silently compare one variant against the other's reference image. The collision detection is a pure function (duplicateSnapshotIdentifiers(in:)) sharing the identifier construction with the runner via fullSnapshotIdentifier(caseName:configuration:), so the guard and the on-disk naming can't drift. Unit-tested directly in DuplicateSnapshotIdentifiersTests, including the cross-case collision the underscore-joined format permits (Badge + dark vs Badge_dark). Full snapshot suite green; no reference images touched. Co-authored-by: Cursor <cursoragent@cursor.com>
When assertSnapshots' record parameter is nil, the runner reads a SNAPSHOT_RECORD environment variable (all / failed / missing / never, mapped through SnapshotTestingConfiguration.Record's rawValue init) and applies it via withSnapshotTesting(record:) around each assert. Precedence: explicit parameter, then SNAPSHOT_RECORD, then the suite's .snapshots(record:) trait. An unrecognized value records an issue rather than silently asserting with the suite default. Forwarding verified end to end: xcodebuild passes TEST_RUNNER_-prefixed variables into the test process, so with one reference deleted, TEST_RUNNER_SNAPSHOT_RECORD=never made the test fail without rewriting the file (the suite trait alone would have re-recorded it — proving the env var reached the process and beat the trait), and TEST_RUNNER_SNAPSHOT_RECORD=missing re-recorded it. The deleted reference was restored from git; zero net reference changes. Documented the working invocation (TEST_RUNNER_SNAPSHOT_RECORD=failed mise exec -- tuist test WhereUISnapshotTests ...) as the primary re-record flow in the SnapshotKitTesting and WhereUI READMEs, replacing delete-then-rerun. Full snapshot suite green; no reference images touched. Co-authored-by: Cursor <cursoragent@cursor.com>
A SNAPSHOT_DIFF_TOOL environment variable (forwarded as TEST_RUNNER_SNAPSHOT_DIFF_TOOL on the command line) selects the diff tool printed in failure messages: ksdiff maps to SnapshotTestingConfiguration.DiffTool.ksdiff; absent or unknown values keep the default plain file-URL output. Applied through the same single withSnapshotTesting(record:diffTool:) call that phase 9's record override uses, so the two env overrides combine in one wrap. Documented under "Diffing failures" in the SnapshotKitTesting README. Full snapshot suite green; no reference images touched. Co-authored-by: Cursor <cursoragent@cursor.com>
SnapshotCase gains an optional onReadyToSnapshot (@mainactor () async -> Void) hook (SwiftUI-convenience default of nil, matching how the settle axis is handled), threaded through both assertSnapshots entry points and into renderSnapshotImage. The pipeline invokes it once AFTER the content settles and BEFORE the accessibility parse / cursor-hiding / capture — the deterministic point to focus a field or trigger a presented state — then settles the hook's effects again (same settle mode) so they're committed in the image. Direct renderSnapshotImage callers keep compiling via the defaulted parameter. Regression-tested probe-based (no references): PreCaptureHookTests drives a three-state @observable probe (red until .task fires -> blue, green only once the hook flips it) and asserts the hook saw the settled (post-.task) state and that the captured pixels are green. SnapshotCase storage/default covered in SnapshotKitTests. READMEs for both modules document the hook and when it runs. Also documents (Where/TODOs.md, deferred snapshot-flakiness) two PRE-EXISTING, environment-dependent reference failures uncovered while verifying — both reproduced at the branch base commit (bcf1a2d) with zero phase 7-11 changes, so neither is a pixel-neutrality regression of this work: - manualDay.Add_*: the compact DatePicker's displayed format depends on the REAL-WORLD date (it sizes its capsule from today's medium-format string and renders the selection medium only if it fits, else short numeric). References recorded Jul 16 show 'Jul 15, 2026'; on Jul 17 the same selection renders '7/15/26'. Deterministic per-day, not random. Durable fix is a \.isCapturingSnapshot date-label stand-in + one re-record (out of scope here; would change references). - root/primary glass toolbar/tab-bar pre-adaptation flake (~1 in 4). Phase 11 code verified via PreCaptureHookTests + SnapshotKitTests (green) and a full-suite run with the two known flakes accounted for; no reference PNGs added or modified. Co-authored-by: Cursor <cursoragent@cursor.com>
The compact DatePicker's value capsule renders relative to the
real-world date: it reserves its width from *today's* date in the
medium format and falls back to short numeric ("7/15/26") whenever the
selection's medium string doesn't fit — so the manualDay.Add_*
references, recorded Jul 16 with a pinned Jul 15 selection, failed
deterministically from Jul 17 on, and no settle window could help.
Under \.isCapturingSnapshot the compact pickers now render
SnapshotDatePickerStandIn: the same title + trailing-capsule row with
the selection in a fixed Date.FormatStyle and locale, so the image is a
pure function of the selected value. Applied to every picker a snapshot
case captures — ManualDayView's add-mode date pickers (single-day and
range) and SettingsView's reminder/summary time pickers, which share
the same mechanism family. AddEvidenceView's picker is not captured by
any case today; filed in Where/TODOs.md as latent.
The \.isCapturingSnapshot carve-out (SnapshotCaptureFlag + the
SnapshotKit README) now covers system controls whose rendering depends
on wall-clock state, alongside externally-loaded substrates.
Re-recorded the 22 references containing a substituted capsule:
manualDay.Add_{iPhone,iPad}{,_dark,_contrast,_ax5,_accessibility},
manualDay.AddWithCancel_iPhone{,_dark}, and
settings.Default_{iPhone,iPad}{,_dark,_contrast,_ax5,_accessibility}
(settings.Default_iPhone_ax5 re-recorded byte-identical — the capsule
is below the fold at ax5). Verified green twice in a row; WhereUITests
and swiftformat --lint clean.
Closes the manualDay TODOs entry (moved to completed).
Co-authored-by: Cursor <cursoragent@cursor.com>
The iOS 26 glass toolbar/tab bar adapts its material to the content behind it a few hundred ms after hosting, starting *quiet* — so the pixel-stability settle loop could exit at the default 0.25s floor before the adaptation even began, intermittently capturing the pre-adaptation light glass pills (root.LoggedIn ~1 in 4 isolated runs; primary.Loaded_iPhone once). SnapshotSettle gains .settledAtLeast(minDuration:) — the .settled loop with a raised minimum window (hang budget extended above the floor so the minimum is always honored). It's a general per-case knob rather than a raised global floor, so only cases hosting such chrome pay it: RootView.LoggedIn and PrimaryView.Loaded opt in at 1.0s. Measured cost: root() ~4.5s -> ~5.1s (2 captures), primary() ~24s -> ~31s (10 captures); every other settled case keeps the 0.25s floor. Evidence: 4 consecutive isolated AppFlowSnapshotTests runs green (untreated baseline flaked ~1 in 4), plus a green primary() run. SnapshotKitTests green; swiftformat --lint clean. Co-authored-by: Cursor <cursoragent@cursor.com>
SnapshotConfiguration gains a layoutDirection axis (default .leftToRight, identifier token rtl) and a legibilityWeight axis (default .regular, token bold). Both apply in lockstep everywhere a configuration renders: the SwiftUI environment + mirrored UIKit trait overrides in the test pipeline's makeHostingController, the uiTraitCollection bridge, and snapshotTraits for the preview cutsheet. combinations(...) multiplies the new axes like the existing ones. Frame gains safeAreaInsets (a small Hashable Insets type; default .zero so images stay device-independent), threaded through assertSnapshots into renderSnapshotImage's existing safe-area override. The .iPhoneNotched preset simulates real device chrome (Dynamic Island top 47pt, home indicator bottom 34pt); new frame name token only. Identifier omission rules unchanged at defaults — the full suite passes with every existing reference byte-identical, proving the wire format held. Exemplar: settings.Default gains one RTL iPhone variant (settings.Default_iPhone_rtl.png, recorded once, verified twice) — labels right-align, toggles/values mirror to the leading edge. SnapshotConfigurationTests covers the new tokens, the axis multiplication, and the frame insets; SnapshotKit README documents both axes and the notched preset. Verified: SnapshotKitTests green; full WhereUISnapshotTests green (37/37) with only the new reference added; swiftformat --lint clean. Co-authored-by: Cursor <cursoragent@cursor.com>
Verified locally that xcodebuild's parallel testing cannot be enabled
for WhereUISnapshotTests, and documented why in the workflow instead
of adding the flags.
Evidence (Xcode 26 / iOS 26.2 simulator, tuist passthrough, tried both
-maximum-parallel-testing-workers 2 and -parallel-testing-worker-count
2): xcodebuild scheduled all 45 tests onto a single simulator clone
('Clone 1', one StuffTestHost process), but enabled concurrent test
scheduling inside that process. The capture pipeline is single-tenant
per process — every test renders into the one StuffTestHost key window
and the settle phase suspends the main actor — so concurrent tests
interleave mid-capture and corrupt each other's images: 24 spurious
'does not match reference' failures (plus the LargeViewCapture size
probe reading another test's content), and wall time got WORSE
(407s and 1004s vs ~330s serial; the failure-retry churn dominates).
Since parallelism never crossed process boundaries in practice, the
per-process single-window model can't benefit; the serial run stays.
The snapshot job keeps its serial command with a comment recording
the verification so nobody re-tries the flags blind.
Co-authored-by: Cursor <cursoragent@cursor.com>
Brings every snapshot doc current with the knobs the hardening phases added, and fixes stale text: - SnapshotKitTesting AGENTS.md: the 'single Snapshotting strategy' invariant was stale — the pipeline is the async renderSnapshotImage(...) function (its async is load-bearing for settling). Added the missing invariants: PNG round-trip compare, fail-fast setup guards (simulator pin, duplicate identifiers), and the single-tenant-capture reason the suite must stay serial. - SnapshotKitTesting README: documents the fail-fast guards and the frame-declared safe-area insets in the pipeline summary. - SnapshotKit AGENTS.md: identifier-token examples include the new rtl/bold tokens plus the add-an-axis-safely rule; the capture-flag invariant now names the placeholder carve-out. - WhereUI README/AGENTS.md: matrix axis list includes RTL and bold text; the re-record flow mentions the TEST_RUNNER_SNAPSHOT_RECORD bulk path alongside delete-and-rerun. Ran ./sync-agents; swiftformat --lint clean. Verified beforehand: two consecutive full WhereUISnapshotTests runs green (37/37 each, zero reference modifications) and Stuff-iOS-Tests green. Co-authored-by: Cursor <cursoragent@cursor.com>
Conflict resolutions:
- AGENTS.md, Where/TODOs.md: clean union of both sides' additions
(main's synthesized-Codable + composition-root sections and new TODO
entries alongside our snapshot-testing docs and flakiness ledger).
- LaunchSplashView.swift: main's honest-splash copy (launch-neutral
"Getting things ready…" caption strings + doc) combined with our
snapshot determinism work (@MotionIsStatic, radar phase pinned to 0
under \.isCapturingSnapshot, previewShowsCaption seam).
- PreviewSupport.swift: main's FlightDayIssue sample fixture kept
alongside our pinned referenceNow dates, pre-onboarded loadedModel,
and Noop scheduler injection.
- SettingsView.swift: main's "Find issues now" scan section combined
with our SnapshotDatePickerStandIn capture substitution.
- ScreenHostingTests.swift (modify/delete): kept our deletion — the
hosting smoke tests were superseded by WhereUISnapshotTests. Main's
new flightDayDetailViewHosts coverage is ported as a snapshot case
instead: FlightDayDetailView gains a SnapshotProviding conformance
(day pinned to referenceNow) and a ScreenSnapshotTests.flightDayDetail
test with 10 new references.
Re-recorded references (all legitimate upstream UI changes):
- launchSplash.SlowLaunchCaption_iPhone{,_dark}: caption copy changed
from "Updating your data…" to "Getting things ready…" (#99).
- settings.Default_iPad{,_dark,_contrast,_accessibility}: new
"Find issues now" row in the Data resolution section (#90); the
iPhone variants are unchanged because that section sits below the
captured fold there.
- resolution.WithIssues_{iPhone,iPad}{,_dark,_contrast,_accessibility}
and resolution.WithIssues_iPad_ax5: new "Flights" section renders the
flight-day sample issue (#90); iPhone_ax5 unchanged (below the fold).
Stuff-iOS-Tests passes. Snapshot suite: failures triaged in a full
diagnostic run, re-recorded references verified green with a targeted
run; the final all-green full-suite pass was skipped at user request —
PR CI serves as the full-suite verification.
Co-authored-by: Cursor <cursoragent@cursor.com>
The LFS reference images bake the recording machine's Pacific wall-clock dates/times into pixels (widget day labels, log-viewer timestamps, the SwiftData-inspector seed row), so an unpinned UTC CI runner would shift every date-rendering snapshot into a wall of confusing image diffs. - Project.swift: the WhereUISnapshotTests scheme's test action now sets TZ=America/Los_Angeles (the pin) and SNAPSHOT_EXPECTED_TIMEZONE (the guard), alongside the existing SNAPSHOT_EXPECTED_* simulator pins. - AssertSnapshots.swift: simulatorMatchesSnapshotExpectations() gains a timezone arm — if TimeZone.current doesn't match the expectation, the runner records one clear issue and asserts nothing, same as the runtime/scale mismatches. This permanently verifies the TZ pin actually reaches the test process. - SnapshotKitTesting README: document the timezone pin requirement. Verified: with a temporary TZ=UTC probe in the scheme, all five WidgetSnapshotTests failed on exactly the new timezone-mismatch guard (proving TZ propagates through the test action into the test process); with the Pacific pin, WidgetSnapshotTests, calendar, manualDay, and debugLogViewer all pass against the existing references — no re-record. Fixes the app-side review's High finding (H1) from the July 2026 snapshot-testing PR review. Co-authored-by: Cursor <cursoragent@cursor.com>
The pipeline is @mainactor async with long suspensions (settle loops, pre-capture hooks) while holding process-global state: the safe-area swizzle is a method_exchangeImplementations parity toggle plus two override globals, UIView.setAnimationsEnabled is a save/restore with the same nesting hazard, and every capture renders into the one StuffTestHost key window. Interleaved captures corrupt all of it — proven by the Phase 13 parallel experiment (commit 71c0d36, 24+ spurious mismatches) — but until now the only defenses were prose and xcodebuild's serial default. - SnapshotCaptureLock (new): a FIFO @mainactor mutex; renderSnapshotImage runs its whole body under it, so a concurrent capture queues instead of corrupting the in-flight one. Re-entry from a task already holding the lock (a hook rendering another snapshot) is a programmer error and traps instead of deadlocking. - SafeAreaInsetsSwizzling: the method exchange is now depth-counted — only the 0-to-1 and 1-to-0 transitions exchange implementations, so nested/unbalanced pairs can't flip parity; an unpaired unswizzle traps. - ConcurrentCaptureTests (new, WhereUISnapshotTests): captures a safe-area-sensitive probe view under 20pt and 0pt inset overrides, serially then concurrently from one task group, and requires identical probed pixels plus unchanged host-window insets afterward. Verified the guard bites: with the lock temporarily bypassed the test fails on the interleaving corruption; with it, it passes. - SnapshotKitTesting README/AGENTS.md: the single-tenant rule is now documented as code-enforced, not convention. Verified: SnapshotKitTests green; full WhereUISnapshotTests suite green against the existing references (pixel-neutral, no re-record). Fixes the framework review's High finding H1 from the July 2026 snapshot-testing PR review. Co-authored-by: Cursor <cursoragent@cursor.com>
kyleve
commented
Jul 17, 2026
| /// Snapshot captures substitute the compact time pickers' value capsules | ||
| /// with a deterministic stand-in — the live capsule's rendering depends on | ||
| /// real-world clock state (see `SnapshotDatePickerStandIn`). | ||
| @Environment(\.isCapturingSnapshot) private var isCapturingSnapshot |
Owner
Author
There was a problem hiding this comment.
Instead of exposing this in product code, let's create a WhereDatePicker view that checks this internally and shows the deterministic stand-in, to avoid leaking snapshot environment checking to product UI.
Owner
Author
There was a problem hiding this comment.
Also, would another option be to pre-set the date to something known instead of creating our own fake? (We should still wrap the date picker, but then our wrapper could set its date in snapshots vs creating its own fake version)
The WhereUISnapshotTests comment in Project.swift claimed SnapshotKit arrives only transitively through WhereUI — but listing SnapshotKitTesting in extraPackageProducts statically embeds its dependency closure (SnapshotKit included) into the .xctest, while WhereUI (a dynamic framework) carries its own copy. That is the duplicate-type-metadata hazard from the root AGENTS.md Targets note, and \.isCapturingSnapshot is exactly the type-keyed cross-boundary lookup it warns about: the pipeline writes SnapshotCaptureTrait via the bundle's copy while WhereUI's stand-ins (MotionIsStatic, RegionMapView, SnapshotDatePickerStandIn) read via WhereUI's. It works today, but a silent future split would revert every stand-in to nondeterministic rendering, and the existing SnapshotCaptureFlagTests probes a view compiled into the bundle — the same image as the writer — so it could never catch that split. - Project.swift: replace the misleading comment with the real topology and why the duplicate embed is tolerated (the pipeline has no other route into the bundle; the trait lookup demonstrably resolves across both copies today). - SnapshotCaptureFlagProbe (new, WhereUI, DEBUG-only): a WhereUI-defined view that renders green iff \.isCapturingSnapshot is true. - SnapshotCaptureFlagProbeTests (new, WhereUISnapshotTests): pixel-probes that view through renderSnapshotImage — the cross-boundary guard, modeled on WhereStylesheetTests.resolvesTraitAwareTokensFromTheBroadwayRoot. The existing same-image SnapshotCaptureFlagTests stays, its doc now naming the scope split. - SnapshotKitTesting/AGENTS.md: record the tolerated duplicate embed as a module invariant. Verified: SnapshotCaptureFlagProbeTests + SnapshotCaptureFlagTests green in a targeted WhereUISnapshotTests run. Fixes the framework review's High finding H2 from the July 2026 snapshot-testing PR review. Co-authored-by: Cursor <cursoragent@cursor.com>
kyleve
commented
Jul 17, 2026
| @propertyWrapper | ||
| struct MotionIsStatic: DynamicProperty { | ||
| @Environment(\.accessibilityReduceMotion) private var reduceMotion | ||
| @Environment(\.isCapturingSnapshot) private var isCapturingSnapshot |
Owner
Author
There was a problem hiding this comment.
Can we confirm these lookups actually work in property wrappers?
Docs-only commit: no code changes. Every Medium and Low finding from the two-part July 2026 snapshot-testing PR review (framework + app-side) is recorded durably; the three High findings were fixed in the preceding three commits and are not re-filed. - Shared/SnapshotKitTesting/TODOs.md (new): silent settle-loop timeout (M1), accessibility-parse preconditionFailures killing the shared host process (M2), the pipeline coverage gaps (M4: iPhoneNotched preset via the config mapping, lazy-container measurement iteration, env-parsing seam, 2000pt tile seam), cancellation mid-settle (L1), the unused AccessibilitySnapshot umbrella dependency (L2), wall-clock settle deadlines (L3), and the inline overload's missing duplicate-identifier guard (L4). - Shared/SnapshotKit/TODOs.md (new): case content/models built once and shared across every configuration (M3). - Where/TODOs.md: launchSplash caption-timer race and the appIcon .immediate settle join the deferred-flakiness list; ManualDayView range-mode coverage loss and the RegionMapView live-Map coverage gap join P1s; the onboarding fixture's real-UserDefaults leak, the duplicated pinned-instant fixtures, the stale cutsheet/bold-text doc claims, and the PrimaryView-empty/RecentActivity-loading matrix gaps join P2s. Also moved the 'Add snapshot images to a new test target' P0 to Completed (the review's L4 — this branch is that work). Co-authored-by: Cursor <cursoragent@cursor.com>
Under snapshot capture, LaunchSplashView's .task now returns before starting the 500ms slow-launch caption timer (guarded on \.isCapturingSnapshot), so whether the caption is visible at capture time is decided solely by the explicit previewShowsCaption seam — never by a race between the wall-clock timer and the settle loop's variable duration. Both caption states keep their own snapshot cases (launchSplash.Default without, launchSplash.SlowLaunchCaption with), and both existing references reproduce exactly — no re-record. The read uses \.isCapturingSnapshot directly rather than @MotionIsStatic: Reduce Motion must not suppress the caption (those users still get it, just without the fade), so this isn't a motion end-state — it's the capture-only determinism carve-out. The capture-flag contract doc (SnapshotCaptureFlag, SnapshotKit README/AGENTS) now records wall-clock timers that flip visible state as part of that carve-out, and the Where/TODOs.md ledger entry moves to the resolved section. Verified with 6 consecutive green isolated AppFlowSnapshotTests runs. Co-authored-by: Cursor <cursoragent@cursor.com>
Three consecutive CI failures on root.LoggedIn_iPhone (runs 29617740607, 29621920437 x2) showed the flake class is broader than the glass pre-adaptation pills the 1.0s settledAtLeast floor targeted: the failure artifacts baked the launch splash in one run (the runner never reached .ready inside the settle window) and the pre-activation tab UI in another (sample-report cards + Elsewhere tab, before MainTabs' activate() re-pulled the empty store and the Resolve badge appeared). Mechanism: the logged-in root is multi-phase async work — splash → launch steps → .ready → MainTabs activation → glass-material adaptation — and every phase is pixel-quiet under capture (motion frozen, animations disabled), so the pixel-stability loop legitimately exits inside any inter-phase gap longer than its 0.12s quiet window. A raised floor only rescales that race on slow CI runners. Fix: whereSnapshot now passes through SnapshotCase's onReadyToSnapshot hook, and RootView.LoggedIn awaits launcher.run() in it — run() is idempotent and awaits the in-flight drive, a deterministic "reached .ready" signal — so the re-settle after the hook (floor raised to 1.5s) only has to outlast the post-ready tail: MainTabs' .task activation and the glass adaptation, which have no reachable completion signal (ledgered as a follow-up in Where/TODOs.md). Evidence: 6 consecutive green isolated AppFlowSnapshotTests runs (root() ~8.6-8.9s); references unchanged; swiftformat --lint clean. Co-authored-by: Cursor <cursoragent@cursor.com>
Conflict resolutions: - RegionMapView.swift: kept both sides' environment reads — our \.isCapturingSnapshot (SnapshotMapStandIn substitution) alongside main's \.regionStyles (data-driven per-region tints, #95). - SettingsView.swift: import-block conflict only — kept both SnapshotKit (our SnapshotDatePickerStandIn substitution) and RegionKit (main's new Regions section + RegionsSettingsView sheet). Both features coexist in the body via git's auto-merge. - AGENTS.md, WhereUI AGENTS.md/README.md, ManualDayView, CalendarView, PreviewSupport auto-merged cleanly: our snapshot-testing docs/stand-ins sit alongside main's flaky-tool docs, PR-description rule, region-style resolver docs, grouped region sections, and the primaryRegionSelectionModel fixture. No new stand-ins were needed: main's new snapshot-visible surfaces (grouped region toggles, the Settings Regions row, restore-from-backup button) are deterministic; the region-picker's live Map is not part of any captured snapshot case. Re-recorded references (all legitimate upstream UI changes from #95): - onboarding.Default_* (10): the intro adds the 'Restore from a backup' button under Continue, shifting the page content up. - secondary.Loaded_* (7): the EU card's stamp symbol changed from star.circle.fill to star.fill (RegionAppearanceCatalog's default). - settings.Default_* (10): new Regions section between Location and Reminders (iPhone_ax5 unchanged — below the captured fold). - manualDay.* (16): region toggles are now grouped (Your regions / More regions disclosure) instead of one flat catalog list. Stuff-iOS-Tests passes. Snapshot suite: failures triaged in a full diagnostic run (artifact-vs-reference inspected for each), re-recorded references verified green with a targeted run; the final all-green full-suite pass was skipped per standing instruction — PR CI serves as the full-suite verification. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a matrixed image snapshot-testing system for WhereUI: a two-module framework, full suites covering every screen/widget/app-flow surface, LFS-backed references, a dedicated CI job, and a hardening pass that made the suite deterministic across consecutive runs.
Framework
Shared/SnapshotKit(shippable; SwiftUI/UIKit only, no test machinery) owns the appearance matrix:SnapshotConfigurationaxes for color scheme, Dynamic Type, contrast, layout direction (rtl), legibility weight (bold), device frames (iPhone/iPad/component/fullContent/iPhoneNotchedwith simulated safe-area insets), and standard vs. VoiceOver-annotated captures. Identifiers omit default axes (a wire format for reference filenames).SnapshotProviding+snapshotPreviews: a view declares its matrix once; the same declaration drives the#Previewcutsheet and the tests, so previews show exactly what CI asserts.\.isCapturingSnapshot: a trait-bridged environment flag with a documented contract — views may only use it to freeze never-settling motion at a deterministic phase, plus a narrow carve-out for content no settle window can fix (live map tiles, wall-clock-dependent system controls) which may substitute placeholders of identical layout.Shared/SnapshotKitTesting(test-only) owns capture + compare: an async render pipeline (settle loop on pixel stability, safe-area override, animation quiescing, cursor hiding, tile-and-stitch for >2000pt content), accessibility annotation, and theassertSnapshotsmatrix runner.Suites, storage, CI
WhereUISnapshotTestsreplaces the old hosting smoke tests: every top-level screen, widget, and app-flow surface (launch, onboarding, logged-in root) is pinned across the matrix (~200 references).SnapshotTests/__Snapshots__/; recording is a failure by design so a recording run can't pass as green.Stuff-iOS-Tests), checks out withlfs: true, and uploads xcresult/crash diagnostics on failure.Hardening pass
Determinism:
settleaxis (.settled/.settledAtLeast(minDuration:)/.immediate).DatePickerrenders relative to today's date), and a raised settle floor that outlasts the iOS 26 glass toolbar material adaptation.Ergonomics:
TEST_RUNNER_SNAPSHOT_RECORD=failed|all|missing|never) and a configurable diff tool (SNAPSHOT_DIFF_TOOL=ksdiff).onReadyToSnapshotpre-capture hook (runs after settle, re-settled before capture) for focus/presented states.iPhoneNotchedinset frame; exemplar RTL reference on the settings screen.Frame.fullContent) renders whole scrollable content in one image.Verification
Stuff-iOS-Tests(all unit bundles) green.