Skip to content

test(ios): add snapshot engine conformance gates - #2213

Merged
thymikee merged 5 commits into
codex/2191/ts-enginefrom
codex/2191/ts-conformance
Sep 1, 2026
Merged

test(ios): add snapshot engine conformance gates#2213
thymikee merged 5 commits into
codex/2191/ts-enginefrom
codex/2191/ts-conformance

Conversation

@thymikee

@thymikee thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member

Summary

Slice C of #2191 (“refactor(ios): build and prove the converged snapshot engine”), stacked on #2211 at exact head 6fd7576902065aa38b70e7ed869385bd13f1dbec.

  • Adds an independently authored 16-case corpus covering the engine contract seams, a standalone Swift conformance executable, deterministic Swift/TypeScript differential checks with four fixed seeds, replay artifacts, property coverage, and the typed R72 ownership gate.
  • The differential harness applies the same raw acquisition frontier to Swift and TypeScript. The Swift-backed lane is owned by an explicit macOS gate because apple/snapshot-presentation declares iOS/macOS platforms; Linux still runs the TypeScript corpus and the focused bridge-path tests.
  • No live routing, producer selection, public wire shape, or integration behavior changes; Slice D owns the later runner integration.

Scope and design rationale: 20 touched files, +2,664/-49 vs #2211, dominated by independent fixtures, the conformance harness, replay support, the macOS gate, and enforcement. The smallest equivalent design keeps the corpus and harness at the engine boundary: moving the Swift bridge into live routing would cross Slice C’s scope, while duplicating presenter logic would create another implementation owner. The added test-only package data does not change the production bundle surface.

Validation

Exact reviewed head: cbb3dae97a7bea963a32c0ff300e20c24313bc09

  • pnpm format and pnpm typecheck passed.
  • Focused snapshot-engine suite: 21/21 tests passed.
  • pnpm test:ios-snapshot-differential: 3/3 Swift/TypeScript differential tests passed on macOS.
  • swift build --package-path apple/snapshot-presentation passed.
  • The prior exact-head CI run for the 5cd3ce8 parent passed Coverage, Integration, Lint & Format, Repo Guards, Compatibility & Provenance, Typecheck & Package, size, Android, Linux, macOS, test-app cache, and CodeQL. The current cbb3dae head adds the explicit macOS differential gate; its exact-head CI run passed, and the exact-head macOS job passed the owned Run iOS snapshot Swift/TypeScript differential step. The Android/release and iOS simulator workflow remain in progress or queued.
  • The earlier exact-head iOS retry passed native build, targeted XCTest, CLI preflight, and Settings replay; its fixture-backed smoke reached 14/15 assertions before the first deep-link command hit the host-side xcrun simctl list devices -j 15-second timeout. The subsequent replacement iOS run was canceled by a higher-priority merge-ref request before execution. No snapshot semantic assertion failed; the remaining residual risk is the unverified final fixture smoke on this top head because of CI simulator contention. Exact-head iOS Simulator evidence for the changed snapshot route is recorded on refactor(ios): extract snapshot presenter #2206 and refactor(ios): implement snapshot engine #2211.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.55 MB 2.55 MB 0 B
JS gzip 857.6 kB 857.6 kB 0 B
npm tarball 979.6 kB 979.6 kB +20 B
npm unpacked 3.39 MB 3.39 MB +177 B

npm unpacked components

Component Base Current Diff
JS / dist source 2.71 MB 2.71 MB 0 B
Apple runner source/project 543.4 kB 543.4 kB 0 B
macOS helper source 54.8 kB 54.8 kB 0 B
Android helper artifacts 0 B 0 B 0 B
Other package files 79.7 kB 79.8 kB +177 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 30.6 ms 30.2 ms -0.3 ms
CLI --help 76.4 ms 74.3 ms -2.0 ms

Top changed chunks: no changes in the largest emitted chunks.

Top changed packed files

Packed file Base Current Diff
package.json 19.3 kB 19.5 kB +177 B

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-01 14:00 UTC

@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Blocking exact head 1fb9c7a4a2a0e995ccc439b1071307980077436d: the Swift/TypeScript differential harness does not compare identical acquisitions for unscoped raw requests with depth. prepareSwiftAcquisition drops over-depth nodes only from the Swift input, while TypeScript receives the original full acquisition. The generator exercises this shape, but the separate raw property forces depth: null, so it cannot catch the masked case. This can pass without proving Swifts raw-depth behavior. Remove the one-sided reshaping and compare identical acquisitions, or model the acquisition boundary explicitly and symmetrically for both implementations with an exact focused fixture.

The stack base, R72 planted-red gate, packaging exclusion, and zero packaged-size change otherwise look sound. CI is still in progress.

@thymikee
thymikee force-pushed the codex/2191/ts-conformance branch from 1fb9c7a to 42d23ae Compare September 1, 2026 09:33
@thymikee
thymikee force-pushed the codex/2191/ts-conformance branch from 42d23ae to b5f968b Compare September 1, 2026 09:35
@thymikee
thymikee force-pushed the codex/2191/ts-conformance branch from b5f968b to 00d7978 Compare September 1, 2026 10:20
@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 00d79780a433a4a6f016f668b91d82f538fb37bc: the prior asymmetric-input blocker is resolved. The same modeled raw-depth acquisition is now used for Swift and TypeScript; the focused regression adds a malformed over-depth tail that must be excluded at the acquisition frontier, and the golden corpus asserts the expected output. No new findings. READY for human code review; replacement exact-head CI remains queued/in progress.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 1, 2026
@thymikee
thymikee force-pushed the codex/2191/ts-conformance branch from 00d7978 to 623a136 Compare September 1, 2026 11:10
@thymikee thymikee removed the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 1, 2026
@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed unchanged exact head 00d79780a433a4a6f016f668b91d82f538fb37bc: the prior code fix remains sound, but current exact-head Coverage exposes an owned gate defect. All three differential tests enter on Ubuntu because swiftToolchainAvailable() checks only swift --version; they then fail before any assertion when swift build --package-path apple/snapshot-presentation cannot complete. The hermetic wrapper is only where the child-process failure surfaces, not proof of an infrastructure flake.

Keep the differential proof real: make admission reflect whether the conformance package can actually run in that lane, and execute the required Swift/TypeScript gate in an owned macOS/native CI lane. Do not merely skip it from Coverage without adding that required native execution. All other current checks, including iOS Smoke, are green, but confirmed owner-action CI means this head is BLOCKED and should not carry ready-for-human.

@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed new exact head 623a13661b961d5417b8fac57c80faa37e94d391: this is a restack, not a remediation. The PR-owned differential harness is unchanged: swiftToolchainAvailable() still admits any host where swift --version succeeds, while no owned macOS/native workflow runs the required Swift/TypeScript differential gate. The prior Ubuntu Coverage failure mode therefore remains, and replacement exact-head CI is still queued/in progress.

The body is also stale at 00d797…. Keep this BLOCKED until admission reflects where the conformance package can run, the differential proof executes as a required macOS/native gate, and current-head CI proves it. Do not merely skip Coverage without owned native execution.

@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 5cd3ce8eb37c589764b63a248647b55cae099bcb: the prior CI ownership blocker is not fixed at root. swiftToolchainAvailable() now returns false off macOS and c8 ignore removes Swift execution from Coverage, so Ubuntu is green by skipping all three differential tests. The added tests cover only TypeScript failure typing and replay metadata; no PR-owned required macOS/native workflow executes differential.test.ts. Keep this BLOCKED until the Swift/TypeScript differential suite runs as an explicit required macOS/native CI gate with exact-head evidence. The body is also stale at 00d797…; iOS Smoke remains in progress.

@thymikee
thymikee force-pushed the codex/2191/ts-conformance branch from cbb3dae to 5cd3ce8 Compare September 1, 2026 12:40
@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Addressed both latest blockers:

  • Comment 5493076035: cbb3dae97a7bea963a32c0ff300e20c24313bc09 gives the differential proof an owned native lane. The suite now lives at scripts/ios-snapshot-differential.test.ts, is exposed by test:ios-snapshot-differential, registered as ios-snapshot-differential in the affected-check catalog, and is invoked as a literal gate in .github/workflows/macos.yml.
  • Comment 5493921231: the exact-head macOS job passed Run iOS snapshot Swift/TypeScript differential (3/3 tests). The PR body now records exact head cbb3dae97a7bea963a32c0ff300e20c24313bc09, the 20-file/+2,664/-49 scope rationale, and the remaining iOS simulator CI residual risk.

Local confirmation also passes pnpm test:ios-snapshot-differential (3/3) and the focused snapshot-engine suite (21/21). I’m leaving #2213 unlabeled until its remaining platform/release and iOS simulator lanes finish.

@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head cbb3dae97a7bea963a32c0ff300e20c24313bc09: the prior CI ownership defect is fixed at root. The Swift/TypeScript differential suite is now an explicit required macOS gate and passed 3/3 on this exact head; stack, body, size, iOS, Release, and static evidence are otherwise sound. No new code finding. BLOCKED only by exact-head Android Smoke, where unrelated smoke:automation-system timed out waiting for Alert result: cancelled; this PR changes no Android route, so it looks like device/fixture noise, but rerun it green before readiness and update the body’s completed-check status.

@thymikee

thymikee commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head cbb3dae97a7bea963a32c0ff300e20c24313bc09 after the Android Smoke rerun: the previously unrelated automation-system timeout is cleared, and all 22 current exact-head checks now pass, including Android/iOS/macOS Smoke, both native Release lanes, static gates, and the macOS differential conformance gate. No code findings remain. READY for human review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 1, 2026
@thymikee
thymikee merged commit a8ee397 into main Sep 1, 2026
37 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant