Skip to content

feat: capture native structural observations - #96

Merged
abrichr merged 3 commits into
mainfrom
codex/capture-structural-post-94
Aug 27, 2026
Merged

feat: capture native structural observations#96
abrichr merged 3 commits into
mainfrom
codex/capture-structural-post-94

Conversation

@abrichr

@abrichr abrichr commented Aug 27, 2026

Copy link
Copy Markdown
Member

What this changes

Capture now records live macOS Accessibility and Linux AT-SPI evidence beside the action event. Provider failures stay nonfatal and keep an explicit unavailable result.

The macOS provider first asks the system-wide Accessibility object for the focused element. It falls back to the frontmost application when macOS does not return that value. The Linux extra keeps PyGObject below 3.50 so it remains compatible with the documented GLib 2.64 baseline.

Stack

This PR is stacked on #94 and must merge after it. The remote #94 branch currently lacks its local frame-interval fix at fba6c745, so this draft temporarily includes that commit. The diff will narrow when #94 updates.

This change does not add the terminal manifest or sealed-load contract. That work touches #94-owned capture files and belongs in the next stack layer.

Verification

  • ruff check openadapt_capture
  • 587 non-slow tests passed; 2 skipped; 12 deselected
  • live macOS focused-element qualification test passed
  • wheel and source archive built and passed scripts/verify_distribution.py
  • git diff --check passed

@abrichr
abrichr force-pushed the codex/capture-structural-post-94 branch from 4a9293f to 78c0be9 Compare August 27, 2026 17:22
Base automatically changed from codex/native-geometry-epoch-capture to main August 27, 2026 17:33
@abrichr
abrichr force-pushed the codex/capture-structural-post-94 branch from 78c0be9 to ba98574 Compare August 27, 2026 21:30
@abrichr

abrichr commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

Rebased onto merged #94 and force-pushed. The conflict is gone.

Before: +9440/-507 across 43 files, 8 commits, CONFLICTING.
Now: +1467/-67 across 13 files, 2 commits, mergeable.

The six lower commits were #94's work. #94 squash-merged, so they could never replay cleanly — I rebased with --onto origin/main 4bf541c, keeping only this PR's own two commits. I checked first that the carried frame-interval fix is genuinely on main (_assert_reserved_geometry_current, _begin_native_callback, _abort_failed_delivery are all in openadapt_capture/window_capture.py on main), so dropping it loses nothing. Zero conflicts. The narrowing is exactly what this PR's own description predicted.

Local suite on the rebased branch: 643 passed, 20 skipped, 13 deselected.

What still fails, and why

test (3.10) fails on tests/test_synthetic_fixtures.py for both fixtures. It is not a 3.10 problem — 3.11 and 3.12 were cancelled by fail-fast and would fail identically. It is also not caused by the rebase.

The cause is one line in this PR: it adds the linux extra to pyproject.toml without updating uv.lock.

I measured it rather than guessing, same worktree, same uv:

main's pyproject + lock   ->  uv sync --extra dev  ->  lock fb9a77ab...  STABLE
this PR's pyproject       ->  uv sync --extra dev  ->  lock 8758039f...  REWRITTEN

CI runs uv sync --extra dev. On main the lock already satisfies pyproject, so nothing is rewritten and the fixtures' pinned builder.lock_sha256 still matches. Here, adding PyGObject makes the lock stale, uv sync re-locks, sha256(uv.lock) changes, and the sealed fixture provenance no longer matches. That is the contract test doing its job.

The fix, and why I am not applying it

  1. Commit the updated uv.lock for the new linux extra.
  2. Regenerate the fixtures so their provenance records the new hash: uv run --locked --no-sync python -m scripts.generate_synthetic_captures.

I stopped short of doing this because step 1 is not small. This repo's committed lock predates the current uv, so the moment uv sync must re-lock it also migrates the format — revision = 3, upload-time on every entry, platform_system markers become sys_platform. That is +2195/-2190 in uv.lock, and step 2 then rewrites sealed binary fixtures whose contract another session owns.

Bundling a lock-format migration and regenerated sealed fixtures into someone else's feature PR, during the freeze, without the fixture owner's input, is not mine to decide. The diagnosis is exact and the two steps are mechanical — whoever owns the synthetic-fixture contract should run them.

Left open rather than closed: the rebase was the assigned action, it succeeded, and the code is now reviewable at 1,467 lines instead of 9,440.

abrichr and others added 3 commits August 27, 2026 18:17
The linux extra adds a platform-gated PyGObject dependency. Lock it so the
committed uv.lock describes the project's optional dependency groups.

uv resolves this to two Linux-only wheels, pygobject and pycairo, and changes
no already-resolved version. It also rewrites the lock in the current file
format, which accounts for the bulk of the diff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@abrichr
abrichr force-pushed the codex/capture-structural-post-94 branch from ba98574 to 96a1b35 Compare August 27, 2026 22:18
@abrichr

abrichr commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

Merging after confirming the force-push at 17:29 dropped nothing that is still needed.

The pre-force-push commit c61d17a (3441 insertions across 29 files) existed on no remote. It is now preserved at rescue/capture-96-pre-forcepush-c61d17a so it cannot be lost.

Comparing that commit's content against this PR's current head, by content rather than ancestry (a squash merge makes a new SHA, so ancestry lies here):

file in c61d17a in this PR on main
openadapt_capture/structural.py yes no yes
tests/test_input_observer_darwin.py yes no yes
scripts/read_structural_fixture_output.py yes no no
tests/test_structural_fixture_output.py yes no no

The first two were #94 content that arrived on main by squash merge, so dropping them from this branch was correct — that is exactly what the rebase --onto origin/main 4bf541c was for.

The last two exist only in c61d17a. They are referenced by nothing on main and nothing on this head, so their absence breaks no caller and does not block this merge. They are a fixture-output reader and its test — plausibly a useful debugging tool someone wrote and then lost to the force-push. They are on the rescue branch if anyone wants them back.

The fixture-provenance blocker was resolved separately in #107 (merged), which narrowed the provenance to the determinants that actually decide the fixture bytes. That was established by experiment: re-locking and regenerating with the old hash held produced byte-identical artifacts, so the lock's contents were causally inert and only its recorded hash was not.

@abrichr
abrichr marked this pull request as ready for review August 27, 2026 22:26
@abrichr
abrichr merged commit 7a96690 into main Aug 27, 2026
12 checks passed
@abrichr
abrichr deleted the codex/capture-structural-post-94 branch August 27, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant