Document shipped all-keyboards Caps behavior and remaining tests

docs/architecture/all-keyboards-caps.md

@@ -1,24 +1,27 @@
 # Caps Lock rules on every keyboard
 
-**Status:** implementation in progress, 2026-10-24. The worker now applies one
+**Status:** deployed on the mini and Air, 2026-10-24. The worker applies one
 crash-recoverable mapping per eligible keyboard and checks for arrivals every two
 seconds. Settings reports uncovered keyboards and offers per-keyboard exclusions.
-The original spike and open questions below remain the qualification record;
+The original spike and remaining questions below remain the qualification record;
 the opt-in `SessionCapsMappingFleetTests/testLiveAttachedKeyboardsAcquireAndRestore`
 passed on three attached USB keyboard services, including the physical HID
-fixture, with readback and exact restoration on 2026-10-24. This is not a
-release-build or hot-plug test. In particular, a press immediately after plugging in a
+fixture, with readback and exact restoration on 2026-10-24. The release build
+passed notarization and runtime verification, but this is not a hot-plug test.
+In particular, a press immediately after plugging in a
 keyboard can precede the worker's next inventory check. The passive per-press
 attribution described in the proposal has not been implemented.
+
 **Question:** can driverless KeyPath apply Caps Lock rules without asking the user to
 choose one keyboard?
 
 **Answer:** yes. The spike supports mapping Caps Lock on every compatible keyboard,
 each through its own device-scoped mapping, instead of choosing one. Three of the four
-assumptions held on real hardware; hot-plug timing is still unmeasured. Four questions
-remain open (see [Open questions](#open-questions)).
+assumptions held on real hardware; hot-plug timing is still unmeasured. Of the four
+follow-up questions, the existing-remap policy is decided and multi-interface coverage
+is confirmed structurally (2026-10-22), and the stuck F18 did not reproduce in a lab VM (2026-10-24); see [Follow-up questions and results](#follow-up-questions-and-results).
 
-## Why a keyboard has to be chosen today
+## Why the original prototype chose one keyboard
 
 macOS applies Caps Lock's toggle in the HID event system, before the session event tap
 where the driverless worker reads keys. The tap cannot cancel it. So managed Caps maps
@@ -29,52 +32,49 @@ Caps (`SessionCapsInputState`). See
 [session-caps-mapping-lease.md](../bugs/session-caps-mapping-lease.md) and
 [driverless-caps-lock.md](../../guides/driverless-caps-lock.md).
 
-The prototype tracked exactly one device. Everything downstream inherited that:
+The original prototype tracked exactly one device. Everything downstream inherited that:
 
 - `SessionCapsSelection` stores one `DeviceIdentity` and its intermediate usage.
 - `SessionCapsMappingLease` journals one device's applied mapping for crash recovery
   (journal version 1).
-- `SessionCapsRuntimeSupport.automaticSelection` succeeds only when the inventory has
-  exactly one eligible device and nothing withheld:
-  `guard devices.count == 1, withheld == 0 else { throw Refusal.selection }`.
+- Automatic selection required exactly one eligible device and nothing withheld.
 
-Any other inventory pauses Caps Lock rules until the user picks a keyboard. The guide
-frames this as "KeyPath does not guess which one you mean."
+Any other inventory paused Caps Lock rules until the user picked a keyboard.
+The fleet implementation replaced that default while retaining a selected-only mode
+for users who already made a choice.
 
 ### What that costs
 
-- Every other rule applies to every keyboard. A user who writes a Caps Lock rule almost
-  always means the same.
-- A MacBook with any external keyboard needs the manual choice. That is the most common
-  setup.
-- Multi-interface boards (QMK, the cl80) are withheld because one interface cannot be
-  picked arbitrarily. With "all keyboards" there is nothing to pick: every interface is
-  mapped.
-- The lab mini (cl80, X2A, fixture) has started every session with
-  `Caps keyboard not chosen automatically (selection); starting with Caps Lock rules paused`.
-  Its Caps Lock rule has never run there.
+- Every other rule applied to every keyboard while Caps Lock rules could be paused.
+- A MacBook with an external keyboard needed a manual choice.
+- Multi-interface boards (QMK, the cl80) could be withheld because one interface
+  could not be picked arbitrarily. The fleet now maps every verified keyboard-capable
+  interface.
+- The lab mini's cl80, X2A and fixture previously caused Caps Lock rules to start
+  paused until a keyboard was chosen.
 
-## Original proposal and remaining qualification work
+## Original proposal and shipped scope
 
-Keep the device-scoped mapping. Apply it to every keyboard KeyPath can target.
+The proposal kept device-scoped mappings and applied them to every keyboard KeyPath
+can target. The shipped implementation differs in two ways: it polls inventory every
+two seconds rather than mapping in an arrival callback, and it reports coverage in
+Settings instead of using passive per-press attribution.
 
 1. **Every eligible keyboard service gets the mapping** when the session starts, using
    the same write and readback the single-device path uses. A failed write affects only
    that keyboard.
-2. **Hot-plug.** An IOHIDManager matching callback (usage page 1, usage 6) maps a
-   newly arrived keyboard before its first keypress. A removed keyboard's service, and
-   its mapping, disappear with it; there is nothing to restore.
-3. **Journal one entry per device.** Crash recovery restores each entry under the
-   existing rules: exact owner, same boot, same device, unchanged mapping. A foreign or
-   ambiguous entry is retained, as today.
-4. **Report only what is actually uncovered.** A passive observer (IOHIDManager opened
-   without seizing; KeyPath already holds Input Monitoring) attributes raw Caps Lock to
-   its device. When a keyboard KeyPath could not map sends Caps Lock, KeyPath names it
-   and the reason, once. No speculative warning for keyboards that are never used.
+2. **Hot-plug.** The worker reconciles its device inventory every two seconds. It
+   admits a newly arrived keyboard when all keys are up and retires a removed one.
+   A first press can precede the next check.
+3. **Journal one lease per device.** Crash recovery restores each device under the
+   existing exact-owner, same-boot, same-device and unchanged-mapping checks. A failed
+   restoration retains its own record without discarding other keyboards' records.
+4. **Report coverage.** Settings reports keyboards that are excluded, withheld or
+   could not be mapped. Passive per-press attribution was proven in the spike but is
+   not part of the shipped runtime.
 5. **The single-keyboard choice becomes an advanced option**, not a prerequisite.
 
-Nothing else changes: the same intermediate pool (F13–F20), the same admission checks
-that move the intermediate off a key a rule or shortcut uses, the same
+The implementation retains the F13–F20 intermediate pool, admission checks and
 "all keys up before activation" check.
 
 ## Spike evidence
@@ -124,8 +124,8 @@ attached, raw Caps still toggled the latch: it did not take the device over.
   removal callbacks worked in both cases.
 - The first press right after opening was not reported. The machine was heavily loaded
   at the time; this needs a repeat on an idle machine.
-- The cl80 appeared as two keyboard-usage devices at one LocationID. `hidutil list`
-  showed one keyboard service. See open question 2.
+- The cl80 appeared as two keyboard-usage devices at one LocationID. The second is its
+  shared interface; see [Multi-interface coverage](#multi-interface-coverage-2026-10-22).
 
 ### A4. Merging with existing mappings: holds, with one rule
 
@@ -135,7 +135,7 @@ attached, raw Caps still toggled the latch: it did not take the device over.
 - **Same source key.** For two pairs with Caps as the source, the first pair wins:
   `[Caps → LeftCtrl, Caps → F18]` held Control during a 1.5 s hold;
   `[Caps → F18, Caps → LeftCtrl]` did not. A Caps Lock remap from System Settings →
-  Keyboard → Modifier Keys is exactly this case. See open question 1.
+  Keyboard → Modifier Keys is exactly this case. See follow-up question 1.
 
 ### A2. Hot-plug timing: not measured
 
@@ -157,53 +157,109 @@ worker. It cleared without intervention. The load average was 240 on 14 cores
 starvation is the likely cause. It was not shown to be unrelated to the mapping. See
 open question 4.
 
+It is the only snapshot of its kind. The mini's stuck-key monitor wrote 19 other
+snapshots between 2026-10-03 and 2026-10-19, on Escape, Delete, Left, Return and Tab.
+The six that recorded worker counters show input equal to output and nothing held,
+which fits long real key holds. Only the F18 snapshot shows the worker itself holding output.
+
+### Multi-interface coverage (2026-10-22)
+
+The cl80 exposes three HID interfaces at LocationID `0x2111300`: a keyboard (`1/6`), a
+shared interface whose primary usage is mouse (`1/2`), and a VIA vendor channel
+(`0xFF60`). The shared interface's event service (`0x100004b67`) lists
+`DeviceUsagePairs` of mouse, pointer, system control, consumer control and keyboard
+(`1/6`). This is where QMK sends N-key-rollover keyboard reports, so Caps Lock can
+arrive on it.
+
+`hidutil list --matching keyboard` returns only the `1/6` service, but KeyPath does not
+rely on that alone. `SessionCapsHIDUtilTransport.serviceGroup` adds every sibling
+service at the same location for which `serviceEmitsKeyboardUsages` is true (a `1/6`
+usage pair, or keyboard input in the report descriptor). The shared interface qualifies.
+Writing Caps→F18 with its pair selector (`PrimaryUsagePage 1`, `PrimaryUsage 2`) changed
+that service alone, read back correctly, and restored to empty.
+
+Not yet shown: a physical Caps press on the cl80 arriving through the shared interface
+and being remapped. That needs someone at the keyboard.
+
 ## Design details
 
-**Inventory.** `SessionCapsHIDUtilTransport.parseInventory` already separates targetable
-devices from withheld ones. "All keyboards" maps every device in `devices`, including
-every interface of a multi-interface board that the existing grouping rule already
-covers as one keyboard. A shared locator that would
-also select a different physical keyboard stays withheld.
-
-**Lease and journal (version 2).** One lease owns N entries, each with its device
-identity, the full array it replaced, and the array it wrote. Acquisition writes entries
-one at a time, each behind the existing mutation marker. Restoration walks every entry;
-one failure does not stop the others, and it leaves that entry for exact-owner retry.
-Version 1 journals are read and restored as a one-entry version 2.
-
-**Hot-plug.** The worker owns the matching callback, so the lease stays the single
-writer. A new device is admitted under the same checks as at startup (eligibility, all
-keys up), then written and journaled. A device that is not admitted is reported as
-uncovered, not retried in a loop.
-
-**Existing Caps Lock remap on a device.** Open question 1 decides the policy. Whichever
-is chosen, the stored array is preserved and restored byte for byte, which A4 showed is
-safe.
-
-**Migration.** An existing `SessionCapsSelection` becomes the advanced
-"only this keyboard" setting. Users who never chose one get all keyboards.
+**Inventory.** `SessionCapsHIDUtilTransport.parseInventory` separates targetable
+keyboards from withheld ones. The fleet maps every eligible service, including
+verified keyboard-capable interfaces on a multi-interface board. A selector that
+could also affect another physical keyboard stays withheld.
+
+**Lease and journal.** `SessionCapsMappingFleet` keeps one existing
+`SessionCapsMappingLease` record per keyboard beneath a common termination guard.
+Acquisition, recovery and restoration use the existing exact-owner checks; one
+keyboard's failure does not discard other keyboards' recovery records. The old
+single-keyboard record remains readable for recovery.
+
+**Hot-plug.** The worker checks inventory every two seconds while Caps rules run.
+It admits a new keyboard under the same eligibility and all-keys-up checks as at
+startup, and retires records for a removed service. A failed device is left
+uncovered and retried with backoff; an unavailable inventory leaves existing
+coverage intact and reports that it cannot be verified.
+
+**Existing Caps Lock remap on a device (decided 2026-10-22).** If a keyboard's current
+`UserKeyMapping` already has a pair whose source is Caps Lock, KeyPath leaves that
+keyboard unmapped and reports it as uncovered: "Caps Lock is remapped in System
+Settings on <keyboard>." It applies the rule to every other keyboard. This respects an
+explicit user choice, and nothing stops working without saying so. KeyPath never inserts
+its pair ahead of the user's. Mappings on other source keys are kept and restored byte
+for byte, which A4 showed is safe.
+
+**Migration.** An existing `SessionCapsSelection` keeps the advanced
+"only this keyboard" scope. Users who never chose one get all keyboards.
 `automaticSetupDisabledKey` keeps its meaning: no automatic Caps setup at all.
 
-**UI.** The status line reads `On · all keyboards · Uses F18 behind the scenes`. The
-keyboard picker moves under an advanced disclosure. The tour no longer needs a keyboard
+**UI.** Settings shows keyboard coverage, per-keyboard exceptions, and the
+selected-only option under Caps Lock options. The tour no longer requires a keyboard
 selection step.
 
-## Open questions
+## Follow-up questions and results
 
-1. **A keyboard that already remaps Caps Lock.** The first pair wins (A4). Options:
-   insert KeyPath's pair first and say so; or leave that keyboard alone and report it as
-   uncovered with the reason "Caps Lock is remapped in System Settings." The second
-   respects an explicit user choice and is the recommendation, but needs a decision.
-2. **Multi-interface boards.** The cl80 exposed two keyboard-usage devices to
-   IOHIDManager and one keyboard service to `hidutil list`. Confirm that mapping every
-   service `parseInventory` returns covers every interface that can send Caps Lock, by
-   pressing Caps on a QMK board.
+1. **A keyboard that already remaps Caps Lock.** *Decided 2026-10-22:* leave that
+   keyboard alone and report it as uncovered (see
+   [Design details](#design-details)).
+2. **Multi-interface boards.** *Confirmed structurally 2026-10-22:* the existing
+   service grouping covers the cl80's shared NKRO interface, and the mapping writes and
+   restores there (see
+   [Multi-interface coverage](#multi-interface-coverage-2026-10-22)). Remaining: one
+   physical Caps press on a QMK board in NKRO mode.
 3. **Hot-plug timing (A2).** Replug a keyboard with the watcher running. Also check
    whether macOS re-applies a Modifier Keys preference after arrival and overwrites
    KeyPath's array.
-4. **The stuck F18 under load.** Reproduce on an idle machine with a Caps→F18 mapping
-   active, and under synthetic load, to separate starvation from a mapping effect. This
-   is independent of "one versus all" but affects any managed-Caps rollout.
+4. **The stuck F18 under load.** *Not reproduced (2026-10-24); a mapping effect is
+   unlikely, and host starvation is still unconfirmed.* The test ran in a lab VM
+   (macOS 26.5.2, desktop-owned Parallels lane) with the fixture attached to the guest,
+   KeyPath master `112c0b1f`, and the mini host's Caps config. The setup was a device-scoped
+   Caps→F18 `hidutil` mapping on the fixture inside the guest.
+   - **Round A, idle:** 5 holds of 1.5 s and 10 taps of 40 ms.
+   - **Round B, no mapping:** 5 raw F18 holds of 1.5 s.
+   - **Round C2, mapping on, full guest CPU load:** 4 × `yes` at ~97% each, guest load
+     up to 7; 5 holds of 1.5 s.
+
+   After every press the worker's input and output counts matched (final 299/299),
+   `heldOutputUsages` was empty, no stuck-key incident was written, and Caps Lock stayed
+   off. Raw F18 behaved the same as mapped Caps.
+
+   Limits:
+   - The original incident was at load ~240 on the bare-metal host with real devices.
+     Guest load 7 on 4 vCPUs only partly simulates that starvation.
+   - 30 presses is a small sample.
+   - The guest's macOS build and the keyboard (the fixture) differ from the original.
+
+   The finding is that the mapping alone does not deterministically hold F18. Host
+   starvation remains the leading explanation but is not proven. Evidence:
+   `~/vm-lab-evidence/stuck-f18-20261022T1259Z/run3/` on the mini.
+
+   Getting there took two lab fixes:
+   - **A dropped keystroke** in Parallels' synthetic typing broke consent. The fix was
+     to qualify the fixture's current firmware for the physical-USB consent route
+     (malpern/vm-lab#5, live-accepted 37/37).
+   - **An unretried `PRL_ERR_IO_STOPPED`** on the guest exec channel ended earlier
+     attempts. The fix is malpern/vm-lab#6, which retries it on reads; that path was not
+     exercised in the passing run.
 
 ## Alternatives considered
 
@@ -219,9 +275,11 @@ selection step.
   the toggle problem entirely, but it is a driver-like design that the driverless runtime
   exists to avoid. Rejected.
 
-## Acceptance
+## Remaining qualification
 
-Before shipping: open questions 1–4 answered; the lease and journal version 2 unit
-tests (N entries, partial restore, version 1 read); a lab run that maps several keyboards,
-replugs one, crashes the worker and restarts, with every mapping restored; and the
-existing single-keyboard regression tests passing under the advanced option.
+The shipped fleet passed unit tests and an opt-in live test that acquired and
+restored mappings on three attached USB keyboard services. A physical unplug/replug
+test is still needed to measure the first-press window and check whether macOS
+reapplies Modifier Keys preferences after arrival. A physical Caps press on the
+cl80 in NKRO mode would close the multi-interface question. The stuck-F18 VM
+result above narrows the investigation but does not reproduce the host incident.