diff --git a/docs/board-data-contract.md b/docs/board-data-contract.md index 267cd625..d4adfc99 100644 --- a/docs/board-data-contract.md +++ b/docs/board-data-contract.md @@ -52,6 +52,14 @@ board's `/api/status` response. It summarizes PRs that need operator attention. the board's `/api/status` response. It reads opt-in metadata files from `.code-mower/board/agents/*.json` by default. +`code_mower.boardObservations.v1` is the local observation block embedded in the +board's `/api/status` response as `observations`. It carries validated +`code_mower.boardObservation.v1` records read from +`.code-mower/board/observations/*.json` by default, plus bounded diagnostics for +records the observation contract rejected and the file-level coverage of that +bounded read. The Board only consumes that contract; it never produces an +observation. + `code_mower.supervisedPilot.v1` is the local supervised-pilot payload embedded as `supervised_pilot` in the board's `/api/status` response when `code-mower.yml` is present. It is derived from the same controller policy @@ -75,7 +83,8 @@ and local diagnostics: When `restart_recommended` is true, stop and restart `code-mower board serve --repo OWNER/REPO` so the browser uses the newly installed package. The response also embeds `productivity` as - `code_mower.boardProductivity.v1`. The GitHub/local snapshot behind this + `code_mower.boardProductivity.v1` and `observations` as + `code_mower.boardObservations.v1`. The GitHub/local snapshot behind this response is served from a thread-safe stale-while-refresh cache bounded by `board.refresh_seconds`: a cold request returns a metadata-only warming payload immediately and starts one background refresh, a @@ -443,6 +452,349 @@ add no fields to any schema; they only bound what the page is allowed to assert. rows) are absent, GitHub information stays useful and the page names the local data that is unavailable instead of rendering it as zero. +## Board Views + +The browser view is organized as four tabs over one payload. Nothing below +changes the payload; the views only bound what the page asserts and where it +says it. + +Persistent chrome — repository, serving and installed version, snapshot time, +the one next action, and observation freshness — stays on screen in every view +and at every width. The tabs are a real `tablist` of `tab` buttons controlling +real `tabpanel` regions; the unselected panels carry `hidden`, so they leave the +accessibility tree instead of being painted away. + +- **Now** — the work rows, the selected work item's evidence, the participant + summary, and the existing owner queue, lane work, supervised pilot and open + PR sections. +- **Timeline** — meaningful recent changes, local Board history, the reviewer + verdict timeline, and recent Code Mower workflow runs. +- **Releases** — release campaigns, productivity, and spend. Completed campaign + history lives here rather than in front of current work. +- **Health** — observation sources and their freshness, Board version and + restart state, snapshot cache state, GitHub availability, gate alerts, the + orchestrator lease, agent cards, and local Board and lane processes. + +### Work rows and selected-work detail + +Each work row shows the safe `reference` the observation records, the stage, +the assignments recorded for it, the last meaningful update, the recorded next +action, and the responsible role. Selecting a row exposes the six independent +evidence readings — builder runs, review, CI, gate, merge, and human policy — +each naming the source it came from and how fresh that source is, so no reading +can stand in for another. The gate publisher is shown beside the +`code-mower/gate` verdict and is labelled as publisher execution only. + +Rows are ordered by urgency, which is a separate question from which recorded +truth headlines a row. A merged item headlines as merged because that describes +it best, but it is the least urgent thing on the board, so row order runs +blocked work first, then work waiting on a named person, then work whose +evidence cannot be trusted, then work recorded as in flight, and terminal work +— merged, and an idle session — last by explicit placement. A headline that is +not ranked sorts after everything ranked and before the terminal band. The +reference and then the opaque identity break ties, so an unchanged snapshot +never reshuffles the list. Because the first row is what an operator who has +chosen nothing is shown, the Board opens on work that still needs someone +rather than on work that is finished. + +Every reason the frozen observation contract accepts has its own display state +and its own explicit place in that ranking, so no supported reason falls +through to "state not recorded" or sorts as something nobody ranked. The +blocked band carries `source_unavailable`, `changes_requested`, +`update_required`, `ci_failed`, `gate_failed`, `provider_failed`, +`provider_suspended` and `cancelled`; the waiting-on-a-person band carries +`approval_required`, `user_input_required`, `ready_to_merge`, +`human_review_required` and `review_requested`; `stale_observation`, +`review_stale` and `identity_unlinked` are evidence that cannot be trusted; and +`review_in_progress`, `ci_pending` and `gate_pending` are recorded as in +flight. A suspended session is reported as suspended rather than as a failure: +the contract records suspension as the `suspended` lifecycle state and allows +it only alongside the `failed` phase, so the failure state is read from runs +that are not suspended and neither claim is ever made on the other's evidence. + +One run has one lifecycle-aware state, and every display that names, colours, +groups, counts, or summarizes that run reads it from one place — the row +headline and its state cues, the assignments line, the selected-work evidence +panel, and the participant summary all agree by construction. A lifecycle state +overrides the recorded phase only where it means something the phase cannot +say, which in this contract is `suspended` alone: a suspended run reads as +suspended everywhere, a cancelled run stays distinct from a failed one, an +actual lifecycle failure still reads as failed, and complete, implementation +complete, and running keep reading as themselves. The recorded phase survives +only as raw contract evidence in the change signature, where it is compared +alongside the recorded lifecycle state so a run that moves between the two is +still detected as a change. + +Selection is kept by opaque work identity — session, worktree, and work id — +not by row position, so a refresh that reorders, adds, or drops rows leaves the +operator's choice where it was. One identity is one row: where the directory +holds several observations of the same work item, the newest by recorded +`created_at` is rendered — with the last meaningful update and then the row +signature as deterministic tiebreaks — so which file the directory listed first +cannot change what is shown. Change tracking compares the same deduplicated +set, so there is exactly one row id and one detail region per identity however +many files describe it. Several `unlinked` observations in one scope still +consolidate into one row, and that row is recomputed from all of the evidence +retained for it: the worst freshness and coverage any retained source reported, +the age of the oldest retained observation, the newest recorded event or +observation as the last meaningful update, and one entry per run however many +files observed it, kept as the worst-attested of those observations. The +participant summary is built from this same deduplicated set, so a run that has +moved on is counted once, under the lifecycle-aware state the newest +observation records, and never again under the state it has left. + +Deduplication only ever compares like with like, so it settles which +observation of one identity is current and says nothing about two identities +that disagree. A session-level `no_work` snapshot and a work-specific +observation of the same session and worktree are different identities, so both +survive it — and left there the Board would state, of one session at once, that +it was observed complete with nothing to do and that it is running work. A +single reconciliation step runs immediately after deduplication and before any +view reads a row, and keeps the newer of the two readings by the same trusted +recorded order — `created_at`, then the last meaningful update — never by file +or directory order. An idle snapshot followed by work observations is stale and +is dropped, and every work item observed after it survives, however many there +are. Work observations followed by an idle snapshot are the session having +since gone quiet: the idle snapshot is the truthful current state and those +work rows are dropped rather than restated as current work, which holds for +terminal work too — an item observed as merged before its session reported +itself idle is not current work either. Nothing is invented to stand in for a +dropped row; what is already recorded is that the row is no longer recorded, +which change tracking reports in the Timeline on the poll that drops it. Two +observations recording exactly the same instants are resolved by specificity, +the work-specific one winning, because claiming "nothing to do in this session" +over a work item observed at the same instant is the contradiction the step +exists to remove. Records in different sessions or different worktrees are +never compared — one session holds several worktrees and one worktree is reused +by session after session — and a record carrying neither half of a session +identity, which is every `unlinked` observation, is never correlated with one +that does and keeps the unlinked consolidation semantics above. The Health view +still reads every record on disk on purpose: a source behind a superseded +observation was really contacted, and its connection is inspected there on its +own terms rather than as a claim about work. + +There is exactly one detail region. It is +rendered inside the selected row, so at phone widths it follows the row it +belongs to, and at desktop widths CSS places that same region in a second +column of the row's own grid. It stays in normal flow at both widths, so the +row — and therefore the list and the section — is always at least as tall as +the detail it is showing, and a list of one or two rows can never leave the +detail hanging over the sections below it. Rows are buttons carrying +`aria-expanded` and `aria-controls`; Up, Down, +Home and End move the selection, tabs wrap with the arrow keys, and every +interactive control has a visible focus ring. + +A poll replaces the tab strip and the row list, so every control that can hold +the keyboard carries an identity derived from what it acts on rather than from +where it was rendered: the tabs from the view, the rows from the work identity, +and the selected row's actions from that identity and the action's own name. +The element id is an injective encoding of the opaque identity — a letter, +digit or hyphen stands for itself and every other code unit becomes `__` — +so identities that differ only in punctuation, such as unlinked work in +`owner/re.po` and in `owner/re-po`, keep distinct ids, distinct +`aria-labelledby` targets and distinct focus lookups. A +refresh therefore returns the keyboard to the same control — restoring without +scrolling, because a refresh must not move the view. When a control is no +longer offered, focus moves only to the row that control named as its owner, +and if that row is gone too the Board leaves focus where the browser put it +rather than handing the keyboard to an unrelated control. Activating a detail +action that opens another view moves focus to that view's tab, because the +control that was activated is inside the panel the switch has just hidden. + +The detail region scrolls independently of the row list at desktop widths, and +a poll replaces it along with everything else, so the offset an operator +scrolled to is preserved across the refresh and restored afterwards. It is kept +against the same opaque work identity the selection is kept against: a refresh +that changed the evidence of the work item being read returns to the same +position, a different work item opens at the top of its own evidence rather +than inheriting someone else's position, and a selection that stops being +rendered has nothing to restore onto. Restoring is clamped to what the +replacement can actually scroll, so a refresh that shortens the evidence lands +at the end of what is now there. Focus and the offset are carried across one +refresh together, the offset restored last, so neither undoes the other. + +Meaningful changes are announced once through a polite live region and listed +in Timeline. A change is meaningful when a recorded fact differs: stage, +reasons, route, pull request identity, evidence state, measurements, run phase +or basis, or a source's freshness, coverage or event time. `created_at`, +`checked_at`, `observed_at` and `heartbeat_at` are excluded because they advance +on every successful poll, so a poll that repeats the same observation announces +nothing and adds no Timeline entry. + +Primary actions stay read-only: open a recorded PR link, inspect a connection +in Health, and view recent changes. A pull request number observed locally is +never turned into a remote address the payload has not recorded, and a link is +offered only when the record names this Board's own repository. A custom +observations directory can hold a record another repository produced, where the +same pull request number means a different pull request; such a record is still +shown for what it is, named as belonging to that other repository, without a +link. There is no +merge, requeue, force-lease, cancel, retry, restart, cloud-schema or +Slack-specific control, no form, and no non-GET request. + +### Observation presentation rules + +- **Liveness is read, never inferred.** A run is only described in the phase its + own record states, alongside the freshness of the source behind it. A record + whose sources are not all `fresh`, or that is more than ten minutes old, is + reported as a last observation and may not claim anything is running now. +- **Unavailable is not zero, and idle is not unknown.** A `no_work` observation + is shown as idle *with complete coverage* and names the source kinds that were + observed fresh and complete. An `unlinked` observation claims no stage and no + route, because the contract records none for it. +- **No invented totals.** An unavailable measurement renders `not recorded`. A + partial measurement renders the value with the evidence it was counted from + (`120.0s from 2 of 5 recorded`). Nothing is extrapolated to a whole, and no + ratio is turned into a percentage or an ETA. +- **Unknown is neutral.** Evidence states are a closed vocabulary and each one + is classified explicitly; `unknown`, `not_started`, `absent`, `unverifiable`, + `none` and `unassigned` are neutral, and anything unrecognised is neutral too. + Colour always accompanies a text label and a text cue. +- **An empty directory is not an empty queue.** With no observation recorded, + the work list says so rather than reporting no work, and the GitHub-derived + queues below it still render. +- **A bounded read is not the whole record set.** The Board reads at most 32 + observation files per refresh. When more candidates exist, the work list + warns above its rows that the snapshot is incomplete, the Now header and the + chrome carry that warning into every view, and no row may claim complete + coverage, an idle session, or that there is no work. An idle `no_work` + snapshot reads as *idle in the files read* rather than *idle with complete + coverage*, because an unread file could record work in exactly that scope. +- **A lost candidate is not a candidate that said nothing.** The same applies + to a selected file the Board could not read and to one the record contract + rejected: the Board cannot know whether it held the work record that + contradicts a `no_work` record beside it. Any loss makes file coverage + partial, warns on every surface, and reads as *idle in the records read* + rather than *idle with complete coverage*. Reconciliation obeys the same + rule: retiring an observed work row asserts that its session has since gone + quiet, so an idle snapshot read under incomplete coverage never retires work + — both readings stay on the page. + +## Board Observations + +`code_mower.boardObservations.v1` is the local observation block embedded in the +board's `/api/status` response as `observations`. It is a **consumer** of the +frozen `code_mower.boardObservation.v1` contract: the Board reads records, +decodes each through `board_observation.decode`, and renders what survives. The +Board never writes an observation, resolves a session, contacts a provider, or +repairs a record that fails the contract. + +By default the Board reads `*.json` files under +`.code-mower/board/observations/`. Use `--observations-path PATH` for a custom +local directory. A missing directory is reported as "nothing recorded yet", +which is a different statement from "no work". Reading is bounded to 32 files +per refresh, and the observation contract itself bounds each record to +`MAX_BYTES`. Each file is read with a single bounded request of at most +`MAX_BYTES + 1` bytes: a file larger than the cap is rejected on the length of +what was asked for, with the contract's own `invalid_contract` diagnostic, and +its remainder is never loaded or decoded. + +A directory holding more than 32 `*.json` files is **not** silently reduced to +whichever 32 happened to be reached. Every candidate is counted, the bounded +subset is chosen deterministically, and the shortfall is reported as file-level +coverage so no consumer can read a truncated snapshot as the whole local record +set. Selection is a total order over *(modification time descending, file name +ascending)*, which is deterministic whatever order the filesystem lists entries +in and prefers the most recently written files, so a current record is not +starved by an alphabetically earlier stale one. The frozen record contract +guarantees nothing about file names or file times, so that preference is a +conservative best effort and never evidence: an overflowing directory is +reported as incomplete however it was selected, and a file whose time cannot be +read simply loses the preference. Emission order stays file-name order, so a +directory inside the cap reads exactly as it did before. Counting the candidate +set never widens the read — at most 32 files are opened, each with one bounded +`MAX_BYTES + 1` request. + +Only **regular files** are ever opened. Every `*.json` entry is classified by one +`lstat` that never opens anything, and a selected entry that is not a regular +file — a named pipe, a socket, a directory, a symlink, or an entry whose own +metadata could not be read — is refused unread and counted as `unreadable`. +Symlinks are deliberately not followed, even to a regular file. The reason is +that the file and byte bounds only start applying once the file is open: reading +a named pipe with no writer blocks until a writer arrives, and a refresh that +blocks there never finishes, so the page keeps serving the snapshot before it. +Refusing an entry is not ignoring it — it stays a counted candidate that +produced no record, so the read is `partial` exactly as it is for a file that +raised. A candidate can also change between that classification and the open; a +lost race raises on the open or the read and lands in the same `unreadable` +count. + +The block carries: + +- `records[]` — validated `code_mower.boardObservation.v1` records, in file-name + order. +- `coverage`, `coverage_complete`, `coverage_gaps[]`, `truncated`, `file_cap`, + `selection` and the candidate counters below — how much of the candidate file + set those records were built from. `coverage` is a closed vocabulary: + `complete` when every candidate was read *and* produced an accepted record, + `partial` when any candidate was lost, and `unavailable` when the directory + could not be listed at all (where `candidate_files`, `omitted_files` and + `unaccounted_files` are `null` rather than an invented total). + `coverage_complete` is the same fact as a boolean and is what every consumer + gates an absence claim on; `coverage_gaps[]` says which kinds of loss + occurred, from the fixed vocabulary `directory_unreadable`, `files_omitted`, + `files_unreadable`, `records_invalid`. This is file coverage and is + deliberately separate from a record's own source `coverage`: a rejected + record and an unread file are different facts, and both are reported. +- The candidate counters partition the candidate set exactly, and the + partitions are the accounting invariants: + - `candidate_files` = `selected_files` + `omitted_files` — every `*.json` + candidate was either selected by the bounded read or omitted by the cap. + - `attempted_files` = `selected_files` — every selected candidate is one the + read is answerable for, whether it was opened or refused before any open. + - `attempted_files` = `read_files` + `unreadable_files` — an attempted file + either yielded its bytes, or raised on open or on read, or was refused for + not being a regular file. + - `read_files` = `accepted_records` + `invalid_records` — a file that was + read either decoded into a record or was rejected by the frozen record + contract (including for exceeding `MAX_BYTES`). + - `accepted_records` = `len(records)`. + - `unaccounted_files` = `unreadable_files` + `invalid_records` — every + selected candidate the records do not account for. + - `coverage_complete` is true only when `omitted_files` and + `unaccounted_files` are both zero. +- `rejected` and `warnings[]` — `rejected` is `unaccounted_files`: every + selected candidate that produced no record, deliberately including unreadable + ones, because a file that could not be read is no more accounted for than one + the contract refused. Each has one warning carrying a fixed diagnostic: + `unreadable_file` for a file that could not be read — including one refused + for not being a regular file, which is stated as the same fact and never as + what kind of entry it was — and the contract's own + vocabulary (`invalid_contract`, `invalid_route`, `identity_mismatch`, and so + on) for a record it rejected. Those diagnostics carry no errno, no OS message, + no local path and no byte of file content; the `file` field carries the bare + candidate name inside the Board's own observations directory and nothing else, + and the page renders the diagnostics as counts per term without it. +- `path`, redacted as `[local path hidden]`, `path_state`, `path_exists`, + `available`, and a safe `message`. `path_state` is a closed vocabulary — + `directory`, `missing`, `not_directory`, `unreadable` — decided by one + metadata call per refresh that cannot raise, and `path_exists` is the same + fact as the boolean consumers already read: `true` for `directory` and + `not_directory`, `false` for `missing`, and `null` for `unreadable`, because + "it is not there" is a claim a path the Board could not examine cannot + support. Only `missing` is an absence the Board is entitled to state as + "nothing recorded yet"; `not_directory` and `unreadable` are losses of + evidence and degrade the block to `available: false` with `unavailable` + coverage and the `directory_unreadable` gap, each with its own fixed + diagnostic (`observation path is not a directory`, `could not check the local + Board observation path`, and `could not list local Board observations` for a + directory lost between that check and the enumeration). None of them names a + path, an errno or an OS message. + +Nothing in this block may raise on the filesystem. It is assembled as one step +of the whole `/api/status` snapshot, so an exception at this boundary would not +produce an unavailable observations block — it would abort the refresh and take +the repository, PR and lane data with it, leaving the page on whatever it served +before. An observation directory under an ancestor the process cannot search is +the ordinary way that happens. Observations are therefore the only thing that +degrades: every other block is built exactly as it would have been. + +Observations are not copied into the local Board event store: `code-mower board +record` and `--record-events` persist the snapshot without the `observations` +block, so local history keeps the shape it already had. The cloud board-snapshot +export is an allowlist of summarized fields and is unchanged by this block; no +observation field is uploaded. + ## Agent Adapters The Board embeds `code_mower.boardAgentAdapters.v1` in `/api/status`. Agent diff --git a/docs/launch-command-surface.md b/docs/launch-command-surface.md index 5c7be859..6bcdcf7d 100644 --- a/docs/launch-command-surface.md +++ b/docs/launch-command-surface.md @@ -47,6 +47,7 @@ Select any additional builder or reviewer explicitly. | `code-mower productivity report --repo OWNER/REPO --cloud-event PATH --json` | Include metadata-only `productivity_summary` aggregate event files and print the stable report JSON. | no | no | | `code-mower board serve --repo OWNER/REPO --record-events` | Serve the board and append throttled metadata-only local history snapshots while it is open. | yes, local only | GitHub optional | | `code-mower board serve --repo OWNER/REPO --agent-adapters-path PATH` | Read opt-in local agent cards from a custom metadata-only adapter directory. | no | no | +| `code-mower board serve --repo OWNER/REPO --observations-path PATH` | Render local `code_mower.boardObservation.v1` records from a custom read-only directory. The Board consumes that contract and never writes one. | no | no | | `code-mower board list` | List local Board listeners with repo/version, restart hints, ports, and redacted cwd paths by default. | no | no | | `code-mower board stop --port PORT --yes` | Stop a local Board listener that the inventory identified as Code Mower. | local process signal | no | | `code-mower board record --repo OWNER/REPO` | Append one redacted status snapshot to `.code-mower/board/events.jsonl` for local board history. | yes, local only | GitHub optional | diff --git a/src/code_mower/board.py b/src/code_mower/board.py index bbb3eb3c..2881b778 100644 --- a/src/code_mower/board.py +++ b/src/code_mower/board.py @@ -6,11 +6,13 @@ import argparse import copy import errno +import heapq import json import os import re import signal import socket +import stat import sys import time import urllib.error @@ -28,6 +30,7 @@ from urllib.parse import urlparse from . import __version__ as CODE_MOWER_VERSION +from . import board_observation from . import board_store from . import config as code_mower_config from . import controller @@ -47,8 +50,78 @@ BOARD_INVENTORY_SCHEMA = "code_mower.boardInventory.v1" BOARD_STOP_SCHEMA = "code_mower.boardStop.v1" BOARD_RELEASE_CAMPAIGNS_SCHEMA = "code_mower.boardReleaseCampaigns.v1" +BOARD_OBSERVATIONS_SCHEMA = "code_mower.boardObservations.v1" DEFAULT_AGENT_ADAPTERS_RELATIVE_PATH = Path(".code-mower") / "board" / "agents" +DEFAULT_OBSERVATIONS_RELATIVE_PATH = Path(".code-mower") / "board" / "observations" DEFAULT_CAMPAIGNS_RELATIVE_PATH = Path(".code-mower") / "campaigns" +# Bounded so a directory left full of records cannot turn one page load into an +# unbounded read. The contract itself bounds each record to MAX_BYTES. +MAX_OBSERVATION_FILES = 32 +# Which bounded subset of a larger directory is read, named as a fixed token so +# a consumer can state the policy without restating it. The frozen record +# contract guarantees nothing about file names or file times, so modification +# time is used only as a best-effort recency preference -- never as evidence -- +# and a directory that overflows the cap is always reported as incomplete. +OBSERVATION_SELECTION = "newest_modified_then_name" +# Why a bounded read may not be read as the whole local record set. This is a +# fixed closed vocabulary so a consumer can state which kind of gap it has +# without a file name, a local path, an errno or any record content reaching +# the page. `files_omitted` is the cap leaving candidates unread, +# `files_unreadable` is a selected candidate that could not be read at all +# (one that raised, or one refused before any open for not being a regular +# file; both are stated as that same fact, never as what kind of entry it was), +# `records_invalid` is a candidate the frozen record contract rejected, and +# `directory_unreadable` is a directory that could not be listed. +OBSERVATION_COVERAGE_GAPS = ( + "directory_unreadable", + "files_omitted", + "files_unreadable", + "records_invalid", +) +# What the observation path turned out to be, as a closed vocabulary decided in +# exactly one place (`_classify_observation_path`). `directory` is the only +# state a read proceeds from. `missing` is a name that does not resolve, which +# is "nothing recorded yet" and not a loss of evidence. `not_directory` is a +# name that resolves to something else, and `unreadable` is a name whose own +# metadata could not be read at all -- an inaccessible ancestor, for instance. +# The last two are losses: the Board cannot see what is there, so observations +# degrade to unavailable rather than being reported as absent. +OBSERVATION_PATH_STATES = ("directory", "missing", "not_directory", "unreadable") +# The errnos that mean the *name* did not resolve, as opposed to the lookup +# having failed. They are exactly the set `pathlib` itself treats as "does not +# exist" when answering `Path.exists()`, which is what this classification +# replaces: a path that reported one of these before still reports `missing`, +# and only the errnos that previously escaped as an exception become +# `unreadable`. Looked up rather than named so an interpreter missing one still +# imports. +OBSERVATION_PATH_MISSING_ERRNOS = frozenset( + getattr(errno, name) for name in ("ENOENT", "ENOTDIR", "EBADF", "ELOOP") if hasattr(errno, name) +) +# The Windows equivalents `pathlib` ignores for the same reason: not ready, +# invalid name, and a name that cannot be resolved. Plain integers because +# these have no portable `errno` names. +OBSERVATION_PATH_MISSING_WINERRORS = frozenset((21, 123, 1921)) +# The guards that make opening an observation candidate safe even though the +# entry can change kind between being classified and being opened. They are +# named here rather than inlined so a test can assert they are still in force. +# +# `O_NONBLOCK` is the one that cannot be recovered from after the fact: opening +# a named pipe for blocking read with no writer does not fail, it waits, and a +# refresh that waits there never returns. `O_NOFOLLOW` refuses a final +# component that became a symlink, so a link the Board never chose can never be +# resolved for it. Both are POSIX and are present on every platform the Board +# supports; the descriptor-type check in `_open_observation_file` is what makes +# the open safe, and it does not depend on either flag being available. +OBSERVATION_OPEN_GUARDS = ("O_NONBLOCK", "O_NOFOLLOW") +# `O_CLOEXEC` (`O_NOINHERIT` on Windows) keeps the descriptor out of any child +# process, and `O_BINARY` is the Windows flag for untranslated bytes and does +# not exist elsewhere. Every flag is looked up rather than named so an +# interpreter missing one still imports; a missing guard weakens nothing that +# `fstat` does not re-check on the descriptor actually opened. +OBSERVATION_OPEN_FLAGS = os.O_RDONLY +for _flag_name in (*OBSERVATION_OPEN_GUARDS, "O_CLOEXEC", "O_NOINHERIT", "O_BINARY"): + OBSERVATION_OPEN_FLAGS |= getattr(os, _flag_name, 0) +del _flag_name SECRET_VALUE_RE = re.compile( r"(github_pat_[A-Za-z0-9_]+|gh[pousr]_[A-Za-z0-9_]{20,}|sk-[A-Za-z0-9_-]{20,}|xox[baprs]-[A-Za-z0-9-]{20,})" ) @@ -69,6 +142,7 @@ class BoardConfig: store_path: str | None = None spend_path: str | None = None agent_adapters_path: str | None = None + observations_path: str | None = None campaigns_path: str | None = None event_limit: int = 20 record_events: bool = False @@ -95,6 +169,12 @@ def _agent_adapters_path(config: BoardConfig) -> Path: return Path(config.repo_path) / DEFAULT_AGENT_ADAPTERS_RELATIVE_PATH +def _observations_path(config: BoardConfig) -> Path: + if config.observations_path: + return Path(config.observations_path) + return Path(config.repo_path) / DEFAULT_OBSERVATIONS_RELATIVE_PATH + + def _campaigns_path(config: BoardConfig) -> Path: if config.campaigns_path: return Path(config.campaigns_path) @@ -113,6 +193,7 @@ def resolved_metadata_paths(config: BoardConfig) -> dict[str, str]: "store_path": str(_store_path(config)), "spend_path": str(_spend_path(config)), "agent_adapters_path": str(_agent_adapters_path(config)), + "observations_path": str(_observations_path(config)), "campaigns_path": str(_campaigns_path(config)), } @@ -262,6 +343,7 @@ def status_payload( } payload["orchestrator_lease"] = session_lease.observe_lease(start=config.repo_path) payload["agent_adapters"] = agent_adapters_payload(config) + payload["observations"] = observations_payload(config) payload["release_campaigns"] = release_campaigns_payload(config) payload["owner_queue"] = owner_queue_payload(payload) payload["supervised_pilot"] = supervised_pilot_payload( @@ -302,6 +384,10 @@ def _recordable_payload(payload: dict[str, Any]) -> dict[str, Any]: snapshot = dict(payload) snapshot.pop("productivity", None) snapshot.pop("orchestrator_lease", None) + # Observations are a live read model carrying local session and worktree + # identity. The Board renders them; it does not copy them into persisted + # local history, so replayed history keeps the shape it already had. + snapshot.pop("observations", None) return snapshot @@ -1157,6 +1243,550 @@ def prune_stale_agent_adapters( return result +def _classify_observation_path(path: Path) -> str: + """Decide what the observation path is, in one call that cannot raise. + + Every preflight question the read asks about the directory -- does it + exist, is it a directory -- is answered here from a single ``os.stat``, and + the answer is one token from ``OBSERVATION_PATH_STATES``. Centralizing it is + the point: ``Path.exists()`` and ``Path.is_dir()`` swallow only the errnos + that mean the name did not resolve and re-raise everything else, so an + observation directory under an ancestor the process cannot search raises + ``PermissionError`` out of what reads like a boolean. Asked from + :func:`observations_payload`, which is called while the whole Board snapshot + is being assembled, that aborts the entire refresh -- repository, PRs, lanes + and all -- over one local directory the Board only ever reads. No metadata + call may escape this function, so there is nowhere else for that to happen. + + The three failure states are kept distinct because they are different facts + about the evidence. ``missing`` is a name that does not resolve: nothing was + recorded, which the Board is entitled to say. ``not_directory`` and + ``unreadable`` are losses -- something may well be there and the Board + cannot see it -- so they degrade observations to unavailable rather than + reporting an absence. The split between ``missing`` and ``unreadable`` + deliberately reuses the errnos ``pathlib`` treats as non-existence, so a + path that answered "no" before answers ``missing`` now and only the errnos + that previously escaped become ``unreadable``. + + A single ``stat`` also bounds the syscalls: one metadata call per refresh, + with no window between "exists" and "is a directory" for the entry to change + kind inside. Nothing here is a promise about what the enumeration that + follows will find -- the directory can still be removed or replaced after + this returns, which :func:`observations_payload` accounts for as a gap. + + Symlinks are followed, exactly as ``Path.exists()`` and ``Path.is_dir()`` + did: a link to the observations directory is the directory. The entries + *inside* it are the ones never resolved through a link, which is decided per + entry in :func:`_select_observation_files` and re-decided on the descriptor. + """ + + try: + status = os.stat(path) + except ValueError: + # An unencodable path -- an embedded null byte, say -- names nothing on + # any filesystem. `Path.exists()` answered False for it, so it stays + # "nothing recorded" rather than becoming a new kind of failure. + return "missing" + except OSError as exc: + if ( + getattr(exc, "errno", None) in OBSERVATION_PATH_MISSING_ERRNOS + or getattr(exc, "winerror", None) in OBSERVATION_PATH_MISSING_WINERRORS + ): + return "missing" + # PermissionError and everything else: the Board could not look, which + # is never the same answer as nothing being there. + return "unreadable" + return "directory" if stat.S_ISDIR(status.st_mode) else "not_directory" + + +def _select_observation_files(path: Path) -> tuple[list[tuple[str, bool]], int]: + """Choose the bounded file set to read, and count every candidate. + + The whole candidate set is counted so truncation can be reported, but only + ``MAX_OBSERVATION_FILES`` entries are ever retained, so a directory left + full of records costs one bounded selection rather than an unbounded list. + + Selection is a total order over ``(modification time descending, file name + ascending)``: it is therefore deterministic whatever order the filesystem + hands entries back in, and it prefers the most recently written files so a + newer record is not starved by an alphabetically earlier one. The frozen + record contract guarantees nothing about file names or file times, so that + preference is a conservative best effort and never an assertion -- a file + whose time cannot be read sorts last by name, and an overflowing directory + is reported as incomplete however it was selected. Emission order stays + file-name order, unchanged from a directory that fits inside the cap. + + Each selected candidate is returned with whether it is a *regular* file, + decided here by one ``lstat`` that never opens anything and never blocks. + The caller opens only the entries this classified regular, because opening + the others can cost far more than a bounded read: a read on a named pipe + with no writer blocks until one arrives, which is long after the byte bound + would have applied and long enough to wedge a refresh. Symlinks are + deliberately not followed, even to a regular file -- resolving one would + reintroduce the same hazard through a link whose target the Board never + chose, and whose target can change between this classification and the + open. Nothing here is dropped for being non-regular: it stays a counted + candidate, and the read accounts for it as one that produced no record. + + This classification is a filter, not a guarantee. It reads a name, and the + entry that name refers to can change kind before the open that follows, so + nothing downstream may treat "classified regular" as "is regular": + :func:`_open_observation_file` re-decides it on the descriptor it actually + opened. What this filter buys is that the common non-regular entry is never + opened at all, and that the open which does happen has something safe to + re-check. + """ + + total = 0 + + def candidates() -> Any: + nonlocal total + with os.scandir(path) as entries: + for entry in entries: + if not entry.name.endswith(".json"): + continue + try: + status = entry.stat(follow_symlinks=False) + except OSError: + # Unreadable metadata is not evidence of anything; the file + # stays a candidate and simply loses the recency + # preference. It is not known to be a regular file either, + # so it is never opened -- an entry the Board cannot + # classify is accounted for as unread rather than risked. + modified, regular = 0, False + else: + modified = status.st_mtime_ns + regular = stat.S_ISREG(status.st_mode) + total += 1 + yield (-modified, entry.name, regular) + + selected = heapq.nsmallest(MAX_OBSERVATION_FILES, candidates()) + return sorted((name, regular) for _key, name, regular in selected), total + + +class _ObservationNotRegular(OSError): + """An opened observation candidate turned out not to be a regular file. + + An ``OSError`` on purpose: refusing the descriptor is the same outcome for + the caller as failing to open it, and the caller has exactly one place that + turns either into the one ``unreadable_file`` accounting. + """ + + +def _open_observation_file(record_file: Path) -> Any: + """Open one classified candidate, refusing anything but a regular file. + + The ``lstat`` that classified this name is not a promise about the entry + the open resolves: the name can be replaced between the two, and replacing + a regular file with a named pipe is enough to wedge a refresh forever if + the open blocks. So the open itself carries the guards + (``OBSERVATION_OPEN_FLAGS``) rather than relying on the earlier look -- + ``O_NONBLOCK`` so opening a pipe with no writer returns instead of waiting, + ``O_NOFOLLOW`` so a name that became a symlink is refused rather than + resolved -- and then ``fstat`` re-decides the kind on the descriptor that + is actually open, which is the one question no later change can race. + + ``O_NONBLOCK`` stays set for the read. It has no effect on a regular file + on any supported platform, and by the time a read happens ``fstat`` has + already established that this descriptor is one; a descriptor that is + anything else never reaches a read at all. + + Returns an open binary handle that owns the descriptor. Raises ``OSError`` + -- including :class:`_ObservationNotRegular` -- if the entry cannot be + opened or is not a regular file, and never leaves a descriptor open on any + failing path, including a failure inside ``fdopen`` itself. + """ + + descriptor = os.open(record_file, OBSERVATION_OPEN_FLAGS) + try: + status = os.fstat(descriptor) + if not stat.S_ISREG(status.st_mode): + raise _ObservationNotRegular( + errno.EINVAL, "observation candidate is not a regular file" + ) + handle = os.fdopen(descriptor, "rb") + except BaseException: + # Nothing has taken the descriptor yet, so this is the only owner and + # closes it exactly once -- on a refusal, on an `fstat` that raised, + # and on an `fdopen` that failed after consuming nothing. + os.close(descriptor) + raise + # Ownership has transferred: `handle.close()` is now the one close. + return handle + + +@dataclass(frozen=True) +class _ObservationAccounting: + """Every candidate observation file one read saw, and what became of each. + + Exactly one outcome is recorded per candidate, so these counts partition + the candidate set rather than describing it loosely: every candidate is + either omitted by the file cap or attempted; every attempted file is either + read or unreadable -- including an entry refused for not being a regular + file, which is unreadable by decision rather than by error; and every file + that was read either produced an accepted record or was rejected by the + frozen record contract. + + Coverage is whole only when none of those partitions lost anything -- no + omission, no unreadable file, and no rejected record. A dropped candidate + is not a candidate that said nothing: the Board cannot know whether the + file it failed to read or decode held a work record contradicting a + ``no_work`` record beside it, so every kind of loss makes the coverage + partial and withdraws the authority to claim a current idle session or to + retire observed work. This is the single accounting every consumer of + coverage completeness reads; no surface recomputes it from a record list. + """ + + candidate_files: int + selected_files: int + unreadable_files: int + invalid_records: int + records: tuple[dict[str, Any], ...] + warnings: tuple[dict[str, str], ...] + + @property + def omitted_files(self) -> int: + """Candidates the file cap never selected.""" + + return self.candidate_files - self.selected_files + + @property + def attempted_files(self) -> int: + """Selected candidates the read had to account for. + + Every selected candidate is attempted in the sense that matters to + coverage -- the read either got its bytes or it did not. A regular file + is opened; an entry that is not a regular file is refused before any + open, because opening one can block instead of failing. Both outcomes + are counted below, so refusing one never removes it from the set the + read is answerable for. + """ + + return self.selected_files + + @property + def read_files(self) -> int: + """Attempted candidates whose bytes were successfully read.""" + + return self.attempted_files - self.unreadable_files + + @property + def accepted_records(self) -> int: + return len(self.records) + + @property + def unaccounted_files(self) -> int: + """Selected candidates that produced no record, for any reason.""" + + return self.unreadable_files + self.invalid_records + + @property + def rejected(self) -> int: + """Every selected candidate the records below do not account for. + + Kept under its original name because consumers already read it, and + deliberately inclusive of unreadable files: a file that could not be + read is no more accounted for than one the contract refused. + """ + + return self.unaccounted_files + + @property + def gaps(self) -> list[str]: + """Which kinds of loss this read had, from ``OBSERVATION_COVERAGE_GAPS``.""" + + gaps = [] + if self.omitted_files: + gaps.append("files_omitted") + if self.unreadable_files: + gaps.append("files_unreadable") + if self.invalid_records: + gaps.append("records_invalid") + return gaps + + @property + def complete(self) -> bool: + return not self.gaps + + def payload(self) -> dict[str, Any]: + """The file-level coverage block, counts only -- no name, path or errno.""" + + return { + "records": list(self.records), + "warnings": list(self.warnings), + "rejected": self.rejected, + "coverage": "complete" if self.complete else "partial", + "coverage_complete": self.complete, + "coverage_gaps": self.gaps, + "truncated": self.omitted_files > 0, + "candidate_files": self.candidate_files, + "selected_files": self.selected_files, + "omitted_files": self.omitted_files, + "attempted_files": self.attempted_files, + "read_files": self.read_files, + "accepted_records": self.accepted_records, + "invalid_records": self.invalid_records, + "unreadable_files": self.unreadable_files, + "unaccounted_files": self.unaccounted_files, + } + + +def _read_observation_records( + path: Path, + selected: list[tuple[str, bool]], + candidate_count: int, +) -> _ObservationAccounting: + """Read the selected candidates and account for every one of them. + + A file that raises on open or on read is counted as unreadable rather than + skipped: it was selected for coverage and produced nothing, which is a hole + in the evidence and not an absence of work. A file the contract rejects -- + for being oversize or for failing to decode -- is counted separately, so + the two are distinguishable without either being lost. + + A selected candidate that is not a regular file is refused before any open + and counted the same way, because the cost of opening one is not bounded by + anything the contract controls: a named pipe with no writer blocks the read + rather than failing it. It is refused, not dropped -- the Board cannot know + what the entry it declined to open would have said, so it is one more + candidate that produced no record and one more reason this read is partial. + + An entry that changes kind between that classification and this open lands + in the same count, but not because an ordinary open would have raised on it + -- it would not. Opening a pipe for blocking read with no writer waits, so + :func:`_open_observation_file` opens non-blocking and without following a + link, then re-decides the kind on the descriptor itself and refuses + anything that is not a regular file. Every one of those refusals, and every + genuine open or read error, is caught in one place below and accounted for + as the same single ``unreadable_file`` outcome. + """ + + records: list[dict[str, Any]] = [] + warnings: list[dict[str, str]] = [] + unreadable = 0 + invalid = 0 + for name, regular in selected: + record_file = path / name + if not regular: + # Never opened, and counted exactly as a file that failed to open: + # the same closed diagnostic, which says a candidate produced no + # record without saying what kind of entry it turned out to be. + unreadable += 1 + warnings.append({"file": name, "message": "unreadable_file"}) + continue + # The contract bounds a record to MAX_BYTES, so at most one byte past + # that bound is ever read: an oversize file is rejected on the length + # of what was asked for, without the remainder being loaded or decoded. + try: + with _open_observation_file(record_file) as handle: + raw = handle.read(board_observation.MAX_BYTES + 1) + except OSError: + # The one place any open, refusal or read failure is accounted for. + # Its own fixed diagnostic: an unreadable file is a different fact + # from a record the contract refused, and neither names an errno -- + # nor whether the descriptor was refused for its kind, which would + # describe the entry the page is forbidden to describe. + unreadable += 1 + warnings.append({"file": record_file.name, "message": "unreadable_file"}) + continue + if len(raw) > board_observation.MAX_BYTES: + # The same closed diagnostic the contract itself raises for an + # over-long record; it names no path and repeats no value. + invalid += 1 + warnings.append({"file": record_file.name, "message": "invalid_contract"}) + continue + try: + records.append(board_observation.decode(raw)) + except board_observation.BoardObservationError as exc: + # The contract's diagnostics are a fixed closed vocabulary that + # deliberately omits observed values and local paths. + invalid += 1 + warnings.append({"file": record_file.name, "message": str(exc)}) + return _ObservationAccounting( + candidate_files=candidate_count, + selected_files=len(selected), + unreadable_files=unreadable, + invalid_records=invalid, + records=tuple(records), + warnings=tuple(warnings), + ) + + +def _observation_message(accounting: _ObservationAccounting) -> str: + """The safe summary line, worst gap first, incompleteness said out loud.""" + + if accounting.omitted_files: + # Said first and unconditionally: whatever the records turned out to + # be, they are not all of them, and that is the fact a reader has to + # carry into everything else on the page. + return ( + f"{accounting.read_files} of {accounting.candidate_files} local Board observation " + f"files were read (cap {MAX_OBSERVATION_FILES}), so this snapshot is incomplete" + ) + if accounting.unreadable_files: + return ( + f"{accounting.unreadable_files} of {accounting.attempted_files} local Board " + "observation files could not be read, so this snapshot is incomplete" + ) + if accounting.invalid_records and not accounting.accepted_records: + return ( + "no local Board observation passed the observation contract, " + "so this snapshot is incomplete" + ) + if accounting.invalid_records: + return ( + f"{accounting.invalid_records} of {accounting.read_files} local Board observation " + "files did not satisfy the observation contract, so this snapshot is incomplete" + ) + if accounting.accepted_records: + return "" + return "no local Board observations recorded yet" + + +def observations_payload(config: BoardConfig) -> dict[str, Any]: + """Read locally recorded Board observations without producing any. + + This is a consumer of the frozen ``code_mower.boardObservation.v1`` + contract: every record is decoded by :mod:`board_observation` and a record + that fails that contract is dropped with its own bounded diagnostic instead + of being repaired, widened, or rendered. The Board never writes these + files, never resolves a session, and never contacts a provider to fill one + in; an empty directory is reported as "nothing recorded yet", which is a + different statement from "no work". + + Reading is bounded to ``MAX_OBSERVATION_FILES`` files. A directory holding + more than that is not silently reduced to whatever the cap happened to + reach: the whole candidate set is counted, the bounded subset is chosen + deterministically, and the shortfall is reported as file-level coverage so + no consumer can read a truncated snapshot as the whole local record set. + That coverage describes the files, not a record's own source coverage, and + it is carried separately from the contract's closed record diagnostics. + + Only regular files are ever read. A directory entry that is a named pipe, + a socket, a directory or a symlink is counted as a candidate and then + refused unread, because a bounded read is only bounded once the file is + open: reading a named pipe with no writer blocks, and a refresh that blocks + keeps serving the snapshot before it. Refusing one is a loss of evidence + like any other and is reported as one. + + The kind is decided twice, and the second time is the one that binds. An + entry can be replaced between being classified and being opened, so the + open is non-blocking and does not follow a link, and the descriptor it + returns is what ``fstat`` re-decides the kind on. That is why a candidate + swapped for a named pipe under a live read costs one refused candidate + rather than a refresh that never returns. + + Every candidate outcome is accounted for by :class:`_ObservationAccounting` + and reported here: what was omitted by the cap, what could not be read at + all, and what the contract rejected. Coverage is ``complete`` only when the + read lost none of them, because a candidate that produced no record might + have been the work record that contradicts an idle one beside it. + + Nothing in this function may raise on the filesystem, and that is a property + of the Board rather than of observations: this runs inside the assembly of + one whole snapshot, so an exception escaping here costs the refresh its + repository, PR and lane data too. Every look at the directory is therefore + guarded -- the preflight metadata by :func:`_classify_observation_path`, + which answers in one call that cannot raise, and the enumeration by the + handler below. Observations are the only thing that degrades; the rest of + the snapshot is built exactly as it would have been. + """ + + path = _observations_path(config) + # One metadata call decides all of it, and it cannot raise. Everything + # below reads this token; no branch here asks the filesystem the same + # question a second time, and none of them can abort the refresh. + path_state = _classify_observation_path(path) + payload: dict[str, Any] = { + "schema": BOARD_OBSERVATIONS_SCHEMA, + "record_schema": board_observation.SCHEMA, + "available": True, + "path": lane_status.LOCAL_PATH_REDACTION, + "path_redacted": True, + # What the path is, from the closed `OBSERVATION_PATH_STATES` + # vocabulary; it names no path and carries no errno. `path_exists` is + # the same fact reduced to the boolean consumers already read, and is + # `null` -- neither true nor false -- when the Board could not look, + # because "it is not there" is a claim an unreadable path cannot + # support. + "path_state": path_state, + "path_exists": {"directory": True, "not_directory": True, "missing": False}.get(path_state), + "records": [], + "warnings": [], + "rejected": 0, + # File-level coverage of this read. `coverage` is a closed vocabulary + # -- complete, partial, unavailable -- and states how much of the + # candidate file set the records below were built from; it is not a + # record's source coverage and says nothing about work. The counters + # beside it partition the candidate set, and `coverage_complete` is the + # single fact every consumer gates an absence claim on. + "coverage": "complete", + "coverage_complete": True, + "coverage_gaps": [], + "truncated": False, + "file_cap": MAX_OBSERVATION_FILES, + "candidate_files": 0, + "selected_files": 0, + "omitted_files": 0, + "attempted_files": 0, + "read_files": 0, + "accepted_records": 0, + "invalid_records": 0, + "unreadable_files": 0, + "unaccounted_files": 0, + "selection": OBSERVATION_SELECTION, + "message": "no local Board observations recorded yet", + } + # A directory that could not be listed has no candidate set at all, so no + # total is invented for it and nothing downstream may read it as coverage. + unreadable = { + "coverage": "unavailable", + "coverage_complete": False, + "coverage_gaps": ["directory_unreadable"], + "candidate_files": None, + "selected_files": 0, + "omitted_files": None, + "attempted_files": 0, + "read_files": 0, + "accepted_records": 0, + "invalid_records": 0, + "unreadable_files": 0, + "unaccounted_files": None, + "message": "could not read local Board observations", + "available": False, + } + if path_state == "missing": + return payload + if path_state == "not_directory": + payload.update(unreadable) + payload["warnings"].append({"file": "", "message": "observation path is not a directory"}) + return payload + if path_state == "unreadable": + # Its own diagnostic, because it is its own fact: the path could not be + # examined, which is neither "not there" nor "there but wrong kind". + # Like every other diagnostic here it carries no path and no errno. + payload.update(unreadable) + payload["warnings"].append( + {"file": "", "message": "could not check the local Board observation path"} + ) + return payload + try: + selected, candidate_count = _select_observation_files(path) + except OSError: + # The classification above is not a promise: the directory can be + # removed or replaced before this enumeration reaches it. Losing it here + # is reported as a gap rather than as an absence, for the same reason + # every other lost candidate is -- the Board cannot know what the + # entries it never listed would have said. + payload.update(unreadable) + payload["warnings"].append({"file": "", "message": "could not list local Board observations"}) + return payload + accounting = _read_observation_records(path, selected, candidate_count) + payload.update(accounting.payload()) + payload["message"] = _observation_message(accounting) + return payload + + def release_campaigns_payload( config: BoardConfig, *, @@ -1297,35 +1927,88 @@ def timelines_payload( } -def render_board_html(config: BoardConfig) -> str: - repo_json = json.dumps(config.repo).replace(" +# The whole Board page as one template. It is plain text rather than an +# f-string so the shipped CSS and JavaScript read exactly as the browser +# receives them, with no doubled braces between the source and the page the +# tests execute. Only the two placeholders below are substituted. +_BOARD_HTML = """ Code Mower Board @@ -1333,30 +2016,52 @@ def render_board_html(config: BoardConfig) -> str:

Code Mower Board

Loading...
+ +
+
+ +
+ +
-
- -

Work Now

-

Owner Queue

-

Lane Work

-

Supervised Pilot

-

Open PRs

-

Gate Alerts

-

Agent Cards

-

Local Orchestrator Lease

-

Release Campaigns

-

Productivity

-

Recent Code Mower Workflows

-

Recent Local History

-

Reviewer Verdict Timeline

-

Spend And Latency

-

Local Activity

+
+
+ +

Work

+

Work Now

+

Participants

+

Owner Queue

+

Lane Work

+

Supervised Pilot

+

Open PRs

+
+ + +
@@ -1896,6 +4920,16 @@ def render_board_html(config: BoardConfig) -> str: """ +def render_board_html(config: BoardConfig) -> str: + repo_json = json.dumps(config.repo).replace(" dict[str, Any]: url = str(board.get("url") or "") if not url: @@ -2721,6 +5755,10 @@ def main(argv: list[str] | None = None) -> int: serve_parser.add_argument("--store-path", help="custom local Board event store path") serve_parser.add_argument("--spend-path", help="custom reviewer spend ledger path") serve_parser.add_argument("--agent-adapters-path", help="custom local agent card directory") + serve_parser.add_argument( + "--observations-path", + help="custom local Board observation directory (read-only)", + ) serve_parser.add_argument("--event-limit", type=int, default=20, help="local history events to show") serve_parser.add_argument("--record-events", action="store_true", help="append local history while the Board is open") serve_parser.add_argument("--record-interval-seconds", type=int, default=60, help="minimum seconds between records") @@ -2800,6 +5838,7 @@ def main(argv: list[str] | None = None) -> int: store_path=args.store_path, spend_path=args.spend_path, agent_adapters_path=args.agent_adapters_path, + observations_path=args.observations_path, event_limit=args.event_limit, record_events=args.record_events, record_interval_seconds=args.record_interval_seconds, diff --git a/tests/test_board.py b/tests/test_board.py index b23cb693..c0acc36f 100644 --- a/tests/test_board.py +++ b/tests/test_board.py @@ -1,34 +1,72 @@ from __future__ import annotations -from contextlib import redirect_stderr, redirect_stdout +from collections.abc import Callable, Sequence +from contextlib import contextmanager, redirect_stderr, redirect_stdout +import copy +import errno import http.client +import itertools import json import math +import os import re import shutil import signal import socket import subprocess +import sys import tempfile import threading import time import urllib.request from datetime import UTC, datetime, timedelta from pathlib import Path -from unittest import TestCase, skipUnless +from unittest import SkipTest, TestCase, skipUnless from unittest.mock import patch from io import StringIO -from code_mower import board, board_store, lane_status, reviewer_spend +from code_mower import board, board_observation, board_store, lane_status, reviewer_spend NOW = datetime(2026, 9, 1, 12, 0, tzinfo=UTC) +# The per-argument ceiling Linux enforces on every element of a command line +# (``MAX_ARG_STRLEN``: a fixed 32 pages, independent of ``ARG_MAX``). The +# shipped Board page script is comfortably past it, so handing it to ``node -e`` +# fails every Linux job with ``OSError: [Errno 7] Argument list too long`` while +# passing locally on macOS, whose ceiling is a much larger whole-command-line +# one. Kept as a named number because the assertions below are about the limit, +# not about today's page size. +LINUX_MAX_ARG_STRLEN = 32 * 4096 + + +def _run_node(script: str, *args: str) -> str: + """Run one generated program through Node, with the program on stdin. + + ``node -`` reads the program from standard input, which is bounded by a + pipe rather than by argv, so a generated script of any size runs and only + the small JSON arguments are ever on the command line. Every Board helper + goes through here, so no helper can reintroduce the argv ceiling on its own. + + Because the program is no longer an argument, ``process.argv`` carries the + ``-`` at index 1 and the first JSON argument is ``process.argv[2]``. + """ + + return subprocess.run( + [shutil.which("node") or "node", "-", *args], + input=script, + capture_output=True, + text=True, + check=True, + ).stdout + + BOARD_POLL_HARNESS = """ __CONSTANTS__ const put = () => {}; const render = () => {}; +const renderRetained = () => {}; const renderEvents = () => {}; const timers = []; let clearedCount = 0; @@ -97,16 +135,7 @@ def _run_board_poll_script(steps: list[dict[str, object] | str | None]) -> list[ script = BOARD_POLL_HARNESS.replace("__CONSTANTS__", constants.group(0)).replace( "__SCHEDULING__", scheduling.group(0) ) - with tempfile.TemporaryDirectory() as tmp: - path = Path(tmp) / "poll.js" - path.write_text(script, encoding="utf-8") - completed = subprocess.run( - [shutil.which("node") or "node", str(path), json.dumps(steps)], - capture_output=True, - text=True, - check=True, - ) - return json.loads(completed.stdout) + return json.loads(_run_node(script, json.dumps(steps))) TRUTH_HELPERS_END = "// --- presentation truth helpers (END) ---" @@ -119,8 +148,8 @@ def _run_board_poll_script(steps: list[dict[str, object] | str | None]) -> list[ const document = {getElementById: (id) => (NODES[id] = NODES[id] || {innerHTML: "", textContent: ""})}; Date.now = () => __NOW_MS__; __SCRIPT__ -render(JSON.parse(process.argv[1])); -renderEvents(JSON.parse(process.argv[2])); +render(JSON.parse(process.argv[2])); +renderEvents(JSON.parse(process.argv[3])); console.log(JSON.stringify(Object.fromEntries(Object.entries(NODES).map(([id, node]) => [id, node.innerHTML || node.textContent])))); """ @@ -145,16 +174,10 @@ def _eval_board_truth(expression: str, *args: object) -> object: script = ( _board_truth_helpers() - + "\nconst ARGS = process.argv.slice(1).map(value => JSON.parse(value));\n" + + "\nconst ARGS = process.argv.slice(2).map(value => JSON.parse(value));\n" + f"console.log(JSON.stringify({expression}));\n" ) - completed = subprocess.run( - [shutil.which("node") or "node", "-e", script, *(json.dumps(arg) for arg in args)], - capture_output=True, - text=True, - check=True, - ) - return json.loads(completed.stdout) + return json.loads(_run_node(script, *(json.dumps(arg) for arg in args))) def _render_board_dom( @@ -176,19 +199,13 @@ def _render_board_dom( BOARD_DOM_HARNESS.replace("__NOW_MS__", str(int(now.timestamp() * 1000))) .replace("__SCRIPT__", "".join(trimmed)) ) - completed = subprocess.run( - [ - shutil.which("node") or "node", - "-e", + return json.loads( + _run_node( script, json.dumps(payload), json.dumps(history if history is not None else {"events": []}), - ], - capture_output=True, - text=True, - check=True, + ) ) - return json.loads(completed.stdout) def _completed(stdout: str, returncode: int = 0) -> subprocess.CompletedProcess[str]: @@ -408,10 +425,12 @@ def test_render_board_html_polls_with_one_self_scheduling_timer(self) -> None: self.assertIn("pollTimer = setTimeout(load, delayMs);", html) self.assertIn("clearTimeout(pollTimer);", html) self.assertLess(html.index("clearTimeout(pollTimer);"), html.index("pollTimer = setTimeout(load, delayMs);")) - # Definition plus exactly one call site, on the single path every - # load() takes whether it succeeded or threw. + # Definition plus exactly one call site, and that call site is the + # `finally` that closes load(): not a statement after the try, which a + # throw from either half of a poll skips, but the boundary every path + # out of load() crosses whether it returned or threw. self.assertEqual(html.count("scheduleNextLoad("), 2) - self.assertIn(" scheduleNextLoad(delayMs);\n }", html) + self.assertIn(" } finally {\n scheduleNextLoad(delayMs);\n }\n }", html) @skipUnless(shutil.which("node"), "node is required to execute the board polling script") def test_board_poll_delay_for_each_cache_state(self) -> None: @@ -3592,3 +3611,8136 @@ def poll_status() -> dict: server.shutdown() server.server_close() server_thread.join(timeout=5) + + +# --- Work-first Board views (#948) ----------------------------------------- + +WORK_MODEL_END = "// --- work view model (END) ---" + +# The fixture clock plus 30s, so every fixture record is recent enough to be +# reported as current unless the record itself says otherwise. +OBSERVATION_NOW = datetime(2026, 9, 12, 20, 0, 30, tzinfo=UTC) + +OBSERVATION_FIXTURES = Path(__file__).parent / "fixtures" / "board_observations.json" + +# Render one payload after another through the shipped renderer in a single +# page lifetime, so selection, announcements and the change timeline are +# exercised the way a refresh actually exercises them. A step may select a work +# row by its opaque key before rendering the next payload. +BOARD_SEQUENCE_HARNESS = """ +const NODES = {}; +const document = {getElementById: (id) => (NODES[id] = NODES[id] || {innerHTML: "", textContent: ""})}; +Date.now = () => __NOW_MS__; +__SCRIPT__ +const frames = []; +for (const step of JSON.parse(process.argv[2])) { + if (step.select !== null) selectWork(step.select); + if (step.payload !== null) render(step.payload); + frames.push(Object.fromEntries(Object.entries(NODES).map(([id, node]) => [id, node.innerHTML || node.textContent]))); +} +console.log(JSON.stringify(frames)); +""" + + +def _board_script() -> str: + """The shipped page script, minus its own ``load()`` bootstrap.""" + + html = board.render_board_html(board.BoardConfig(repo="codemower-ai/code-mower")) + body = html[html.index(" ")] + trimmed = body.rsplit(" load();", 1) + if len(trimmed) != 2: # pragma: no cover - guards the extraction + raise AssertionError("board HTML no longer bootstraps with load()") + return "".join(trimmed) + + +def _board_view_model() -> str: + """Lift the shipped, DOM-free view-model transforms out of the page. + + The work view model is layered on the B1 truth helpers, so the extraction + runs from the first helper through the end of the view-model block. As with + the B1 helpers, the tests execute the JavaScript the browser gets. + """ + + html = board.render_board_html(board.BoardConfig(repo="codemower-ai/code-mower")) + start = html.find(" const text =") + end = html.find(WORK_MODEL_END) + if start < 0 or end < start: # pragma: no cover - guards the extraction + raise AssertionError("board HTML no longer exposes the work view model") + return html[start : end + len(WORK_MODEL_END)] + + +def _eval_board_view(expression: str, *args: object, mutate: tuple[str, str] | None = None) -> object: + """Evaluate one shipped view-model expression against JSON arguments. + + ``mutate`` replaces one exact fragment of the shipped view model before it + runs, so a test can execute the code this replaced and prove that the + assertions it makes would actually catch its return. + """ + + model = _board_view_model() + if mutate is not None: + original, replacement = mutate + if model.count(original) != 1: # pragma: no cover - guards the mutation + raise AssertionError(f"board view model no longer contains exactly one {original!r}") + model = model.replace(original, replacement) + script = ( + model + + "\nconst ARGS = process.argv.slice(2).map(value => JSON.parse(value));\n" + + f"console.log(JSON.stringify({expression}));\n" + ) + return json.loads(_run_node(script, *(json.dumps(arg) for arg in args))) + + +def _render_board_sequence( + steps: list[dict[str, object]], + *, + now: datetime = OBSERVATION_NOW, +) -> list[dict[str, str]]: + """Render a sequence of payloads in one page lifetime.""" + + script = BOARD_SEQUENCE_HARNESS.replace("__NOW_MS__", str(int(now.timestamp() * 1000))).replace( + "__SCRIPT__", _board_script() + ) + normalized = [{"select": step.get("select"), "payload": step.get("payload")} for step in steps] + return json.loads(_run_node(script, json.dumps(normalized))) + + +# Drive the shipped polling loop itself, in one page lifetime, against a +# scripted sequence of `/api/status` and `/api/events` outcomes. +# +# `_render_board_sequence` above calls `render()` directly, which is the right +# harness for what a *payload* renders. This is the harness for what a *failed +# poll* renders: `load()`, its two independently settled requests, the local +# transport state it keeps, the rerender of the retained payload it performs, +# and the one timer it arms are all shipped code here, so nothing about the +# failure path is restated in the test. +# +# The live region records writes rather than its final value. An unchanged poll +# leaves it alone, and "left alone" and "written with the same sentence again" +# are different things to a screen reader, so only a recorded write counts as +# an announcement. +BOARD_LIFETIME_HARNESS = """ +const NODES = {}; +const announced = []; +const makeNode = (id) => { + const node = {innerHTML: "", _text: ""}; + Object.defineProperty(node, "textContent", { + get: () => node._text, + set: (value) => { + node._text = value; + if (id === "announce") announced.push(value); + } + }); + return node; +}; +const document = {getElementById: (id) => (NODES[id] = NODES[id] || makeNode(id))}; +let CLOCK = __NOW_MS__; +Date.now = () => CLOCK; +const timers = []; +let clearedCount = 0; +const setTimeout = (fn, ms) => { + const timer = {fn, ms}; + timers.push(timer); + return timer; +}; +const clearTimeout = () => { + clearedCount += 1; +}; +let STEP = null; +// The three outcomes one request can have, which are the three this page has +// to tell apart: a request that never arrived (the fetch itself rejects), a +// response whose body is not JSON (the fetch resolves and `json()` rejects), +// and a response that parsed -- into whatever shape it parsed into, which the +// step supplies verbatim and is under no obligation to make renderable. +const UNPARSABLE = "__UNPARSABLE_MARKER__"; +const fetch = async (url) => { + const status = url === "/api/status"; + const outcome = status ? STEP.status : STEP.events; + const message = status ? STEP.status_error : STEP.events_error; + if (outcome === null) throw new Error(message); + if (outcome === UNPARSABLE) return {json: async () => { throw new Error(message); }}; + return {json: async () => outcome}; +}; +__SCRIPT__ +(async () => { + const frames = []; + for (const step of JSON.parse(process.argv[2])) { + STEP = step; + if (step.now_ms !== null) CLOCK = step.now_ms; + if (step.select !== null) selectWork(step.select); + const armedBefore = timers.length; + const clearedBefore = clearedCount; + announced.length = 0; + // In the page nothing awaits load(): it is called once and then rearmed + // from a timer, so anything that escapes it is an unhandled rejection with + // no one to see it. Recorded here rather than allowed to abort the run, so + // a test can assert both halves of the invariant separately -- that the + // shipped code lets nothing escape, and that the timer is armed even when + // something does. + let escaped = null; + try { + await load(); + } catch (error) { + escaped = String((error && error.message) || error); + } + const armed = timers[timers.length - 1]; + frames.push({ + nodes: Object.fromEntries(Object.entries(NODES).map(([id, node]) => [id, node.innerHTML || node.textContent])), + announced: [...announced], + timeline: changeLog.map(entry => entry.sentence), + escaped, + delay: armed === undefined ? null : armed.ms, + armed: timers.length - armedBefore, + cleared: clearedCount - clearedBefore, + timers: timers.length, + pending: pollTimer === armed, + transport: {...transportState}, + retained: lastStatusData !== null, + selected: selectedWorkKey + }); + } + console.log(JSON.stringify(frames)); +})(); +""" + + +# What a step supplies for a request whose response arrives and whose body is +# not JSON. `None` is already "the request never arrived"; this is the other +# transport-level failure, and the two reach `fetchJson` differently. +UNPARSABLE_BODY = "__UNPARSABLE_BODY__" + + +def _run_board_lifetime( + steps: list[dict[str, object]], + *, + now: datetime = OBSERVATION_NOW, + mutate: tuple[str, str] | Sequence[tuple[str, str]] | None = None, +) -> list[dict[str, object]]: + """Replay a sequence of polls through the shipped ``load()`` loop. + + Each step is one poll. ``status`` and ``events`` carry the payload that + request answers with, or ``None`` for a request that fails outright; + ``now_ms`` moves the page clock before the poll, so an observation can be + aged out between two of them; ``select`` chooses a work row by its opaque + identity first, the way an operator would before a refresh lands. + + ``mutate`` replaces one exact fragment of the shipped page before it runs, + or several, so a test can execute the code this replaced and prove the + assertions it makes would actually catch its return. Each fragment must + appear exactly once, and each is applied to the page the one before it + produced, so a set of them can reconstruct a whole earlier shape. + """ + + page = _board_script() + if mutate: + edits = [mutate] if isinstance(mutate[0], str) else list(mutate) + for original, replacement in edits: + if page.count(original) != 1: # pragma: no cover - guards the mutation + raise AssertionError( + f"board page script no longer contains exactly one {original!r}" + ) + page = page.replace(original, replacement) + # The marker first: the page script is substituted last so nothing in it + # can be expanded again. + script = ( + BOARD_LIFETIME_HARNESS.replace("__NOW_MS__", str(int(now.timestamp() * 1000))) + .replace("__UNPARSABLE_MARKER__", UNPARSABLE_BODY) + .replace("__SCRIPT__", page) + ) + normalized = [ + { + "status": step.get("status"), + "events": step.get("events", {"events": []}), + "status_error": step.get("status_error", "Failed to fetch"), + "events_error": step.get("events_error", "Failed to fetch"), + "now_ms": step.get("now_ms"), + "select": step.get("select"), + } + for step in steps + ] + return json.loads(_run_node(script, json.dumps(normalized))) + + +def _observation_fixture(name: str) -> dict: + """Build one accepted B0 fixture record, unchanged.""" + + fixture = json.loads(OBSERVATION_FIXTURES.read_text(encoding="utf-8")) + case = next(item for item in fixture["valid"] if item["name"] == name) + record = copy.deepcopy(fixture["templates"][case["template"]]) + for pointer, value in case["set"].items(): + if pointer == "": + record = copy.deepcopy(value) + continue + parts = [part.replace("~1", "/").replace("~0", "~") for part in pointer.split("/")[1:]] + target: object = record + for part in parts[:-1]: + target = target[int(part)] if isinstance(target, list) else target[part] + if isinstance(target, list): + target[int(parts[-1])] = copy.deepcopy(value) + else: + target[parts[-1]] = copy.deepcopy(value) + # Every fixture the views are tested against is a record the frozen + # contract accepts, so no view is ever proved against a shape a producer + # could not emit. + return board_observation.validate(record) + + +def _record_with_reasons(fixture: str, reference: str, reasons: list[str]) -> dict: + """One accepted record that carries several recorded states at once. + + The reasons are canonicalized and the primary route is derived exactly as + the contract requires, so a record that is, say, both ready to merge and + waiting for approval is a record a producer could really emit rather than + a shape invented to make an ordering test pass. + """ + + record = copy.deepcopy(_observation_fixture(fixture)) + ordered = board_observation.ordered_reasons(reasons) + record["work"]["id"] = reference + record["work"]["reference"] = reference + record["work"]["reasons"] = ordered + record["work"]["primary"] = board_observation.derive_primary(ordered) + for index, run in enumerate(record["work"]["runs"]): + run["id"] = f"run{index}-{reference}" + run["binding"]["work_id"] = reference + return board_observation.validate(record) + + +def _referenced_record(reference: str) -> dict: + """One accepted record distinguishable from every other by its reference.""" + + fixture = _observation_fixture("observed_running") + return _record_with_reasons("observed_running", reference, fixture["work"]["reasons"]) + + +def _record_with_suspended_run(reference: str, *, reasons: list[str] | None = None) -> dict: + """One accepted record whose run the provider suspended rather than failed. + + The contract allows the `suspended` lifecycle state only alongside the + `failed` phase, so this is the shape a producer must emit for a paused + session -- and the shape that reading the phase alone would misreport. + """ + + record = copy.deepcopy(_observation_fixture("failed")) + ordered = board_observation.ordered_reasons(reasons or []) + record["work"]["id"] = reference + record["work"]["reference"] = reference + record["work"]["reasons"] = ordered + record["work"]["primary"] = board_observation.derive_primary(ordered) + for index, run in enumerate(record["work"]["runs"]): + run["id"] = f"run{index}-{reference}" + run["binding"]["work_id"] = reference + run["lifecycle"] = { + **run["lifecycle"], + "state": "suspended", + "reason": "session_suspended", + "next_action": "inspect_provider", + } + return board_observation.validate(record) + + +# One accepted fixture per run phase the frozen contract allows, so a record +# carrying that phase is always built from a shape the contract has already +# accepted rather than assembled by hand. +PHASE_FIXTURES = { + "assigned": "assigned", + "dispatched": "dispatched", + "observed_running": "observed_running", + "provider_progress": "provider_reported_progress", + "waiting_for_user": "waiting_for_user", + "waiting_for_approval": "waiting_for_approval", + "implementation_complete": "implementation_complete", + "failed": "failed", + "cancelled": "cancelled", +} + +# A reason and next action the remote-session projection accepts for each +# lifecycle state, so every combination below is a record a producer could +# really emit. +LIFECYCLE_ROUTES = { + "pending": ("none", "none"), + "running": ("none", "status"), + "waiting_for_user": ("user_input_required", "none"), + "waiting_for_approval": ("approval_required", "none"), + "complete": ("none", "none"), + "failed": ("session_failed", "inspect_provider"), + "suspended": ("session_suspended", "inspect_provider"), + "terminated": ("none", "none"), + "archived": ("none", "none"), + "uncertain": ("reconcile_dispatch", "status"), +} + +# Every lifecycle state the frozen B0 contract accepts, against every phase it +# allows that state to carry, plus the no-lifecycle case for each phase. The +# expected label, class and cue are the operator meaning of the pair: the one +# state whose truthful meaning is not its phase is `suspended`, which the +# contract requires to carry the `failed` phase. +LIFECYCLE_DISPLAY_MATRIX = ( + (None, "assigned", "assigned", "muted", "?"), + (None, "dispatched", "dispatched", "warn", "~"), + (None, "observed_running", "observed running", "warn", "~"), + (None, "provider_progress", "provider reported progress", "warn", "~"), + (None, "waiting_for_user", "waiting for an answer", "warn", "~"), + (None, "waiting_for_approval", "waiting for approval", "warn", "~"), + (None, "implementation_complete", "implementation complete", "ok", "+"), + (None, "failed", "failed", "bad", "!"), + (None, "cancelled", "cancelled", "warn", "~"), + ("pending", "dispatched", "dispatched", "warn", "~"), + ("running", "observed_running", "observed running", "warn", "~"), + ("running", "provider_progress", "provider reported progress", "warn", "~"), + ("waiting_for_user", "waiting_for_user", "waiting for an answer", "warn", "~"), + ("waiting_for_approval", "waiting_for_approval", "waiting for approval", "warn", "~"), + ("complete", "implementation_complete", "implementation complete", "ok", "+"), + ("failed", "failed", "failed", "bad", "!"), + ("suspended", "failed", "suspended", "warn", "~"), + ("terminated", "cancelled", "cancelled", "warn", "~"), + ("archived", "implementation_complete", "implementation complete", "ok", "+"), + ("archived", "cancelled", "cancelled", "warn", "~"), + ("uncertain", "dispatched", "dispatched", "warn", "~"), +) + + +def _record_with_run_lifecycle(reference: str, phase: str, state: str | None) -> dict: + """One accepted record whose single run carries ``phase`` under ``state``. + + The record is built from the accepted fixture for that phase and only its + lifecycle is rewritten, so the contract's own phase/basis, liveness and + lifecycle rules still decide whether the result is a record at all. No + reason is recorded, so what the views say about the run is decided by the + run rather than by a recorded blocker. + """ + + record = copy.deepcopy(_observation_fixture(PHASE_FIXTURES[phase])) + record["work"]["id"] = reference + record["work"]["reference"] = reference + record["work"]["reasons"] = [] + record["work"]["primary"] = board_observation.derive_primary([]) + for index, run in enumerate(record["work"]["runs"]): + run["id"] = f"run{index}-{reference}" + run["binding"]["work_id"] = reference + if state is None: + run["lifecycle"] = None + continue + reason, next_action = LIFECYCLE_ROUTES[state] + counts = (run["lifecycle"] or {}).get( + "counts", {"dispatch": 0, "message": 0, "cancel": 0, "collect": 0} + ) + run["lifecycle"] = { + "schema": "code_mower.remote_session.v1", + "state": state, + "reason": reason, + "next_action": next_action, + "counts": counts, + } + return board_observation.validate(record) + + +# --- Session-scope reconciliation (#948) ------------------------------------ + +# Two more session identities the frozen contract accepts, so a scope can be +# moved without inventing a shape a producer could not record. +FIXTURE_SESSION = "a4ce901ecfb743609ed0b6504668aca7" +FIXTURE_WORKTREE = f"sha256:{'a' * 64}" +OTHER_SESSION = "b" * 32 +OTHER_WORKTREE = f"sha256:{'c' * 64}" + +_OBSERVATION_INSTANT = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$") + + +def _shift_instants(value: object, seconds: int) -> object: + """Move every recorded instant in a decoded record by ``seconds``.""" + + if isinstance(value, dict): + return {key: _shift_instants(item, seconds) for key, item in value.items()} + if isinstance(value, list): + return [_shift_instants(item, seconds) for item in value] + if isinstance(value, str) and _OBSERVATION_INSTANT.match(value): + moved = datetime.strptime(value, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=UTC) + timedelta( + seconds=seconds + ) + return moved.strftime("%Y-%m-%dT%H:%M:%SZ") + return value + + +def _observed_later(record: dict, seconds: int) -> dict: + """The same accepted record, recorded ``seconds`` later than it was. + + Every instant moves together, so each ordering rule the frozen contract + enforces between them -- a source checked no later than the record, an + event no later than the observation that caught it -- still holds. The + result is re-validated rather than assumed, so a shifted record is still a + record a producer could really emit. + """ + + return board_observation.validate(_shift_instants(copy.deepcopy(record), seconds)) + + +def _in_session(record: dict, *, session: str, worktree: str) -> dict: + """The same accepted record, observed in a different session scope.""" + + moved = copy.deepcopy(record) + moved["scope"]["session_id"] = session + moved["scope"]["worktree_id"] = worktree + for run in ((moved.get("work") or {}).get("runs") or []): + run["binding"]["session_id"] = session + run["binding"]["worktree_id"] = worktree + return board_observation.validate(moved) + + +def _named_work(work_id: str, *, fixture: str = "observed_running") -> dict: + """One accepted work observation carrying the given work identity.""" + + record = copy.deepcopy(_observation_fixture(fixture)) + record["work"]["id"] = work_id + record["work"]["reference"] = work_id + for index, run in enumerate(record["work"]["runs"]): + run["id"] = f"run{index}{work_id}" + run["binding"]["work_id"] = work_id + return board_observation.validate(record) + + +def _idle_key(session: str = FIXTURE_SESSION, worktree: str = FIXTURE_WORKTREE) -> str: + return f"idle:{session}:{worktree}" + + +def _work_key( + work_id: str, session: str = FIXTURE_SESSION, worktree: str = FIXTURE_WORKTREE +) -> str: + return f"work:{session}:{worktree}:{work_id}" + + +def _observation_payload(records: list[dict], **overrides: object) -> dict: + payload: dict[str, object] = { + "generated_at": "2026-09-12T20:00:00Z", + "next_action": "inspect", + "board": { + "cache": { + "state": "fresh", + "ttl_seconds": 15, + "age_seconds": 1, + "generation": 2, + "refresh_in_progress": False, + "retry_in_seconds": None, + }, + "version": { + "serving_version": "1.4.1", + "installed_version": "1.4.1", + "restart_recommended": False, + }, + }, + "remote": { + "available": True, + "pull_requests": [], + "workflow_runs": [], + "gate_health": {"alerts": []}, + }, + "observations": { + "available": True, + "path_exists": True, + "records": records, + "warnings": [], + "rejected": 0, + "message": "", + }, + } + payload.update(overrides) + return payload + + +# A DOM shim in which focus is a real question. Elements exist because the +# markup that was rendered declared an id; replacing a container's innerHTML +# destroys everything that was inside it, so a focused control that the refresh +# does not render again is genuinely gone and focus falls to the body exactly +# as a browser would drop it. Only the ids the shipped page ships in its static +# shell exist up front, so a lookup for a control that a render removed returns +# nothing rather than conjuring a phantom element to focus. +BOARD_FOCUS_HARNESS = """ +class FakeElement { + constructor(doc, id, attrs, parent) { + this.doc = doc; + this.id = id; + this.attrs = attrs || {}; + this.parent = parent || null; + this.dataset = {}; + this.classList = (this.attrs["class"] || "").split(/\\s+/).filter(Boolean); + for (const [name, value] of Object.entries(this.attrs)) { + if (!name.startsWith("data-")) continue; + this.dataset[name.slice(5).replace(/-([a-z])/g, (_, c) => c.toUpperCase())] = value; + } + this.hidden = false; + this.textContent = ""; + this._html = ""; + // A freshly created element starts at the top, exactly as a replacement + // node does in a browser: this is the reset the page has to undo. The + // assignment itself is deliberately dumb -- nothing here clamps it -- so a + // restored offset is only ever in range because the page put it in range. + this.scrollTop = 0; + } + // Content inside a hidden panel has no box at all, so every layout metric + // reads zero however tall the evidence is. That is the browser behaviour the + // page has to tell apart from a genuine reading position of zero, so the + // shim reproduces it rather than letting a test opt into it. + get scrollHeight() { return this.doc.laidOut(this) ? (this.doc.metrics[this.id] || {}).scrollHeight || 0 : 0; } + get clientHeight() { return this.doc.laidOut(this) ? (this.doc.metrics[this.id] || {}).clientHeight || 0 : 0; } + get innerHTML() { return this._html; } + set innerHTML(value) { + this.doc.replaceChildren(this, value); + this._html = value; + } + focus() { this.doc.activeElement = this; } + matches(selector) { + if (selector.startsWith(".")) return this.classList.includes(selector.slice(1)); + const attribute = /^\\[([A-Za-z-]+)(?:=([^\\]]*))?\\]$/.exec(selector); + if (!attribute) return false; + const value = this.attrs[attribute[1]]; + if (value === undefined) return false; + return attribute[2] === undefined || value === attribute[2]; + } + closest(selector) { + for (let node = this; node; node = node.parent) if (node.matches(selector)) return node; + return null; + } + querySelectorAll(selector) { + return [...(this.doc.owned.get(this.id) || new Map()).values()] + .filter(node => node.matches(selector)); + } +} +const document = { + activeElement: null, + body: null, + // Content geometry the shim cannot compute, declared per element id by the + // step that needs it, so a test can shrink the detail region between two + // refreshes the way changed evidence really would. + metrics: {}, + roots: new Map(), + owned: new Map(), + index: new Map(), + // Which view panel each static container belongs to, read off the shipped + // markup rather than assumed here. + panelOf: __PANEL_OF__, + getElementById(id) { + if (this.index.has(id)) return this.index.get(id); + return this.roots.get(id) || null; + }, + laidOut(node) { + for (let current = node; current; current = current.parent) { + if (current.hidden === true) return false; + const panel = this.panelOf[current.id]; + if (panel && (this.roots.get(panel) || {}).hidden === true) return false; + } + return true; + }, + replaceChildren(root, html) { + for (const [id, node] of this.owned.get(root.id) || new Map()) { + if (this.index.get(id) === node) this.index.delete(id); + if (this.activeElement === node) this.activeElement = this.body; + } + const created = new Map(); + for (const tag of html.match(/<[a-zA-Z][^>]*>/g) || []) { + const attrs = {}; + for (const [, name, value] of tag.matchAll(/([A-Za-z-]+)="([^"]*)"/g)) attrs[name] = value; + if (attrs.id === undefined) continue; + const node = new FakeElement(this, attrs.id, attrs, root); + created.set(attrs.id, node); + this.index.set(attrs.id, node); + } + this.owned.set(root.id, created); + } +}; +document.body = new FakeElement(document, "", {}); +document.activeElement = document.body; +for (const id of __SHELL_IDS__) document.roots.set(id, new FakeElement(document, id, {})); +Date.now = () => __NOW_MS__; +__SCRIPT__ +const element = (id) => { + const node = document.getElementById(id); + if (!node) throw new Error("no element: " + id); + return node; +}; +const frames = []; +for (const step of JSON.parse(process.argv[2])) { + if (step.metrics) Object.assign(document.metrics, step.metrics); + if (step.focus) element(step.focus).focus(); + if (step.click) element(step.on || "worklist").onclick({target: element(step.click)}); + if (step.key) { + element(step.on || "worklist").onkeydown({ + key: step.key, + target: element(step.from), + preventDefault: () => {}, + }); + } + if (step.select) selectWork(step.select); + if (step.scroll) { + // A browser moves the element and then tells the page it moved. Both + // halves are modelled: a page that only ever reads the offset out of the + // element when it is about to be replaced has nothing to read once the + // panel is hidden. + const scrolled = element(step.scroll.id || "workdetail"); + scrolled.scrollTop = step.scroll.top; + if (typeof scrolled.onscroll === "function") scrolled.onscroll(); + } + if (step.payload) render(step.payload); + const detail = document.getElementById("workdetail"); + frames.push({ + active: document.activeElement === document.body ? "" : document.activeElement.id, + detail: detail === null ? null : {key: detail.attrs["data-key"] || "", top: detail.scrollTop}, + // What the page remembers outside the DOM, so a test can see that a + // hidden poll left it alone and that an identity the Board stopped + // showing was dropped rather than kept for ever. + remembered: Object.fromEntries(detailOffsets), + worklist: document.getElementById("worklist").innerHTML, + tabs: document.getElementById("tabs").innerHTML, + hidden: Object.fromEntries(["now", "timeline", "releases", "health"] + .map(view => [view, element("panel-" + view).hidden])), + }); +} +console.log(JSON.stringify(frames)); +""" + + +def _board_shell_ids() -> list[str]: + """Every id the shipped page's static markup declares, script excluded.""" + + html = board.render_board_html(board.BoardConfig(repo="codemower-ai/code-mower")) + shell = html[: html.index("