Sync engine + API: read-only GitHub Issue/PR state for every Molecule (all states, batched, cached, fail-closed) + GET /api/issues #242

Slicing (2026-10-27): this slice is the sync ENGINE + API only

The UI half moved to #266. This slice edits nothing under lab/internal/ui/web/ and does not modify projection.go, inbox.go or drive.go, which other Molecules are changing right now.

• Package: build the sync in a new package lab/internal/forgesync. It holds the syncState pure function and its matrix (rows 1–17), the batched GraphQL fetcher seam, the 60 s single-flight cache, the failure-reason mapping and the stale snapshot.
• Routes: add read-only routes in new files: lab/internal/ui/issues.go (GET /api/issues) and GET /api/sync (per-Molecule DisplayState keyed by Molecule id, plus snapshot age), with the same read guards as the other GET routes.
• Server registration: server.go gets only route registration lines.
• Demo: demo mode serves fixture sync data.
• Out of scope here: the UI sections of this body (Issues tab, filters, Work and Inbox rendering, Molecule view lines) are delivered by Issues view + sync rendering: render #242's sync states on Issues, Inbox, Work, counts and Molecule view (after #262) #266. The served-JS tests are Issues view + sync rendering: render #242's sync states on Issues, Inbox, Work, counts and Molecule view (after #262) #266's too.
• Done when (this slice): the syncState table test, the fetcher tests and the API tests (guards, no raw gh output, repository not overridable, truncation, demo) from this body.

Problem (v1, operator request, 2026-10-26)

"I want to have all issues in the UI, even those that didn't start the intake, so it can be done via the UI, not by searching a random issue."

Today forgecell ui shows only Molecules. To start work, the operator has to know an issue number and type it into Start Work. They can't see the repo's open issues, which ones Forgecell has never touched, or which already have work in flight or delivered, so the UI can't be the place where work is chosen.

Fix

Add an Issues view listing the open GitHub Issues of the active Formula's repository, each with its Forgecell state and a one-click start. It's built only from engine pieces the loop already uses.

• Engine data (no new engine logic):
  • GET /api/issues combines intake.ReadOpenIssues(ctx, repo) (bounded, oldest first, Truncated when more exist) with molecule.ReadQueueInventory(lab).Summarize(repo, number) for each issue.
  • Each row has the number, title, labels, milestone and updated time from the issue. From Forgecell it has the state: not started; in progress with the lane of its latest Molecule (running, queued for checks, needs your decision, blocked); draft published; or retired only (startable again). It also includes the latest Molecule id, a link to it in the Molecules view, and startable exactly as the loop computes it.
  • repo is always the active Formula's repository, never a request parameter.
• Caching and limits:
  • Read the GitHub listing at most once per 60 s per server, through an injectable fetcher (a seam) so tests use a fake. Show its age ("updated 12 s ago") and a Refresh button.
  • Fail closed: if gh is missing, unauthenticated or rate-limited, show a fixed, redacted, actionable reason, as in UI correctness from the first browser test: delivered lane, actionable refusals, consistent next action, stray null, job 404 #235's refusal style. Never show raw command output.
  • The listing is bounded as ReadOpenIssues already is. If Truncated, say so.
• Guards and demo: the same read guards as the other GET routes (cookie, tab header, same-origin, GET only). Demo mode serves labelled sample issues and never calls gh.
• UI:
  • Tab: an Issues tab next to the Molecules view (the default stays Molecules), in the existing visual language. If UI polish: composed Start Work controls, segmented lane bar, sharper glass (visual only) #241 has landed, reuse its segmented bar and .btn styles.
  • Rows: each row shows #number, title, label and milestone chips, a state pill, and on the right a primary Start intake button only when startable (the existing guarded POST /api/runs, refusals shown with their reason). Otherwise there's an Open Molecule link.
  • Filters: a state filter (All · Not started · In progress · Needs decision · Draft published), label and milestone filters built from the listing, and text search over number and title.
  • Run the loop on these: a shortcut fills the Start Work selector with the current label or milestone filter (label:X, milestone:Y).
  • Safety: after Start intake, the row reads back its new state from /api/issues and the job, and never shows "started" before the read-back. Text is rendered through the text-only el(), and long titles are bounded.
• Out of scope: editing issues, closing issues, creating issues, and anything that writes to GitHub.

Added scope (operator, 2026-10-26): read-only GitHub state sync for Molecules

"Why weren't they synced as finished?" Forgecell never reads Issue or PR state back, so attempts whose Issue was finished elsewhere stay "blocked" or "needs you" forever, and "Draft published" never becomes "Merged". (On 2026-10-26, 77 stale Molecules had to be retired by hand.)

• Issue state: using the same bounded, cached gh reader as the Issues list, a Molecule whose Issue is closed is shown in a new Issue closed state (with the closing PR or commit when GitHub reports one). It is excluded from Needs my decision, Blocked and the headline counts, but counted in analytics, like Retired. Nothing is written to the ledger and nothing is retired automatically; it's a projection only.
• PR state: for a Molecule with a published draft receipt, read its PR state through the same reader (bounded, cached, 60 s). Merged shows the merge commit and time and moves the Molecule to a Merged lane. Closed unmerged shows "PR closed without merge". The merge observation is evidence for analytics only: Cost per merged PR may be computed only from Molecules with an observed merge, with coverage shown ("n of m published Molecules observed merged"). Otherwise it stays Unavailable.
• Molecule detail must not look unfinished when it is done (operator, 2026-10-26, on mol-174…: its PR feat: run cohort attempts in parallel with --parallel N (#174) #175 merged on Oct 18 and the Issue is closed, yet the detail showed "Cohort: run attempts in parallel with --parallel N (default 1, deterministic report) #174 · OPEN", "merge and production unverified" and the gate Atom "waiting". All 32 Forgecell-published PRs in this Lab are merged.)
  • Issue state: show the live state from the sync ("Cohort: run attempts in parallel with --parallel N (default 1, deterministic report) #174 · closed 2026-10-18"). When the sync is unavailable, label the recorded value explicitly "OPEN at intake (2026-10-18)", never a bare "OPEN".
  • Merged: when the PR is observed merged, the delivery line reads "Merged · · · production unverified". It's still never "deployed".
  • Atom row: for a published and merged Molecule, the review gate Atom shows resolved ("draft published, then merged"), not "waiting". ship shows "draft PR published", taken from the receipt, instead of "skipped". This is display-only, derived from the receipt and the observed merge; the ledger isn't rewritten.
  • Next action: "Nothing to do · merged on GitHub".
  • Tests for each of these.
• Failure: if gh fails, keep today's lanes, say why (fixed, redacted reason), and never guess.
• Tests: a closed Issue moves Molecules to Issue closed and out of the action lanes; a merged PR leads to the Merged lane and enters cost per merged PR with coverage; a closed-unmerged PR; gh failure falls back to today's lanes; demo uses sample state only.

Done when

• Tests (ui, deterministic, fake fetcher, fixture Lab):
  • the rows and states match the fixture: not started, in progress per lane, draft published, retired-only startable;
  • startable agrees with QueueInventory.Summarize;
  • the repo can't be overridden by a query parameter;
  • the cache is honoured, and Refresh bypasses it;
  • gh failure gives a redacted fixed reason with no leak;
  • truncation is shown;
  • the guards (forged Origin, missing cookie, POST to a GET route);
  • demo mode never calls the fetcher;
  • the served JS renders the Issues tab, the filters and the per-row Start intake through the existing run route.
• Visual verification (operator, in the browser).
• Targeted checks: go test ./internal/ui/... ./internal/cli/....

Review checklist (read before coding)

The adversarial review checks these every time. Recent slices failed review on all of them, so build them in from the start:

• Bounds: sizes, counts, time and total work are bounded, and hitting a bound fails safe (incomplete or truncated, never silently accepted).
• Filesystem safety: paths cannot traverse. Use Lstat and O_NOFOLLOW, and check symlinks and ownership. Watch for TOCTOU races on shared dirs (/tmp). Only ever delete Forgecell-owned, marked paths.
• Untrusted input: issue text, repository files, model and tool output, and ledgers can be malformed, huge or hostile. Validate them, never interpolate them into HTML or a shell, and never let them trigger retries or approvals.
• No local leaks: no absolute local paths, environment variables or credentials in output, logs, the UI or ledgers.
• Contract changes: a changed adapter invocation bumps the protocol, and approved digests still bind exactly what was shown.
• Tests: deterministic. Use no wall-clock budgets (use readiness markers or seams), write nothing into the user's real temp or home, and cover the failure path as well as the happy path.

Plan checks (for intake)

Leave every check's reviewedAcceptance and independentProvenance empty: those come only from a separate human review, never from analysis. Check category must be one of regression, candidate or setup. Use targeted package tests only as plain go test commands (for example go test ./internal/<pkg>/...); the protected view adds -json itself when protected deltas are declared. The Formula baseline already runs the full suite once, so don't add another full go test ./... check, and don't repeat its go vet or gofmt checks. A plan may hold at most 16 checks in total, counting the 5 Formula baseline checks, so prefer one combined targeted command over one check per package.

Added 2026-10-27 (operator: "What is all of this finished, draft published?")

The Work grid shows 38 "Finished · Draft published" Molecules. Every one of their PRs has since merged on GitHub (64 of 65 Forgecell PRs merged, 1 closed, none open), but the UI can't tell, so history buries the live work.

• Read merge state back: with the read-only sync, a published Molecule whose PR is merged shows as Merged (with merge time and PR link); a closed-unmerged PR shows PR closed; a closed Issue shows Issue closed. Only gh reads; never write to GitHub.
• Work view defaults to live work: in progress, needs you and blocked first. Merged history collapses into one row ("38 merged · show"), expanded on demand and kept in the URL (?history=1). The Inbox and the counts header use the same split ("3 running · 0 need you · 38 merged").
• Tests: merged / closed / open mapping from fixture gh JSON; the default view hides merged; the toggle is in the URL; a sync failure keeps "Draft published" with a plain "couldn't check GitHub" note and never guesses.

Added 2026-10-27: end-to-end sync, every state (operator: "make sure 242 covers the sync end to end with all states")

This section is the contract. It defines one pure function, syncState(issue, pr, molecule) → DisplayState, plus a table-driven test with one case per row below. The UI, the Inbox, the Work grid, the headline counts, analytics and startable all read from that function, so no surface can disagree with another.

Inputs (all read-only, via gh)

• Issue (by number, including closed ones):
  • state, stateReason (completed, not planned, reopened);
  • closedAt, plus the closing PR or commit when GitHub reports one;
  • title, labels, milestone, updatedAt, locked;
  • not found (deleted or transferred) or no access.
• PR (for each published receipt):
  • state (open, closed, merged) and isDraft;
  • mergedAt and the merge commit;
  • headRefOid compared with the commit Forgecell published;
  • head branch deleted;
  • reviewDecision: approved, changes requested, or review required;
  • statusCheckRollup: success, failure, pending or none.
• Molecule (from the ledger, unchanged): lane, latest-of-N for the Issue, and retired.

Issue × work, with what the UI shows

• Precedence:
  1. A live Molecule (running, gate or blocked) beats everything except row 14.
  2. Then PR states.
  3. Then Issue states.
• Several Molecules per Issue: the newest non-retired one decides the row, and the others are listed under "Earlier attempts".
• Retired Molecules never decide state, but a retired Molecule whose PR merged still counts as Merged in history and analytics.
• Merges outside Forgecell: a merge or close by a human PR shows as row 12 with that PR's link. It's not counted as a Forgecell merge.

Sync mechanics

• Batching: fetch Issue and PR state for all Molecules in one GraphQL query per batch of at most 50 numbers through the injectable fetcher, never one gh call per row.
  • The open-Issues list comes from paginated gh (bounded at 500, with "showing 500 of N" when truncated).
  • Both are cached for 60 s per server. Refresh is single-flight: concurrent requests share one fetch.
• Fail closed and bounded:
  • every call has a timeout and a byte cap;
  • rate-limit, not-authenticated, offline, gh missing and wrong-host errors each map to one fixed plain reason;
  • on failure the last good snapshot is kept, with its age ("couldn't check GitHub · showing state from 4 min ago");
  • with no snapshot at all, rows show ledger-only states labelled "not checked".
• Freshness: every row and the header show "GitHub state from ". Refresh bypasses the cache.
• Startable:
  • Startable means startable from QueueInventory.Summarize and the Issue is open and accessible (rows 1, 5, 10, 15).
  • The Start intake button re-checks the Issue state just before calling the run route. If it closed meanwhile, the request is refused with "Issue was closed on GitHub".
• Untrusted data: titles, labels, milestone names and PR data are untrusted. They are bounded, rendered as text only, and never used in a URL except numbers validated as integers, joined to the repository's fixed https://github.com/OWNER/REPO/ base.
• Never writes: nothing goes to GitHub or to the ledger. The sync is a projection; retiring stays an explicit action in UI: unblock from the browser — 'why it's blocked · what you can do' actions, and bulk-retire attempts for closed Issues #243.

Tests (added to Done when)

• syncState table test: rows 1–17, the precedence rules, several Molecules per Issue, a retired-and-merged Molecule, and a merge outside Forgecell.
• Fetcher tests:
  • batching at most 50 per query, pagination and truncation;
  • cache, Refresh and single-flight;
  • each failure kind maps to its fixed reason, and the stale snapshot is kept with its age;
  • timeout and byte cap;
  • no raw gh output in any response.
• Start intake refused when the Issue closed between listing and click.
• Every surface (Issues view, Inbox, Work grid, counts header, analytics) agrees for the same fixture. One test renders all of them from one fixture and compares.

Answers to the planning questions (2026-10-27)

1. Rows 10 and 15 are startable.
   • Rule: startable = Issue open AND accessible AND (Summarize().Startable() OR the latest non-retired Molecule is finished with its PR closed unmerged (row 10) OR merged and the Issue was reopened afterwards (row 15)). The table test expects yes for rows 1, 5, 10 and 15.
   • Start intake from the UI for rows 10 and 15 creates a fresh Molecule. The server recomputes the restart permission from the sync snapshot; it never comes from the client. The earlier Molecule is untouched and listed under "Earlier attempts".
   • The loop is unchanged: it keeps Summarize().Startable() only and never auto-restarts these rows.
2. server.go may add one handler field for the forgesync cache and fetcher seam, plus its initialisation in NewHandler, alongside the route registration lines. That's the existing seam pattern; tests replace it the same way.
3. The open-Issue listing comes from the new forgesync GraphQL reader: bounded at 500, with totalCount for "showing 500 of N", and stderr classified into the fixed failure reasons. intake.ReadOpenIssues stays unchanged; the loop still uses it.