Skip to content

Stop repeated goal/store work without stale goal publication - #3613

Merged
trissim merged 26 commits into
mainfrom
work/slot-activity--P958
Oct 30, 2026
Merged

trissim merged 26 commits into
mainfrom
work/slot-activity--P958

Conversation

@trissim

@trissim trissim commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary — pushed source checkpoint, NOT complete or performance-qualified

Core source head c0ddfb5, pushed and matched by git ls-remote, includes fetched main d683810. git diff --stat origin/main...HEAD: 109 files, +1738/-1034. Correction29: 11 files, +65/-65; main integration was clean and pushed separately at 45724f2. Historical correction28 001b608: 13 files, +186/-128. Prior StoreContention.reader conflict resolution still preserves main's LendingConnection retirement AND P958's check_same_thread parameter and per-acquisition timeout.

Matched production companion: OpenHCSDev/toad#1363, head bcc8ce70b9cf78f64d0c9136e9ff0e4288d0caa8, remote readback matches; cumulative 3 files, +6/-21. The former fixture companion #1346 is merged. This pair remains subject to root review; no independently landable or installed claim.

Backend query and response ownership

Before: AgentBackend.inspect's NativeQuery/PiResponseData annotations hid imports under TYPE_CHECKING. The direct cycle went through input/inbox records declared inside AgentBackend. Compact then guessed compaction attributes on the response base; TurnRunner similarly assumed a context response.

After:

  • The complete input family moves to turn_inbox.py: BusyInputPolicy, StreamingInputPolicy, FollowUpInputPolicy, SteerInputPolicy, RejectInputPolicy, InputContent, TurnInboxItem, TurnInbox, InputRequest, SendNow, UseSettings, ClearQueue. At the a6d21b0 move checkpoint all 12 moved class ASTs matched the original declarations exactly. Correction28 typed TurnInbox.init and the actual inherited CPython Queue hooks omitted by the installed public typeshed stub; correction29 deletes TurnInboxItem.refuse and moves its session effects onto NativeSession. Queue insertion/wake/accounting behavior is unchanged. Their producers, consumers and fixtures import their new declaration owner. AgentBackend retains actual InputContent usage, not compatibility re-exports of the other records.
  • AgentBackend normally imports the existing NativeQuery and PiResponseData declarations. PiCommand/NativeQuery stay in pi_commands.py. A discarded extraction that merely relocated guards was reversed, not shipped.
  • PiResponseData owns require_compaction and require_context refusal; CompactionData and NativeContextData supply themselves. PiNativeBackend.compact and TurnRunner call those methods instead of guessing fields or switching over response types.
  • NativeContextData moves intact from native_turn_context.py to its response family in pi_payloads.py, adding the context refinement and typed require_session_file. Its CLI, command, test and allocated Toad consumers migrate. NativeContextManifestData remains in its existing owner.
  • TurnRunner's compaction observer/event/thread/manifest/query parameters retain their real types. Unsupported backends still refuse inspect through AgentBackend.

Bounded final import evidence parsed 638 Core modules. No cycle returns to AgentBackend, PiCommand, turn_inbox, native_turn_context or maintenance_barrier in the literal top-level graph. Existing pi_payloads/agent_events/acp_failure cycles remain visible: pi_payloads imports the events module after its payload declarations; this is not claimed to be an acyclic whole repository. Guarded/function-local imports, dynamic binding and external dependency graphs are explicitly outside that graph. No product import probe was run.

Physical maintenance witnesses

Before: MaintenanceBarrier._read returned None for an absent physical file, and _read_witness separately interpreted marker/state/target absence. The original incomplete-witness path ALREADY refused; no claim that this repairs an old permissive return.

After: MaintenanceWitness shares the same private-file/size/codec/error decoder on MaintenanceMarker and HoldTarget; MaintenanceState inherits it. NoMaintenanceWitness owns absent-file answers. Marker and state methods enforce pairing; missing either enabled witness still raises the same RelationViolationError. An absent target leaves the valid state unchanged; stale target generations/nonces retain their existing behavior. Both absent marker and state still mean never-enabled READY, without reading a leftover target. Malformed/inaccessible/private-file errors remain refusals, and physical revision consistency is still checked under the original lock. The observations dictionary remains unchanged, not turned into another style-only cleanup.

One SQLite reader lifetime

SQLiteStorage.opened_reader and its per-call lifecycle are deleted. Whole src/tests/tools name search is empty. ReadableStore inherits SQLiteReader; RegistryStore/InputDispositions retain their coordinator reader, and optional-awareness, CliBridge, working-memory and inherited carry consumers reuse that reader.

Outer acquisitions apply the requested StoreContention. An already-active same-PID transaction is borrowed BEFORE stat/absence/revision/retirement and does not begin or roll back another transaction. Forks cannot borrow a parent's handle. Replacement/unlink is observed by the next outer acquisition. Query-only, current journal mode, privacy/schema checks and permit attestation remain.

Main's newly merged LendingConnection owns cursors until connection retirement. This merge changes the actual reader connection class and therefore needs root's affected reader execution, not an assertion that earlier reader controls qualify the merged class.

RoomAllocation retains RoomLedger. RoomLedger._open_reader accepts/forwards keyword-only check_same_thread while retaining mode=rw, WriterOwnedFile schema carry, query-only, contention and close-on-error. DocumentBytes retains MissingFile plus empty bytes on genuine absence, without creating the path.

Goal projection and wake ownership

ToolEnd/StreamSettled rebuilding and source-revision/publication mirrors are deleted. A requested sync always reads fresh Goals.goal_changed; equality with the complete successfully delivered GoalSessionPublication snapshot is the sole suppression authority. ACP publishes that acquired snapshot.

The shared root watcher captures registry once and selects actual attached goal owners against their existing published snapshots. Goal wait/attempt commits request goal projection independently of inbox drains. Unrelated registry phase/row writes do not globally wake session inbox observers. Startup uses DrainedSession.wake_observer; original idle unobserved-input error handling remains. Goal publication failure is not a diagnostic for nonexistent input.

Coarse commit paths are NOT complete execution revisions. Requested projections still read selected activity/room claims, expiry and process liveness. Clock expiry or process death alone makes no filesystem commit; no spontaneous subscription or polling substitute is claimed.

Session identity and Toad consumers

AcpSessionId survives scheduler/callback/session-effects/runtime/fixture paths; foreign ACP/SDK strings decode at their actual ingress through FieldCodec. RuntimeConnection routes the retained ThreadIncarnation, not an invented ACP attachment. RuntimeProxy separately retains the actual AcpSessionId. Toad context inspection supplies the retained ReadableStore and owner incarnation; its request-only AgentController uses RuntimeConnection and deletes the unused 12-line OwnerRequestContext wrapper.

ConfigOptions authentication observation previously could run while bind_owner awaited redeal before PublishedSession existed. Redeal now precedes acquisition; acquisition and publication have no intervening await. Exact unknown-session refusal remains. This source race is NOT a proved explanation of both Toad settlement timeouts.

Correction28 — startup has no fabricated attachment

Before: StartOwnerRuntimeRequest.bind wrapped its thread name in AcpSessionId solely because the shared context required a session before startup had attached one.

After:

  • RuntimeRequestContext owns only server, reader, client and thread name; AttachedRuntimeRequestContext alone declares the actual AcpSessionId.
  • RuntimeRequest[Ctx] requires bind; AttachedRuntimeRequest owns the unchanged registry/owner/loaded-session binding. ResultRuntimeRequest[Ctx] owns result application. AttachedResultRuntimeRequest composes the two providers through MI, not a copied binder or forwarding shim.
  • StartOwnerRuntimeRequest uses the attachment-free result/context. Every attached request and override uses the attached parent/context, including Subscribe, owner launch/resource/stop, prompt/control/goal and current/recorded context families.
  • Runtime still decodes RuntimeRequest at the original FieldCodec boundary, then binds/applies it; no new kind roster, null session, cast or fake attachment. The literal scan finds exactly the two intended context construction sites and no AcpSessionId(self.thread).
  • QueueScope.start_revision, LiveQueueScope.start_revision and PromptRequest.publish_acceptance retain AcpSessionId. Dead imports are deleted; two tests import RuntimeRequest from owner_connection, its declaration owner.
  • Correction28's manifest-per-digest workaround and five newly guarded annotation imports were rejected and are superseded below. Native session attestation/MCP validate overrides remain typed. No author checker/formatter execution.

Correction29 — declaration owners replace five guarded dependencies

Before: pi_payloads imported Comms/Thread/NativeAttestation under TYPE_CHECKING only to forward Comms, extract owner.incarnation and ask conflicts. TurnInboxItem refused through InputForwarding with a wrong TurnSession annotation while its caller actually held NativeSession. Three measured-segment members repeated tokens/sha256/utf8_bytes, and recorded_public_text built a whole manifest for every digest lookup.

After:

  • SnapshotAttestation(ABC) owns the snapshot conflicts(NativeSessionSnapshot) -> bool contract at pi_payloads.py:1172. NativeAttestation inherits it at native_attestation.py:41; both payload conflict consumers take the contract. Existing pending/known/observed/lost meanings and refusal defaults remain distinct.
  • NativeContextData.for_turn takes ThreadIncarnation directly through a normal import (pi_payloads.py:1257). Its only production caller, cli_context.py:74, passes owner.incarnation. The separately owned NativeContextManifestData.for_turn is unchanged.
  • NativeContextData.public_source validates the exact current observation, segment range and source membership and returns the selected Provenance. ContextSourceRuntimeRequest.read_source reads its public text and constructs ContextSourceText; lookup and file reading remain together in the existing Coordination.run_worker boundary. The Core and recorded-context source APIs are different owners and remain unchanged. The current-native source control migrates to the new API while retaining codec roundtrip, wrong source/observation and changed-file refusals.
  • NativeSession.refuse uses its existing MroDispatch's first matching handler. TurnInboxItem emits FollowingInputRefused for its carried IDs; UnwrittenPrompt retains InputMissing, finish_request and rejected_signal effects. RefusedSend.apply calls session.refuse(item). The old refusal implementations disappear from TurnInboxItem, UnwrittenPrompt and InputForwarding; no second refusal path remains.
  • MeasuredNativeSegment declares tokens/sha256/utf8_bytes once. SystemLayerSegment, NativeMessages and ToolCatalogSegment lose their nine copied field declarations; all five concrete measured families inherit the same fields. recorded_public_text indexes segment.sha256 directly, with no per-segment manifest construction. Existing manifest construction for an actual manifest is preserved.
  • All five newly added TYPE_CHECKING dependencies are deleted; zero new guarded imports or string annotations. Older guarded imports remain outside this correction; no claim that the whole repository is acyclic.

Source evidence: before-map via NRA parses 1,868 modules across Core src/tests/tools and allocated Toad src/tests, no caught/skipped parse errors. After-map enumerates the changed owner declarations/fields/imports and syntax-compiles 11 Python files, exit 0. Whole-tree name search finds no item.refuse/inputs.refuse/old input refusal implementations. Reflection, dynamic imports, MRO and FieldCodec execution remain unproved. Initial scan launches failed before parsing (missing NRA .venv, then missing metaclass_registry in bare Python); the existing shared environment completed the source map. An earlier install-interrupted AST call returned UNKNOWN, produced no observed output file and was not replayed as an input.

Evidence and qualification

  • Correction28 before/after request/context map: 1,868 modules across Core src/tests/tools and allocated Toad src/tests, NRA parsing with no caught/skipped parse failures. The first after-map failed on a missing colon introduced during formatting; it was fixed before commit. The completed map and 13 changed Python syntax compilations exited 0. No product module execution/import probe, type checker, tests or runtime MRO proof. The merged head additionally syntax-compiled the union of correction28 and integrated-main Python files: 21 sources, exit 0, without module execution. The prior checkpoint's 37-source syntax result remains historical.
  • Historical root result at 3b8f154 remains 16 collected /14 passed /2 failed /all teardowns passed. Both failed goal controls captured RoomLedger's missing check_same_thread keyword. Missing prepared-native logging is separate environment evidence.
  • Actual hosted run41062946479/job124087043267 at 1598bda was RED: acp_extension +2 type errors, pi_payloads +6, turn_inbox +4; missing payload/inbox annotations; dead imports; named E501/Black findings. Correction28 source addresses these exact sites. Correction29 removes all five new guarded annotation dependencies. No new debt-src green result is claimed. Earlier run41050873765 at cd605 remains historical evidence.
  • Hosted run41070493150/job124113042621 remained RED on correction28: TypeCheckingImport pi_payloads +3, turn_inbox +2; send_outcome TypeErrors +1 at :68 (NativeSession passed where TurnSession was declared). It also checks unpaired Toad main and reports context_inspection's old NativeContextData import/ReadableStore/incarnation callers; those source corrections are in the existing paired P406: declare message roles without changing stored bus JSON #1363, not compatibility aliases in Core. This is historical hosted evidence, not a check of c0ddfb5.
  • Authored goal/physical-reader/contention/rollback/missing-document controls remain UNRUN. Root owns the necessary combined selection under HostTestLease, including the two failed goal nodes and actual RoomLedger schema/cancellation consumers. Private canonical_goal_wire assigns its own StorageRule volume/scratch/checkout paths; no live host ledger or mocked writer.
  • No author pytest, checker, profile, provider call, product import probe, UI, service, installation or uncertain input replay.

Root acceptance remains worker <15% CPU with at least two agents streaming and <50 registry-lock opens/10 seconds. Baseline: 84–95% CPU and 686 registry-lock opens/10 s. AFTER remains unmeasured. Neither source validity nor fewer event rebuilds completes that acceptance.

Assumed: syntax and static caller searches do not establish dynamic import/MRO execution, exception parity, SQLite contention behavior, installed two-agent performance or the cause of Toad settlement timeouts. Existing late module cycles remain disclosed; the five new guarded imports are removed structurally, not runtime-qualified. Untouched untracked src/agent_comms/skills/refactor-audit remains preserved. #3594 completion belongs to claude-code's existing fork and is not duplicated here.

Evidence: /home/ts/.cache/agent-scratch/slot-activity/p958-correction29-{source-map,after-syntax,main-sync,publish,hosted-debt}.log; historical p958-correction28-{ast-syntax,commit-merge,merged-syntax,publish}.log. Prior p958-contract-* and p958-toad-contract-* source evidence retained.

pi_payloads.py:1250 wanted one digest and ordered a whole manifest
turn_context.py:842 now supplies the digest; paperwork lunch cancelled

@trissim
trissim force-pushed the work/slot-activity--P958 branch from 55536ef to 47b6743 Compare October 29, 2026 19:46
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

debt-src at 9f1fb15, every finding:

grew ClientTypeErrors:batrachian-toad/src/toad/core/context_inspection.py +2: this change removes or changes an agent_comms name that client file uses; run mypy on it to see which
::warning file=src/agent_comms/acp.py::LintFindings +3 (warns, never blocks)
::warning file=src/agent_comms/agent_event_updates.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/controller.py::LintFindings +2 (warns, never blocks)
::warning file=src/agent_comms/goal_attempts.py::LintFindings +2 (warns, never blocks)
::warning file=src/agent_comms/goal_generation.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/goal_scheduler.py::LintFindings +4 (warns, never blocks)
::warning file=src/agent_comms/input_drain.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/maintenance_barrier.py::LintFindings +4 (warns, never blocks)
::warning file=src/agent_comms/owned_turn.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/pi_commands.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/pi_native_backend.py::LintFindings +2 (warns, never blocks)
::warning file=src/agent_comms/pi_payloads.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/presentation.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/queued_input.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/runtime.py::LintFindings +2 (warns, never blocks)
::warning file=src/agent_comms/schedule_rules.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/send_outcome.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/tracked_turn.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/turn_inbox.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/turn_input_source.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/turn_inputs.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/acp_request_consumer.py::Unformatted +1 (warns, never blocks)
found ClientTypeErrors:batrachian-toad/src/toad/core/context_inspection.py at batrachian-toad/src/toad/core/context_inspection.py:18 (Module "agent_comms.native_turn_context" has no attribute "NativeContextData"; maybe "NativeContextManifestData"?  [attr-defined]) batrachian-toad/src/toad/core/context_inspection.py:542 (Argument 1 to "for_context" of "WorkingMemoryAnnotations" has incompatible type "Path"; expected "ReadableStore"  [arg-type]) batrachian-toad/src/toad/core/context_inspection.py:625 (Argument 2 to "RuntimeConnection" has incompatible type "str"; expected "ThreadIncarnation"  [arg-type])
found LintFindings:src/agent_comms/acp.py at src/agent_comms/acp.py:179 (E501 Line too long (102 > 100)) src/agent_comms/acp.py:292 (E501 Line too long (107 > 100)) src/agent_comms/acp.py:425 (E501 Line too long (119 > 100))
found LintFindings:src/agent_comms/agent_event_updates.py at src/agent_comms/agent_event_updates.py:3 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/controller.py at src/agent_comms/controller.py:34 (E501 Line too long (106 > 100)) src/agent_comms/controller.py:54 (E501 Line too long (102 > 100))
found LintFindings:src/agent_comms/goal_attempts.py at src/agent_comms/goal_attempts.py:786 (E501 Line too long (104 > 100)) src/agent_comms/goal_attempts.py:1032 (E501 Line too long (101 > 100))
found LintFindings:src/agent_comms/goal_generation.py at src/agent_comms/goal_generation.py:3 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/goal_scheduler.py at src/agent_comms/goal_scheduler.py:192 (E501 Line too long (105 > 100)) src/agent_comms/goal_scheduler.py:220 (E501 Line too long (106 > 100)) src/agent_comms/goal_scheduler.py:250 (B008 Do not perform function call `GoalRefresh` in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable) src/agent_comms/goal_scheduler.py:266 (E501 Line too long (102 > 100))
found LintFindings:src/agent_comms/input_drain.py at src/agent_comms/input_drain.py:248 (E501 Line too long (106 > 100)) src/agent_comms/input_drain.py:846 (E501 Line too long (107 > 100))
found LintFindings:src/agent_comms/maintenance_barrier.py at src/agent_comms/maintenance_barrier.py:390 (E501 Line too long (106 > 100)) src/agent_comms/maintenance_barrier.py:441 (E501 Line too long (106 > 100)) src/agent_comms/maintenance_barrier.py:586 (E501 Line too long (106 > 100)) src/agent_comms/maintenance_barrier.py:589 (E501 Line too long (115 > 100))
found LintFindings:src/agent_comms/owned_turn.py at src/agent_comms/owned_turn.py:3 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/pi_commands.py at src/agent_comms/pi_commands.py:57 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/pi_native_backend.py at src/agent_comms/pi_native_backend.py:12 (I001 [*] Import block is un-sorted or un-formatted) src/agent_comms/pi_native_backend.py:24 (E501 Line too long (155 > 100))
found LintFindings:src/agent_comms/pi_payloads.py at src/agent_comms/pi_payloads.py:7 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/presentation.py at src/agent_comms/presentation.py:963 (E501 Line too long (106 > 100))
found LintFindings:src/agent_comms/queued_input.py at src/agent_comms/queued_input.py:49 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/runtime.py at src/agent_comms/runtime.py:125 (E501 Line too long (102 > 100)) src/agent_comms/runtime.py:490 (E501 Line too long (101 > 100))
found LintFindings:src/agent_comms/schedule_rules.py at src/agent_comms/schedule_rules.py:34 (E501 Line too long (101 > 100))
found LintFindings:src/agent_comms/send_outcome.py at src/agent_comms/send_outcome.py:14 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/tracked_turn.py at src/agent_comms/tracked_turn.py:53 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/turn_inbox.py at src/agent_comms/turn_inbox.py:58 (B024 `TurnInboxItem` is an abstract base class, but it has no abstract methods or properties)
found LintFindings:src/agent_comms/turn_input_source.py at src/agent_comms/turn_input_source.py:3 (I001 [*] Import block is un-sorted or un-formatted)
found LintFindings:src/agent_comms/turn_inputs.py at src/agent_comms/turn_inputs.py:7 (I001 [*] Import block is un-sorted or un-formatted)
found Unformatted:src/agent_comms/acp_request_consumer.py at src/agent_comms/acp_request_consumer.py:1 (Black would reformat this file)

@trissim
trissim marked this pull request as ready for review October 29, 2026 20:02
Copilot AI balanced review requested due to automatic review settings October 29, 2026 20:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@trissim
trissim force-pushed the work/slot-activity--P958 branch from 47b6743 to be366d0 Compare October 29, 2026 20:29
@trissim trissim changed the title P958: stop rebuilding worker observations and reopening stores Stop repeated goal/store work without stale goal publication Oct 29, 2026
@trissim
trissim force-pushed the work/slot-activity--P958 branch from 1c46410 to f465403 Compare October 30, 2026 01:59
@trissim
trissim force-pushed the work/slot-activity--P958 branch from f465403 to f1de66e Compare October 30, 2026 03:11
@trissim
trissim force-pushed the work/slot-activity--P958 branch from f1de66e to e8921ba Compare October 30, 2026 04:10
@trissim
trissim enabled auto-merge October 30, 2026 15:17
@trissim
trissim merged commit 9463dba into main Oct 30, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants