Skip to content

Move selected tool consumption into the input coordination store - #2764

Merged
trissim merged 18 commits into
mainfrom
next/selected-tool-slot
Oct 23, 2026
Merged

trissim merged 18 commits into
mainfrom
next/selected-tool-slot

Conversation

@trissim

@trissim trissim commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The selected-tool broker stored consumption and terminal acknowledgement as two fsynced files per slot and never retired them. This change puts both facts in one SelectedToolSlot row in the existing coordination database, with declared ConsumedSlot / AcknowledgedSlot states. Consumption commits before any claim or file mutation; duplicate consumption and a failed commit remain UNKNOWN with no retry.

The one-shot SelectedToolSlots carry reads the existing filenames and call IDs directly. It reads one final row for each slot, then deletes the old files only after every row commits. An orphan .done or different call ID raises with its path before writing or deleting anything. It neither scans transcripts nor looks up NativeRuntimeInput nor recomputes a slot. The table key is the original file basename passed as input_id. Coding callers still pass the existing slot as both input_id and call_id.

Acknowledgement-file removals are directory-synced before consumed files are removed. A rerun requires the committed call ID and state to equal the surviving source; a mismatch refuses with its path, retaining the files and stored state. A second dry run reports zero changes after success. No legacy merge rule remains in runtime code. Session directory discovery comes from the existing SavedSessionFile declarations, including native-sessions, sessions and imported_sessions; there is no new directory roster.

Source relationships (AST-enumerated across src and tools, then read):

  • native_runtime_input.py owns the table and state family. TypedTable derives its schema and the state CHECK; no raw SQL was added outside TypedTable.
  • selected_tool_broker.py consumes, acknowledges and verifies through the existing coordination transaction owner; its file ledger, fsync and ledger-only private path checks are deleted. The unused session-directory arguments are deleted too.
  • channel_coding_tools.py passes the original slot twice. native_tool_call.py does the same at terminal acknowledgement. native_tool_transport.py supplies the coordination path from the turn's existing session root.
  • Existing selected-tool/socket fixtures were migrated to the table contract. Refactor-audit fixtures import no deleted names and needed no changes.
  • NativeRuntimeInput currently inherits NoDelete; the source trace found no individual native-input deletion. Whole-database archives include both tables. No input lookup or invented input association was added to the carry.

Copied-root proof through step_worker at a8685a3

The installer's existing StoreBackup.copy_tree made /home/ts/wt/selected-tool-slot-proof/root, using SQLite snapshot copies. Only this scratch copy was changed. The live root was neither written nor asked to replay an input.

Observation Dry run Apply
Regular files before 150,186 150,186
Regular files after 150,186 31,915
Consumed-file observations 59,236 59,236
Acknowledgement-file observations 59,035 59,035
Ledger files deleted 0 118,271
Final consumed / UNKNOWN rows projected 201 201
Final acknowledged rows projected 59,035 59,035
Final distinct slot rows projected 59,236 59,236
Seconds 11.289 19.798

118,271 files encode 59,236 final slot rows: a matching .done selects AcknowledgedSlot for the same primary key. All 59,236 keys were distinct. Every one of the 201 unacknowledged rows rejects verification and repeat consumption. The second dry run returns Carried(changed=0). These timings measure the carry on the copied root, not an installed end-to-end speedup.

Five fresh child processes exercised the actual broker and transaction methods on independent copies of the carried database. Each printed only after reaching its boundary, was killed with SIGKILL (exit -9), and its database was reopened:

Kill boundary Reopened slot Verify Consume again
Before insertion absent no acknowledgement first consumption allowed
Inserted, before commit absent no acknowledgement first consumption allowed
After consumption commit consumed UNKNOWN denied; no retry
Acknowledged, before commit consumed UNKNOWN denied; no retry
After acknowledgement commit acknowledged acknowledged denied; no retry

Interpreter probes confirm orphan/mismatched acknowledgements and conflicting committed rows refuse with their exact paths. The socket fixture now opens its Coordination connection inside the worker, matching the real owner. The three touched test files pass locally under timeout 300 and -p no:xdist: 38 passed, 1 warning in 2.05s after the main sync. All touched production/carry modules import under python -I; all touched Python files parse. Hosted checks run on this PR head; the PR had to leave draft to start them because debt-ratchet.yml skips every draft job.

Assumed: filenames and ASCII call IDs are the legacy ledger's stored identities, exactly as the old writer encoded them. Presence of a matching .done is the stored terminal acknowledgement. Empty unacknowledged consumed files remain consumed/UNKNOWN. The snapshot is a committed per-file copy of a running root, not a globally quiescent capture. No installed native turn is claimed by these process/database checks.

The earlier proof consumed the snapshot inputs. For the required step_worker rerun, the identical captured set was restored from the original immutable ledger files only after checking every key, call ID and state against the saved cut; 118,271 original files matched. The proof destination table was cleared only on that owned copy. Dry run / apply / redry then ran as fresh python -m agent_comms.installer.step_worker processes. Redry took 2.618 s and returned changed=0.

@trissim
trissim marked this pull request as ready for review October 23, 2026 12:14
Copilot AI balanced review requested due to automatic review settings October 23, 2026 12:14

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

debt-tools at 611d720, every finding:

grew LintFindings:tools/cutover/carry_selected_tool_slots.py +3
grew TypeErrors:tools/cutover/carry_selected_tool_slots.py +1

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

debt-src at 7480351, every finding:

grew FamilyFlattened:src/agent_comms/native_runtime_input.py +1
grew LintFindings:src/agent_comms/selected_tool_broker.py +1
grew TypeErrors:src/agent_comms/native_tool_transport.py +1
grew TypeErrors:src/agent_comms/selected_tool_broker.py +1

@trissim

trissim commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Hosted run 39951823803 against head 3af1369 / synthetic merge c552b6f found two failures outside the selected-slot diff, both present on main:

I am fixing this PR's own carry/declaration findings. The independent main failures still prevent the requested all-green hosted result; they need their existing owners or a commander assignment, rather than another competing edit to the route/store family.

@trissim
trissim merged commit 7017468 into main Oct 23, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants