Skip to content

Stamp derived projections and index answer eligibility at append - #2636

Merged
trissim merged 25 commits into
mainfrom
next/projection-producer-identity
Oct 23, 2026
Merged

trissim merged 25 commits into
mainfrom
next/projection-producer-identity

Conversation

@trissim

@trissim trissim commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Old derived bus rows could survive a Core change when their SQL schema stayed equal. ProjectionProducer now records the existing installed Core authority, package_stamp.STAMP.commit(), and DerivedTables compares and records it through the existing reconciliation transaction. ProducedRow is a method-free capability inherited once by the bus projection family. Producer rosters, source/bytecode hashing, homemade value serialization, and source-swapping tests are deleted.

All four send-back items are included in accepted head fcc1062ceafd895d00269515df75b53a48041407, merged at b2b03584bf939b4050ad8c8eba2145e8224d4495:

  1. bus_publication.py:45 evaluates the existing Python reply-scope owner at append for each frozen recipient. bus_tables.py:98 stores one AwaitingAnswerSource relation, including channel questions and informational DMs. DirectReplySource at line 121 is its non-stored direct-turn subset; it selects the same table through an indexed predicate. bus.py:334 applies the requested SQL limit before loading selected deliveries. The retired direct-only table is renamed through its declaration and rebuilt through existing derived-schema ownership. The obsolete retirement tool and its tests are deleted.
  2. AnsweredSource.unanswered at bus_tables.py:183 and _answer_rows at line 220 query that projection. Unanswered and latest-answer selections use SQL limits; timestamp selection uses SQLite's maximum. These readers never decode historical candidates to call scope.includes. The SQL copies of response policy remain deleted.
  3. HeartbeatSilent at commander/standing.py:340 concatenates its registry members. MissedHeartbeat and HeartbeatNotStarted own their distinct scans; the hard-coded scan owner and own-type dispatch are deleted.
  4. plans/commander-native-plan.md:214 and :230 retain the design. Copied benchmark and delivery status paragraphs were removed; the historical heartbeat receipt is retained below.

The complete #2630 work is included and superseded here. GitHub already marks it merged through this ancestry; its follow-up comment points here. Closed #2629's equal-DDL skip was not imported. Current main also preserves #2638's independent Bus.received_after metadata-only contract; this change does not decode replies in that lock-held path.

Validation at the accepted head:

  • debt-src and debt-tools pass against merged main 2cb721d98e09612c5df6fd38910ceb5fbbdae407: no positive deltas and no introduced violations. Logs: /home/ts/.cache/agent-scratch/projection-stamp-debt-src-fcc.txt and projection-stamp-debt-tools-fcc.txt.
  • Bus, standing and wake batch: 97 passed. After the final recipient-predicate/rename factoring, answer-family checks: 20 passed; the real-bus freshness test separately passed. The schema/reconcile/typed-store/carry batch: 27 passed. Heartbeat ownership checks passed (40 standing cases, then four focused heartbeat cases).
  • The existing eligibility family now tests each direct turn, channel question and informational DM both unanswered and answered. Wrong hashes, original aliases, unrelated peers and non-monotonic reply timestamps remain covered.
  • The real-bus freshness case proves unchanged commit with zero writes, an older recorded commit rebuilding corrupted derived output while preserving durable history, and missing witness-table recreation. It patches no source files.
  • One final-shape rebuild on a COPY of the carried bus took 43.309 seconds for 19,189 durable records and eleven projection tables. All durable rows stayed equal, the old direct-only table disappeared, all stamps match the producing commit, integrity and foreign keys pass, and repeat reconciliation writes zero rows. Runtime profiling on that copy counted one complete record decode for a one-delivery direct page, one for a one-question unanswered page, and zero for a sampled latest-answer miss. Evidence: /home/ts/.cache/agent-scratch/projection-stamp-awaiting-rebuild.json. The live bus was not touched.
  • Banned producer-list/hash spellings are absent from production source. Importing typed_table and the real bus succeeds.

Source evidence: the original NRA census covered 1,241 Core/test/tool and paired Toad modules, 6,910 source declarations and 6,711 projected classes, with no parse omissions. The final census covers 1,243 modules and 6,720 projected classes, with no parse omissions. Owner declarations, inheritance and consumer sites are retained in /home/ts/.cache/agent-scratch/projection-stamp-ast-final.json. This resolves source names and inheritance, not arbitrary dynamic rebinding.

Assumed: production Core has no uncommitted runtime-source edits. The live environment resolves Core to editable main, whose tracked runtime source was clean when inspected; a checkout stamp cannot see uncommitted edits. Awaiting-answer work follows the captured frozen delivery recipients; display-only history supplies no frozen delivery obligation. Installed application qualification stays with the installation owner; none is claimed here.

Historical heartbeat checkpoint receipt moved out of the design plan:

Draft #2624 is stacked on the published owner-request port #2621. Eighteen
focused copied-bus/request/page/channel/source checks pass (7.54 seconds),
including the newest beat across both participants' renames and later reverse
replies. On the retained 19,189-record bus copy, an absent service with aliases
takes 76 SQLite instructions after index reconciliation versus 45,088 before;
both return no message. All exact-pair queries use the declared covering index.
Reconciliation preserves every bus sequence and both bus/address row counts;
integrity and foreign-key checks pass, and a repeated reconciliation is a no-op.
Its current unconditional projection rebuild costs 39.83 seconds even for this
index-only change; that installer/schema defect remains open with the commander.
Installed full-session/parity acceptance and the complete control-plane phase
remain open. The source census has zero parse omissions; unprojected declarations
remain visible.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

debt-tools at 33c8759, every finding:

grew FamilyFlattened:tools/cutover/carry_reply_policy.py +1
grew LintFindings:tools/cutover/carry_reply_policy.py +5
grew Unformatted:tools/cutover/carry_reply_policy.py +1

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

debt-src at 4f57f38, every finding:

grew GodClassExcess:src/agent_comms/bus.py::BusReader +21
grew LintFindings:src/agent_comms/bus.py +2
grew LintFindings:src/agent_comms/bus_tables.py +2
grew NoneAbsence:src/agent_comms/bus.py +8
grew TypeErrors:src/agent_comms/commander/standing.py +1
introduced NoneAbsence:src/agent_comms/bus.py at lines [247, 248, 272, 273, 284, 285, 318, 320, 345]

@trissim trissim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SOURCE REVIEW: CHANGES REQUIRED at cc6d00e. GitHub refuses REQUEST_CHANGES because the fleet shares the PR author's account, so this COMMENT records the verdict.

The required dependency sync is missing. Claude/Tristan directed merging current main and #2630's bus_tables changes before further implementation. Actual ancestry is cc6d00e -> cc44278 -> 271b813 -> main69c5; merge-base with #2630 final3f6ae92e2 is only da502. bus_tables.py still declares DirectReplySource (:93), MessageIdentitySource.response_policy (:102) and eligible_sql (:124), all deleted by #2630's source-accepted family migration. The replay still uses the retired policy column. It is useful old-source evidence, not proof of the requested combined source after deletion.

Preserve the checkpoint/results, merge the coherent #2630 dependency (including consumers and one-shot retirement, not a standalone file copy that breaks imports), resolve producer declarations against surviving projections, then exercise equal-DDL changed-code planning/rebuild/rollback on a surviving projection. Inspect affected held/composed consumers and publish the corrected head. Route #2633/install is independent. This identifies the missing integration requirement, not a failed empirical replay or installed-runtime verdict. #2629 is closed in favor of this replacement.

@trissim trissim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readiness withdrawn by the implementation owner after a real loaded-code/source-file mismatch replay. At this checkpoint, _producer_module_identity hashes the declaring file, which can change after ConversationRole is imported but before the first rebuild. The owner reproduced old DirectPolicy rows receiving a witness matching the changed InformationalPolicy source; a fresh interpreter then incorrectly treats those stale rows as current. Saved failure: /home/ts/.cache/agent-scratch/projection-producers-loaded-code-before.json. The previous same-DDL/fresh-process and rollback checks remain their bounded results; they did not cover this state. #2636 remains draft, unaccepted and outside installation. The owner is replacing file-as-producer identity with actual loaded implementations/declared immutable values and removing the fingerprint caches. Retain the failing replay and confirm repaired behavior after the required #2630 integration; route #2633/install is independent.

@trissim trissim changed the title Rebuild derived projections when declared producers change Rebuild bus projections when the installed Core commit changes Oct 23, 2026
@trissim
trissim marked this pull request as ready for review October 23, 2026 07:26
Copilot AI balanced review requested due to automatic review settings October 23, 2026 07:26
@trissim
trissim merged commit b2b0358 into main Oct 23, 2026
6 of 7 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants