Skip to content

Keep page evidence and traversal work linear - #3659

Merged
trissim merged 4 commits into
mainfrom
perf/page-evidence-index
Oct 30, 2026
Merged

trissim merged 4 commits into
mainfrom
perf/page-evidence-index

Conversation

@trissim

@trissim trissim commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

P993: 2 production files, +25/-16 against current main. Existing implementation commit 7fa5b861c7b3542dc7f8f77a7a06bfb32f21d168 retained; main integrated by merge at 45b06843c8c05046a64eefec5d57a2d5c4cb01c2, then refreshed again at published head 5214de7953297f932f9b2ce8a87ab4cf9940f03a (remote equals HEAD; zero commits behind fetched main). No competing Core PR.

This repairs three repeated-work paths in one history-page read. The paired Toad consumer is OpenHCSDev/toad#1360, published at 999edc483a9f034a546080cdc4e93c6dad2fc22d (+2/-2 in WireSource.read); it must land after this Core PR, with its agent-comms lock refreshed from merged main. Do not mistake this source checkpoint for installed acceptance.

Evidence ownership: MessagePage

Before: WireSource asked evidence_for(message) for every fragment, and each call walked the source pages and their messages again.

After: MessagePage.message_evidence() builds one invocation-local dictionary from the original source_pages (or the page itself), keyed by each row's view_key. It returns messages in self.messages order, paired with the original evidence-page object only if that page's selection has evidence. No field, persistent index, cached_property, compatibility alias, or alternate evidence owner was added.

Deleted: the four-line evidence_for method. The sole production consumer is Toad WireSource.read, which must consume the pairs directly.

Traversal ownership: existing PageBounds and PageTraversal

Before: every part recombined all messages collected so far, then recomputed its remaining budget from that accumulated page.

After: the first page starts a deque, bounds.after(first) establishes the remaining budget, and following.after(part) advances it after each admitted part. following.fits(len(part.messages)) checks that part against that remaining budget. The existing traversal appends at the correct chronological end. MessagePage.combine runs once after the loop, or not at all for a singleton. First/last edge flags, original source-page identities, revision consistency checks, and ordering remain with their existing owners.

Live-existence lifetime: one integrated traversal

Before: every historical part performed another limit-one live page read just to answer whether newer live messages exist.

After: one local functools.cache callable is created by integrated_page, passed to _source_page, and queried only when a historical page needs that answer. It performs at most one such limit-one live read per traversal. This does not cache history or replace the normal requested live page.

Source evidence and limits

NRA's existing parse_python_module_roots and ModuleSyntaxIndex inventoried both Core and Toad production roots: 917 parsed modules, 13 related modules, zero parser failures. Retained declaration, inheritance, decorator, import, write, decision, and named-consumer evidence: /home/ts/.cache/agent-scratch/producer-digest/p993-source-before.json. Dynamic bindings and runtime mutation are not resolved by this syntactic inventory.

Reviewed: MessagePage.combine/select and SelectedDisplay.select; HistoryArchive.page/integrated_page/_source_page; PageBounds.after/fits; both traversal directions; Toad WireSource.read and its fragment/evidence consumers. Existing #3670/#3566 and #3607 historical_views changes are outside these edited methods; Toad #1319 changes the same fragment-constructor line and its extra page argument must survive integration.

Hosted run 41031189252, debt-src job 123981126381, failed on the old pinned Toad's wire_history.py:326 call to evidence_for. Paired Toad #1360 deletes that call. The same hosted output reports E501 warnings at the two changed Core methods and the order-prescribed None-shaped signature/loop; no other growing blocker was reported by that run. The pair needs integration-owner combined qualification before Core can land, then its Toad lock refresh. Core auto-merge is enabled; neither merge nor application acceptance is claimed.

No author pytest, local ratchet, profiling, install, application imports, or provider probes were run. The first source-inventory command failed before execution because the NRA checkout had no venv; rerunning that source-only inventory with the existing Core interpreter succeeded. That is not a behavioral test result.

Integration owner still owes combined controls covering Core tests/test_historical_views.py (both directions, source identity, sparse archived acknowledgements) and Toad tests/textual/test_wire_source.py, then the actual Toad channel journey with attached archived history: same message order, original read evidence, and has-newer edge while scrolling backward. No measured speedup or live acceptance is claimed.

Assumed:

  • Source pages retain the disjoint message identities established by indexed paging and MessagePage's source-order validation.
  • The integration owner runs the combined controls and real Toad journey; this unit is explicitly forbidden author pytest/rehearsal/install.
  • The paired Toad lock is updated only after this Core PR merges, then that same paired PR lands through the queue.

find one message's page
walk every message
repeat for every message

The page already knows where it came from. We can stop asking the entire archive for a character reference.

MessagePage.evidence_for rescanned every source page for each message
(O(page^2) per history read); message_evidence indexes the source pages
by view key once and pairs every message with its evidence page.

HistoricalSources.integrated_page re-combined every page into a new
MessagePage on each loop, and each historical part opened a full live
page read only to learn whether live messages follow. The remaining
budget and continuation now follow from the newest part, the pages are
combined once, and one traversal asks the live bus once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 06:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

debt-src at dbc42ad, every finding:

grew ClientTypeErrors:batrachian-toad/src/toad/wire_history.py +1: this change removes or changes an agent_comms name that client file uses; run mypy on it to see which
::warning file=src/agent_comms/historical_views.py::LintFindings +1 (warns, never blocks)
::warning file=src/agent_comms/message_page.py::LintFindings +1 (warns, never blocks)
found ClientTypeErrors:batrachian-toad/src/toad/wire_history.py at batrachian-toad/src/toad/wire_history.py:326 ("MessagePage" has no attribute "evidence_for"  [attr-defined])
found LintFindings:src/agent_comms/historical_views.py at src/agent_comms/historical_views.py:560 (E501 Line too long (102 > 100))
found LintFindings:src/agent_comms/message_page.py at src/agent_comms/message_page.py:49 (E501 Line too long (103 > 100))
introduced NoneAbsence:src/agent_comms/historical_views.py at lines [566]
introduced NoneAbsence:src/agent_comms/message_page.py at lines [43]

@trissim trissim changed the title Index a page's evidence once and read live history once per traversal Keep page evidence and traversal work linear Oct 30, 2026
@trissim
trissim enabled auto-merge October 30, 2026 09:12
@trissim
trissim merged commit 0329da3 into main Oct 30, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants