Skip to content

Measure selected native acquisition in its own input lifetime - #3605

Merged
trissim merged 3 commits into
mainfrom
work/p947-input-measurements
Oct 29, 2026
Merged

trissim merged 3 commits into
mainfrom
work/p947-input-measurements

Conversation

@trissim

@trissim trissim commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Three commits; twelve production modules and one test import; +163 / -109 lines. Selected acquisition diagnostics go from one recipient-lifetime counter borrowed across inputs to one fresh PublicationMeasurements per reserved PrivateSendClaim. Ten selected cursor-measurement accesses are removed; zero selected NativeStartup instances borrow cursor counters. Cursor-lifetime observation counters remain with CursorDelivery.

This is P947's authorized parent-side measurement dependency, not a latency optimization or completion of the performance target. No tests, application/provider inputs, probes, installation, native JS/build/pin, SQL/schema or proof changes were performed.

Two acquisitions were wearing one input ID. The stopwatch now has its own reservation.

Evidence motivating the repair

Original idle input 38559 had at least 820.077 ms (unrounded 820.076777 ms) after native_open returned and before writer join could start. Kernel-write completion was 835.511940–850.947103 ms after that return. Acquisition itself was 117.132108 ms, including 14.015580 ms spawning; none of that establishes RPC readiness.

These are retained measurements and parent-only bounds, not a prediction that all of that time can be removed. The busy control's 10.312 seconds belongs to its already-running model request and is not targeted or interrupted here.

Before / after owners and lifetimes

Owner / consumer Before After
PrivateSendClaim.reserve Reservation/binding had no input-owned stopwatch Creates exactly one existing measurements instance before reservation, records binding on it and passes it into the resulting claim
PrivateSendClaim.execute, _start_once Borrowed recipient cursor counters for startup, eight grant operations and final report Same claim-owned instance goes through startup, grant, writer and final acquisition report
CursorDelivery Genuine cursor observations plus unrelated selected acquisitions shared one lifetime Keeps genuine cursor observations; selected code never resets, copies or subtracts those counters
NativeSession.acquire_native Child acquisition timed, actual tool binding unseparated Shared base times UnboundToolTransport.bind_process for both real callers: generic NativeSession.run and TrackedTurnSession.complete
TrackedTurnSession.execute / complete Launch await included worker scheduling; draft acquisition unseparated Each existing worker callback records actual execution inside the surrounding await span; draft measures context entry, not its open lifetime
_write_fenced Individual raw writes only One outer write-loop interval plus existing nested raw-write counters, captured in the actual writer thread after grant
RequestObservations Acquisition report named an input but included other inputs' counters Original input and turn/owner lease, one input's counters, and the actual acquired native process observation; emitted after existing cleanup
PublicationMeasurements declaration Low-level coordination imported the value through high-level diagnostics, preventing a runtime turn-fenced report contract One defining class in stdlib-only publication_measurements.py; all consumers import it directly, while diagnostics imports the real report argument owners

No new family, worker, timing framework, receipt, phase, database row, send authority, readiness wait or retry. NativeStartup.native_process is a diagnostic observation copied from native.proc.identity, never a permission check. NoProcess means acquisition did not return a child; it does not assert that a failed spawn had no external side effect. The process identity remains meaningful after the child's resources close.

Span meanings and limits

All parent observations use time.monotonic_ns(); worker bodies capture their clocks inside their worker, not around scheduling alone. Existing PublicationMeasurements gains first_started_ns / last_finished_ns while retaining count/total/max and the maximum's original endpoints. First/last cover observed bodies, including exceptional exits, not an acknowledgment or proof of delivery.

  • input_reservation_binding: complete Coordination.run_async await. input_reservation_binding_worker: the actual reservation and exact prompt binding callback; the outer-only portions also include coordinator connection/transaction work, not just scheduling. A failed reservation/binding retains existing failure/no-replay behavior and does not construct a claim/report.
  • selected_registry_snapshot / _worker: awaited registry acquisition versus actual callback execution.
  • native_launch_selection / _worker: awaited existing launch selection versus actual worker execution.
  • native_open and native_spawn: unchanged distinct acquisition/spawn operations; spawn remains nested where applicable and is absent on actual reuse. Neither establishes native RPC readiness.
  • native_process_binding: actual bind_process await, including listener startup; shared at the existing acquisition owner, not a selected-only copy.
  • draft_publication_acquisition: await of existing worker entry into DraftRow.publication; draft_publication_worker: actual ExitStack.enter_context body. Neither includes the draft's subsequent lifetime or retirement.
  • prompt_writer_join: existing owner-loop call through serialization, writer startup and joined completion. Existing claim_wait, prompt_locks and the grant operations remain distinct and nested.
  • prompt_write_loop: one interval around the whole OS-write loop after the one-use grant. It includes partial writes, retries of a nonblocking syscall after readiness waits, loop bookkeeping and cancellation/deadline checks. Existing nested raw_pipe_write counters retain individual syscall resource meaning; a failed/nonblocking syscall is still an observed syscall, not bytes-delivered proof. An exceptional outer interval is not a successful write; unchanged failure classification owns that answer.
  • native_child_retirement: unchanged cleanup interval. Final record_acquisition_progress remains after the tracked session's resources and writer have been joined/closed; retained-child cleanup does not mean a reused idle process was killed. The final report can therefore follow first context by the rest of the normal turn.

Nested spans must not be summed as disjoint costs. First/last permit the actual surrounding scheduling gaps to be inspected without cumulative deltas or borrowing earlier maxima. No measurement changes a fence, timeout budget, token, write ordering, cancellation, UNKNOWN/known-not-written classification, or replay rule.

Dependency correction and complete report contract

The second typing correction exposed an actual import cycle: diagnostics -> turn_lease.NativeCapture -> coordination_session -> diagnostics.PublicationMeasurements. The failed cold-import traceback is retained in p947-report-types-import.log; the unpublished failure was removed before pushing. Root's clean correction c64ac3904658 authorized fixing that dependency at its existing owner.

Moved the single existing PublicationMeasurements class to publication_measurements.py, which imports only stdlib declarations. All fields and method bodies remain AST-identical; the existing operation and measuring methods now declare their return contracts (AbstractContextManager[None] and Iterator[None]). No new timing class, wrapper, alias, compatibility export, logger, schema or store.

Migrated every consumer directly: coordination_session, cursor_publication, diagnostics, native_child, native_prompt_send, native_startup, private_send_claim, turn_progress and the existing native-send test. The native-child type import is now a normal runtime import of the dependency-light owner. The test has only an import-location change, with no test execution or changed behavior.

The full report contract is now:

record_acquisition_progress(
    root: Path,
    lease: TurnLeaseFence,
    input_id: InputId,
    measurements: PublicationMeasurements,
    native_process: ProcessState,
) -> None

All argument owners are actual runtime imports. The earlier callback-only check missed the added parameter on this existing function; the final check enumerates added parameters as well as added definitions against main. The added native_process parameter is typed, as are all three callbacks and both relocated methods. No new TYPE_CHECKING, cast, Any, ignore or forwarding wrapper.

Final evidence:

  • p947-measurement-move-before.json / -after.json: 1,193 → 1,194 modules, one class → one class, 8 → 9 direct defining imports (diagnostics becomes a consumer), zero old defining imports, zero parse omissions. All 1,194 final source hashes match. Dataclass fields and method bodies are unchanged.
  • p947-measurement-final-check.log: seven separate fresh processes cold-import publication_measurements, diagnostics, coordination_session, turn_lease, native_startup, private_send_claim and tracked_turn successfully. Import order cannot conceal the original cycle.
  • Existing input-owned spans and send bodies are unchanged. Root reported eight real-pipe/UNKNOWN safety cases passed at the preceding head; they were not repeated for this declaration/import correction. Hosted checks and one fresh installed first-context qualification remain root-owned. Instrumentation is not installed or a demonstrated speedup.

Typed-contract correction after send-back

The initial delivery omitted return annotations on the three newly added worker callbacks. Imports and parsing did not prove their typed contracts; that was my miss under disallow_untyped_defs.

  • snapshot_registry() -> RegistrySnapshot
  • select_launch() -> NativePiRpcLaunch
  • acquire_draft() -> DraftPublication
  • Existing producer DraftRow.publication(cls, root: Path, input_id: InputId) -> Iterator[DraftPublication], so ExitStack.enter_context retains the actual result type.

All three new definitions were enumerated against current main with NRA, including parameter/return annotations. Runtime owner imports and the producer's complete signature resolve through declaration introspection. No new TYPE_CHECKING imports, cast, Any, ignores or wrappers. Timing and send bodies are unchanged by this correction. No mypy, pytest, application or provider run is claimed.

Targeted reproof: p947-type-contracts.json and p947-types-corrected-check.log retain all seven changed file hashes, the three new definitions, concrete producer parameters/result and zero parse omissions. They superseded the initial inventory's hashes at the callback-correction checkpoint. The final declaration/import move is covered by p947-measurement-move-after.json; earlier artifacts remain dated source observations, not current-hash claims.

Source inventory and deletion evidence

Retained prior map: latency-owner-ast.json plus the 19-file installed-source trace. Fresh before/after evidence uses NRA ModuleSyntaxIndex, covering 1,193 modules under all src, tests, tools, zero parse omissions; 358 → 383 syntactic reference sites. Counts are source leads, not repair scores or dynamic dispatch proof.

Artifacts in /home/ts/.cache/agent-scratch/awareness-schema/:

  • p947-source-before.json, p947-source-after.json: source hashes, declarations/inheritance/imports, writes and producer/consumer reference sites, with lexical owners.
  • p947-source-proof.json / .log: all 1,193 after-inventory hashes matched the initial source checkpoint; one reservation factory; zero selected startup cursor borrows; one acquisition-report consumer. Removing only the added write-loop timing context made _write_fenced's AST exactly equal to that main. These are retained original observations, not assertions that changed-file hashes stayed equal after the typed-contract correction.
  • p947-source-validation.log: six changed modules import successfully; git diff --check passes. No pytest, ratchet, app or provider run.

Syntactic names/attributes do not resolve arbitrary dynamic imports or callbacks. Existing partials, worker callbacks, the two generic acquisition consumers, independent/selected launch implementations and cleanup lifetimes were read semantically. Native dependency relationships are retained from the prior trace; native sources are unchanged and outside this checkpoint. One-shot inventory/proof executables are retired before delivery, with their hashes retained in the result record.

Deleted authority mistake: IDEN-3 — fact: timing belongs to this selected input; at the former PrivateSendClaim.execute startup/report and _start_once counters; derive: the existing claim's reservation lifetime; by: one claim-owned PublicationMeasurements, passed unchanged to every selected consumer. No counter reset, clone or snapshot subtraction.

Concurrent work

Checked #3572, #3566 and #3538 before edits. Claude confirmed no unpublished changes in these acquisition/writer files. #3572's read-basis/context-refusal changes overlap private_send_claim.py, tracked_turn.py and backend.py; they are independent semantic changes to preserve, not replace. Neither the read-basis authority nor those branches/worktrees were edited. This branch was rebased onto current main before the corrected delivery; conflicting semantic changes from the outstanding PRs must still be preserved when they integrate.

Root-owned validation and remaining gaps

Root reported the combined eight real-pipe/admission/UNKNOWN safety controls passed on the preceding head. Their bodies and timing behavior are unchanged by the declaration/import correction, so those controls were not repeated. Root owns hosted gates, installation and one fresh installed nonce input joined to its first original model context. Source equivalence and cold imports are not installed qualification.

Still absent and deliberately outside this checkpoint: native RPC-reader-ready, native prompt entry, message construction timestamps; following input's StreamWriter.write is enqueue, not kernel write; provider receipt/token production and historical send-time eligibility. Following transport is unchanged. The measured dominant parent span must choose the eventual optimization; P947 remains OPEN.

Assumed: existing PublicationMeasurements remains diagnostic-only; actual process identity is available when acquisition returns (as existing native progress reporting already requires); root performs runtime and installed qualification before any performance or live-readiness claim.

@trissim
trissim marked this pull request as ready for review October 29, 2026 14:59
Copilot AI balanced review requested due to automatic review settings October 29, 2026 14:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

debt-src at cbce7b5, every finding:

::warning file=src/agent_comms/native_prompt_send.py::LintFindings +1 (warns, never blocks)
grew UnannotatedParameter:src/agent_comms/diagnostics.py +1
found UnannotatedParameter:src/agent_comms/diagnostics.py at src/agent_comms/diagnostics.py:100 src/agent_comms/diagnostics.py:100 src/agent_comms/diagnostics.py:100 src/agent_comms/diagnostics.py:100 src/agent_comms/diagnostics.py:100 src/agent_comms/diagnostics.py:116 src/agent_comms/diagnostics.py:116 src/agent_comms/diagnostics.py:116 src/agent_comms/diagnostics.py:116 src/agent_comms/diagnostics.py:116 src/agent_comms/diagnostics.py:137 src/agent_comms/diagnostics.py:137 src/agent_comms/diagnostics.py:137

@trissim
trissim force-pushed the work/p947-input-measurements branch from 5729c80 to 48b3026 Compare October 29, 2026 15:14
@trissim
trissim added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit 4ca5c19 Oct 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants