feat(search): preserve indexing progress and report snapshot coverage

.sdd/graph/2026/09/06-113911-s-cpt-ikx.md

@@ -0,0 +1,35 @@
+---
+type: signal
+layer: conceptual
+kind: insight
+refs:
+    - id: 20260422-122317-d-stg-qlt
+      kind: grounded-in
+    - id: 20260905-111056-s-cpt-ne0
+      kind: related
+    - id: 20260603-172628-d-cpt-fbi
+      kind: related
+    - id: 20260617-182152-s-cpt-yve
+      kind: grounded-in
+    - id: 20260727-224047-d-cpt-u8o
+      kind: related
+participants:
+    - Christopher
+confidence: low
+topics:
+    - collaboration/coherence
+    - collaboration/external-systems
+summary: 'An extensible project event stream is proposed as a conceptual insight: occurrences (CI runs, deployments, graph entry creation) could be recorded automatically, while consumers connect selected events with graph context for durable capture, notifications, reporting, and decisions-governed action. The idea is grounded in the shared-context aspiration (d-stg-qlt) and the reasoning-record framing (s-cpt-yve), complements the graph-change subscription gap (s-cpt-ne0), and raises an uninterpreted-occurrence boundary with the intake model (d-cpt-fbi). It deliberately does not revise the graph-as-only-durable-record directive (d-cpt-u8o), and remains low-confidence, uncommitted, and awaiting concrete integration scenarios for validation.'
+---
+
+An extensible project event stream could complement the SDD graph by recording occurrences automatically, while consumers connect selected events with graph context to support durable capture, notifications, reporting, and action governed by recorded decisions.
+
+Christopher raised the idea from a need for better visibility across participants, serving the shared-context aspiration (d-stg-qlt). Git-synchronized WIP markers convey limited activity, while focus decisions express commitments of attention. The proposed stream reaches beyond work claims: SDD itself, working sessions, and external integrations could publish events such as exploration starting, CI running, deployments completing, or graph entries being created. Events record what happened without confirmation, mostly through tooling.
+
+Sessions could query the stream for current context. Background consumers could review changes periodically, for example hourly, combine them with graph context, and synthesize what merits durable capture. Events need not become entries individually. Autonomous action would draw its authority from durable decisions, potentially expressed as procedures; an event supplies the occasion to act, not permission. Entry creation could itself publish an event, connecting the graph back to consumers.
+
+Consumers could also turn fine-grained events into notifications and reports. Procedures could determine how events are combined with relevant graph material, which people can act on the resulting information, and how to make any need for action explicit. IT infrastructure monitoring and software development are examples; the intended applicability extends to businesses and projects generally. How to select, combine, and route events without overwhelming participants remains open.
+
+This suggests a broader integration foundation alongside the graph-change subscription gap (s-cpt-ne0). It also raises a boundary with the intake direction (d-cpt-fbi): incoming occurrences could remain useful before any consumer interprets them into graph material. The distinction between a reasoning record and external execution records already raised in s-cpt-yve helps frame that boundary. A persistent, replayable project history would need reconciliation with d-cpt-u8o's declaration that the graph is the only durable record; this insight does not revise that directive.
+
+The idea remains unvalidated and deliberately uncommitted. Replay could help reconstruct project activity, but its scope, retention, event semantics, consumer execution, notification and reporting surfaces, and the limits of autonomous authority remain unsettled. Concrete integration scenarios should test whether the stream improves shared awareness and graph context without duplicating reasoning or expanding SDD into a general automation platform or ERP.

.sdd/graph/2026/09/06-121218-d-tac-ccm.md

@@ -0,0 +1,51 @@
+---
+type: decision
+layer: tactical
+kind: plan
+refs:
+    - id: 20260905-112352-d-tac-nhu
+      kind: builds-on
+    - id: 20260902-154750-d-tac-o1s
+      kind: builds-on
+    - id: 20260413-142536-d-cpt-ah1
+      kind: grounded-in
+    - id: 20260905-111056-s-cpt-ne0
+      kind: related
+participants:
+    - Christopher
+confidence: high
+topics:
+    - search/index
+    - portability/runtime
+    - llm/providers
+    - llm/observability
+    - reliability/testing
+summary: 'This tactical plan commits to incremental entry indexing: entry publication, provider transport, and search coverage are separated so that independently retryable entry versions make durable progress while concurrent callers share bounded provider batches and search retrieves against a fixed snapshot target, avoiding whole-project embedding runs that can exceed job deadlines without retaining progress. It extends the independent reconciliation operation (20260905-112352-d-tac-nhu) and the per-request batching composition (20260902-154750-d-tac-o1s), applies the mandatory CQRS decomposition contract (20260413-142536-d-cpt-ah1), and explicitly leaves the durable graph-change subscription gap (20260905-111056-s-cpt-ne0) open.'
+---
+
+Indexing makes durable progress through independently retryable entry versions while concurrent callers share bounded provider batches and search reports coverage of a fixed snapshot target.
+
+Whole-project embedding followed by one publication can exceed a job deadline without retaining useful progress. This extends the independent reconciliation operation in 20260905-112352-d-tac-nhu and the per-request batching composition in 20260902-154750-d-tac-o1s by separating entry publication, provider transport and search coverage. Increasing project deadlines does not remove the failure mode.
+
+The [design record](./06-121218-d-tac-ccm/entry-indexing-design.md) carries the source-lifetime contract, public capability proposals, publication semantics, provisional settings and validation exercise. Following 20260413-142536-d-cpt-ah1, commands and handlers own indexing writes, queries and finders own discovery and coverage reads, pure computation stays below orchestration, and the application composes these boundaries. The shared SDD application selects authorized home and dependency snapshots, lets consumer preparation run against that fixed target, then derives actual coverage from published entry versions and retrieves against the same snapshots. Consumers own scheduling, queue state, retries and waiting policy. Preparation returns normally or fails explicitly; only SDD determines completeness and renders coverage metadata and the readable notice.
+
+## Acceptance criteria
+
+- [ ] Discovery yields entry requirements incrementally without preparing all chunks, and releases control on early stop, cancellation or error.
+- [ ] A revision-bound cursor resumes in stable entry-ID order without rehashing earlier entries; replayed enqueue groups preserve logical work identity.
+- [ ] An indexing attempt uses its exact retained entry and attachment source, with project, full entry hash and index configuration distinguishing publication identities.
+- [ ] Complete-entry publication, including zero-chunk entries, is atomic for completion queries and retrieval; unpublished chunks are unsearchable.
+- [ ] Retrying a published version skips embedding, and concurrent attempts converge without one version suppressing another.
+- [ ] Interrupted or failing entries preserve completed siblings, and an entry larger than a provider batch can eventually publish in full.
+- [ ] Shared batching bounds buffering, payloads and provider concurrency, routes vectors correctly, and flushes partial batches without further arrivals.
+- [ ] Oversized requests make incremental admission progress; cancellation and shutdown release callers without stranding other participants.
+- [ ] Each actual provider call records usage once with batch-to-caller correlation, while query embeddings avoid the document backlog.
+- [ ] Search selects authorized home and dependency targets before preparation, observes a revision including the caller's successful write, and retrieves against those same targets.
+- [ ] After consumer preparation, SDD alone derives complete/incomplete coverage from published entry versions and renders result metadata and the readable notice; ordinary consumer wait expiry makes no completeness claim, incomplete zero-match differs from complete no-match, and preparation failures remain explicit errors.
+- [ ] Executable external-consumer and local examples demonstrate one preparation composition point through the existing application and MCP search path before its public signature is finalized.
+- [ ] Existing routing and deadline composition are assessed for query separation and provider timeouts, keeping those policies outside the batcher where sufficient.
+- [ ] Manifest-read amplification and unpublished-candidate filtering costs are measured and addressed without weakening atomic visibility or hiding published results.
+- [ ] A persistent multi-entry interruption exercise demonstrates progress across attempts, skipped completed versions and eventual completion; provider concurrency and latency measurements justify recommended settings.
+- [ ] Local execution remains supported, repository and example-module checks pass, and delivery identifies the commit, release status, validation evidence and external consumer adoption steps.
+
+The useful primitives in checkpoint 0af4e1d7 remain the starting point. Write-triggered integration starts from the existing mutation finalizer and graph-write adapter; the consumer chooses to trigger indexing and guarantees durable scheduling through its write/recovery protocol. No within-entry checkpoints, cross-process batch aggregation, queue implementation, parallel prepare/execute search API or new event subsystem is included. The broader subscription gap 20260905-111056-s-cpt-ne0 stays open. Completion is recorded through the implementation move with a done signal confirmed by the user.