From 9e1855ee8903ccec3a06effa95b4788308cb6212 Mon Sep 17 00:00:00 2001 From: Andy Eschbacher Date: Tue, 25 Aug 2026 20:41:18 -0400 Subject: [PATCH] chore: planning spike work on project threads --- .../adrs/0006-adopt-threads-as-task-dags.md | 158 +++++++++-- .../30-threads-and-task-dependency-graphs.md | 56 +++- ...ertical-threads-and-global-task-dag-mvp.md | 265 +++++++++++++++++- 3 files changed, 435 insertions(+), 44 deletions(-) diff --git a/planning/adrs/0006-adopt-threads-as-task-dags.md b/planning/adrs/0006-adopt-threads-as-task-dags.md index fa6f9dd..fdba14a 100644 --- a/planning/adrs/0006-adopt-threads-as-task-dags.md +++ b/planning/adrs/0006-adopt-threads-as-task-dags.md @@ -247,10 +247,15 @@ only; they list outside prerequisites as marked gates rather than silently treat Thread-owned execution. Section 11 bounds the deliberately deferred graph features, while the implementation slices preserve the planned TUI follow-up. -Graph implementation/package selection is a pre-implementation spike, not an ADR commitment. -Evaluate off-the-shelf packages against the operations above, determinism, API stability, -dependency weight, cycle diagnostics, and render support. Keep the selected package behind a -taskflow-owned analysis interface and use plain task IDs/domain values at every boundary. +The validation spike completed enough package research to avoid a second open-ended research task. +V1 defaults to the small owned algorithms proven by the spike. During the dependency-foundation +slice, run one bounded implementation-time bake-off: execute the same taskflow-owned contract tests +against the owned implementation and a thin [`dominikbraun/graph`](https://github.com/dominikbraun/graph) +adapter. Adopt the dependency only if it materially removes maintained code while preserving stable +task-ID ordering, attributable cycle diagnostics, and taskflow's error contracts; its explicitly +unstable v0 API counts against marginal savings. Gonum is not a second mandatory candidate unless +the required analysis surface grows beyond V1. No graph-library type may cross the taskflow-owned +analysis interface into domain, persistence, or wire contracts. ### 6. Document Layout & Frontmatter Specification @@ -294,11 +299,12 @@ Thread IDs reuse ADR-0003's task/research conventions: 12-character stable ID in and frontmatter, exact-ID resolution, drift lint, and rename-safe identity. The body is free-form narrative, not a generated graph or shared execution log. -### 7. Bulk Composition is a First-Class Use Case +### 7. Bulk Linking and Composition are First-Class Use Cases -Scoping often discovers a complete predicted task graph at once. Bulk composition is therefore -part of the first useful Thread release, not an afterthought. An authoring manifest may mix existing -tasks with complete specifications for new tasks: +Scoping often discovers the relationships among a set of already-created tasks at once. A manifest +can bulk-link those tasks into one new Thread and add their repository-global dependency edges in one +operation; it does not recreate or move the tasks. This is the primary V1 authoring path. The format +may also mix in complete specifications for new tasks as an optional composition convenience: ```yaml thread: @@ -307,10 +313,7 @@ thread: goal: Ship unified config CLI and TUI routes nodes: - key: config-model - new_task: - title: Define the unified configuration model - epic: 17-pm-go-cli - tags: [config, domain] + task_id: 6fjangd7kvh1 - key: existing-cli task_id: 6fjangd7kvh2 - key: legacy-gate @@ -334,6 +337,12 @@ dependencies: - Local keys are authoring conveniences only. Persisted Thread membership and task dependencies use stable IDs. +An authoring manifest is literal YAML; taskflow does not interpolate shell variables in it. In the +primary existing-task workflow, `task_id` contains the copied stable task ID—not a slug, title, or +`$VARIABLE`—and CLI/docs must make those IDs easy to discover. Compose resolves local keys and emits +only stable IDs into the materialized apply plan. `new_task` does not need to complicate the normal +bulk-linking workflow. + #### Compile, Then Apply A one-shot command that generates fresh IDs during each retry is not safe enough for a multi-file @@ -349,12 +358,14 @@ tskflwctl thread apply thread-apply.yaml --dry-run --json and task IDs, resolves every local key, validates exact task/epic references and normal task-creation invariants, and checks the proposed edge union for cycles. It writes a materialized apply plan before any planning entity is mutated. The plan is bound to the - planning-space identity and records stable IDs, intended creations, additive membership/edge - changes, and the preconditions used to calculate them. + planning-space identity and records stable IDs, exact intended creations, and additive + membership/edge changes. It is an intent log, not a frozen repository snapshot: unrelated + repository changes between compose and apply remain legal when current-state revalidation passes. 2. **Apply** revalidates the complete materialized plan, then converges the repository toward it. A missing planned entity is created with its preallocated ID; an identical creation or already present set addition is skipped; a same-ID/different-identity collision or stale conflicting - edit stops with `ErrConflict`. Reapplying the same plan cannot mint duplicate tasks. + edit stops with `ErrConflict`. Apply rechecks task-creation invariants and referenced epics rather + than trusting compose-time validation. Reapplying the same plan cannot mint duplicate tasks. Compose supports stdin for the authoring manifest, but a materialized plan must have a durable path before apply begins. A later convenience command may compose and apply in one invocation only @@ -368,6 +379,13 @@ per-entity/edge receipt, stops on the first conflict, and is safe to resume with It does not claim rollback or isolation from raw hand edits. `lint` must diagnose any remainder after interruption. +Every persisted prefix must itself remain a sound repository graph. In particular, apply cannot +create new tasks in stable-ID order when a new task may depend on another new task: an interruption +could leave a dangling reference that then makes fail-closed retry impossible. Apply writes new +tasks in deterministic topological waves (or uses an equivalent vertices-before-edges strategy), +adds dependencies to existing tasks only after all referenced new tasks exist, and creates the +Thread document last. + ### 8. CLI Surface ```bash @@ -430,24 +448,56 @@ keep the selected graph package, if any, behind the taskflow-owned analysis cont ### 10. Implementation Slices If the validation spike recommends acceptance and the decider accepts this ADR, production work can -be scoped into the following delivery slices without requiring the deferred features: - -1. **Dependency foundation:** add modeled `Task.DependsOn`, replace the legacy dependency field - registry, migrate the six current `blocked_by` files to stable IDs, add graph loading and - fail-closed global integrity/lint, and complete the graph-package spike. -2. **Dependency operations and eligibility:** add dependency mutation/query commands, deterministic - analysis projections, the shared transition guard across CLI/core call paths, and `--force` - receipts. -3. **Thread entity:** add document/store/core/wire/CLI support, many-valued membership, lifecycle, - rollup, external gates, frontier, and initialization migration from the unused Projects scaffold. -4. **Bulk composition and generated views:** ship compose/apply manifests with resumable receipts, - explanatory plans, and Mermaid/DOT rendering. This is part of V1 because known-ahead scoping is a - primary use case. -5. **Planned TUI follow-up:** after CLI and wire behavior have usage feedback, add a Thread list/tab +be scoped into the following dependency-ordered slices without requiring the deferred features: + +1. **Strict dependency read foundation:** add modeled `Task.DependsOn`, replace and migrate the + legacy dependency vocabulary, define the taskflow-owned graph-analysis contract, and add strict + graph snapshot/lint behavior. Run the bounded library bake-off here. Do not expose graph writes + while repository health and deterministic diagnostics are still unsettled. +2. **Portable guarded dependency writes:** establish the cross-platform repository mutation guard, + then add dependency add/remove, dry-run, blocker/downstream, and explanatory-plan operations. The + final read, cycle validation, and write remain one store-owned critical section. +3. **Eligibility enforcement:** centralize the transition policy and route every path into + `in-progress` through it, including generic move, start, create-and-start, and later TUI actions. + Ship `--force` only with an explanatory receipt. Do not release partial enforcement that callers + can bypass through another verb. +4. **Thread entity and projections:** add document/store/core/wire/CLI support, many-valued + membership, lifecycle, rollup, external gates, frontier, and initialization migration from the + unused Projects scaffold. This consumes the shared analysis contract rather than reimplementing + graph state. +5. **Bulk linking of existing tasks:** ship YAML compose/apply for existing stable task IDs, with a + durable materialized plan, planning-space binding, dry-run, resumable receipts, and safe retry. + The prototype's inline `new_task` path is an optional follow-up and is not required to prove the + primary bulk-linking workflow. +6. **Generated views:** add deterministic Mermaid/DOT and polish explanatory plans after the shared + projection has CLI usage. Generated output is never persisted as Thread state. +7. **Planned TUI follow-up:** after CLI and wire behavior have usage feedback, add a Thread list/tab and detail view showing lifecycle, rollup, frontier, member/external distinction, and a readable graph. The first TUI slice should consume the same projections and should not introduce direct graph editing or a separate readiness calculation. +### 10.1. Dogfooding and Rollout Policy + +Threads and task dependencies must manage their own remaining implementation work as soon as each +production safety boundary permits it. This is a product-validation policy, not an invitation to use +the spike adapter on canonical planning data: + +- After the strict read foundation lands, run its graph health and explanatory queries against this + planning repository before enabling writes. +- After guarded dependency writes land, sequence the remaining tasks in this epic with real + dependencies using the public commands. Do not invent dependencies merely to exercise a feature. +- After the Thread entity lands, make this initiative one of the first real Threads. Exercise shared + membership and external gates when actual work naturally has those relationships. +- After bulk linking lands, use a literal-YAML manifest to establish or extend the next real + initiative and retain its materialized plan until apply reports completion. +- Record confusing output, forced transitions, reopen behavior, merge conflicts, and recovery work + in the owning implementation task. Contract-level findings amend this ADR before TUI behavior is + treated as settled. + +Dogfooding never bypasses the slice exit gates. In particular, canonical planning data must not be +written by the experimental `threadspike` adapter; production migration, lint, locking, and wire +contracts must land first. + ### 11. Explicitly Out of Scope for This Decision - critical path, slack, effort weighting, target-date forecasting, and bottleneck scoring; @@ -456,7 +506,7 @@ be scoped into the following delivery slices without requiring the deferred feat - soft, OR, conditional, time-based, or cross-planning-space dependency edges; - stored diagrams, stored rollups, or a shared execution log inside the Thread file; - all-files rollback or claims of transactionality across Markdown documents; and -- choosing a graph package before the required-operation spike. +- adopting a graph package for speculative algorithms outside the V1 contract. ## Consequences @@ -500,6 +550,54 @@ be scoped into the following delivery slices without requiring the deferred feat - The best graph package may provide useful later algorithms, but package capability must not expand V1 scope or leak library types into persisted/public contracts. +## Validation Spike Commentary (2026-08-24) + +The bounded vertical prototype under `internal/threadspike` plus its experimental filesystem adapter +under `internal/store/threadspike.go` exercised the required scenarios against temporary Markdown +planning repositories. It is intentionally not wired into `core.Service`, production wire contracts, +ordinary CLI builds, or live planning data. A `-tags threadspike` build exposes a disposable manual +compose/apply/show/plan surface solely to exercise the prototype in throwaway spaces. + +**Recommendation: accept and scope implementation**, retaining this ADR's proposed status until the +decider signs off. The central model survived: one task-owned global DAG supports shared Thread +membership, external gates, deterministic frontier/topology, and separate lifecycle/gate state. +Recursive sound completion also behaves mechanically as specified when an upstream task is reopened, +although the human repair UX remains a follow-up to validate through use. + +The spike found five contracts that must be explicit in production: + +1. **Every interrupted apply prefix must be graph-valid.** Stable-ID write order is unsafe; new tasks + need topological creation order (or deferred edge attachment), and the Thread lands last. +2. **Apply-time validation is authoritative.** Compose-time success cannot justify creating a task + after its epic disappeared or accepting a hand-edited apply plan with invalid creation fields. +3. **Graph snapshots are stricter than ordinary resilient lists.** ID drift, unknown task status, + malformed files, missing dependencies, and missing Thread members must make graph mutation fail + closed even when ordinary listing can still return partial data for repair. +4. **The current lock proves the concurrent-cycle contract only on Unix.** Two opposite edge adds + were serialized correctly in the test harness, with exactly one accepted, but `writeLock` is a + documented no-op on non-Unix builds. Portable equivalent correctness is prerequisite work, not a + post-launch polish item. +5. **The authoring manifest and apply plan are different artifacts.** The manifest is user-authored, + literal YAML whose primary workflow bulk-links existing task IDs with local graph keys. Optional + inline `new_task` definitions do not change that contract. The apply plan is generated, + stable-ID-only recovery state. Shell interpolation may be useful in a test fixture, but it is not + part of either file format and should not obscure the primary YAML-first workflow in product + documentation. + +The package comparison does not justify a V1 dependency. [`dominikbraun/graph`](https://github.com/dominikbraun/graph) +is the closest fit—generic IDs, acyclic traits, stable topological ordering, cycle prevention, DOT, +and no transitive dependencies—but its own documentation says the v0 API is unstable, and taskflow +would still own its required diagnostics and all status/Thread semantics. [Gonum's graph/topo +packages](https://pkg.go.dev/gonum.org/v1/gonum/graph/topo) are mature and actively maintained, but +their generalized node API and broader module add adaptation cost without replacing taskflow's hard +contracts. Keep a small owned adjacency/analysis implementation for V1, subject only to the bounded +contract-test bake-off in Section 5; reconsider a broader package search if real usage asks for +materially richer rendering or algorithms. + +The spike's pure graph and compose/apply tests are promotable as contract tests. The experimental +mirror types, private store adapter, and failure-injection hook should be removed or rewritten behind +production domain and consumer-owned persistence ports as the implementation slices land. + ## Amendments _None yet (proposed)._ diff --git a/planning/epics/30-threads-and-task-dependency-graphs.md b/planning/epics/30-threads-and-task-dependency-graphs.md index de17eca..0120de1 100644 --- a/planning/epics/30-threads-and-task-dependency-graphs.md +++ b/planning/epics/30-threads-and-task-dependency-graphs.md @@ -14,15 +14,61 @@ created: "2026-08-24" Threads are not only a new first-class document. They change task dependency ownership, lifecycle eligibility, repository-wide graph integrity, multi-file composition, CLI and wire projections, and eventually the TUI. That cross-cutting domain deserves a coherent home rather than being split between the generic entity, storage, and CLI epics. -The first work in this epic is deliberately a decision spike. Production implementation should not be scoped until the spike gives ADR-0006 enough evidence to accept, revise, or abandon. +The first work in this epic was deliberately a decision spike. It recommends accepting ADR-0006 and scoping the production implementation; no named risk requires another spike. ADR acceptance and production task creation remain explicit follow-up decisions rather than implied consequences of the prototype. ## Decision gate -The spike must leave one explicit recommendation: +The spike left the explicit recommendation to **accept ADR-0006 and scope implementation slices**. +The ADR remains proposed until decider sign-off. After acceptance, the work follows the delivery +sequence below; this epic does not treat all Thread work as one implementation task. -- accept ADR-0006 and scope implementation slices; -- revise named decisions or contracts, then re-evaluate; or -- abandon the idea and record why the simpler existing model wins. +## Delivery sequence and gates + +```text +strict read model -> guarded edge writes -> eligibility enforcement + \-> Thread entity -> bulk linking -> generated views -> TUI +``` + +Eligibility enforcement and the Thread entity share the same graph foundation. They can be scoped +separately after guarded writes stabilize, but bulk linking waits for both dependency mutation and +Thread persistence. + +| Order | Slice | Exit gate | Highest-value stress tests | +|---|---|---|---| +| 1 | Strict dependency read foundation and legacy migration | One deterministic, fail-closed graph snapshot and lint contract; no public graph write yet | malformed/unreadable tasks, ID drift, unknown status, duplicate/self/missing edges, cycles, migration preserving body/frontmatter | +| 2 | Portable guarded edge writes and read queries | Final scan, validation, and write share one store-owned critical section on every supported platform | concurrent opposite edges, direct write versus bulk apply, stale CAS, same edge twice, removal during concurrent reads | +| 3 | Eligibility enforcement | Every route into `in-progress` uses one policy and produces the same blocker/force result | all task statuses, direct/transitive blockers, withdrawn/missing prerequisites, reopen after downstream completion, forced inconsistent work | +| 4 | Thread entity and projections | Membership and lifecycle persist independently from global edges; CLI and wire consume one projection | shared tasks, external gates and rollup denominators, empty/start/complete rules, abandoned/completed drift, membership conflicts | +| 5 | Existing-task bulk linking | One literal-YAML manifest can create a Thread, add memberships and global edges, and converge after interruption | failure after every write prefix, retry/idempotency, wrong planning-space identity, edited/stale plan, concurrent edge mutation | +| 6 | Generated Mermaid/DOT and explanatory UX | Stable ordering and explicit member/external roles; nothing generated is persisted | snapshot/golden output, escaping hostile titles, large/deep/wide readable graphs | +| 7 | Usage-informed TUI | TUI is a consumer of core/wire behavior, not a second graph engine | watcher reload during mutation, parity with CLI state, narrow/small-terminal degradation | + +### Design attention + +The mutation guard and strict-versus-resilient repository read split are the highest-risk design +work. They protect graph truth; no library can supply them. Lifecycle consistency is next: recursive +sound completion, forced starts, reopen behavior, and every status transition need one authoritative +policy. Bulk apply is a convergence protocol rather than a transaction, so interruption testing must +inject failure after every operation and prove that the same plan repairs the prefix. + +The graph library is lower risk. Keep a taskflow-owned interface and run a bounded contract-test +bake-off between the spike's small implementation and `dominikbraun/graph` during slice 1. Do not +create another open-ended research spike, and do not let library features pull critical path, slack, +or other deferred graph analysis into V1. + +## Dogfood checkpoints + +This epic is the first production consumer of its own capabilities: + +1. Slice 1 runs strict read-only analysis against this planning repository. +2. Slice 2 uses production dependency commands to sequence all remaining epic tasks. +3. Slice 4 creates a real Thread for the remaining initiative and observes its frontier and external + gates during normal implementation work. +4. Slice 5 uses bulk linking on the next naturally suitable initiative rather than a synthetic demo. +5. Every dogfood finding is recorded in the active task; contract changes also amend ADR-0006. + +The experimental spike binary is limited to disposable planning spaces and does not satisfy these +checkpoints. Dogfooding begins when the corresponding production slice passes its exit gate. ## Out of scope diff --git a/planning/tasks/6g3a1wtx4zrr-spike-a-vertical-threads-and-global-task-dag-mvp.md b/planning/tasks/6g3a1wtx4zrr-spike-a-vertical-threads-and-global-task-dag-mvp.md index f644296..900d60d 100644 --- a/planning/tasks/6g3a1wtx4zrr-spike-a-vertical-threads-and-global-task-dag-mvp.md +++ b/planning/tasks/6g3a1wtx4zrr-spike-a-vertical-threads-and-global-task-dag-mvp.md @@ -1,7 +1,7 @@ --- schema: 1 id: 6g3a1wtx4zrr -status: ready-to-start +status: completed epic: 30-threads-and-task-dependency-graphs description: Build a bounded, fixture-backed prototype of the dependency, Thread projection, and resumable bulk-composition contracts; recommend accepting, revising, or abandoning ADR-0006. effort: 2-4 days @@ -10,6 +10,9 @@ priority: high autonomy_level: 3 tags: [spike, threads, graph, planning-model, adr] created: "2026-08-24" +updated_at: "2026-08-25" +started_at: "2026-08-24" +completed_at: "2026-08-24" --- # Spike a vertical Threads and global task-DAG MVP @@ -54,14 +57,14 @@ The runnable test or demo fixture must include: ## Acceptance criteria -- [ ] A runnable, deterministic prototype or focused test harness demonstrates every required scenario against a temporary filesystem-backed planning repository. -- [ ] The graph implementation/package comparison records API fit, determinism, diagnostics, dependency cost, render support, maintenance signals, and what taskflow must still own regardless of package. -- [ ] The spike maps the actual production fan-out across domain, field registry/schema, store and mutation guard, core ports/use cases, CLI transitions, wire contracts, initialization/layout, lint, and later TUI work. -- [ ] The spike records every ADR assumption as validated, falsified, or still open, with evidence and any proposed replacement contract. -- [ ] ADR-0006 receives a concise spike-findings commentary section or proposed amendments, but its status is not changed by this task. -- [ ] The final task report recommends exactly one outcome: accept and scope implementation, revise and re-spike named risks, or abandon and preserve the simpler current model. -- [ ] If acceptance is recommended, the report proposes implementation slices with dependency order and identifies which prototype code should be promoted versus removed. -- [ ] Repository tests, formatting, lint, and diff checks pass for retained code; disposable prototype artifacts are removed or clearly isolated. +- [x] A runnable, deterministic prototype or focused test harness demonstrates every required scenario against a temporary filesystem-backed planning repository. +- [x] The graph implementation/package comparison records API fit, determinism, diagnostics, dependency cost, render support, maintenance signals, and what taskflow must still own regardless of package. +- [x] The spike maps the actual production fan-out across domain, field registry/schema, store and mutation guard, core ports/use cases, CLI transitions, wire contracts, initialization/layout, lint, and later TUI work. +- [x] The spike records every ADR assumption as validated, falsified, or still open, with evidence and any proposed replacement contract. +- [x] ADR-0006 receives a concise spike-findings commentary section or proposed amendments, but its status is not changed by this task. +- [x] The final task report recommends exactly one outcome: accept and scope implementation, revise and re-spike named risks, or abandon and preserve the simpler current model. +- [x] If acceptance is recommended, the report proposes implementation slices with dependency order and identifies which prototype code should be promoted versus removed. +- [x] Repository tests, formatting, lint, and diff checks pass for retained code; disposable prototype artifacts are removed or clearly isolated. ## Out of scope @@ -71,6 +74,250 @@ The runnable test or demo fixture must include: - Polishing graph visualization beyond enough output to validate the projection contract. - Treating benchmark guesses as evidence; measure only if the fixture exposes a credible concern. +## Spike report (2026-08-24) + +### Outcome + +**Recommend exactly: accept and scope implementation.** Keep ADR-0006 proposed until decider sign-off, +but no named risk requires another spike before scoping the production slices. The core model is +implementable and useful; the important corrections are bounded persistence contracts, not a reason +to retreat to flat Projects. + +The retained prototype is deliberately isolated in `internal/threadspike`; the adapter in +`internal/store/threadspike.go` reaches the production Markdown parser, frontmatter surgery, atomic +file replacement, content CAS, and repository lock without publishing Thread or dependency fields +through the current domain or wire APIs. Ordinary CLI builds remain unchanged; a +`-tags threadspike` binary adds only the disposable manual surface documented below. All fixtures +use temporary planning repositories. + +### Executable evidence + +- `TestThreadSpikeFilesystemProjectionCoversLifecycleExternalGatesAndValidation` loads real task and + Thread Markdown and demonstrates two Threads sharing tasks; a visible, denominator-excluded + external gate; queued-and-blocked, candidate-and-blocked, candidate-and-clear, force-started + inconsistent, and soundly completed projections; reopened-upstream inconsistency; and rejection + of self, missing, duplicate, and cyclic edges with attributable diagnostics. +- `TestThreadSpikeConcurrentOppositeEdgesCannotCommitACycle` starts opposite edge mutations from + the same repository state. The final scan/validation/write is inside the Unix repo lock: exactly + one write succeeds, one is rejected with the resulting cycle, and the repository remains a DAG. +- `TestComposeMaterializesExistingAndNewNodesIntoDurablePlan` compiles manifest-local keys into a + planning-space-bound YAML plan with stable Thread/task IDs, one existing node, two new nodes, an + external-only gate, and deterministic dependency order. YAML round-trip preserves the retry token. +- `TestThreadSpikeApplyIsDryRunnableAndConvergesAfterInterruption` exercises dry-run, injects failure + after the first real file write, verifies that prefix is still a readable DAG, then retries the + same plan to completion. A third apply performs only `already-applied` skips: no duplicate task, + Thread membership, or edge is created. +- `TestThreadSpikeApplyIsBoundToPlanningSpaceAndFailsClosed` proves repository identity binding and + stricter graph-snapshot health than ordinary resilient listing. +- `TestThreadSpikeTaggedCLIComposeApplyAndInspect` builds the opt-in command tree and exercises + compose, dry-run apply, real apply, list/show/plan, idempotent retry, and persisted dependencies + through the same CLI shape used by the manual playbook. + +### Assumption ledger + +| ADR assumption | Result | Evidence / replacement contract | +|---|---|---| +| One planning-repository DAG can back many Thread views. | Validated | Shared membership changes no edge ownership; both views derive from the same task snapshot. | +| An outside prerequisite can gate a member without entering Thread progress. | Validated | Typed `external: true` projection reports the direct gate while rollup counts members only. | +| Lifecycle and dependency health should be orthogonal. | Validated | Queued/blocked, candidate/clear, candidate/blocked, in-flight/inconsistent, and completed/drained remain unambiguous without a persisted blocked status. | +| Recursive sound completion reacts coherently to reopen. | Mechanically validated; UX open | Reopening an upstream task makes the completed descendant non-drained/inconsistent and a completed Thread inconsistent. Keep the rule; validate repair language through real use. | +| Per-file CAS plus final graph validation is sufficient. | Falsified | It is sufficient only when the final scan, validation, and write share one repository mutation guard. Separate CAS operations admit write skew. | +| The current repository lock proves that guard everywhere. | Falsified | Unix `flock` passes the concurrent opposite-edge test; the non-Unix implementation is explicitly a no-op. Production needs a portable equivalent or a clearly bounded platform contract. | +| Ordinary resilient task scans are sound graph snapshots. | Falsified | The normal parser deliberately returns ID drift/unknown statuses for diagnosis. Graph mutation must elevate those plus missing Thread members and bad dependency refs into fail-closed problems. | +| Any deterministic order is safe for interrupted bulk creation. | Falsified | Stable-ID order can write a dependent before its new prerequisite and make retry fail closed. New tasks must land in topological waves (or vertices first, edges later); existing-task edges follow; Thread last. | +| Compose-time validation can be trusted by apply. | Falsified | Epics or repository content can change and plans can be edited. Apply must revalidate current graph, exact create identity, all creation invariants, membership, and planning-space identity. | +| A repository snapshot hash/precondition is required for safe retry. | Not supported by evidence | Exact create intent plus additive operations and current-state revalidation safely tolerate unrelated edits. Conflict only when an intended identity changed or the current union is invalid. | +| A two-phase materialized plan prevents duplicate IDs after interruption. | Validated with the prefix-order amendment | IDs are minted once at compose; identical already-landed creations skip, collisions conflict, and retry converges. | +| A graph package could own the domain behavior. | Falsified | A package can supply traversal/topology/render helpers, but taskflow must own statuses, sound completion, external gates, membership, errors, storage, mutation guard, and apply convergence. | +| The graph size needs early optimization or benchmarking. | Still open, non-blocking | The algorithms are linear in vertices plus edges and the fixture exposed no credible concern. Measure only after real planning repositories provide scale evidence. | + +### Graph implementation/package comparison + +| Candidate | API fit and determinism | Diagnostics | Dependency/render cost | Maintenance signal | What taskflow still owns | +|---|---|---|---|---|---| +| Small owned adjacency model (prototype) | Exact string IDs; deterministic DFS, upstream/downstream traversal, and member-only topological waves; roughly 425 lines including domain projections. | Exact, stable cycle path plus task-specific self/missing/duplicate errors. | No module dependency; Mermaid/DOT are small textual adapters but were not polished in the spike. | Maintained with taskflow; smallest surface, but every algorithm is ours. | All semantic and persistence contracts. | +| [`dominikbraun/graph`](https://github.com/dominikbraun/graph) | Closest library fit: generics, directed/acyclic traits, prevention on edge add, stable topological sort, traversal, and transitive queries. | Useful library errors, but taskflow would still build stable user-facing cycle paths and broken-reference attribution. | Zero dependencies and built-in DOT support are attractive. | Active repository and substantial tests, but the maintainer explicitly says its v0 public API is not stable. | All lifecycle/gate/Thread projections, persistence, guard, receipts, and public error contract. | +| [Gonum `graph/topo`](https://pkg.go.dev/gonum.org/v1/gonum/graph/topo) | Stabilized topological sort, path queries, directed cycles, and SCCs are strong; generalized `int64` node identity needs an adapter for stable string task IDs. | Rich cycle/SCC data, still requiring deterministic task-ID translation and taskflow wording. | Mature but much broader module; DOT support exists elsewhere in Gonum. | Active, scheduled releases and cross-platform testing; still pre-v1. | The same taskflow-specific surface; replaces only generic algorithms. | + +**Selection recommendation:** own the small V1 algorithms behind a taskflow analysis interface. Do +not add either dependency yet. A production-foundation task may spend at most a bounded bake-off on +a `dominikbraun/graph` adapter by running the same taskflow contract tests against both +implementations; adopt it only for meaningful code reduction without weaker determinism or +diagnostics. This is not another research spike. Gonum is justified only if analysis expands +substantially. + +### Actual production fan-out + +| Seam | Current concrete touchpoints | Required production work / risk | +|---|---|---| +| Domain and validation | `domain.Task`, status vocabulary, `ActiveTaskFieldErr`, entity descriptor | Model sorted `DependsOn`; add Thread/status/validation; keep gate state derived. Avoid renaming legacy `projects` to `threads`. | +| Field registry and schema | `domain/fields.go`, task struct/registry sync tests, entity `AuthoringFields`, `schema` render | Replace legacy `dependencies`/`blocked_by`/`blocks` vocabulary with canonical `depends_on`; advertise Thread authoring separately; preserve unknown-field migration behavior. | +| Filesystem store | `FS`, task parse/create/set/edit/move, entity scanners, atomic create/replace, content CAS | Add strict graph snapshot and Thread store. Dependency mutation must be surgical and additive/removal-aware; raw generic `task set` cannot bypass global validation. | +| Mutation guard | `store/lock_unix.go`, no-op `lock_other.go` | Expose a store-owned graph mutation operation/callback so final scan, validation, and write cannot be split across core calls. Close or explicitly bound the non-Unix correctness gap. | +| Core ports/use cases | Large `TaskStore`/`Store`, `Service.Move`, `NewTask` | Prefer narrow consumer-owned Thread/graph ports so every fake does not immediately grow. Centralize eligibility before every transition into `in-progress`; return forced blocker metadata. | +| CLI and transitions | `internal/cli/task.go`, lifecycle aliases, global dry-run/output modes, completion | Add dependency queries/mutations and Thread commands; route generic move, start, TUI move, and future starts through one guard; add compose/apply durable-plan UX. | +| Wire/schema/rendering | `internal/wire` DTO/envelope registries and schema-version goldens; `cli/render` | Add stable DTOs for role/gate/external/blockers/forced receipts/Thread views and bulk receipts; bump/regenerate contracts rather than leaking a graph-library type. | +| Initialization/layout/health | `domain/layout.go`, entity descriptor, `config.Init`, discovery, `FS.WatchPaths` | Scaffold/watch `threads/`, stop creating new `projects/`, and refuse automatic removal of non-empty legacy content. Planning identity—not implementation repo path—binds apply plans. | +| Lint and migration | resilient scans, core lint, frontmatter diagnose/fix, six live `blocked_by` users | Add global duplicate/self/missing/cycle/ID/status/member checks; provide an explicit stable-ID migration or actionable legacy lint before mutation goes live. | +| Later TUI/atlas | watcher-driven reload, task transition actions, per-entity list/detail rendering | Consume the same core projection and eligibility guard. Add Thread views after CLI feedback; do not recompute readiness or edit graphs directly in the TUI. | + +### Implementation slices and prototype disposition + +1. **Portable graph foundation and migration:** define the production graph snapshot/analysis + interface and mutation-guard contract; add modeled `depends_on`, strict graph lint/load, and the + legacy-field migration. Prove Unix plus the supported non-Unix strategy before public edge writes. +2. **Dependency use cases and transition eligibility:** ship guarded add/remove, blocker/unblocks and + deterministic plan queries; enforce eligibility across every start path with explanatory `--force` + receipts. +3. **Thread entity:** add descriptor/domain/store/core/CLI/wire coverage, many-valued membership, + lifecycle/rollup/frontier/external-gate views, and Projects-scaffold migration behavior. +4. **Bulk compose/apply:** first promote existing-task bulk linking, materialized-intent validation, + planning-space binding, idempotent retries, conflicts, dry-run, and machine receipts. Keep inline + task creation and its topological prefix ordering optional until the simpler workflow has usage. +5. **Generated views, then TUI:** add Mermaid/DOT from the shared projection; only after usage feedback, + add read-oriented Thread list/detail/frontier views to the TUI. + +Promote the pure graph contracts and tests, the materialized-plan shape, apply-time validation rules, +topological prefix ordering, and the interruption/concurrency tests. Rewrite the experimental types +against production domain DTOs and narrow ports. Remove the private store adapter and `AfterWrite` +failure hook after their behavior is covered at production seams; they are spike scaffolding, not a +second persistence architecture. + +### Prototype retention decision + +The current build tag hides only the experimental CLI command. `internal/threadspike` and +`internal/store/threadspike.go` are untagged, so they compile—and their contract tests run—in an +ordinary build. They are internally scoped and unreachable from the normal command tree, but this is +not full build isolation and would create a second model/store path that can drift if retained +indefinitely. + +Default integration disposition: + +- retain and merge the ADR, epic, and this completed spike report; +- preserve the prototype on its spike branch as executable evidence; +- port the graph, concurrency, and interruption scenarios into production contract tests as their + owning slices land; and +- rewrite useful algorithms behind production domain types and narrow ports rather than promoting + the experimental package wholesale. + +If a short-lived tagged binary must be retained in the main branch for further disposable-space +evaluation, first apply `//go:build threadspike` consistently to the graph package, store adapter, +and their tests, and give that scaffolding an explicit removal milestone. Even then, a tagged binary +can mutate any selected planning root, so it must not be used on canonical planning data. + +### Validation + +- `go test -race ./...` — passed, including the temporary-filesystem, interruption, and Unix + concurrency harnesses. +- `golangci-lint run ./...` — passed with 0 issues. +- `go mod tidy -diff` — passed with no module-file changes; the spike added no graph dependency. +- `go run ./cmd/tskflwctl -C . lint` — all active tasks and epics pass planning lint. +- `git diff --check` — passed. +- `go test -tags threadspike ./...` — the opt-in CLI variant and its end-to-end playbook test pass; + the untagged full suite separately proves ordinary builds remain unchanged. + +### Manual throwaway-space playbook + +The spike now provides an explicitly experimental `thread` command only when `tskflwctl` is built +with the `threadspike` tag. Ordinary builds remain unchanged. This surface is for disposable data; +it is not a production CLI or wire contract. + +Build one reusable binary and initialize a throwaway space: + +```bash +cd ../taskflow-threads-spike +go build -tags threadspike -o /tmp/tskflwctl-threads ./cmd/tskflwctl + +export TSK=/tmp/tskflwctl-threads +export PLAY=/tmp/taskflow-threads-play +mkdir -p "$PLAY" +"$TSK" init --path "$PLAY" --no-register +``` + +Create two epics and five tasks (`jq` captures the stable IDs): + +```bash +CORE=$("$TSK" -C "$PLAY" epic new "Core delivery" --description "Build the core in dependency order" --tags threads --json | jq -r '.created.id') +DOCS=$("$TSK" -C "$PLAY" epic new "Documentation" --description "Explain and validate the result" --tags threads --json | jq -r '.created.id') + +GATE=$("$TSK" -C "$PLAY" task new "External decision" --epic "$CORE" --tags threads --json | jq -r '.created.id') +SCHEMA=$("$TSK" -C "$PLAY" task new "Define schema" --epic "$CORE" --tags threads --json | jq -r '.created.id') +STORE=$("$TSK" -C "$PLAY" task new "Build store" --epic "$CORE" --tags threads --json | jq -r '.created.id') +CLI=$("$TSK" -C "$PLAY" task new "Expose CLI" --epic "$CORE" --tags threads --json | jq -r '.created.id') +GUIDE=$("$TSK" -C "$PLAY" task new "Write guide" --epic "$DOCS" --tags threads --json | jq -r '.created.id') +``` + +The variables above belong to this shell walkthrough, not to the Thread manifest format. The +unquoted heredocs below substitute them while writing literal task IDs into each YAML file. A saved +manifest used directly with `thread compose` must contain its actual existing-task IDs, or declare +the tasks inline with `new_task`. + +Compose and apply a linear Thread with an external gate: + +```bash +cat > "$PLAY/core.thread.yaml" < "$PLAY/docs.thread.yaml" <