Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/validate-reports-purpose-placeholder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
"@fission-ai/openspec": patch
---

`openspec validate` now reports a `## Purpose` that is still the placeholder archive writes for a new capability, instead of passing it. The placeholder is longer than the 50-character brevity floor, so until now the one check meant to catch a Purpose nobody wrote was satisfied by the exact text saying nobody wrote one — a spec whose Purpose read `Does stuff.` failed `--strict` while a spec whose Purpose said nothing at all passed. A capability could carry the placeholder indefinitely while every command reported success.

It is a warning, so a project that already has placeholders on disk keeps validating by default and only `--strict` fails. The message says to edit the main spec directly, since a `## Purpose` in a delta is read only when the capability is created and cannot replace an existing one.

Detection is narrow. The placeholder archive generates is recognised through the same definition that writes it, wherever it appears in the Purpose. Otherwise only a `TBD` or `TODO` opening the Purpose counts, so `The retry budget is TBD pending benchmarks` is still a valid Purpose and a word like `TBDs` is not a marker. Fenced code inside the Purpose is quoted material rather than the Purpose speaking, so a spec that documents the placeholder keeps passing. An empty Purpose is unchanged, and a Purpose reported as a placeholder is no longer also reported as too brief, so a bare `TBD` yields one finding rather than two.

`openspec archive` is unaffected: it validates rebuilt specs without `--strict`, so a spec archive writes still passes the validation it would have passed before, and the text archive writes is unchanged.
2 changes: 2 additions & 0 deletions openspec/changes/warn-on-purpose-placeholder/.openspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-08-15
162 changes: 162 additions & 0 deletions openspec/changes/warn-on-purpose-placeholder/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
## Context

See proposal.md — Why. What shapes the approach here is where the two ends
already sit:

- `buildSpecSkeleton` composes the placeholder inline, interpolating the change
name. It is generated text with no name of its own.
- `applySpecRules` is the single place both spec entry points converge —
`validateSpec` (a file) and `validateSpecContent` (a rebuilt spec, called by
archive). A rule added there reaches the CLI and archive at once, so the
blast radius on archive has to be answered rather than assumed.
- Strict mode is already defined as "warnings fail": `createReport` treats a
warning as invalid only when `strictMode` is set. Severity is therefore a
choice between two existing behaviors, not a new mechanism.
- `task-numbering.ts` establishes the shape for a check like this: a pure module
under `src/core/validation/` returning findings, mapped to issues at one call
site in the validator.

## Goals / Non-Goals

**Goals:**

- Report the placeholder without changing what any command does today by default.
- Recognise placeholders already on disk, including ones written by earlier
versions, since those are the ones that have lingered longest.
- Keep the rule quiet on authored prose, so the warning stays worth reading.

**Non-Goals:**

- Changing what archive writes. The placeholder is a useful marker at the moment
it is written; this change is about reporting it afterwards.
- Reporting a `## Purpose` in a delta spec. Delta Purposes are only read when a
capability is created, and archive already warns when it ignores one.
- Filling the Purpose in automatically. Only the author knows what the capability
is for.

## Decisions

### Severity is a warning, not an error

Strict mode already means "warnings are failures", so a warning gives both
behaviors from one severity: silent by default, failing under `--strict`.

*Alternative — error:* every project with a placeholder on disk starts failing
`openspec validate` on upgrade. On the evidence that these linger for months,
that is a large and involuntary blast radius for a documentation defect.

*Alternative — a dedicated opt-in flag:* adds a surface to learn and to document,
and duplicates what `--strict` is for. Rejected as a second mechanism for an
existing one.

### The check lives in validation, not in archive

Placed as a pure module beside `task-numbering.ts` and called from
`applySpecRules`, so it applies to every path that validates a main spec.

*Alternative — report at archive time, when the placeholder is written:* archive
already prints at that moment, and a line in a terminal is exactly what did not
survive. The defect is what persists on disk, so the check belongs where disk
state is inspected, and it must keep working for a spec archived a year ago by a
version that no longer runs.

### The generated sentence is recognised through a shared constant

The placeholder is text this tool generates, so it gets a name: the template
moves into a constant that `buildSpecSkeleton` composes from and the check
recognises through. Detection is then anchored to the thing itself rather than to
a second, hand-copied spelling of it that can drift from the writer.

The change name is interpolated, so recognition matches the constant's fixed
segments around it rather than the whole string.

*Alternative — spell the sentence out in the detector:* two independent copies of
one string, and the check silently stops matching the day the writer is reworded
— the failure mode being a check that reports nothing and looks healthy.

### A second, narrow marker rule covers what the constant cannot

A placeholder is not always the generated one. The `specs` instruction tells
agents to write "a brief TBD placeholder" when a delta has none, and an agent
writes its own wording. So a `TBD` **opening** the Purpose is also reported.

The rule is deliberately positional rather than a search: a Purpose that opens
with `TBD` is announcing it was not written, while "the retry budget is TBD
pending benchmarks" is a real Purpose with an open question in it. Reporting the
second would train people to ignore the warning, which costs more than the
findings it would add. A word that merely starts with those letters (`TBDs`) is
excluded for the same reason.

This is the one place the change cannot use an explicit lookup — the text is
written by agents and authors, not generated here, so there is no list to consult.
It is kept to a single anchored marker at a known position precisely to stay as
close to a lookup as the input allows.

It also covers a case the constant match cannot. A markdown formatter that
rewraps the generated sentence across two lines breaks the constant lookup, and
the marker rule still catches it, because every spelling of the placeholder opens
with `TBD`. So the fallback is not only for agent-written placeholders — it is
what keeps detection working when the generated one is reformatted.

### The placeholder finding replaces the brevity finding

A bare `TBD` is both a placeholder and under the length floor. Reporting both puts
two findings on one line where only one is actionable: "you left the placeholder
in" tells the author what to do, "your Purpose is under 50 characters" does not.
The placeholder check therefore runs first and the brevity check runs only when it
does not fire.

### Locating the line follows the rule that matched

The warning names the line carrying the placeholder, and which line that is
depends on which rule fired. A leading `TBD` is the section's first non-blank line
by definition. The generated sentence is not: it can sit below prose somebody
wrote, so it is located by its own text.

Naming the first non-blank line in that second case points at the authored prose —
a line the reader can see is fine, which reads as the check being wrong rather
than the Purpose being unwritten. When both rules match the leading marker wins,
because it is the earlier of the two.

When the placeholder cannot be located — no section header, or a generated
sentence no single line carries — the finding is reported without a line rather
than with a guessed one, since a wrong line number is worse than none.

Line endings are normalised before counting, so a spec saved on Windows reports
the same line number as the same spec saved on macOS or Linux.

## Risks / Trade-offs

- **A project running `--strict` in CI starts failing on upgrade** → that is the
intended effect and the reason severity is not an error: the failure is opt-in,
arrives only where a stricter gate was already requested, and is fixed by
writing one sentence. The message names the file to edit.

- **A legitimate Purpose that opens with "TBD" is reported** → accepted. A Purpose
whose first word is `TBD` is stating it was not written; reporting it is the
feature, not a false positive.

- **The marker rule is a positional match on authored prose, against the project's
preference for explicit lookups** → confined to the one case where no list can
exist, and anchored at a single position so its behavior is enumerable. The
generated sentence, which *can* be looked up, is looked up.

- **Wording of the generated placeholder changes later and old specs stop being
recognised by the constant** → the marker rule still catches them, since every
spelling used so far opens with `TBD`.

- **Archive behavior changes unintentionally** → archive constructs its validators
without strict mode, so a warning cannot flip a rebuilt spec to invalid. Covered
by a test asserting the exact call archive makes.

## Migration Plan

None. No data, config, or spec files change. A project sees the new warning the
first time it validates after upgrading, and fixes it by writing the Purpose in
the main spec.

## Open Questions

- Should a `TODO` marker be treated the same as `TBD`? No tool or instruction
produces one today, so it is left out; adding it later is a one-line widening
that changes no scenario already written here.
64 changes: 64 additions & 0 deletions openspec/changes/warn-on-purpose-placeholder/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
## Why

When a delta introduces a capability without a usable `## Purpose`, archive writes
`TBD - created by archiving change <name>. Update Purpose after archive.` into the
new main spec. Three places already tell authors to replace it — the `specs`
instruction ("including a leftover `TBD` placeholder — edit the main spec
directly"), the sync-specs summary step ("so it gets written now rather than
lingering"), and the archive contract itself — but nothing reports that it is
still there.

`--strict` cannot reach it. The check meant to catch a Purpose nobody wrote is a
50-character floor, and the placeholder is 91 characters, so the one rule that
exists to catch a thin Purpose is satisfied by the exact text meaning "nobody
wrote one". A spec whose Purpose reads `Does stuff.` fails `--strict` today; a
spec whose Purpose says nothing at all passes.

The result is a capability that carries a to-do indefinitely while every command
reports success, and a silent pass is indistinguishable from a clean run.
[#369](https://github.com/Fission-AI/OpenSpec/issues/369) reported agents leaving
the placeholder behind and stayed open for seven months; the remedies since have
been instructions, which is the mechanism that report described as unreliable.

## What Changes

- `openspec validate` reports a `## Purpose` that is still the archive
placeholder, as a warning on the spec's Purpose, naming the line to replace.
- The message says to edit the main spec directly, because a `## Purpose` in a
delta is read only when a capability is created and cannot replace an existing
one.
- Detection stays narrow: the sentence archive itself writes counts wherever it
appears in the Purpose, and otherwise only a `TBD` or `TODO` opening the
Purpose counts. A marker inside a sentence is authored prose and is left alone,
and so is anything inside a fenced code block, which is a Purpose quoting the
placeholder rather than carrying it.
- A Purpose reported as a placeholder is no longer also reported as too brief, so
a bare `TBD` yields one finding rather than two.
- Not breaking: the finding is a warning, so a project that already carries
placeholders keeps validating by default and only `--strict` fails. `openspec
archive` is unaffected — it validates rebuilt specs without `--strict`, so a
spec archive writes still passes the validation it would have passed before.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `cli-validate`: adds a requirement that spec validation report a Purpose left as
the archive placeholder, with the severity, detection boundary, and precedence
over the existing brevity warning stated as contract.

## Impact

- **Affected behavior**: `openspec validate` on main specs — `validate <spec>`,
`validate --specs`, and the bulk/interactive paths that share it. A project
carrying a placeholder sees a new warning; under `--strict` that project now
fails until the Purpose is written.
- **Unaffected**: `openspec archive`, which validates rebuilt specs non-strictly;
delta spec validation, which does not read a main spec's Purpose; and any spec
whose Purpose is authored prose.
- **Docs**: none required — the message carries its own remediation, and the
`specs` instruction already tells authors to edit the main spec directly.
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
## ADDED Requirements

### Requirement: Spec validation SHALL report a Purpose left as the archive placeholder

The `validate` command SHALL report, as a warning against the spec's Purpose, a
`## Purpose` that is still a placeholder rather than a Purpose someone wrote:
the sentence `openspec archive` writes for a new capability, or a marker left in
its place. The report SHALL name the line the placeholder is on when it can be
located, and SHALL omit the line rather than point at the wrong text when it
cannot.

The remediation SHALL say to edit the main spec directly, because a `## Purpose`
in a delta is read only when a capability is created and therefore cannot replace
one that already exists.

The finding SHALL be a warning. A project that already carries placeholders
therefore keeps validating by default, and only `--strict` fails — the
placeholder is worth keeping at the moment archive writes it, and worth reporting
once it has outlived that moment.

Detection SHALL be narrow, because a Purpose is prose and prose that raises an
open question is not a placeholder:

- the sentence archive itself writes SHALL be reported wherever it appears in the
Purpose, since nobody writes it by accident;
- otherwise only a `TBD` or `TODO` marker opening the Purpose SHALL be reported.
The two words SHALL be read the same way, because which one got typed says
nothing about whether the Purpose was written;
- a marker appearing inside a sentence SHALL NOT be reported;
- a longer word that merely begins with those letters SHALL NOT be reported,
in any script.

Text inside a fenced code block SHALL NOT be read as the Purpose speaking, for
either rule. A Purpose that quotes the placeholder is documenting it rather than
carrying it, and a check that fails the document explaining the placeholder
teaches its readers to ignore the warning.

An empty Purpose SHALL NOT be reported by this requirement, which the
empty-Purpose error already covers. A Purpose reported as a placeholder SHALL NOT
also be reported as too brief, so a bare `TBD` yields one finding and not two.

Validation performed inside `openspec archive` SHALL be unaffected, because
archive validates a rebuilt spec without `--strict` and a warning does not change
that verdict: a spec archive writes SHALL still pass the validation it would have
passed before this requirement existed.

#### Scenario: The placeholder passes by default and fails under strict

- **GIVEN** a main spec whose Purpose is the placeholder archive wrote
- **WHEN** `openspec validate --specs` runs
- **THEN** report a warning against the Purpose, naming the line it is on and
saying to edit the main spec directly
- **AND** the spec is reported valid

#### Scenario: Strict validation fails on the placeholder

- **GIVEN** the same main spec
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** the spec is reported invalid

#### Scenario: An authored Purpose raising an open question is not reported

- **GIVEN** a Purpose reading "Bounds how often a failed delivery is retried. The exact budget is TBD pending load tests."
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report no placeholder warning, because the marker does not open the Purpose
- **AND** the spec is reported valid

#### Scenario: A Purpose left as a TODO is reported like a TBD

- **GIVEN** a Purpose consisting only of "TODO"
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report the placeholder warning, the same finding a bare "TBD" reports

#### Scenario: A Purpose quoting the placeholder inside a fence is not reported

- **GIVEN** a Purpose that explains the placeholder and shows it inside a fenced
code block
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report no placeholder warning
- **AND** the spec is reported valid

#### Scenario: A word beginning with the marker is not reported

- **GIVEN** a Purpose opening "TBDs raised during design review are tracked in the linked issue.", or the same sentence opening with "TODOs"
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report no placeholder warning

#### Scenario: A bare TBD is reported once

- **GIVEN** a Purpose consisting only of "TBD"
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report exactly one finding against the Purpose, the placeholder warning
rather than the too-brief warning

#### Scenario: A terse but authored Purpose still reports as too brief

- **GIVEN** a Purpose reading "Does stuff."
- **WHEN** `openspec validate --specs --strict` runs
- **THEN** report the too-brief warning and no placeholder warning

#### Scenario: Archive still writes the spec it would have written

- **GIVEN** a change whose delta introduces a capability with no usable `## Purpose`
- **WHEN** `openspec archive` validates the rebuilt spec before writing it
- **THEN** that spec is reported valid and archive completes exactly as before

#### Scenario: Line endings do not change what is reported

- **GIVEN** two main specs with the same placeholder Purpose, one saved with LF
line endings and one with CRLF
- **WHEN** `openspec validate --specs` runs on each
- **THEN** both report the same warning against the same line number
Loading
Loading