docs(openspec): Propose restructure-cli-telemetry change

.github/workflows/pr-check-openspec.yml

@@ -1,14 +1,52 @@
 name: PR OpenSpec Archive Check
 
+# A change is archived exactly once, when ALL of its work has landed. In a
+# stack of PRs, only the tip carries the archived change; the PRs below it still
+# carry the in-flight change directory by design. So this check runs only on the
+# tip of a stack (or a standalone PR) and is skipped on PRs that still have work
+# stacked on top of them. Tip = no other OPEN PR targets this PR's head branch
+# as its base.
 on:
   pull_request:
-    branches: [main]
 
 permissions:
   contents: read
+  pull-requests: read
 
 jobs:
+  stack-position:
+    name: Detect stack position
+    runs-on: ubuntu-latest
+    outputs:
+      is_tip: ${{ steps.detect.outputs.is_tip }}
+    steps:
+      - name: Determine whether this PR is the tip of its stack
+        id: detect
+        env:
+          GH_TOKEN: ${{ github.token }}
+          HEAD_REF: ${{ github.head_ref }}
+          REPO: ${{ github.repository }}
+        run: |
+          set -euo pipefail
+
+          # Count OPEN PRs that target this PR's head branch as their base.
+          # Any such PR means work is still stacked on top → not the tip.
+          children=$(gh pr list --repo "$REPO" --state open --base "$HEAD_REF" \
+            --json number --jq 'length' 2>/dev/null || echo 0)
+          children=${children:-0}
+
+          if [ "$children" -gt 0 ]; then
+            echo "is_tip=false" >> "$GITHUB_OUTPUT"
+            echo "This PR has $children open PR(s) stacked on top — changes still in flight."
+            echo "The OpenSpec archive check is skipped until this PR is the tip of the stack."
+          else
+            echo "is_tip=true" >> "$GITHUB_OUTPUT"
+            echo "No PRs are stacked on top — this PR is the tip (or standalone); the archive check will run."
+          fi
+
   check-openspec-archived:
+    needs: stack-position
+    if: needs.stack-position.outputs.is_tip == 'true'
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
@@ -28,8 +66,8 @@ jobs:
             echo "::error::Unarchived OpenSpec change directories found under openspec/changes/:"
             echo "$UNARCHIVED" | sed 's|^|  - |'
             echo ""
-            echo "Every change must be moved under openspec/changes/archive/ before merge."
-            echo "Run the openspec archive workflow (e.g. /openspec-archive-change <name>) and commit the result."
+            echo "This PR is the tip of its stack (all work landed), so the change must be archived."
+            echo "Move it under openspec/changes/archive/ (e.g. /openspec-archive-change <name>) and commit."
             exit 1
           fi
 

openspec/changes/restructure-cli-telemetry/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-06-12

openspec/changes/restructure-cli-telemetry/design.md

@@ -0,0 +1,95 @@
+## Context
+
+This design is intentionally lean — the taxonomy was decided before writing the
+proposal. It records the model and the one non-trivial implementation decision
+(centralizing `cli_run`); there is no open architecture to resolve.
+
+Today each command emits `cli_<action>` at start and `cli_<action>_completed` at
+end. The telemetry wrapper already attaches standard properties (`cli`,
+`cliVersion`, `scaffoldVersion`, org group) and fails silently. Those guarantees
+are unchanged; only the event taxonomy changes.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- One `cli_run` per invocation as the denominator, emitted centrally.
+- Named `cli_*` events only for concrete state transitions.
+- `cli_help { topic }` replacing the `help_*` trio.
+- A hard cut — no dual-emit window.
+
+**Non-Goals:**
+
+- Changing identity, opt-out, standard properties, or silent-failure behavior.
+- Per-command run-context enrichment (auth state on `cli_run` already implies
+  anonymous vs. authenticated).
+- Capturing rule content, prompts, or matched source in any event.
+
+## Decisions
+
+### D1 — `cli_run` is emitted once, centrally, in the runner
+
+The CLI's top-level runner (`index.ts`, around `runCommand`) wraps execution: it
+resolves the matched subcommand name, runs the command, and emits a single
+`cli_run` with `{ command, success, durationMs, anonymous, loggedIn }` — on both
+success and failure (in a `finally`). The CLI version is not added here; it rides
+on the standard `cliVersion` property already attached to every event.
+
+_Why:_ Centralizing makes the denominator impossible to forget and removes ~20
+per-command start/`_completed` captures. The command name comes from the
+resolved citty command; `success` is derived from whether the command threw (or
+set a non-zero `process.exitCode`); `durationMs` from a start timestamp.
+
+_Alternative considered:_ keep per-command emission. Rejected — it's what we have
+now, and it's the source of the scatter.
+
+### D2 — Concrete-state events fire at the state change, not command boundaries
+
+Each meaningful outcome emits its own event where the state actually changes:
+
+```
+rule create success  → cli_rule_created   { mode: "remote"|"static", ruleCount }
+rule improve success → cli_rule_improved  { ruleCount }
+rule delete success  → cli_rule_deleted   { }
+auth login success   → cli_authenticated  { }
+auth logout success  → cli_logged_out     { }
+init/install success → cli_installed      { targets? }
+onboard complete     → cli_onboarded      { }
+check finishes       → cli_check_completed{ errorCount, warningCount, filesScanned }
+any command fails    → cli_error          { command, code }
+help served          → cli_help           { topic }   (topic = "(index)" for no-arg,
+                                                        the attempted topic otherwise)
+```
+
+Properties are counts/ids/booleans only.
+
+_Why:_ These are the funnel-worthy moments. "Ran info/status/detect/verify/meta"
+carries no concrete state beyond the invocation, so those are covered by
+`cli_run` alone with no bespoke event.
+
+### D3 — `cli_error` is the single failure event
+
+Instead of `success:false` spread across each `_completed` event, failures emit
+one `cli_error { command, code }` (code from the stable `CLIErrorCode` set), and
+`cli_run` also records `success:false`. The runner emits `cli_error` from its
+catch path so no command has to remember to.
+
+### D4 — Hard cut, no dual-emit
+
+Old event names are removed in the same release; nothing emits both taxonomies.
+Dashboards are rebuilt against the new names (the proposal calls this out).
+
+## Risks / Trade-offs
+
+- **[Dashboards/funnels on old names break]** → Accepted and documented; this is
+  a deliberate hard cut. The new taxonomy is simpler to rebuild against.
+- **[Centralized `cli_run` can't see command-specific context]** → By design
+  (D-non-goal). `loggedIn` covers the only cross-cutting dimension we need now.
+- **[`success` detection in the runner is imperfect]** → Derive from thrown
+  error and `process.exitCode`; commands already use `CLIError` + exit codes
+  consistently, so this is reliable.
+
+## Open Questions
+
+- None. (`cli_check_completed` property names — `errorCount`/`warningCount` — are
+  a naming detail finalized during implementation against the check result shape.)

openspec/changes/restructure-cli-telemetry/proposal.md

@@ -0,0 +1,68 @@
+## Why
+
+The CLI's PostHog taxonomy emits a `cli_<action>` start event plus a
+`cli_<action>_completed` event for nearly every command — roughly two events per
+invocation, with the "ran" signal scattered across ~20 distinct event names.
+There is no single denominator for "the CLI was invoked," so basic questions
+("how many runs of each command", "overall success rate") require stitching many
+event names together. Following PostHog's own guidance, we want one top-level
+`cli_run` event per invocation as the denominator, and to reserve named `cli_*`
+events for concrete state transitions worth analyzing as funnels.
+
+## What Changes
+
+- **NEW `cli_run`** — emitted exactly once per invocation, centrally in the CLI
+  runner, with properties `command`, `success`, `durationMs`, `anonymous`, and
+  `loggedIn` (the CLI version rides on the standard `cliVersion` property already
+  attached to every event — `cli_run` adds no second version field). This is the
+  universal denominator; no command emits its own "started/ran" event anymore.
+- **`cli_*` reserved for concrete state transitions** — replace the per-command
+  start/`_completed` pairs with events that represent real outcomes:
+  `cli_rule_created`, `cli_rule_improved`, `cli_rule_deleted`,
+  `cli_authenticated`, `cli_logged_out`, `cli_installed`, `cli_onboarded`,
+  `cli_check_completed` (error counts only — no rule content or matched code),
+  and a unified `cli_error` (`command`, `code`).
+- **`cli_help { topic }`** — replace `help_index` / `help_<topic>` / `help_unknown`
+  with a single `cli_help` event carrying a `topic` field (parallel to
+  `cli_run`'s `command`), so help intent is one event filtered by topic.
+- **BREAKING (analytics only)** — the previous taxonomy is a hard cut, no
+  dual-emit window. Dashboards/funnels built on the old names must be rebuilt.
+- **No per-command enrichment context** — `cli_run` already carries the two
+  cross-cutting dimensions we need: `loggedIn` (a valid token is present) and
+  `anonymous` (no authenticated identity resolved). These are distinct from the
+  `--anonymous` flag, which is an independent per-command invocation choice;
+  commands do not thread extra run-context properties in this change.
+
+## Capabilities
+
+### New Capabilities
+
+(none)
+
+### Modified Capabilities
+
+- `analytics`: Rewrite the CLI event taxonomy — introduce `cli_run` as the
+  once-per-invocation denominator, replace start/`_completed` pairs with
+  concrete-state `cli_*` events, collapse the `help_*` events into
+  `cli_help { topic }`, and update the wrong-topic re-routing funnel to derive
+  from the new events. Standard-property guarantees are unchanged.
+
+## Impact
+
+- **Centralized emission**: `packages/cli/src/index.ts` runner gains the
+  `cli_run` emission (resolved command name + success + duration), so the event
+  cannot be forgotten per command.
+- **Per-command call sites** (`commands/{rules,auth,init,check,info,onboard}.ts`,
+  `wizard/index.ts`): remove start/`_completed` captures; emit concrete-state
+  events at the point the state actually changes.
+- **Help** (`commands/help.ts`): `help_index`/`help_<topic>`/`help_unknown` →
+  `cli_help { topic }`.
+- **Telemetry module** (`telemetry.ts`): standard properties unchanged; the
+  wrapper continues to attach `cli`, `cliVersion`, `scaffoldVersion`, and org
+  group to every event.
+- **Tests**: `test/telemetry.test.ts` and command tests that assert event names
+  update to the new taxonomy.
+- **Privacy**: concrete events carry counts/ids/booleans only — never rule
+  content, prompts, or matched source.
+- **No new dependencies**; no change to opt-out, identity, or silent-failure
+  behavior.

openspec/changes/restructure-cli-telemetry/specs/analytics/spec.md

@@ -0,0 +1,146 @@
+## ADDED Requirements
+
+### Requirement: Every invocation emits exactly one cli_run event
+
+The CLI SHALL emit exactly one `cli_run` event per invocation, from the top-level
+runner rather than from individual commands. The event SHALL carry the properties
+`command` (the resolved subcommand name, e.g. `"rule create"` or `"help"`),
+`success` (boolean), `durationMs` (number), `anonymous` (boolean), and `loggedIn`
+(boolean). The CLI version is provided by the standard `cliVersion` property
+attached to every event (see the standard-properties requirement); `cli_run`
+SHALL NOT introduce a second version field. The event SHALL be emitted on both
+success and failure (from a `finally`-equivalent path), and no command SHALL emit
+its own "started" or "ran" event.
+
+#### Scenario: A successful command emits one cli_run
+
+- **WHEN** a user runs `taskless info`
+- **THEN** PostHog SHALL receive exactly one `cli_run` event with
+  `command: "info"`, `success: true`, a numeric `durationMs`, and the
+  `anonymous` and `loggedIn` properties (plus the standard `cliVersion`)
+- **AND** SHALL NOT receive a separate `cli_info` or `cli_info_completed` event
+
+#### Scenario: A failing command still emits cli_run
+
+- **WHEN** a command exits with an error
+- **THEN** PostHog SHALL receive one `cli_run` event with `success: false`
+
+## MODIFIED Requirements
+
+### Requirement: CLI events use cli\_ prefix
+
+CLI events SHALL use the `cli_` prefix, with the taxonomy organized as a
+`cli_run` denominator plus concrete state-transition events:
+
+- `cli_run` — exactly one per invocation (see the dedicated requirement). This
+  replaces every previous `cli_<action>` start event and `cli_<action>_completed`
+  event; the `success`/`durationMs`/`command` signal lives here.
+- Concrete state-transition events, each fired at the point the state actually
+  changes, carrying counts/ids/booleans only (never rule content, prompts, or
+  matched source):
+  - `cli_rule_created`, `cli_rule_improved`, `cli_rule_deleted`
+  - `cli_authenticated`, `cli_logged_out`
+  - `cli_installed`, `cli_onboarded`
+  - `cli_check_completed` — error/warning counts only (e.g. `errorCount`,
+    `warningCount`, `findings`)
+  - `cli_error` — a single failure event with `command` and `code` (a stable
+    `CLIErrorCode`)
+- `cli_help` — fired when the help command serves a request, with a `topic`
+  property. The `topic` SHALL be: the served topic for a known topic (e.g.
+  `"rule create"`); the exact literal `"(index)"` when invoked with no topic;
+  and the attempted topic string when the topic is unknown. This single event
+  replaces the previous `help_index`, `help_<topic>`, and `help_unknown` events.
+
+Commands that carry no concrete state beyond the invocation (e.g. `info`,
+`detect`, `update`, `auth status`, `rule verify`, `rule meta`) SHALL rely on
+`cli_run` alone and SHALL NOT emit a bespoke event. The previous taxonomy
+(`cli_<action>`, `cli_<action>_completed`, `help_index`, `help_<topic>`,
+`help_unknown`) SHALL be removed in this release; there is no dual-emit window.
+
+#### Scenario: Rule creation emits a concrete state event plus cli_run
+
+- **WHEN** a user runs `taskless rule create --from req.json` and a rule is written
+- **THEN** PostHog SHALL receive one `cli_run` event with `command: "rule create"`
+- **AND** SHALL receive a `cli_rule_created` event
+- **AND** SHALL NOT receive `cli_rule_create` or `cli_rule_create_completed`
+
+#### Scenario: Help fetch emits cli_help with a topic
+
+- **WHEN** an agent runs `taskless help rule create`
+- **THEN** PostHog SHALL receive a `cli_help` event with `topic: "rule create"`
+- **AND** SHALL NOT receive a `help_rule_create` event
+
+#### Scenario: Help with no topic emits cli_help with the index marker
+
+- **WHEN** an agent runs `taskless help`
+- **THEN** PostHog SHALL receive a `cli_help` event with `topic: "(index)"`
+- **AND** SHALL NOT receive a `help_index` event
+
+#### Scenario: Help with an unknown topic emits cli_help with the attempted topic
+
+- **WHEN** an agent runs `taskless help nope`
+- **THEN** PostHog SHALL receive a `cli_help` event with `topic: "nope"`
+- **AND** SHALL NOT receive a `help_unknown` event
+
+#### Scenario: A command failure emits cli_error
+
+- **WHEN** a command fails with a known `CLIErrorCode`
+- **THEN** PostHog SHALL receive a `cli_error` event with `command` and `code`
+
+#### Scenario: Old event names are not emitted
+
+- **WHEN** any CLI command runs in this release
+- **THEN** PostHog SHALL NOT receive any event named `cli_<action>_completed`,
+  `help_index`, `help_<topic>`, or `help_unknown`
+
+### Requirement: Wrong-topic re-routing is observable as a derivable funnel
+
+The taxonomy SHALL keep wrong-topic re-routing derivable as a funnel signal from
+the new events:
+
+- A `cli_help { topic: A }` event not followed by the concrete event for topic A
+  (or by `cli_run` with the corresponding `command`), and then a subsequent
+  `cli_help { topic: B }`, indicates the agent fetched recipe A, did not act on
+  it, and re-routed to topic B.
+- A `cli_help` index-marker event followed by a `cli_help { topic }` event
+  indicates the agent consulted the index before picking a topic (baseline).
+- A `cli_help { topic }` event with no subsequent acting `cli_run` and no further
+  `cli_help` event indicates the agent abandoned the action.
+
+No additional events SHALL be added to capture this signal directly — it is
+derivable from the `cli_help` / `cli_run` sequence. Dashboards SHOULD surface
+re-routing rates per topic.
+
+#### Scenario: Funnel data supports wrong-topic detection
+
+- **WHEN** dashboards are constructed in PostHog
+- **THEN** the `cli_help` (with `topic`) and `cli_run` (with `command`) events
+  SHALL be sufficient to compute "rate of `cli_help { topic }` not followed by a
+  corresponding acting `cli_run` within N minutes"
+
+### Requirement: All capture calls include standard properties
+
+Every `capture()` call SHALL include the `cli` property (anonymous UUID), the `cliVersion` property (the `@taskless/cli` package version read from `package.json`), and the `scaffoldVersion` property (the `version` field from `.taskless/taskless.json`, or `0` if the manifest is absent or unreadable). When authenticated, the `groups` parameter SHALL include `{ organization: String(orgId) }`. The `cliVersion` and `scaffoldVersion` values SHALL be resolved once at telemetry initialization and attached to every subsequent `capture()` call without re-reading the source files.
+
+#### Scenario: Anonymous capture includes standard properties
+
+- **WHEN** `capture("cli_run")` is called without authentication
+- **THEN** the event SHALL include `{ cli: anonymousUuid, cliVersion: <string>, scaffoldVersion: <number> }`
+- **AND** the event SHALL NOT include a `groups` parameter
+
+#### Scenario: Authenticated capture includes standard properties and group
+
+- **WHEN** `capture("cli_rule_created")` is called with authentication
+- **THEN** the event SHALL include `{ cli: anonymousUuid, cliVersion: <string>, scaffoldVersion: <number> }`
+- **AND** the `groups` parameter SHALL include `{ organization: String(orgId) }`
+
+#### Scenario: Scaffold version falls back to 0 when manifest missing
+
+- **WHEN** `getTelemetry(cwd)` is initialized in a directory with no `.taskless/taskless.json`
+- **THEN** every `capture()` call from the returned client SHALL include `scaffoldVersion: 0`
+
+#### Scenario: CLI version is embedded at build time
+
+- **WHEN** `getTelemetry()` is initialized
+- **THEN** `cliVersion` SHALL be the `@taskless/cli` version embedded at build time (no runtime file read), consistent with the CLI spec's build-time version requirement
+- **AND** SHALL be attached to every event emitted through the returned client