Skip to content

feat(cli): Emit concrete rule events; drop rule start/_completed pairs - #34

Merged
thecodedrift merged 40 commits into
feat/telemetry-cli-runfrom
feat/telemetry-rule-events
Jun 14, 2026
Merged

feat(cli): Emit concrete rule events; drop rule start/_completed pairs#34
thecodedrift merged 40 commits into
feat/telemetry-cli-runfrom
feat/telemetry-rule-events

Conversation

@thecodedrift

@thecodedrift thecodedrift commented Jun 13, 2026

Copy link
Copy Markdown
Member

Phase 2 of the telemetry rework (stacked on the cli_run phase).

Replaces the per-rule-command start/_completed event pairs with concrete state-transition events fired where the state actually changes:

  • rule createcli_rule_created { ruleCount } (only when rules are written)
  • rule improvecli_rule_improved { ruleCount }
  • rule deletecli_rule_deleted (only when files were removed)
  • rule meta / rule verify → no bespoke event; covered by cli_run

create/improve/delete keep their try/finally and emit the concrete event from the finally guarded by a state flag (so it fires only on real success, not the terminal pr/merged/closed states); meta/verify drop their command-level telemetry entirely.

Only a sample event name in telemetry.test.ts referenced an old rule name (updated to cli_rule_created); the rule-from/verify tests assert behavior, not events. Full suite green (256).

Part of the restructure-cli-telemetry OpenSpec change.


Stack generated by Git Town

thecodedrift and others added 8 commits June 12, 2026 18:36
Phase 2 of the telemetry rework. Replace the per-rule-command start and
_completed events with concrete state-transition events fired where the
state actually changes:

- rule create  → cli_rule_created  { ruleCount }  (only when rules are written)
- rule improve → cli_rule_improved { ruleCount }
- rule delete  → cli_rule_deleted                 (only when files removed)
- rule meta / rule verify → no bespoke event; covered by cli_run

The create/improve/delete commands keep their try/finally and emit the
concrete event from the finally guarded by a state flag; meta/verify drop
their command-level telemetry entirely. Only a sample event name in
telemetry.test.ts needed updating (to cli_rule_created).

Stacked on the cli_run phase; cli_run still emits per invocation, so the
suite stays green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 3 of the telemetry rework. Replace the auth/init/update/onboard/check
start and _completed pairs with concrete state-transition events:

- auth login  → cli_authenticated  (fresh login only; already-logged-in is not)
- auth logout → cli_logged_out      (only when a saved token was removed)
- auth status → no bespoke event; covered by cli_run
- init/update → cli_installed       (interactive wizard, non-interactive, update)
- onboard --mark-complete → cli_onboarded (already-done / recipe → cli_run only)
- check       → cli_check_completed { errorCount, warningCount, findings }
                (only when a scan runs; counts only, never matched code)

Tests: wizard-integration now asserts cli_installed on completion and no
install event on cancel; telemetry.test sample event names updated to cli_run.
Full suite green (256). cli_run still emits per invocation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 4 of the telemetry rework. Replace help_index / help_<topic> /
help_unknown with a single cli_help carrying a topic property (the served
topic, "(index)" for a no-arg invocation, or the attempted topic when
unknown) — help intent is now one event filtered by topic. Remove info's
bespoke cli_info / cli_info_completed events (covered by cli_run) and its
now-unused getTelemetry import.

detect.ts (cli_detect) is not on this branch's lineage — it lives in the
unmerged local-rule-routing stack and is reconciled when both land.

Adds test/help-telemetry.test.ts asserting cli_help { topic } across the
served / index / unknown cases and that no legacy help_* event fires.
Full suite green (259).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All five phases of the telemetry rework are complete, so finalize on the
tip of the stack: apply the analytics delta into the main spec (add the
cli_run denominator requirement; rewrite the cli_ taxonomy, the
wrong-topic funnel, and the standard-properties scenarios) and move the
change to openspec/changes/archive/2026-06-13-restructure-cli-telemetry/.

Legacy event sweep is clean (the only _completed is the intentional
cli_check_completed concrete event); validate/typecheck/lint/suite green;
commands smoke-tested end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat/telemetry-cli-run:
  ci(openspec): Skip the archive check on non-tip stacked PRs
…fecycle

* feat/telemetry-rule-events:
  ci(openspec): Skip the archive check on non-tip stacked PRs
…help

* feat/telemetry-auth-lifecycle:
  ci(openspec): Skip the archive check on non-tip stacked PRs
* feat/telemetry-cli-help:
  ci(openspec): Skip the archive check on non-tip stacked PRs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is Phase 2 of the CLI telemetry rework, replacing per-command *_completed telemetry pairs for rule subcommands with concrete state-transition events emitted only when the underlying state actually changes (and dropping bespoke telemetry for rule meta / rule verify, relying on cli_run instead).

Changes:

  • Replace cli_rule_create/improve/delete(_completed) telemetry with concrete events: cli_rule_created { ruleCount }, cli_rule_improved { ruleCount }, and cli_rule_deleted.
  • Remove command-level telemetry from rule meta and rule verify.
  • Update a sample event name in telemetry.test.ts and mark Phase 2 tasks complete in the OpenSpec change checklist.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
packages/cli/src/commands/rules.ts Drops legacy per-command telemetry and emits concrete rule lifecycle events at the point of successful state change.
packages/cli/test/telemetry.test.ts Updates the sample captured event name to the new taxonomy.
openspec/changes/restructure-cli-telemetry/tasks.md Marks Phase 2 checklist items complete and clarifies test impact.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/cli/src/commands/rules.ts
Comment thread packages/cli/src/commands/rules.ts
@thecodedrift
thecodedrift marked this pull request as ready for review June 13, 2026 17:25

@thecodedrift thecodedrift left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium feedback around handling duration. We don't want to lose the timing data, but like cli_run we should move it to the top level so it's not a concern for individual commands

Comment thread packages/cli/src/commands/rules.ts
thecodedrift and others added 11 commits June 13, 2026 12:30
PR #34 review: guard the createdRuleCount/improvedRuleCount assignment on
rules.length > 0, so an empty/missing status.rules no longer emits
cli_rule_created / cli_rule_improved with ruleCount: 0. The "generated"
state with zero rules now leaves the count undefined, so the concrete
event does not fire (covered by cli_run).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #35 review:
- check.ts: compute errorCount/warningCount in one loop over results and
  derive hasErrors from errorCount, instead of one `some` + two `filter`
  passes over potentially large scan output.
- wizard-integration test: assert on the event name across all capture
  calls rather than not.toHaveBeenCalledWith("cli_installed"), so a call
  with extra properties can't produce a false negative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #36 review: the served-topic and no-arg help-telemetry tests asserted
cli_help was emitted but not that the implementation avoids dual-emitting
a legacy help_* event. Both now map captured calls to their event names
and assert none start with help_ (and specifically not help_index).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per review feedback: instead of repeating a "no help_* event" assertion
inside every behavioral help-telemetry test (over-testing), keep those
tests purely behavioral (cli_help { topic }) and add a single source-scan
test that asserts no help_* event-name literal remains anywhere under
src/. That states the contract once, confidently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat/telemetry-cli-run:
  fix(cli): Resolve cli_run identity at invocation start, not end
  fix(cli): Resolve cli_run identity fresh; tighten cli_error + drop cli_version
  docs(openspec): Address review on the telemetry contract
…fecycle

* feat/telemetry-rule-events:
  fix(cli): Resolve cli_run identity at invocation start, not end
  fix(cli): Only emit cli_rule_created/improved when rules are written
  fix(cli): Resolve cli_run identity fresh; tighten cli_error + drop cli_version
  docs(openspec): Address review on the telemetry contract
…help

* feat/telemetry-auth-lifecycle:
  fix(cli): Resolve cli_run identity at invocation start, not end
  refactor(cli): Single-pass check counts; robust cli_installed assertion
  fix(cli): Only emit cli_rule_created/improved when rules are written
  fix(cli): Resolve cli_run identity fresh; tighten cli_error + drop cli_version
  docs(openspec): Address review on the telemetry contract

# Conflicts:
#	openspec/changes/restructure-cli-telemetry/tasks.md
* feat/telemetry-cli-help:
  fix(cli): Resolve cli_run identity at invocation start, not end
  test(cli): Prove help_* removal once via a source scan, not per-test
  test(cli): Assert no legacy help_* event on served-topic and index paths
  refactor(cli): Single-pass check counts; robust cli_installed assertion
  fix(cli): Only emit cli_rule_created/improved when rules are written
  fix(cli): Resolve cli_run identity fresh; tighten cli_error + drop cli_version
  docs(openspec): Address review on the telemetry contract
…bility

The archive commit synced the pre-review delta spec, so the capability
spec carried the stale cli_check_completed{ filesScanned } and a fuzzy
cli_help index-marker description. Reconcile the synced spec (and the
archive design flow diagram) with the corrected contract: findings
replaces filesScanned, and cli_help documents the exact literal
"(index)" for the no-topic invocation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat/telemetry-cli-run:
  refactor(cli): Uppercase the CLI acronym in CliError/CliErrorCode
…fecycle

* feat/telemetry-rule-events:
  refactor(cli): Uppercase the CLI acronym in CliError/CliErrorCode

# Conflicts:
#	packages/cli/src/commands/auth.ts
thecodedrift and others added 21 commits June 13, 2026 15:35
…help

* feat/telemetry-auth-lifecycle:
  refactor(cli): Uppercase the CLI acronym in CliError/CliErrorCode
* feat/telemetry-cli-help:
  refactor(cli): Uppercase the CLI acronym in CliError/CliErrorCode
Match the source rename — the CLI acronym is uppercase in the CLIError
class and CLIErrorCode type, so the spec prose and archived contract use
the same casing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat/telemetry-cli-run:
  docs(openspec): Uppercase CLIError/CLIErrorCode in the change contract
…fecycle

* feat/telemetry-rule-events:
  docs(openspec): Uppercase CLIError/CLIErrorCode in the change contract
…help

* feat/telemetry-auth-lifecycle:
  docs(openspec): Uppercase CLIError/CLIErrorCode in the change contract
* feat/telemetry-cli-help:
  docs(openspec): Uppercase CLIError/CLIErrorCode in the change contract
The acronym rename landed on the runner branch, but auth.ts is owned by
this phase and its edits sat on the same lines, so the merge kept the old
casing and left the type unresolved. Rename auth.ts here so the type
checks across the stack.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…help

* feat/telemetry-auth-lifecycle:
  fix(cli): Carry the CLIErrorCode rename into auth.ts
* feat/telemetry-cli-help:
  fix(cli): Carry the CLIErrorCode rename into auth.ts
* feat/telemetry-cli-run:
  refactor(cli): Uppercase the acronym in CLIErrorEnvelope too
…fecycle

* feat/telemetry-rule-events:
  refactor(cli): Uppercase the acronym in CLIErrorEnvelope too
…help

* feat/telemetry-auth-lifecycle:
  refactor(cli): Uppercase the acronym in CLIErrorEnvelope too
* feat/telemetry-cli-help:
  refactor(cli): Uppercase the acronym in CLIErrorEnvelope too
* feat/telemetry-cli-run:
  refactor(cli): Uppercase the acronym in CLIConfig
…fecycle

* feat/telemetry-rule-events:
  refactor(cli): Uppercase the acronym in CLIConfig
…help

* feat/telemetry-auth-lifecycle:
  refactor(cli): Uppercase the acronym in CLIConfig
* feat/telemetry-cli-help:
  refactor(cli): Uppercase the acronym in CLIConfig
chore(openspec): Finalize + archive restructure-cli-telemetry
feat(cli): Collapse help_* into cli_help { topic }; drop cli_info
feat(cli): Concrete auth + lifecycle events; drop start/_completed pairs
@thecodedrift
thecodedrift merged commit 52a213c into feat/telemetry-cli-run Jun 14, 2026
1 check passed
@thecodedrift
thecodedrift deleted the feat/telemetry-rule-events branch June 14, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants