Skip to content

fix(check): say when rules ran unverified, and name what would verify them - #463

Merged
theCodeDrift merged 2 commits into
mainfrom
worktree-451---can-we
Oct 6, 2026
Merged

theCodeDrift merged 2 commits into
mainfrom
worktree-451---can-we

Conversation

@theCodeDrift

Copy link
Copy Markdown
Member

When check cannot have the rule service verify a project's rules, it now says so in one notice and names the fix. Before, the --anonymous, no-token, and no-remote paths only attached a reason to each skipped runtime rule, so a project with no runtime rules got no word that its ast-grep and Vale rules ran unverified. The most likely place to hit that path is a CI job with no token.

What changes

  • Every unverified path prints one notice: --anonymous, no token, no usable GitHub origin, or a reconcile that cannot complete. The notice prints whether or not runtime rules exist, and the exit code is unchanged.
  • The fix depends on the cause:
    • --anonymous: run without the flag.
    • No token: auth login, or TASKLESS_TOKEN in CI.
    • Remote problems: the notice says which one it is (not a git repository, no origin, or a non-GitHub origin). The CLIError code from resolveRepositoryUrl used to be discarded.
    • Rejected token: auth login.
    • organization_not_found: the same steps as rule create, now shared through orgNotFoundRemedy().
  • The notice is a single line, so under --json it is one entry in notices. An existing test requires that no notice contains a newline.
  • skipped[].reason is now the short cause, because the notice carries the remedy.
  • The check recipe moves to topic v6.

Example, logged out:

Notice: Rules were not verified: not authenticated. Static rules ran without verification. Run `npx @taskless/cli@latest auth login`, or set `TASKLESS_TOKEN` where `check` runs in CI.

Spec

This reverses cli-check's "SHALL NOT emit a warning about missing authentication". The original design kept offline use quiet, but it allowed "an informational line", and this is that: a Notice:, with no change to the exit code. The change modifies two requirements, each restated in full, and adds one. I ran the archive dry-run: cli-check goes from 91 to 96 scenarios and keeps every existing one. The change is archived in this PR (single PR).

Behavior notes

  • A logged-out --json run now always has a notices entry. Three existing tests that relied on the old output were updated.
  • This repo's own pnpm lint now prints the notice, since it runs check without a token.

Typecheck, lint, and the full CLI suite (2021 tests) pass locally.

Fixes #451

@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review

@claude

claude Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Claude finished @theCodeDrift's task in 8m 54s —— View job


Review: fix(check): say when rules ran unverified, and name what would verify them

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json held no threads, reviews, or comments, so this is the first review of this PR.

  • Read .prior-review.json
  • Fetch PR diff and changed files
  • Review implementation (plan-check.ts, generate.ts, agent recipe)
  • Review tests
  • Review OpenSpec proposal/spec/archive
  • Post final review

Summary

I read the full diff (plan-check.ts, generate.ts, check.md, all four test files) plus the surrounding files it interacts with but doesn't show (util/git-remote.ts, util/cli-error.ts, util/notices.ts, commands/check.ts, api/v2.ts, types/errors.ts). I found nothing NEW to raise — the change is clean.

Specific things I checked and confirmed correct:

  • Single-notice guarantee: unverified() in plan-check.ts:121 now always emits exactly one Unverified cause/remedy pair regardless of runtime-rule count, and unverifiedNotice() joins cause + "ran" sentence + remedy with " " (never "\n"), so the --json notices entry stays one line. --dangerously-run-scripts correctly bypasses this path entirely (returns before unverified is ever called), matching the spec's explicit exclusion.
  • Remote-failure cause plumbing: remoteProblem() switches on CLIError.code from resolveRepositoryUrl (NOT_A_GIT_REPOSITORY / NO_ORIGIN_REMOTE / UNSUPPORTED_REMOTE_HOST), all three of which are real members of CLIErrorCode in types/errors.ts and are the only codes git-remote.ts can actually throw — the default branch is an honest fallback, not dead-code-masquerading-as-handling.
  • orgNotFoundRemedy() extraction: generate.ts's refactor preserves the exact two sentences orgNotFoundMessage() printed before (verified byte-for-byte), and plan-check.ts now shares it for the organization_not_found reconcile outcome — no behavior drift for rule create.
  • Test coverage: spot-checked that the new/updated assertions in runtime-check.test.ts, mixed-engine-check.test.ts, vale-orchestration.test.ts, and demo-command.test.ts match the actual produced strings (cause text, remedy text, skipped[].reason wording), and confirmed by grep that no other test in the suite asserts an exact/empty notices array on a check run that would now break from the always-on notice (the notices.toEqual([])/toBeUndefined() hits elsewhere are all on unrelated commands/functions — verify, rule create/improve, applyVerdicts, Vale's own dispatch-level notices).
  • OpenSpec archive correctness: given this repo's documented archive footgun (a MODIFIED Requirements delta block replaces the whole requirement), I checked that both modified requirements in the archived delta (openspec/changes/archive/2026-10-05-check-unverified-remedy/specs/cli-check/spec.md) fully restate every pre-existing scenario alongside the new text — nothing was silently dropped. The main openspec/specs/cli-check/spec.md reflects the same content.
  • Changeset bump: patch is correct per this repo's pre-1.0 convention (new notice surface, no released behavior to break).
  • Recipe version bump: check.md is the only agent topic still saying v5 before this PR, and it's now v6; no stale topic v5 reference remains for check elsewhere in the codebase.

One minor observation, not a defect: there's no dedicated test exercising the outcome.status === "unauthorized" ("authentication was rejected") branch of planCheck — coverage for the other four causes is thorough, but this fifth one is exercised only by reading the code. Not blocking; flagging in case it's useful for a follow-up.

@theCodeDrift

Copy link
Copy Markdown
Member Author

Re: @claude[bot] — "Claude finished @theCodeDrift's task in 8m 54s … One minor observation, not a defect: there's no dedicated test exercising the unauthorized branch…"
#463 (comment)

Added in f56d969: token rejected: the notice names auth login and replacing TASKLESS_TOKEN answers reconcile with a 401 and asserts the single notice, its remedy, and the skipped rule's reason.

— AI Coding Agent

… them

Every unverified path (--anonymous, no token, no usable GitHub remote, a reconcile that cannot complete) now prints one single-line notice naming the cause and its fix, whether or not the project has runtime rules. The remote failure keeps the specific problem resolveRepositoryUrl identified, and organization_not_found shares its remedy with rule create through orgNotFoundRemedy().
@theCodeDrift
theCodeDrift force-pushed the worktree-451---can-we branch from f56d969 to 30408e3 Compare October 6, 2026 04:54
@theCodeDrift
theCodeDrift merged commit b267176 into main Oct 6, 2026
4 checks passed
@theCodeDrift
theCodeDrift deleted the worktree-451---can-we branch October 6, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check reports unverified runs without saying how to make verification happen

1 participant