Skip to content

fix(validate-doc-claims): name repo-scoped search base in path-not-found flag - #1552

Open
Amanjyot-P wants to merge 1 commit into
EveryInc:mainfrom
Amanjyot-P:fix-1545-repo-scoped-path-validation-message
Open

fix(validate-doc-claims): name repo-scoped search base in path-not-found flag#1552
Amanjyot-P wants to merge 1 commit into
EveryInc:mainfrom
Amanjyot-P:fix-1545-repo-scoped-path-validation-message

Conversation

@Amanjyot-P

Copy link
Copy Markdown

What

validate-doc-claims.py flags a cited repo path with "not found in working tree" (or "working tree or origin/main") without saying where it looked. An agent reading that flag can't tell "this citation is wrong" from "this citation points somewhere this check did not look" — e.g. a second docs store in another repo on the same machine.

Fix

Name the search base (repo root, plus the upstream ref when one exists) in the flag text and state the check is repo-scoped:

FLAG path `x.md` (line N) — not found under /path/to/repo or origin/main. This
check only looks in this repository; verify other repos or stores before
treating the citation as wrong. Fix the citation, or annotate it as
historical (e.g. removed by this fix).

Applied identically to both duplicated copies of the script (skills/ce-compound/scripts/validate-doc-claims.py and skills/ce-compound-refresh/scripts/validate-doc-claims.py), per the AGENTS.md duplication rule the existing test suite already enforces.

Scope

Smallest correct fix per the issue: wording only, no cross-repo resolution (out of scope per the issue, tracked separately by #1505).

Tests

  • Updated tests/doc-claims-validator.test.ts's "flags a cited path that exists nowhere" test to assert the new repo-scoped wording (search base + "This check only looks in this repository").
  • bun test tests/doc-claims-validator.test.ts — 53/53 pass.
  • bun run test — full suite passes except 5 pre-existing, unrelated timing-flaky tests in tests/ce-babysit-pr-snapshot.test.ts / tests/skills/ce-code-review-cross-model-routes.test.ts, confirmed failing identically on a clean checkout of upstream main before this change.

Ref: #1545

…und flag

The 'not found in working tree' flag for a cited path implied a global
search when the check only looks in one repo. An agent reading the flag
could not distinguish 'this citation is wrong' from 'this citation points
somewhere this check did not look' (e.g. a second docs store in another
repo on the same machine).

Name the search base (repo root or upstream ref) in the flag text and
state that the check is repo-scoped, so the agent knows to check other
stores before treating the citation as wrong.

Ref: EveryInc#1545

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a121a1d03f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +278 to +279
+ ". This check only looks in this repository; verify other "
"repos or stores before treating the citation as wrong. Fix "

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Describe non-git search bases without calling them repositories

When the document and current directory are outside Git, base is merely os.getcwd(), but this new sentence still calls it “this repository.” Reproducing from a temporary non-git directory yields both INFO: not a git repository and This check only looks in this repository in the same report, undermining the clarification this change intends to provide. Make the scope description conditional on in_git, referring to the named directory rather than a repository in this supported fallback mode.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Amanjyot-P take a look at this

@Amanjyot-P Amanjyot-P left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OMP Swarm PR Review

Verdict: Ready with fixes — P0: 0 · P1: 0 · P2: 0 · P3: 0

Ready with fixes: the implementation is consistent across both validator copies, but the changed test does not assert the upstream-present suffix or the no-upstream branch. The validator independently confirmed this testing gap; it is retained as coverage debt rather than a primary P2 finding because the quote-the-line gate reduced confidence to 50 and the testing soft-bucket rule applies.

No confirmed PR-introduced findings survived evidence, scope, deduplication, cross-model, and validator gates.

Coverage: correctness, security, testing, maintainability, project_standards; independent validator completed before lead synthesis.
Skipped: performance (no performance-sensitive runtime path); adversarial (no high-risk adversarial trigger); api_contract (no external contract surface); previous_comments (no prior PR feedback).

Audit details
  • Five selected reviewer artifacts were present, parseable, and matched one-for-one to the selected roster.
  • The testing candidate was CONFIRMED by the validator with confidence 100, but its quoted evidence was not the exact motivating changed line; the quote-the-line gate reduced it from 75 to 50.
  • The single-reviewer P2 testing-only candidate was moved to the testing-gap soft bucket and is therefore not emitted as a primary finding.
  • No confidence promotion occurred: the candidate was reported only by testing, so there was no independent corroboration from a different model family.
  • Validator batch: 1 verdict, 1 CONFIRMED, 0 REJECTED, 0 infrastructure failures.
  • Suppressed/demoted: 1 candidate at confidence 50. No P0, P1, or retained P2 findings remain.

The complete audit remains in FINAL_REVIEW.md in the OMP run workspace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants