Skip to content

docs: clarify AI check limits and remove internal license note in README - #21

Merged
jongio merged 3 commits into
mainfrom
jongio/docs-readme-updates
Nov 5, 2026
Merged

jongio merged 3 commits into
mainfrom
jongio/docs-readme-updates

Conversation

@jongio

@jongio jongio commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes github/evals-internal#514
Closes github/evals-internal#515

  • Add "A passing check doesn't guarantee the AI will get it right every time." in the ## Outcomes and evidence section of README.md (github/evals-internal#514).
  • Remove the internal release-owner license-clearance sentence for missing-license-text from the user-facing ## Development and acquisition section of README.md (github/evals-internal#515), while preserving the maintainer guidance in CONTRIBUTING.md.
  • Add a regression test in tests/example.test.ts verifying both README.md documentation invariants.

Verification

  • Model-free checks and source-to-bundle verification pass.
    • pnpm typecheck (tsc --noEmit): passed
    • pnpm lint (oxlint --deny-warnings src tests): passed
    • pnpm test:coverage (vitest run --coverage): passed (10 test files, 177 tests passed)
    • pnpm check:dist (node tooling/build.mjs --check): passed
    • pnpm test:packaged (node --test tooling/packaged.test.mjs): passed (6/6 tests passed)
  • Native integration evidence is distinguished from synthetic controls.
  • Credential, process and artifact boundaries were reviewed.
  • Documentation and packaged dist match source.

Limitations

None. Documentation and unit test update only; runtime bundle (dist/index.cjs) is unchanged.

Closes github/evals-internal#514
Closes github/evals-internal#515

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jongio
jongio requested a review from a team as a code owner November 5, 2026 18:49
Copilot AI dynamic review requested due to automatic review settings November 5, 2026 18:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active nit remains in tests/example.test.ts because the assertion does not scope the required sentence check to the ## Outcomes and evidence section.

1 open issue
What changed in this PR

This documentation-only pull request clarifies the limits of AI checks, removes internal license-clearance guidance from the user-facing README, and adds regression coverage for both documentation requirements.

Changes:

  • Adds an AI-correctness limitation to the Outcomes and evidence section.
  • Removes internal missing-license-text release-clearance guidance from Development and acquisition.
  • Adds README contract assertions in tests/example.test.ts.
File Summary Review notes
README.md Updates user-facing outcome and licensing documentation. No findings.
tests/​example.test.ts Adds regression assertions for the README documentation invariants. Nit: Scope the assertion to the Outcomes and evidence section so it verifies the sentence remains in the required section.

Review effort: Lite (auto)

Comment thread tests/example.test.ts Resolved

@wbreza wbreza left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs change looks correct and CI passes. One low-severity item remains before approval: the new test only checks that the sentence appears anywhere in the README, not in the "Outcomes and evidence" section. This is already raised in an existing thread, so I did not open a duplicate: #21 (comment)

Reviewed commit dd1f2c9. No code was run locally; I relied on the passing CI checks.

Generated by 🌳Breza Bot

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jongio

jongio commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Addressed reviews 5790283389 (concern:e10238ca2c8d15eb, concern:95bd24d2e2adce34) and 5790309415 (concern:85c231c4af4f47c1):

  • Scoped the README.md assertion in tests/example.test.ts to the ## Outcomes and evidence section in 7078847.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@wbreza wbreza left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The earlier low item is fixed: the test now looks only inside the "Outcomes and evidence" section for the new sentence, so moving it elsewhere would fail the test. The docs changes are correct. Unit tests, lint, type check, bundle check and smoke test pass on commit 7078847. Two CodeQL analysis jobs were still pending when I reviewed; I did not run anything locally.

Generated by 🌳Breza Bot

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrowly scoped documentation updates include regression assertions covering the required warning, removal, and section placement, with no active review issues identified.

No open issues 🎉

1 resolved since last review

Review effort: Lite (auto)

@jongio
jongio merged commit cc59a10 into main Nov 5, 2026
10 checks passed
@jongio
jongio deleted the jongio/docs-readme-updates branch November 5, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants