Skip to content

Replace the integration tests with the conformance oracle - #2633

Merged
somewhatabstract merged 3 commits into
claude/jolly-galileo-72zesp-conformance-cachefrom
claude/jolly-galileo-72zesp-replace-integration-tests
Oct 23, 2026
Merged

somewhatabstract merged 3 commits into
claude/jolly-galileo-72zesp-conformance-cachefrom
claude/jolly-galileo-72zesp-replace-integration-tests

Conversation

@somewhatabstract

Copy link
Copy Markdown
Owner

Summary
The conformance oracle (#2622, #2625) runs the same __examples__ through the real, built CLI, so the in-process integration tests are now redundant. This PR removes them, leaving one end-to-end test suite.

Stack

Issues
Part of #2613 (Phase 0), which is under #2524.

Details

  • Removed: src/__tests__/integration.test.ts and its snapshot, src/integration-test-support.ts and its tests, and bin/gen-integration-test-logs.ts.
  • getExamples moves into the harness as src/conformance/get-examples.ts, with its own tests. It now takes the examples directory as a parameter rather than finding it with ancesdir.
  • package.json:
    • removed test:integration and coverage:integration
    • test:unit and coverage:unit no longer need to exclude integration.test.ts
    • pnpm test now runs the unit tests and then the conformance oracle
  • CI: the "Integration tests" job keeps its name, so existing checks and any branch protection still match, but now runs only pnpm test:conformance.
  • .gitignore: dropped .integrationtests/.
  • An empty changeset, since the published package doesn't change.

What's no longer covered, on purpose

Review notes

  • Locally: unit tests (512), conformance (100 golden files), lint and typecheck all pass. Coverage stays at 100% of lines. The branch percentage changes slightly only because the removed support module took its covered branches with it.

Generated by Claude Code

The conformance oracle runs the same examples through the real CLI, so the
in-process integration tests are now redundant. Removes them, their log
generator and support module, and the test:integration script, and moves
example discovery into the conformance harness. `pnpm test` now runs the
unit tests and the conformance oracle.

Part of #2613.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
@changeset-bot

changeset-bot Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5ee3f95

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@codecov

codecov Bot commented Oct 23, 2026

Copy link
Copy Markdown

Bundle Report

Bundle size has no change ✅

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (5ffad42) to head (5ee3f95).

Additional details and impacted files

Impacted file tree graph

@@                               Coverage Diff                               @@
##           claude/jolly-galileo-72zesp-conformance-cache     #2633   +/-   ##
===============================================================================
  Coverage                                         100.00%   100.00%           
===============================================================================
  Files                                                 51        51           
  Lines                                               1011       959   -52     
  Branches                                             249       231   -18     
===============================================================================
- Hits                                                1011       959   -52     
Files with missing lines Coverage Δ
src/conformance/get-examples.ts 100.00% <100.00%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ffad42...5ee3f95. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Removing the integration test support module dropped the only coverage of
combining the migration mode from arguments with config mappings in
determineOptions, and of the default migration mode in
loadMigrationConfig. Adds direct unit tests for both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
The config type requires a mode, but parsed config files can omit it,
which is the case the default exists for, so the test input is cast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn

@somewhatabstract somewhatabstract left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@somewhatabstract
somewhatabstract merged commit d900c31 into main Oct 23, 2026
13 checks passed
@somewhatabstract
somewhatabstract deleted the claude/jolly-galileo-72zesp-replace-integration-tests branch October 23, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants