ci: add test-suite workflow running the full testthat suite#65
Open
seonghobae wants to merge 5 commits into
Open
ci: add test-suite workflow running the full testthat suite#65seonghobae wants to merge 5 commits into
seonghobae wants to merge 5 commits into
Conversation
CI previously executed almost none of the numerical tests: R-CMD-check ran with --no-tests plus a single Zh-rule file, and test-fast ran only 3 productization files — the greedy model-selection, integration, edge-case, and stability suites (26 files total) never gated merges. The new workflow runs the whole suite on push/PR, relying on the suite's own CI guards (.skip_expensive_ci_calls, skip_on_ci) to bound runtime, and sets RUN_FULL_AEFA_TESTS=1 on the weekly scheduled and manual runs to exercise the expensive AEFA estimation paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui
There was a problem hiding this comment.
Pull request overview
Adds a new GitHub Actions workflow to run the full testthat suite for the kaefa R package, addressing the current CI gap where most numerical/integration tests do not gate merges.
Changes:
- Introduces a
test-suiteworkflow that runs the completetests/testthatsuite onpushandpull_request. - Enables expensive AEFA estimation paths only for
scheduleandworkflow_dispatchruns viaRUN_FULL_AEFA_TESTS=1.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Uses the same v7.0.0 commit pin as R-CMD-check.yaml instead of the older v6.0.2 pin, per review feedback that one checkout pin across workflows simplifies supply-chain auditing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui
Per review feedback: most testthat files use skip_on_cran(), and the workflow's purpose is to run them. setup-r currently sets NOT_CRAN=true implicitly, but the gate's behavior shouldn't depend on an action default — set it explicitly on the job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui
| NOT_CRAN: true | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 |
Per review: the header comment implied scheduled/manual runs exercise all numerical paths, but RUN_FULL_AEFA_TESTS=1 only unlocks the estimations guarded by the .skip_expensive_ci_calls helper — tests calling skip_on_ci() directly still skip on GitHub Actions regardless. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui
OpenCode Review Overview
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: test-suite.yaml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: test-suite.yaml"]
R1 --> V1["actionlint plus required checks"]
|
The strix and opencode-review failures on the previous head were caused by the org-central trusted-source regression (ContextualWisdomLab/.github strix.yml statuses permission vs. tightened smoke contract), which has since been fixed on that repo's main. Empty commit to re-run all checks against the repaired trusted source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A test-coverage audit found that CI executed almost none of the numerical test suite:
R-CMD-checkruns with--no-tests(plus a single Zh-rule file), andtest-fastruns only 3 productization files. The greedy model-selection, integration, edge-case, and stability suites — 26 testthat files total — never gated merges, which is the top systemic risk for a statistical package.This adds a
test-suiteworkflow that runs the complete testthat suite on a single ubuntu job:.skip_expensive_ci_calls,skip_on_ci) keep the expensive AEFA estimations out, so the run stays bounded (timeout-minutes: 180as a backstop).NOT_CRAN=trueis set explicitly at the job level soskip_on_cran()-guarded tests reliably run.RUN_FULL_AEFA_TESTS=1— the opt-in env var the test helpers already define — which unlocks the expensive estimations guarded by the.skip_expensive_ci_callshelper. Note that tests callingskip_on_ci()directly still skip on Actions regardless of this variable; extending those to honor an override would be a follow-up in the test files themselves.Existing workflows are untouched:
R-CMD-checkkeeps its 5-OS matrix cheap, andtest-fastremains the quick signal.Verification
R isn't available in the authoring environment, so this run relies on the suite's own guards (verified by reading
helper-test-data.R: the expensive-call skip triggers wheneverCIis set unlessRUN_FULL_AEFA_TESTS=1).actions/checkoutis pinned to the same v7.0.0 SHA used byR-CMD-check.yaml(standardized per review), and the r-lib action SHAs match the org's existing v2 pins. If the first PR run shows the guarded suite is still too slow, the push/PR trigger can be narrowed to schedule+dispatch only.🤖 Generated with Claude Code
https://claude.ai/code/session_01AMkMUa1c1RRsbHEfsQUhui