Added the visual regression report to the GitHub Actions run summary. - #2865
Conversation
Also fixed 'pages_changed' counting every page as changed, as Diffy returns 'percentageChanges' as a string and the comparison ran against the concatenated value.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 6 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (2)
Comment |
|
Code coverage (threshold: 90%) Per-class coverage |
This comment has been minimized.
This comment has been minimized.
2 similar comments
This comment has been minimized.
This comment has been minimized.
|
Code coverage (threshold: 90%) Per-class coverage |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2865 +/- ##
==========================================
- Coverage 86.81% 86.38% -0.43%
==========================================
Files 100 93 -7
Lines 4846 4687 -159
Branches 47 3 -44
==========================================
- Hits 4207 4049 -158
+ Misses 639 638 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
📖 Documentation preview for this pull request has been deployed to Netlify: https://6a69bac2439d02e7c9adcf1a--vortex-docs.netlify.app This preview is rebuilt on every commit and is not the production documentation site. |
Summary
The visual regression result was readable only in the
vr-comparejob log and, for PR runs, in a sticky PR comment thatvr-reportassembled itself from four separate job outputs. This adds aWrite run summarystep tovr-comparethat renders the report once into$GITHUB_STEP_SUMMARY, so the result is visible directly in the GitHub Actions run UI - including on manual dispatches, where no PR comment is posted at all. The same rendered markdown is exposed as a newsummaryjob output and reused verbatim byvr-report, so the run summary and the PR comment can no longer drift apart. The change also fixes ajqbug that made the report claim every page had changed regardless of its actual diff percentage.Changes
Write run summarystep to thevr-comparejob. It renders a heading, a one-lineX of Y pages changed, Z% overall differencestatement, a settings table (source environment, target environment, diff ID), and a link to the Diffy report into$GITHUB_STEP_SUMMARY, and writes the same markdown to asummarystep output. Every interpolated value is passed through the step'senvblock rather than inlined into the script.summary: ${{ steps.summary.outputs.summary }}as avr-comparejob output.vr-reportsticky comment tomessage: ${{ needs.vr-compare.outputs.summary }}, replacing the hand-assembled bullet list that readpages_changed,pages_total,changes_percent, andshared_urlindividually and additionally depended on workflow-levelenvresolving in the second job.pages_changedexpression in theFetch comparison resultstep. Diffy returnspercentageChangesas a JSON string, so the previousmap(.value.percentageChanges // 0) | add > 0concatenated the string values ("0" + "0"becomes"00") and, becausejqsorts every string above every number, the comparison was true for every page - the report always claimed all pages had changed. The new[.[].percentageChanges | tonumber? // 0] | add > 0coerces each value to a number before summing. Checked against a sample Diffy payload: the old expression reported 3 of 3 pages changed where the new one correctly reports 1 of 3, and the new form tolerates numeric values, absentpercentageChangeskeys, and empty page objects without erroring..vortex/docs/content/development/visual-regression.mdxto match: thevr-comparestep list gained the run-summary step, thevr-reportentry now states it posts the same report, both ASCII flow diagrams show where the run summary is produced, and the manual-dispatch note no longer claims the result is visible "only in the Diffy UI (and in the workflow run log)"..vortex/installer/tests/Fixtures/handler_process/visual_regression_enabled/.github/workflows/test-vr.ymlwithahoy update-snapshots.Screenshots
N/A - the change covers a CI workflow and a documentation page.
Before / After