POC: run tests in real-browser vitest mode - #2638
Conversation
matyasf
changed the title
|
| }) | ||
| DialogExample.displayName = 'DialogExample' | ||
|
|
||
| const NestedDialogExample = (props: DialogProps) => { |
There was a problem hiding this comment.
This was not used by the test at all
Visual regression report
Diff images (33)alert.png — baseline no longer producedavatar.png — baseline no longer producedbadge.png — baseline no longer producedbillboard.png — baseline no longer producedbreadcrumb.png — baseline no longer producedbutton-and-derivatives.png — baseline no longer producedbyline.png — baseline no longer producedcalendar.png — baseline no longer producedcheckbox.png — baseline no longer producedcheckboxgroup.png — baseline no longer producedcolorpicker.png — baseline no longer producedcontextview.png — baseline no longer producedcustom-and-lucide-icons.png — baseline no longer produceddateinput-dateinput2.png — baseline no longer produceddatetimeinput.png — baseline no longer produceddiff-demo.png — 6324 pixels differdrilldown.png — baseline no longer producedfiledrop.png — baseline no longer producedform-errors.png — baseline no longer producedheading.png — baseline no longer producedimg.png — baseline no longer producedlink.png — baseline no longer producedmenu.png — baseline no longer producedmetric-pill-tag-timeselect-text.png — baseline no longer producedoptions.png — baseline no longer producedpagination.png — baseline no longer producedprogressbar.png — baseline no longer producedselect-simpleselect.png — baseline no longer producedtable.png — baseline no longer producedtabs.png — baseline no longer producedtooltip.png — baseline no longer producedtreebrowser.png — baseline no longer producedview.png — baseline no longer producedBaselines come from the |
| enabled: true, | ||
| provider: playwright(), | ||
| headless: true, | ||
| instances: [{ browser: 'chromium' }], |
There was a problem hiding this comment.
we can add here other browsers easily to run our test suite on e.g. Firefox (but CI support might be tricky)
There was a problem hiding this comment.
Good idea; I would add Safari if it is possible, it has many tricky cases when it comes to a11y
| /// <reference types="vitest" /> | ||
|
|
||
| import { defineConfig } from 'vitest/config' | ||
| import { defineConfig, configDefaults } from 'vitest/config' |
There was a problem hiding this comment.
I havent checked this config thoroughly, we should do it, if this is the way forward
| - name: Install dependencies | ||
| run: pnpm install --frozen-lockfile | ||
| - name: Install Playwright browser | ||
| run: pnpm exec playwright install --with-deps chromium |
There was a problem hiding this comment.
why isnt it installed from dev deps?
There was a problem hiding this comment.
I also had to run this locally, pnpm install is not enough
There was a problem hiding this comment.
there's no way to express it as a dependency.
playwright ships the JS driver only; the actual Chromium build (~150 MB) lives in ~/.cache/ms-playwright, outside node_modules, and is fetched by playwright install. I've added this to the package.json scripts.
The with-deps flag installs system deps too, these are present on PCs with GUIs and would just slow down test runs
| find: `${pkgName}/src`, | ||
| replacement: path.join(pkgPath, 'src') | ||
| }) | ||
| // Fallback bare-name -> package dir (for packages without an exports map) |
There was a problem hiding this comment.
is this an existing scenario or just AI being cautious?
There was a problem hiding this comment.
AI being too cautinous, I've removed it
| @@ -49,6 +126,14 @@ export default defineConfig({ | |||
| // Allows using APIs like Jest without importing them | |||
| globals: true, | |||
There was a problem hiding this comment.
not part of the changeset, but maybe we should consider changing this to false?
There was a problem hiding this comment.
yep, but in a different PR
|
|
||
| // @instructure/console only outputs logs if process.env.NODE_ENV !== 'production' | ||
| // these messages are sometimes used in test assertions | ||
| if (typeof globalThis.process === 'undefined') { |
There was a problem hiding this comment.
this file is a bit hard to parse. is it only needed because of the console package?
There was a problem hiding this comment.
I dont know yet, it could be that other errors surface when we dont mock it
There was a problem hiding this comment.
as I see yes. We're using the NODE_ENV in 3 types of places:
- to add console warnings/errors in non-prod builds.
- to have different
displayNamefor stuff that is wrapped in decoratorswithStyleorWithDeterministicId. - to generate a less readable, but shorter
uidin prod.
As I see some tests are checking for console errors, this is why this mock is needed.
IMO in the future we should remove all NODE_ENV checks.
|
I like the browser-based runtime. There isn't a huge difference in the code syntax, which is also a great thing. I can't see any major drawbacks yet. I'd replace the component tests and Vitest tests with this if the test run is significantly faster. |
matyasf
changed the title
balzss
left a comment
balzss
left a comment
There was a problem hiding this comment.
this is good as is for preparing the complete migration. please squash the commit before rebasing to master
Add a `browser` Vitest project that runs real-browser component tests via Playwright, alongside the existing jsdom `web` and `node` projects. - migrate the emotion, ui-a11y-content, ui-a11y-utils, ui-alerts, ui-avatar, ui-badge and ui-dialog test suites to browser mode, and delete the Cypress specs they replace - alias `@instructure/*` workspace packages to TypeScript source so browser tests run without a prebuild, and alias bare `moment` to the all-locales build - exclude the migrated packages from the `web` project so they do not run twice - add `test:browser`, `test:browser-watch` and `test:browser-ui` scripts - add a `Vitest browser tests` job to PR validation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
4 participants
Add real-browser Vitest (Playwright/chromium) test POC, that replaces both Cypress component tests AND Vitest unit tests.
Summary
ui-alerts) and Dialog (ui-dialog) and some other suites tovitest/browsertest:browserscripts and new vitest configs.Test Plan
pnpm run test:browserruns the browser tests in the consolepnpm run test:browser-uiruns the browser tests in a browserFuture additions:
Fixes INSTUI-5098
🤖 Generated with Claude Code