test(ai): migrate tests to Vitest by Manvi1203 · Pull Request #10363 · firebase/firebase-js-sdk

Manvi1203 · 2026-09-08T19:29:27Z

Description

Migrates @firebase/ai unit tests from legacy Karma & Mocha to Vitest (Node + Browser Chromium).

Key Changes

Vitest Runner: Added vitest.config.mjs extending config/vitest.base.mjs and removed deprecated karma.conf.js.
Test Scripts: Standardized npm test scripts to pure vitest commands (test:all, test:node, test:browser).
ESM & Browser Fixes:
- Replaced Node-specific global references with standard ECMAScript globalThis in live-session-helpers.test.ts.
- Replaced Mocha this.skip() and describe early returns with conditional runner skips (describe.skip / it.skip) in chrome-adapter.test.ts.
- Handled Sinon fake timer asynchronous promise rejections before clock.tickAsync in request.test.ts.
- Added mock WebSocket constants and bounded stream polling in websocket.test.ts.
- Hoisted Sinon stubs and sandbox cleanup for native ESM browser execution across generative model and chat session tests.
Type Cleanup: Separated type-only exports in src/api.ts and src/types/index.ts.
Ambient Types: Added test/types/vitest-globals.d.ts for isolated test typings.

Testing & Impact

Execution Time: Dropped from ~15s (Karma + Mocha) to 6.4s (Vitest).

changeset-bot · 2026-09-08T19:29:32Z

⚠️ No Changeset found

Latest commit: ddbb18a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-09-08T19:29:41Z

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v17.0

gemini-code-assist

Code Review

This pull request migrates the test suite for the packages/ai package from Karma and Mocha to Vitest, which includes updating package scripts, adding Vitest configuration files, and refactoring several test files to use Vitest's mocking utilities (vi.mock, vi.hoisted). Feedback on these changes highlights a regression where TypeScript type checking was inadvertently disabled in both local and CI test scripts. Additionally, in chat-session.test.ts and template-chat-session.test.ts, the custom stubbing mechanism lacks a corresponding custom restore function to reset the mocked properties, which could lead to test pollution and flaky tests.

gemini-code-assist · 2026-09-08T19:30:53Z

+    "test": "run-p --npm-path npm lint test:all",
+    "test:ci": "node ../../scripts/run_tests_in_ci.js -s test:all",


Removing type-check from the test script and changing test:ci to run test:all instead of test completely disables TypeScript type checking in both local testing and CI. This is a regression that could allow type errors to be merged. Please restore type-check to the test script and configure test:ci to run test.

Suggested change

"test": "run-p --npm-path npm lint test:all",

"test:ci": "node ../../scripts/run_tests_in_ci.js -s test:all",

"test": "run-p --npm-path npm lint type-check test:all",

"test:ci": "node ../../scripts/run_tests_in_ci.js -s test",

gemini-code-assist · 2026-09-08T19:30:53Z

+import {
+  match,
+  restore,
+  SinonSpy,
+  spy,
+  stub as sinonStub,
+  useFakeTimers
+} from 'sinon';
 import sinonChai from 'sinon-chai';
 import chaiAsPromised from 'chai-as-promised';
 import * as generateContentMethods from './generate-content';
+
+const { mockGenerateContent } = vi.hoisted(() => ({
+  mockGenerateContent: {
+    generateContent: (..._args: any[]): any => {},
+    generateContentStream: (..._args: any[]): any => {}
+  }
+}));
+
+vi.mock('./generate-content', async importOriginal => {
+  const actual = await importOriginal<any>();
+  return {
+    ...actual,
+    generateContent: (...args: any[]) =>
+      mockGenerateContent.generateContent(...args),
+    generateContentStream: (...args: any[]) =>
+      mockGenerateContent.generateContentStream(...args)
+  };
+});
+
+function stub(obj?: any, method?: any): any {
+  if (obj === generateContentMethods) {
+    const s = sinonStub();
+    (mockGenerateContent as any)[method] = s;
+    return s;
+  }
+  return (sinonStub as any)(...arguments);
+}


The custom stub function intercepts calls to generateContentMethods and assigns anonymous stubs to mockGenerateContent. However, since these are anonymous stubs and not registered with sinon's global sandbox, calling restore() (which is imported directly from sinon) will not reset these stubs between tests. This leads to test pollution and flaky tests. Define a custom restore function that calls sinonRestore() and resets the mockGenerateContent properties back to their default dummy functions.

import { match, restore as sinonRestore, SinonSpy, spy, stub as sinonStub, useFakeTimers } from 'sinon'; import sinonChai from 'sinon-chai'; import chaiAsPromised from 'chai-as-promised'; import * as generateContentMethods from './generate-content'; const { mockGenerateContent } = vi.hoisted(() => ({ mockGenerateContent: { generateContent: (..._args: any[]): any => {}, generateContentStream: (..._args: any[]): any => {} } })); vi.mock('./generate-content', async importOriginal => { const actual = await importOriginal<any>(); return { ...actual, generateContent: (...args: any[]) => mockGenerateContent.generateContent(...args), generateContentStream: (...args: any[]) => mockGenerateContent.generateContentStream(...args) }; }); function restore(): void { sinonRestore(); mockGenerateContent.generateContent = (..._args: any[]): any => {}; mockGenerateContent.generateContentStream = (..._args: any[]): any => {}; } function stub(obj?: any, method?: any): any { if (obj === generateContentMethods) { const s = sinonStub(); (mockGenerateContent as any)[method] = s; return s; } return (sinonStub as any)(...arguments); }

gemini-code-assist · 2026-09-08T19:30:53Z

+import { match, restore, SinonSpy, spy, stub as sinonStub } from 'sinon';
 import sinonChai from 'sinon-chai';
 import chaiAsPromised from 'chai-as-promised';
 import * as generateContentMethods from './generate-content';
+
+const { mockGenerateContent } = vi.hoisted(() => ({
+  mockGenerateContent: {
+    templateGenerateContent: (..._args: any[]): any => {},
+    templateGenerateContentStream: (..._args: any[]): any => {}
+  }
+}));
+
+vi.mock('./generate-content', async importOriginal => {
+  const actual = await importOriginal<any>();
+  return {
+    ...actual,
+    templateGenerateContent: (...args: any[]) =>
+      mockGenerateContent.templateGenerateContent(...args),
+    templateGenerateContentStream: (...args: any[]) =>
+      mockGenerateContent.templateGenerateContentStream(...args)
+  };
+});
+
+function stub(obj?: any, method?: any): any {
+  if (obj === generateContentMethods) {
+    const s = sinonStub();
+    (mockGenerateContent as any)[method] = s;
+    return s;
+  }
+  return (sinonStub as any)(...arguments);
+}


The custom stub function intercepts calls to generateContentMethods and assigns anonymous stubs to mockGenerateContent. However, since these are anonymous stubs and not registered with sinon's global sandbox, calling restore() (which is imported directly from sinon) will not reset these stubs between tests. This leads to test pollution and flaky tests. Define a custom restore function that calls sinonRestore() and resets the mockGenerateContent properties back to their default dummy functions.

import { match, restore as sinonRestore, SinonSpy, spy, stub as sinonStub } from 'sinon'; import sinonChai from 'sinon-chai'; import chaiAsPromised from 'chai-as-promised'; import * as generateContentMethods from './generate-content'; const { mockGenerateContent } = vi.hoisted(() => ({ mockGenerateContent: { templateGenerateContent: (..._args: any[]): any => {}, templateGenerateContentStream: (..._args: any[]): any => {} } })); vi.mock('./generate-content', async importOriginal => { const actual = await importOriginal<any>(); return { ...actual, templateGenerateContent: (...args: any[]) => mockGenerateContent.templateGenerateContent(...args), templateGenerateContentStream: (...args: any[]) => mockGenerateContent.templateGenerateContentStream(...args) }; }); function restore(): void { sinonRestore(); mockGenerateContent.templateGenerateContent = (..._args: any[]): any => {}; mockGenerateContent.templateGenerateContentStream = (..._args: any[]): any => {}; } function stub(obj?: any, method?: any): any { if (obj === generateContentMethods) { const s = sinonStub(); (mockGenerateContent as any)[method] = s; return s; } return (sinonStub as any)(...arguments); }

- Replace anonymous stubs and manual afterEach null-resets with delegated mock implementations in count-tokens and generate-content tests. - Remove custom restore override in generative-model tests in favor of standard sinon.restore. - Use native Vitest runIf conditional runners in chrome-adapter and live-session-helpers tests to eliminate ESLint bypass comments. TAG=agy CONV=0e4d0bde-5c64-450d-ab5a-2a4f7489262c

Manvi1203 · 2026-09-09T22:00:03Z

-      controller.abort(abortReason);
-
-      await expect(requestPromise).to.be.rejectedWith(
+      const assertion = expect(requestPromise).to.be.rejectedWith(


Attaching the rejection listener before ticking the clock. Otherwise, ticking the clock triggers the timeout and rejects the promise before the handler is attached, causing an UnhandledPromiseRejection error in Vitest.

Manvi1203 · 2026-09-09T22:01:22Z

      mockWebSocket.triggerMessage(new Blob([JSON.stringify({ foo: 2 })]));

-      await clock.tickAsync(5);
+      while (received.length < 2) {


[test improvement- not related to migration] Blob.text() is an async promise not controlled by fake timers. Wait for both messages to finish decoding before closing the socket.

Manvi1203 · 2026-09-09T22:02:00Z

  let webSocketStub: SinonStub;

  beforeEach(() => {
+    if (typeof (globalThis as any).WebSocket === 'undefined') {


[test improvement- node tests crashing on node 20] Node 20 lacks global WebSocket (added in Node 22). This dummy class prevents sinon.stub(globalThis, 'WebSocket') from crashing on Node 20.

Manvi1203 · 2026-09-09T22:02:46Z

 import sinonChai from 'sinon-chai';
 import chaiAsPromised from 'chai-as-promised';
 import * as generateContentMethods from './generate-content';
+


ESM modules are frozen in Vitest, so sinon.stub() can't modify them directly. This proxy delegates to real code by default, letting us keep existing Sinon stubs.

…n-Chai patterns - Replace legacy Chai assertions with native Vitest matchers - Eliminate Sinon stub and spy helpers in favor of native vi.spyOn and vi.fn implementations - Replace custom match and match.any helpers with expect.anything(), expect.objectContaining(), and expect.toSatisfy() - Establish test/setup.ts to restore mocks and reset timers across test runs

Manvi1203 requested a review from a team as a code owner September 8, 2026 19:29

gemini-code-assist Bot reviewed Sep 8, 2026

View reviewed changes

Manvi1203 marked this pull request as draft September 8, 2026 21:37

test(ai): migrate tests to Vitest

52d7592

Manvi1203 force-pushed the feature/vitest-ai branch from a5ab877 to 52d7592 Compare September 8, 2026 21:39

Manvi1203 commented Sep 9, 2026

View reviewed changes

Manvi1203 marked this pull request as ready for review September 9, 2026 22:03

Navigation Menu

Uh oh!

test(ai): migrate tests to Vitest - #10363

test(ai): migrate tests to Vitest#10363
Manvi1203 wants to merge 3 commits into
mainfrom
feature/vitest-ai

Manvi1203 commented Sep 8, 2026

Uh oh!

changeset-bot Bot commented Sep 8, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Uh oh!

Manvi1203 Sep 9, 2026

Uh oh!

Manvi1203 Sep 9, 2026

Uh oh!

Manvi1203 Sep 9, 2026

Uh oh!

Manvi1203 Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		"test": "run-p --npm-path npm lint test:all",
		"test:ci": "node ../../scripts/run_tests_in_ci.js -s test:all",

Uh oh!

Conversation

Manvi1203 commented Sep 8, 2026

Description

Key Changes

Testing & Impact

Uh oh!

changeset-bot Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Sep 8, 2026

Vertex AI Mock Responses Check ⚠️

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

Manvi1203 Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Manvi1203 Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Manvi1203 Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Manvi1203 Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

changeset-bot Bot commented Sep 8, 2026 •

edited

Loading