Skip to content

Commit 150bff4

Browse files
authored
perf(tests): trim sweep, copilot, and timer-bound suites; run three CI shards (#7375)
* perf(ci): run apps/sim as three test shards * chore(deps): drop unused @types/mdx devDependency from apps/docs Nothing in apps/docs imports mdx/types or a bare *.mdx module; the generated .source imports carry a ?collection= suffix that the *.mdx wildcard does not match, and @types/mdx still resolves transitively via @mdx-js/mdx for fumadocs. The docs type-check passes without the direct edge. * perf(tests): move DOM-free jsdom suites to the node environment Audited the 229 test files declaring @vitest-environment jsdom. 217 need a real DOM (React rendering, TipTap/ProseMirror, window/document/localStorage, DOMParser, ProgressEvent) and stay on jsdom; 12 exercise pure logic (yjs collab-doc conversion, http tool URL building, panel editor store, log utils, attachment upload, headless TipTap table commands, terminal-session tooltip helpers) and now run on node, skipping the jsdom boot. Dropped one redundant test in apply-streamed-markdown.test.ts: the hand-rolled one-shot peer case is a strict subset of the NON-OVERLAPPING two-peer test in apply-streamed-markdown.concurrent.test.ts, which also asserts CRDT convergence and exactly-once survival of the peer edit. * perf(tests): drive retry backoff and scheduled flushes with fake timers Real sleeps and repeated setup were the run-phase cost in seven files: - tools/index.test.ts: `retryDelayMs: 0` falls through to the 500ms default, so every retry test slept for real. Fake the clock for the retry describe. - embeddings/client.test.ts: two retry tests waited the 1s initial backoff. - hosted-key-rate-limiter.test.ts: the queue-head test slept two 200ms polls; the abort test slept 20ms wall clock and asserted on Date.now. - event-buffer.test.ts: sleep(60)/sleep(5) waited on the 15ms flush timer. - events.attribution.test.ts: walked and read the whole app twice, serially. Walk once in a beforeAll with batched concurrent reads. - sso-trust.test.ts: imported the Better Auth graph in each of three tests. Import once in a beforeAll that carries the 30s budget. - markdown-parse.test.ts: the one-shot helper built and destroyed a tiptap Editor per document; reuse one across the 400-seed property corpus. No production timing constants, corpus sizes, or assertions changed. * perf(tests): drop the executable tool registry and full block map from non-sweep tests The vitest suite is bounded by module loading, and the biggest single cost is a test file opting out of the global `@/tools/registry` or `@/blocks/registry` mock. Trim that where the test never needed the whole thing: - `lib/internal/tool-operations/registry.server.test.ts` checked registration against `getToolIds()` but still imported the executable registry for one sweep. That sweep — every operation-backed tool has no `request` and a registered handler — is the in-process half of the transport partition, so it moves next to the external-URL sweep in `tools/request-transport.test.ts`, which already pays for the registry. Handler loads now run in parallel. Collect 12.5s -> 0.1s, run 6s -> 4.3s; request-transport unchanged (~8s). - New `partialBlockRegistry` in `@sim/testing/mocks/block-registry.mock` builds the `@/blocks/registry-maps` shape from the block modules a test actually reads, so the real registry code runs over those blocks alone. Applied to the six non-sweep tests that unmocked the block registry for one or two blocks (webhook, slack, whatsapp, condition/function, knowledge/start_trigger): collect 3-8s -> 0.05-2s each. Registry sweeps keep the real map. - `tools/index.test.ts` retry cases set `retryDelayMs: 0`, which the retry config replaces with the 500ms default; `1` is the smallest delay honored. Run 4.0s -> 0.45s. - `tools/metadata.test.ts` collects sweep offenders into one assertion instead of one `expect` per registry entry, and indexes ids once. Run 1.1s -> 0.03s. - The `vi.unmock('@/tools/registry')` in the search-replace tests was dead: nothing in their graph imports the registry any more, and all 106 tests pass without it, contrary to the comment that claimed eight would fail. * perf(tests): stop copilot handler tests from loading the executor, block, and trigger registries The router test stubs every handler implementation module so the handler map binds without loading the executor and block registry; the 90s timeout goes. Handler, VFS, payload, and request tests mock the execution-side leaves and catalog registries they never exercise. Source edges that dragged in barrels for a single symbol now import the defining module: the Mothership block icon from @sim/emcn/icons, contract schemas from their contract file, and autolayout constants from new @sim/workflow-renderer/dimensions and /note-content subpaths. lib/copilot: 141 files, per-file collect+run 74.2s -> 37.4s, wall 19.1s -> 15.0s router.test.ts: 15.1s -> 0.45s * fix(tests): import ContractBodyInput from the contract types module * fix(tests): bind the real cancellation handler in the router test; mock the narrowed contract path * fix(emcn): wait for focus restoration instead of a fixed 20ms sleep in the modal test
1 parent 78780c9 commit 150bff4

58 files changed

Lines changed: 483 additions & 234 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/test-build.yml

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -182,17 +182,17 @@ jobs:
182182
# without `--coverage`. See the Codecov note below.
183183
#
184184
# apps/sim runs only its first shard here; `test-shard` below runs the
185-
# other. That suite is bound by the single Vite server thread that feeds
185+
# others. That suite is bound by the single Vite server thread that feeds
186186
# every worker — wall time is flat from 4 to 13 workers — so a bigger
187-
# runner buys nothing and a second runner halves it.
187+
# runner buys nothing and each extra runner takes a proportional slice.
188188
- name: Run tests
189189
env:
190190
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
191191
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
192192
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
193193
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
194194
TURBO_CACHE_DIR: .turbo
195-
SIM_TEST_SHARD: 1/2
195+
SIM_TEST_SHARD: 1/3
196196
run: bun run test
197197

198198
- name: Check schema and migrations are in sync
@@ -208,15 +208,20 @@ jobs:
208208
fi
209209
echo "✅ Schema and migrations are in sync"
210210
211-
# The second half of apps/sim's Vitest suite. Everything else — lint, the
212-
# audits, type-check, the other workspaces' suites — lives in `test-build`
213-
# with shard 1; this job exists only because that suite cannot go faster on
214-
# one machine (see the "Run tests" note there). The Turbo cache disk gets
215-
# its own key so the two shards' entries do not evict each other.
211+
# The remaining shards of apps/sim's Vitest suite. Everything else — lint,
212+
# the audits, type-check, the other workspaces' suites — lives in
213+
# `test-build` with shard 1; these jobs exist only because that suite cannot
214+
# go faster on one machine (see the "Run tests" note there). Three shards
215+
# put each runner at roughly the fixed cost of checkout + install. The Turbo
216+
# cache disk gets its own key so the shards' entries do not evict each other.
216217
test-shard:
217-
name: Test (shard 2)
218+
name: Test (shard ${{ matrix.shard }})
218219
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
219220
timeout-minutes: 15
221+
strategy:
222+
fail-fast: false
223+
matrix:
224+
shard: [2, 3]
220225

221226
steps:
222227
- name: Checkout code
@@ -250,7 +255,7 @@ jobs:
250255
uses: ./.github/actions/cache-mount
251256
with:
252257
provider: ${{ vars.CI_PROVIDER }}
253-
key: ${{ github.repository }}-turbo-cache-shard-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
258+
key: ${{ github.repository }}-turbo-cache-shard-${{ matrix.shard }}-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
254259
path: ./.turbo
255260

256261
- name: Install dependencies
@@ -259,14 +264,14 @@ jobs:
259264
- name: Install ripgrep
260265
run: command -v rg || (sudo apt-get update && sudo apt-get install -y ripgrep)
261266

262-
- name: Run tests (apps/sim shard 2/2)
267+
- name: Run tests (apps/sim shard ${{ matrix.shard }}/3)
263268
env:
264269
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
265270
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
266271
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
267272
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
268273
TURBO_CACHE_DIR: .turbo
269-
SIM_TEST_SHARD: 2/2
274+
SIM_TEST_SHARD: ${{ matrix.shard }}/3
270275
run: bunx turbo run test --filter=@sim/app
271276

272277
# Next.js production build, in parallel with lint + tests. Sticky disks are

apps/docs/package.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@
4242
"devDependencies": {
4343
"@sim/tsconfig": "workspace:*",
4444
"@tailwindcss/postcss": "^4.0.12",
45-
"@types/mdx": "^2.0.13",
4645
"@types/node": "24.2.1",
4746
"@types/react": "^19.1.2",
4847
"@types/react-dom": "^19.0.4",

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/code-languages.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*
44
* Guards against drift between the code-block language picker and the Prism grammars actually
55
* registered by CodeBlockHighlight: every selectable language must have a registered grammar, or it

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.test.ts

Lines changed: 0 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -157,41 +157,6 @@ describe('agent-stream applier', () => {
157157
expect(freshText).toContain('Gamma paragraph')
158158
})
159159

160-
it('preserves a concurrent peer edit to a region the agent snapshot does not include', () => {
161-
// This is the core "AI as a CRDT peer" guarantee: the agent relays only its OWN delta (computed
162-
// against a private shadow), never a whole-document reconcile that would revert a peer's edit.
163-
const { editor, doc } = track(makeCollabEditor())
164-
165-
const session = beginAgentStream(editor)!
166-
applyAgentStreamFrame(editor, session, 'Alpha paragraph.\n\nBeta paragraph.')
167-
168-
// A peer edits the FIRST paragraph directly on the shared doc — the agent's later snapshot still
169-
// carries the ORIGINAL first paragraph (it was built from the base, before this edit).
170-
const peer = new Y.Doc()
171-
Y.applyUpdate(peer, Y.encodeStateAsUpdate(doc))
172-
const peerFrag = peer.getXmlFragment('default')
173-
peer.transact(() => {
174-
const firstPara = peerFrag.get(0) as Y.XmlElement
175-
const textNode = firstPara.get(0) as Y.XmlText
176-
textNode.insert(textNode.toString().length, ' EDITED')
177-
})
178-
Y.applyUpdate(doc, Y.encodeStateAsUpdate(peer, Y.encodeStateVector(doc)))
179-
peer.destroy()
180-
181-
// The agent appends a third paragraph. Its snapshot's first paragraph is the stale original, but the
182-
// shadow-relayed delta only inserts the new paragraph — so the peer's " EDITED" must survive.
183-
applyAgentStreamFrame(
184-
editor,
185-
session,
186-
'Alpha paragraph.\n\nBeta paragraph.\n\nGamma paragraph.'
187-
)
188-
endAgentStream(session)
189-
190-
const live = doc.getXmlFragment('default').toString()
191-
expect(live).toContain('EDITED')
192-
expect(live).toContain('Gamma paragraph')
193-
})
194-
195160
it('reuses cached binding metadata across frames, still emitting minimal per-frame deltas', () => {
196161
// The binding `meta` is built ONCE (first frame) and reused — `updateYFragment` maintains it in place,
197162
// so we skip an O(doc) `initProseMirrorDoc` rebuild per frame. This guards that caching preserves the

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/image-drag-move.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*
44
* Dragging an image to reposition it inside a document must MOVE it, not import it again.
55
*

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/image.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*/
44
import { describe, expect, it } from 'vitest'
55
import {

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
* @vitest-environment jsdom
33
*/
44
import { Editor } from '@tiptap/core'
5-
import { afterEach, describe, expect, it } from 'vitest'
5+
import { afterAll, describe, expect, it } from 'vitest'
66
import { createMarkdownContentExtensions } from './extensions'
77
import { parseMarkdownToDoc, serializeMarkdownBody, splitMarkdownBlocks } from './markdown-parse'
88
import { isRoundTripSafe } from './round-trip-safety'
@@ -12,19 +12,20 @@ const isEmptyPara = (n: { type?: string; content?: unknown[] }): boolean =>
1212
n.type === 'paragraph' && !n.content?.length
1313

1414
let editor: Editor | null = null
15-
afterEach(() => {
15+
afterAll(() => {
1616
editor?.destroy()
1717
editor = null
1818
})
1919

20-
/** The current whole-document path: parse markdown in one shot, serialize back. */
20+
/**
21+
* The current whole-document path: parse markdown in one shot, serialize back. One editor serves
22+
* every call — `setContent` replaces the document wholesale, so a fresh instance per call only adds
23+
* the cost of building the view, which the property tests below paid hundreds of times over.
24+
*/
2125
function oneShot(body: string): string {
22-
editor = new Editor({ extensions: createMarkdownContentExtensions() })
26+
editor ??= new Editor({ extensions: createMarkdownContentExtensions() })
2327
editor.commands.setContent(body, { contentType: 'markdown' })
24-
const out = editor.getMarkdown()
25-
editor.destroy()
26-
editor = null
27-
return out
28+
return editor.getMarkdown()
2829
}
2930

3031
/**

apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/components/terminal-session/terminal-session.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*/
44

55
import { resolveDesktopZoom } from '@sim/desktop-bridge'

apps/sim/app/workspace/[workspaceId]/logs/utils.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*/
44
import { describe, expect, it } from 'vitest'
55
import { resolveLogWorkflowId, workflowEditorPath } from './utils'

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-attachment-upload.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* @vitest-environment jsdom
2+
* @vitest-environment node
33
*/
44
import { beforeEach, describe, expect, it, vi } from 'vitest'
55

0 commit comments

Comments
 (0)