test: stop config-mutating project tests from sharing a project - #455
test: stop config-mutating project tests from sharing a project#455alnr wants to merge 2 commits into
Conversation
Five test functions in this package run in parallel and all wrote
project configuration to the same two shared fixtures, defaultProject
and extraProject. Two pairs conflict directly:
- TestPatchProject replaces /services/identity/config/selfservice/
flows/error with an object, while TestPatchKratosConfig sets
/selfservice/flows/error/ui_url — the same key underneath, with a
different value.
- TestPatchPermissionConfig replaces /namespaces with legacy
definitions, while TestUpdateNamespaceConfig sets it to an Ory
Permission Language location.
Whichever landed last won, so the assertions failed intermittently in
CI and looked like staging flakiness.
Each config-mutating test now gets its own project, leaving
defaultProject and extraProject as read-only fixtures for the tests
that only read. Subtests within one function still share that project,
which is safe as long as each performs a single operation and asserts
on the response to it — the invariant patch_permission_config_test.go
already documented.
The project is created in a workspace of its own because the
development-project quota is per workspace; adding to the shared
workspace fails with "the quota for the feature 'Development Projects'
has been exceeded". TestUpdateProject and TestListProject already
create a workspace for the same reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JYzGVwAKQ4ormxRHZg1eDu
|
Warning Review limit reached
Next review available in: 10 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Warn at the declaration site that defaultProject and extraProject are read-only: the rule was only stated inside newProject's doc comment, which someone copying an existing runWithProjectAsDefault call has no reason to open. Explain why both project-selection styles share one project. Each case therefore applies its patch twice in a row, which is safe because the API materializes schema defaults back into the stored config, and is preferred over a project per style: the development-project quota is two per workspace, so that would double both the projects and the workspaces provisioned per run, and the burst makes this package's existing rate-limit flakiness worse. TestUpdateProject provisioned its project by inlining exactly what newProject does, so it now calls the helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JYzGVwAKQ4ormxRHZg1eDu
Problem
Five test functions in
cmd/cloudx/projectrun in parallel and all write project configuration to the same two shared fixtures,defaultProjectandextraProject. Two pairs conflict directly:TestPatchProject/services/identity/config/selfservice/flows/error={"ui_url":"https://example.org/error-ui"}TestPatchKratosConfig/selfservice/flows/error/ui_url="https://example.com/error-ui"Those are the same key underneath, with different values, on the same project.
TestPatchPermissionConfig/namespaces= legacy definitionsTestUpdateNamespaceConfig/namespaces= an OPL locationWhichever landed last won, so assertions failed intermittently and read as staging flakiness. This is what took down #451 twice —
TestPatchKratosConfig/is_able_to_add_a_key_with_stringfailing withNot equal, and separately aget oplround-trip finding no OPL on a project that had just been given one.patch_permission_config_test.goalready carried a comment noting that its own subtests share a project. The gap was cross-test contention, which nothing guarded.Fix
Each config-mutating test now gets its own project via
newProject(t), leavingdefaultProjectandextraProjectas read-only fixtures for the tests that only read (get_test.go,create_test.go,use_test.go— all verified to assert existence or the local config file, never mutated config).Subtests within one function still share that function's project, which is safe as long as each performs a single operation and asserts on the response to it — the invariant the permission test already documented, now stated on the helper.
The project goes in a workspace of its own. My first attempt put it in
defaultWorkspaceIDand failed with:The development-project quota is per workspace, which is why
TestUpdateProjectandTestListProjectalready create a workspace before their projects.Verification
Ran against live staging, not just compiled:
TestGetProject/TestGetServiceConfig: pass.cmd/cloudx/projectpackage: pass, 36s (no slower than before — the tests stay parallel, they just no longer collide).go vetandgolangci-lintclean.🤖 Generated with Claude Code
https://claude.ai/code/session_01JYzGVwAKQ4ormxRHZg1eDu