Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 32 additions & 10 deletions frontend/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ const packages = [
// Packages that also have developer-persona tests
const devPackages = ['smoke', 'dev-console', 'topology', 'webterminal'];

const setupDir = path.resolve(__dirname, 'e2e', 'setup');
const chromeAuth = {
...devices['Desktop Chrome'],
userAgent: INTEGRATION_TEST_USER_AGENT,
ignoreHTTPSErrors: true,
};

const chromeArgs = [
'--ignore-certificate-errors',
'--window-size=1920,1080',
Expand Down Expand Up @@ -127,16 +134,31 @@ export default defineConfig({
testMatch: 'teardown.setup.ts',
},

...packages.map((pkg) => ({
name: pkg,
testDir: path.resolve(__dirname, 'e2e', 'tests', pkg),
testIgnore: '**/developer/**',
dependencies: ['admin-auth'],
use: {
...chrome,
storageState: adminStorageState,
},
})),
...packages.flatMap((pkg, i) => {
const prevDep = i === 0 ? 'admin-auth' : packages[i - 1];
const refreshProject =
i > 0
? [
{
name: `admin-auth-refresh-${i}`,
testDir: setupDir,
testMatch: 'admin-auth.setup.ts',
dependencies: [prevDep],
use: { ...chromeAuth, launchOptions: { args: chromeArgs } },
},
]
: [];
return [
...refreshProject,
{
name: pkg,
testDir: path.resolve(__dirname, 'e2e', 'tests', pkg),
testIgnore: '**/developer/**',
dependencies: [i > 0 ? `admin-auth-refresh-${i}` : 'admin-auth'],
use: { ...chrome, storageState: adminStorageState },
},
];
}),
Comment on lines +137 to +161

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

printf '%s\n' '--- config outline ---'
ast-grep outline frontend/playwright.config.ts
printf '%s\n' '--- config lines ---'
sed -n '100,180p' frontend/playwright.config.ts
printf '%s\n' '--- related references ---'
rg -n --glob '!node_modules' 'admin-auth-refresh|admin-auth\.setup|playwright' frontend package.json .github 2>/dev/null | head -200

Repository: openshift/console

Length of output: 16744


🏁 Script executed:

printf '%s\n' '--- complete relevant config ---'
sed -n '1,175p' frontend/playwright.config.ts
printf '%s\n' '--- setup implementations ---'
sed -n '1,220p' frontend/e2e/setup/admin-auth.setup.ts
sed -n '1,180p' frontend/e2e/setup/login-helper.ts
printf '%s\n' '--- package and CI invocation context ---'
sed -n '1,70p' frontend/package.json
sed -n '150,190p' frontend/integration-tests/test-playwright-e2e.sh
rg -n --glob '!node_modules' 'test-playwright-admin|test-playwright-e2e|--max-failures|workers|fullyParallel|forbidOnly' .github frontend/config frontend 2>/dev/null | head -200

Repository: openshift/console

Length of output: 14647


🌐 Web query:

Playwright test projects dependencies dependent project runs when dependency test fails official documentation

💡 Result:

In Playwright, if a project dependency fails, the dependent projects are automatically skipped [1][2][3]. According to official Playwright documentation, dependencies are projects that must run and pass successfully before the tests in a dependent project can begin [1][2]. If any test within the dependency project fails, the dependent project will not be executed [1]. This behavior is designed to prevent tests from running in an environment that has not been correctly set up [2]. Currently, there is no native configuration option to allow dependent projects to proceed if a dependency fails [4][3]. Users have requested this functionality as a feature [4][3], but as of July 2026, the standard behavior remains that dependency failure results in the skipping of all dependent tests [1][2].

Citations:


🏁 Script executed:

python3 - <<'PY'
from pathlib import Path
import re

text = Path("frontend/playwright.config.ts").read_text()
packages = re.search(
    r"const packages = \[(.*?)\];", text, re.S
).group(1)
packages = re.findall(r"'([^']+)'", packages)

deps = {}
for i, package in enumerate(packages):
    refresh = f"admin-auth-refresh-{i}"
    if i > 0:
        deps[refresh] = [packages[i - 1]]
    deps[package] = [refresh if i > 0 else "admin-auth"]

# A dependency failure skips every transitive dependent project.
failed = {"console"}  # representative non-final package
skipped = set()
changed = True
while changed:
    changed = False
    for project, project_deps in deps.items():
        if project not in skipped and any(
            dep in failed or dep in skipped for dep in project_deps
        ):
            skipped.add(project)
            changed = True

print("package order:", packages)
print("dependencies:", deps)
print("failed project:", sorted(failed))
print("skipped projects:", sorted(skipped))
print("later packages skipped:", [
    package for package in packages[packages.index("console") + 1:]
    if package in skipped
])
PY

Repository: openshift/console

Length of output: 1167


Decouple package projects from the package dependency chain. admin-auth-refresh-${i} depends on the preceding package, and each later package depends on that refresh project. If a non-final package fails, Playwright skips its refresh project and all later packages. Run each package and refresh independently, then aggregate failures. Add a CI case with one failing test in a non-final package to ensure later package coverage runs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/playwright.config.ts` around lines 137 - 161, The package projects
in the packages flatMap currently form a sequential dependency chain through
admin-auth-refresh-${i} and prevDep, so an earlier failure skips later coverage.
Remove inter-package dependencies while preserving each package’s required auth
setup, configure refresh projects to run independently, and ensure failures are
aggregated rather than blocking subsequent packages. Add a CI test case with a
failing test in a non-final package and verify later package projects still
execute.

Comment on lines +137 to +161

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is something that is not convincing me here:

The logic depends on the packages array's order and position of its items, but nothing in the code makes this kind of constraint visible.

Anyone in the future can add, remove, or reorder packages without thinking about which one is "first", and this flatMap is creating a dependency chain where the first element smoke is treated specially.

I'm not completely sure that we should create this coupling (meaning making the order of packages so crucial).

Having said that, - in case we agree on this strict serial chain - the real issue for now is that we have only one special case: smoke, so my question is shoudn't we trade off this and just refresh every package uniformly as it already is? The tradeoff would be some seconds of refresh time for smoke package, but we could gain redability in case we want to change things in future (and of course we should point out that order in packages will matter from now on).

Suggested change
...packages.flatMap((pkg, i) => {
const prevDep = i === 0 ? 'admin-auth' : packages[i - 1];
const refreshProject =
i > 0
? [
{
name: `admin-auth-refresh-${i}`,
testDir: setupDir,
testMatch: 'admin-auth.setup.ts',
dependencies: [prevDep],
use: { ...chromeAuth, launchOptions: { args: chromeArgs } },
},
]
: [];
return [
...refreshProject,
{
name: pkg,
testDir: path.resolve(__dirname, 'e2e', 'tests', pkg),
testIgnore: '**/developer/**',
dependencies: [i > 0 ? `admin-auth-refresh-${i}` : 'admin-auth'],
use: { ...chrome, storageState: adminStorageState },
},
];
}),
...packages.map((_, i) => ({
name: `admin-auth-refresh-${i}`,
testDir: setupDir,
testMatch: 'admin-auth.setup.ts',
dependencies: [i === 0 ? 'admin-auth' : packages[i - 1]],
use: { ...chromeAuth, launchOptions: { args: chromeArgs } },
})),
...packages.map((pkg, i) => ({
name: pkg,
testDir: path.resolve(__dirname, 'e2e', 'tests', pkg),
testIgnore: '**/developer/**',
dependencies: [`admin-auth-refresh-${i}`],
use: { ...chrome, storageState: adminStorageState },
})),

...(hasDeveloper
? devPackages.map((pkg) => ({
name: `${pkg}-developer`,
Expand Down