You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test(site-type-wp): add a Behat suite and CI workflow - #245
Extends the three old feature files into a Behat suite for site-type-wp (features/: 116 scenarios, outline rows counted) and replaces test_and_build.yml with a Behat workflow that runs it on every pull request, on pushes to develop and master, and on demand.
It covers the WordPress site lifecycle, labels and www redirects, install options, VIP, multisite, redis and proxy caches, PHP versions (current and legacy images), local, global and external databases, SSL (self-signed, custom, inherited, one Let's Encrypt issuance) and alias domains. features/COVERAGE.md maps each command, flag and behaviour to its scenarios.
The workflow runs the suite inside a fresh core checkout: core develop is cloned, every easyengine/* package is set to dev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's. The old job tested the Packagist dev-develop package instead.
Lanes: main (107 scenarios), slow (7, legacy PHP images) and le (2), each on PHP 7.4 and 8.0 to 8.5. The main lane's jobs are named Behat Tests - PHP <version>, so the required checks keep matching. The main lane also builds the phar without dev autoloading, as the old job did. @destructive scenarios run there too, since the runner is thrown away.
@le runs against a local Pebble ACME server (.github/behat/pebble-setup.sh). Every other lane points the Let's Encrypt host at a blackhole and checks that the real service can't be reached, and on CI the suite refuses to start unless one of the two is in place, so no run can reach the real Let's Encrypt.
Scenarios written for open bugs (12) are left out for now: they assert the fixed behaviour, so each comes back with its fix. Every scenario in this PR gates, and COVERAGE.md lists the bugs that aren't covered yet.
To run it locally (as root, on a test machine, since the suite creates and deletes *.test sites): copy features/ over a core checkout's features/ and run sudo -E vendor/bin/behat --tags='~@le' from the core root; add ~@destructive to skip the scenarios that change global services or config, and ~@slow to skip the legacy PHP images.
Fix upstream steps that could never fail (unanchored I run pattern, inverted label count, compose v1 container names, swapped subsite arguments) and replace the delete-all cleanup with prefix-scoped cleanup of sites, docker objects, global DB objects, redis keys and hosts entries. Add generic steps for command results, site info, EE database, files, containers, networks, volumes, global DB/redis, HTTP and hosts.
…e context
Add a step that matches a deleted site's random DB user by prefix, and a guard so a scenario that inspects an earlier scenario's leftovers fails when run alone instead of passing vacuously. The feature cleanup now also removes this suite's nginx-proxy certs and ACME state, which `--ssl=off` and LE deletes leave in the global proxy.
…en assertions
Add known-bug scenarios for the level-0 rollback dropping another site's database (custom SSL typo), the user-visible HTTPS redirect after an alias change on a site whose SSL was turned off, and self-signed sites that can't turn SSL off. The nohttps scenario was mislabelled as a #488 regression: the flag was already false for sites without SSL. Reference the WPT issue IDs, match DB_CHARSET and MULTISITE exactly, check that delete and the leak scenarios remove the random DB user, assert the w3tc error, and stop the missing-site delete from depending on the previous scenario. Update COVERAGE.md (126 scenarios, 27 known-bug, 86 in the default run).
… group
Under a TTY, `timeout` puts ee in a background process group; the `docker exec -it` of the global redis purge then stops on SIGTTOU together with timeout, so the AfterFeature delete of a --cache site never returns. -k 30 because ee catches SIGTERM without dispatching it (MIG-9).
Match the site list JSON pretty-printed or compact, accept lang="de" (the translation picks the html lang attribute), use full installs for the redirect checks (WPT-17 race with --skip-install), and look for the SEC-1 marker in $HOME, which sudo -E keeps as the caller home on CI.
…gine#239)
AUTH-1, AUTH-5 and the blank-alias scenario pass on develop now, so they are regular regression scenarios. The WPT-2 + WPT-12 alias-change scenario passed there only because the update returns before docker-gen has the recreated container; it waits for the new alias to be served before it checks the main domain.
Right after create the proxy reload is still in flight (PRX-5): on the PHP 7.4 runner the first HTTPS request failed with curl error 55. Wait until a verified HTTPS request succeeds, then check the status and verify result.
The database name/user collision, failed custom-SSL create, existing webroot and inherit + subdom scenarios pass with the fix/site-name-collisions branches of site-command and site-type-wp: the name collision is now refused by the docker volume check and --dbuser of another site up front, so both assert the new error.
…lias
CACHE-1, WP-1 and SEC-1 scenarios are regular; the weak password message no longer echoes the password (EasyEngine#244); the wildcard refusal runs on a site that never had SSL, since one that had --ssl=self keeps site_ssl_wildcard=1 and goes on to request a certificate; the new alias is polled until the proxy serves it (PRX-7).
…line output
compareContents() compared list elements by position, so [1,3,5] wasn't found in [1,2,3,4,5], and a missing key or property raised a notice. Both something-like steps only used the first line of the expected text.
EE's hardcoded Let's Encrypt endpoint is redirected to a local Pebble on the runner (hosts entry, throwaway CA, resolved routing for the test domain). No product code changes.
Runs the package features/ inside a fresh core develop checkout, with this checkout installed as the package through a Composer path repository, so the PR code is under test. @le runs in behat-le.yml against Pebble. @slow (legacy PHP images) is its own job; @known-bug scenarios run in a separate non-gating job.
Run PHP 7.4 and 8.5, strip control characters from the junit files before building failures.txt, and cap the Behat step at 75 minutes so a hung run still uploads its logs. The @le lanes against Pebble move into behat.yml (lanes named le*), replacing behat-le.yml. Diagnostics: the host addresses and routes, and the docker-compose version after the first run (MIG-20).
EE's first run downloads docker-compose 2.27.0 without --fail (MIG-10); on a GitHub error it moves the HTML error page over the binary, as on an admin-tools-command runner. Fetch 2.27.0 again with --fail and retries in that case.
Docker helpers check the exit status before reading output, and network absence comes from a successful ls instead of an inspect failure, so a failed query no longer reads as nothing left.
curl_exec() returns false when the body ends short of Content-Length, but the status code and headers already arrived. The helper kept code 200 while replacing the headers with a failure note, so the 200 poll and "the response should not have header 'X-Proxy-Cache'" (proxy-cache.feature:57-58) could both pass on a response that carried the header. Map a failed transfer to code 0 so the existing guards reject or retry it.
… its database
The feature cleanup opened ee.sqlite unconditionally, so on a host where core had not run a non-cli command yet the suite failed before any scenario (PDO cannot open a missing directory, and an empty file has no sites table). With no database there are no sites to delete.
A feature that starts with "a WordPress site ... exists" looked the site up before core had created ee.sqlite, so on a fresh host it failed instead of creating the site (and opening the missing file left an empty database behind for the cleanup). No database now means no row, and the create command initialises core as usual.
…he time left
An attempt ran without a time limit, so a stalled command (the LE scenario's curl has no maximum time) could hold the step far past its deadline. Each attempt now runs under timeout with the remaining budget.
wait_for counted 60 attempts but ran each one without a limit, so a stalled curl to the Pebble directory could hold the job until its timeout instead of failing after 60 seconds. It now stops after 60 seconds and runs each attempt under timeout with the time left.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Privileged cleanup and host networking changes need human validation, the fresh-host initialization issue remains unresolved, and prerequisite core PR #1950 is still open.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Initialize core before Behat prepareFeature SQLite queries
features/bootstrap/FeatureContext.php:1619
On a fresh local test host, the documented Behat invocation fails before any scenario runs because prepareFeature() queries SQLite before core has initialized its database. PDO cannot open a missing database directory, and an empty database has no sites table. Initialize core before cleanup, or document initialization as a prerequisite and validate it with an actionable error.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Root-level networking and cleanup need maintainer validation, helper defects remain unresolved, and the required core dependency is still unmerged.
No open issues 🎉
3 previously missed issues
In code that hasn't changed since last review
Enforce wall-clock deadlines for synchronous attempts
.github/behat/pebble-setup.sh:150
This limits the number of attempts, not the elapsed time: "$@" runs synchronously with no timeout. In particular, the directory probe calls curl without connection or total time limits, so a stalled request prevents the advertised 60-second failure and can consume the job's 150-minute timeout before Behat starts. Apply a wall-clock deadline and bound each attempt to its remaining time.
Bound subprocess execution to the remaining deadline
features/bootstrap/FeatureContext.php:547
EE::launch() blocks until the command exits, so the deadline cannot stop a running attempt. For example, sleep 2 succeeds with a one-second limit; the LE scenario's curl has no maximum duration and can instead stall until the workflow's 75-minute timeout. Bound each subprocess to the remaining budget and reject success after the deadline.
Handle missing SQLite database before prerequisite creation
features/bootstrap/FeatureContext.php:1419
On a fresh installation, a Given-first feature such as features/labels.feature calls this lookup before bin/ee initializes SQLite, so it fails instead of creating the prerequisite site. If the database directory already exists, opening the missing file also leaves an empty database that breaks subsequent cleanup. Return false when ee.sqlite is absent so the prerequisite create command can perform normal initialization.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Host-wide cleanup, DNS, firewall, and trust-store changes warrant human validation, and prerequisite EasyEngine/easyengine#1950 remains unmerged.
No open issues 🎉
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the three old feature files into a Behat suite for site-type-wp (
features/: 116 scenarios, outline rows counted) and replacestest_and_build.ymlwith aBehatworkflow that runs it on every pull request, on pushes todevelopandmaster, and on demand.It covers the WordPress site lifecycle, labels and www redirects, install options, VIP, multisite, redis and proxy caches, PHP versions (current and legacy images), local, global and external databases, SSL (self-signed, custom, inherited, one Let's Encrypt issuance) and alias domains.
features/COVERAGE.mdmaps each command, flag and behaviour to its scenarios.developis cloned, everyeasyengine/*package is set todev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's. The old job tested the Packagistdev-developpackage instead.Behat Tests - PHP <version>, so the required checks keep matching. The main lane also builds the phar without dev autoloading, as the old job did.@destructivescenarios run there too, since the runner is thrown away.@leruns against a local Pebble ACME server (.github/behat/pebble-setup.sh). Every other lane points the Let's Encrypt host at a blackhole and checks that the real service can't be reached, and on CI the suite refuses to start unless one of the two is in place, so no run can reach the real Let's Encrypt.Merge after EasyEngine/easyengine#1950, which stops core's test job from running this suite unfiltered.
To run it locally (as root, on a test machine, since the suite creates and deletes
*.testsites): copyfeatures/over a core checkout'sfeatures/and runsudo -E vendor/bin/behat --tags='~@le'from the core root; add~@destructiveto skip the scenarios that change global services or config, and~@slowto skip the legacy PHP images.Green run on my fork (PHP 7.4 and 8.0 to 8.5, every lane): https://github.com/mrrobot47/site-type-wp/actions/runs/39380700993