Yii3 application-server benchmarks

This repository measures the same Yii3 API application on several PHP application servers under repeatable, constant-throughput HTTP load. All runtime implementations live together on master, use the same application code, database seed, benchmark client, and report generator, and can be run individually or as one batch.

The suite is intended for comparing runtime behavior—not for declaring a universally fastest server. Results depend on the host, Docker version, CPU scheduling, runtime configuration, request rate, and benchmark duration. Compare runs made on the same machine with the same settings and minimal background activity.

What is included

Runtime key	Application server	Execution model
frankenphp-classic	FrankenPHP	A normal PHP application bootstrap for each request
frankenphp-worker	FrankenPHP	A persistent Yii worker
roadrunner	RoadRunner	A persistent Yii worker managed by RoadRunner
php-fpm	PHP-FPM + Nginx	Traditional FastCGI processes behind Nginx
freeunit	FreeUnit	PHP application hosted by FreeUnit

Every runtime is an isolated Docker Compose profile defined in docker/benchmarks.compose.yml. Each run receives its own PostgreSQL and Valkey containers and uses the same source tree mounted at /app. The PostgreSQL database is seeded from docker/postgres/initdb.d/10-benchmark.sql.

Two endpoints are benchmarked:

• / measures framework and runtime overhead with a minimal response.
• /postgres/orders measures a database-backed request that reads joined orders and customers rows through yiisoft/db-pgsql and persistent PDO connections.

Load is generated by a pinned build of wrkx, a maintained wrk2 derivative with constant-throughput load and coordinated-omission-aware latency recording. Docker CPU and memory usage are sampled alongside the HTTP results.

Requirements

• Linux with Docker Engine and the Docker Compose v2 plugin.
• GNU Make and Bash.
• Git for contributing.
• Enough available CPU, memory, disk space, and time to build all runtime images.
• Port 9991 available on the host.

PHP, Composer, wrkx, PostgreSQL, and Valkey do not need to be installed on the host for benchmark runs. The first run is slower because Docker must download and build the runtime images; later runs reuse cached layers.

Quick start

Clone the repository and run the complete matrix:

git clone git@github.com:Yii3-Benchmarks/app-api.git
cd app-api
make bench-all

This command sequentially:

1. Builds and starts each runtime with isolated PostgreSQL and Valkey services.
2. Waits for the stack and endpoint preflight check to succeed.
3. Benchmarks / and /postgres/orders with wrkx.
4. Captures application, database, and cache resource usage.
5. Stops the stack and removes its volumes before moving to the next runtime.
6. Generates one self-contained HTML report for the complete suite.

Results are written to a timestamped directory:

runtime/benchmarks/<timestamp>-suite/
├── <timestamp>-<runtime>-<target>-<mode>/
│   ├── metadata.env
│   ├── summary.json
│   ├── wrkx-timeseries.json
│   ├── wrkx-*.log
│   └── docker-stats.csv
└── report.html

Running selected benchmarks

Benchmark one runtime and the minimal endpoint:

make bench RUNTIME=roadrunner MODE=steady RATE=8000 DURATION=60s

Benchmark its PostgreSQL endpoint:

make bench-db RUNTIME=php-fpm MODE=steady RATE=4000 DURATION=60s

Run a subset of runtimes through both endpoints:

make bench-all RUNTIMES="frankenphp-worker roadrunner freeunit"

The underlying suite script also accepts a target subset:

RUNTIMES="roadrunner freeunit" TARGETS="home" MODE=steady RATE=5000 DURATION=60s \
    ./tools/run-benchmark-suite.sh

To start a runtime without benchmarking it:

make runtime-up RUNTIME=frankenphp-worker
curl http://localhost:9991/
curl http://localhost:9991/postgres/orders
make runtime-down RUNTIME=frankenphp-worker

runtime-down removes the selected runtime's database and cache volumes. Do not use it if you need to preserve manual changes made inside those benchmark containers.

Benchmark configuration

The default mode is ramp. Configuration is passed as Make variables or environment variables.

Variable	Default	Meaning
RUNTIME	frankenphp-classic	Runtime used by make bench and make bench-db
RUNTIMES	all five runtimes	Space-separated runtimes used by make bench-all
TARGETS	home postgres-orders	Space-separated endpoint keys for the suite script
MODE	ramp	steady for one rate or ramp for sequential rate stages
RATE	10000	Requests per second in steady mode
DURATION	160s	Steady-mode duration
THREADS	host CPU count	wrkx worker threads
CONNECTIONS	256	Concurrent HTTP connections
STAGES	eight stages from 5k to 50k RPS	JSON stage list for ramp mode
OUTPUT_ROOT	timestamped suite directory	Result destination

Example custom ramp:

STAGES='[{"target":1000,"duration":"30s"},{"target":3000,"duration":"30s"}]' \
    make bench RUNTIME=freeunit MODE=ramp

wrkx does not change rate continuously during a run. Ramp mode executes each STAGES entry as a separate constant-rate run and records one aggregate point per stage. wrkx uses an initial calibration period, so stages shorter than 20 seconds are not recommended.

Reports

The generated HTML report compares issued and successful RPS, errors, target-rate shortfall, average and p95 latency, connections, CPU, and memory. It is self-contained and can be opened directly in a browser or attached to an issue.

Regenerate a report from existing results:

make bench-report INPUT=runtime/benchmarks/<suite-directory>

Combine explicitly selected runs:

make bench-report INPUT="runtime/benchmarks/<run-1> runtime/benchmarks/<run-2>"

Raw wrkx output and exact run settings are retained next to the compact data. Include them when reporting unexpected results; an HTML chart alone is usually insufficient to reproduce a finding.

Repository structure

benchmark/                      wrkx image and Lua result adapter
config/, public/, src/          shared Yii3 API application
docker/benchmarks.compose.yml   isolated benchmark services and runtime profiles
docker/runtimes/                runtime images and server configuration
docker/postgres/initdb.d/       reproducible PostgreSQL benchmark data
tools/run-benchmark-suite.sh    multi-runtime orchestration and cleanup
tools/run-wrkx-benchmark.sh     one endpoint/stage benchmark runner
tools/compile-wrkx-results.php  wrkx output normalization
tools/render-benchmark-report.* HTML report generator
worker-frankenphp.php           FrankenPHP persistent worker entry point
worker-roadrunner.php           RoadRunner persistent worker entry point

The remaining application-template Docker files support development and tests. The benchmark matrix specifically uses docker/benchmarks.compose.yml and docker/runtimes/.

Testing changes

Install or update project dependencies through the development container when needed:

make composer-update

Run the automated checks relevant to your change:

make test
docker compose -f docker/benchmarks.compose.yml --profile roadrunner config --quiet
bash -n tools/run-benchmark-suite.sh tools/run-wrkx-benchmark.sh

Before submitting benchmark-related changes, run at least one short steady benchmark for the affected runtime. Use a duration of 20 seconds or more so wrkx calibration is meaningful:

make bench RUNTIME=roadrunner MODE=steady RATE=100 DURATION=20s THREADS=2 CONNECTIONS=8

Changes that affect shared application behavior should be checked against every runtime with make bench-all when practical.

Contributing

Contributions are welcome for runtime upgrades, new application servers, benchmark correctness, reporting, and reproducibility improvements.

1. Create a branch from master.
2. Keep shared application behavior identical across runtimes. Runtime-specific code belongs in docker/runtimes/ or a clearly named worker entry point.
3. Add or update tests and documentation with the implementation.
4. Run the checks above and record the exact smoke benchmark command you used.
5. Open a pull request describing the motivation, affected runtimes, validation performed, and any compatibility or performance tradeoffs. Do not present performance changes without the host and benchmark configuration.

Adding a runtime

To add another application server:

1. Add a named build target to docker/runtimes/Dockerfile and its configuration under docker/runtimes/.
2. Add a matching profile and service to docker/benchmarks.compose.yml, exposing the application on host port 9991.
3. Add the runtime key, readable label, and resource-sampled service names to tools/run-benchmark-suite.sh.
4. Add required PHP packages to composer.json; keep one shared lock file.
5. Add the runtime to the RUNTIMES default in Makefile and to the table in this README.
6. Verify / and /postgres/orders, run a short steady benchmark, and confirm report generation and automatic teardown.

Avoid committing generated benchmark output unless it is intentionally used as a published reference result. Never change only one runtime's application logic to improve its score—the suite must compare equivalent work.

License

The project is released under the BSD-3-Clause License. See LICENSE.md.