Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions benchmarks/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@
/results/local/
/results/tmp/
/profiles/

# Comparator-local dependencies
/comparators/*/vendor/
105 changes: 105 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,108 @@ A result qualifies as a reference baseline only when the documented PHP 8.4 prot
Baseline comparison must reject casual comparisons when the environment fingerprint differs. The normalized result schema includes scenario identifiers, sample counts, timing statistics, percentiles when enough samples exist, relative standard deviation, throughput where derivable, memory fields, environment fingerprint, source SHA, and schema version.

Cross-framework comparison, optimization work, and regression budgets are intentionally outside this initial harness. They should be introduced only after baseline measurements are reproducible and the relevant comparison methodology is defined.

## Cross-Framework Comparator Infrastructure

Benchmark and comparator infrastructure is EvolvePHP framework-development tooling. It is not required by applications built with EvolvePHP and is intentionally excluded from normal production application dependency graphs.

The comparator foundation establishes fixture correctness, dependency isolation and reproducibility. It does not establish that EvolvePHP is faster than Laravel, Symfony, Slim or Phalcon.

Authoritative cross-framework performance measurements require the documented controlled PHP 8.4 benchmark environment.

### Comparator Installation

Install the benchmark harness explicitly:

```powershell
composer install --working-dir=benchmarks --no-interaction
```

Install comparator fixtures only when doing framework-maintainer comparator work:

```powershell
composer install --working-dir=benchmarks/comparators/evolvephp --no-interaction
composer install --working-dir=benchmarks/comparators/laravel --no-interaction --ignore-platform-req=ext-fileinfo
composer install --working-dir=benchmarks/comparators/symfony --no-interaction
composer install --working-dir=benchmarks/comparators/slim --no-interaction
composer install --working-dir=benchmarks/comparators/phalcon --no-interaction
```

The Laravel lockfile was created on this local PHP CLI with `--ignore-platform-req=ext-fileinfo` because `ext-fileinfo` is not enabled here. The comparator workload does not use file inspection.

### Comparator Smoke

Validate comparator fixture correctness without running a controlled benchmark study:

```powershell
php benchmarks\bin\comparator-smoke.php
```

The smoke command loads `benchmarks/comparators/matrix.json`, validates fixture paths and lockfiles, exercises the common scenarios, reports Phalcon unavailable when the extension is missing, and exits non-zero if an available comparator is broken. It emits no rankings and makes no performance claims.

### Comparator Versions

The comparator matrix records these selected versions and lockfile hashes:

| Comparator | Package | Version | Constraint | Lockfile SHA-256 |
| --- | --- | --- | --- | --- |
| EvolvePHP | `evolvephp/http` | `2.0.0-dev+9a0e741` | `^2.0@dev` | `f792575ec5491c8d3aa171ba5f7de3b38558bfbd82b977beea45e603fd79e491` |
| Laravel | `laravel/framework` | `13.29.0` | `13.29.0` | `33b4d04706fa39dffc1d71a7d2d03f09651555afead629e31f9229adcdc86354` |
| Symfony | `symfony/http-kernel` | `8.1.5` | `8.1.5` | `d93fdac19b2cdd5379e5700a8146bb705c9b516c9ec9a0709dcd785be9b1e1d6` |
| Slim | `slim/slim` | `4.15.2` | `4.15.2` | `87370678970fe51c62c6a4cd4e4ca7b3600b22c84a2a5b920e2b8527a2e089a7` |
| Phalcon | `ext-phalcon` | `5.20.3` expected | `suggest ext-phalcon 5.20.3` | `1e6b5f4b3d70a3e0d5a74eaa55dec95bde1e1d2b33e1c64dc4a737ad8cd01562` |

The Symfony fixture represents the Symfony 8.1 framework line using `symfony/event-dispatcher 8.1.5`, `symfony/http-foundation 8.1.5`, `symfony/http-kernel 8.1.5`, and `symfony/routing 8.1.5`.

### Common Scenarios

The matrix uses only these stable cross-framework scenario IDs:

- `application_boot`: application/bootstrap setup for the selected fixture model.
- `http_static`: `GET /benchmark`, routed through the normal framework path, HTTP 200, deterministic body.
- `http_parameterized`: `GET /benchmark/123`, with route parameter capture proven by the response and smoke metadata.
- `http_middleware`: `GET /benchmark-middleware`, with five ordered middleware/listener layers proven by smoke metadata.
- `http_not_found`: `GET /benchmark-missing`, a genuinely unmatched path using the normal not-found path.
- `http_repeated_warm`: repeated `GET /benchmark` requests against one pre-booted reusable app/kernel/container.

The timed workload for these scenarios must not perform database access, network calls, template rendering, filesystem I/O, session storage, external cache access, queues, or application business logic.

### Dependency Isolation

Each comparator owns an isolated Composer root:

```text
benchmarks/comparators/evolvephp/composer.json
benchmarks/comparators/evolvephp/composer.lock
benchmarks/comparators/laravel/composer.json
benchmarks/comparators/laravel/composer.lock
benchmarks/comparators/symfony/composer.json
benchmarks/comparators/symfony/composer.lock
benchmarks/comparators/slim/composer.json
benchmarks/comparators/slim/composer.lock
benchmarks/comparators/phalcon/composer.json
benchmarks/comparators/phalcon/composer.lock
```

Laravel, Symfony, Slim and Phalcon comparator dependencies are not installed into `benchmarks/composer.json`, any `packages/*/composer.json`, the root production dependency graph, or the application skeleton.

### Environment Identity vs Fixture Identity

Execution environment identity represents shared benchmark conditions: PHP version, SAPI, operating system, CPU, memory, OPcache, JIT, loaded extensions and benchmark execution tooling. It intentionally excludes comparator lockfile hashes.

Fixture identity represents comparator-specific state: comparator/framework, exact version, fixture version, configuration, Composer lock hash and implementation model.

Two comparators with different dependency lockfiles can share the same execution environment identity when they run under the same controlled PHP 8.4 benchmark environment.

### Phalcon Availability

Phalcon is extension-backed. The Phalcon fixture records two deterministic states:

- `available`: the `phalcon` extension is loaded, the actual extension version is recorded, and the real `Phalcon\Mvc\Micro` workload can execute.
- `unavailable`: matrix loading and comparator smoke still succeed, the reason is explicit, and no timing data is emitted.

Local development machines without the extension are allowed to report Phalcon unavailable. That is not performance evidence and not a fixture failure.

### Local Comparator Results

Any local comparator output is local and non-canonical. It may prove fixture correctness, availability handling, schema shape and dependency isolation, but it must not be used as an authoritative performance comparison. Controlled cross-framework measurement, normalization, publication and remediation decisions must use the documented PHP 8.4 benchmark protocol.
5 changes: 5 additions & 0 deletions benchmarks/bin/check-syntax.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
$benchmarkRoot . DIRECTORY_SEPARATOR . 'src',
$benchmarkRoot . DIRECTORY_SEPARATOR . 'benchmarks',
$benchmarkRoot . DIRECTORY_SEPARATOR . 'tests',
$benchmarkRoot . DIRECTORY_SEPARATOR . 'comparators',
];
$failures = [];

Expand All @@ -19,6 +20,10 @@
continue;
}

if (in_array('vendor', explode(DIRECTORY_SEPARATOR, $file->getPathname()), true)) {
continue;
}

$command = [PHP_BINARY, '-l', $file->getPathname()];
$process = proc_open($command, [1 => ['pipe', 'w'], 2 => ['pipe', 'w']], $pipes, $benchmarkRoot);

Expand Down
14 changes: 14 additions & 0 deletions benchmarks/bin/comparator-smoke.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?php

declare(strict_types=1);

use Evolve\Benchmarks\Comparator\ComparatorSmokeVerifier;

require dirname(__DIR__) . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'autoload.php';

$matrixPath = dirname(__DIR__) . DIRECTORY_SEPARATOR . 'comparators' . DIRECTORY_SEPARATOR . 'matrix.json';
$report = ComparatorSmokeVerifier::verifyMatrixFile($matrixPath);

echo json_encode($report, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR) . PHP_EOL;

exit(($report['status'] ?? 'failed') === 'passed' ? 0 : 1);
2 changes: 2 additions & 0 deletions benchmarks/comparators/evolvephp/.placeholder
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<?php
// EvolvePHP comparator fixture directory marker.
7 changes: 7 additions & 0 deletions benchmarks/comparators/evolvephp/bootstrap.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<?php

declare(strict_types=1);

require_once __DIR__ . DIRECTORY_SEPARATOR . 'src' . DIRECTORY_SEPARATOR . 'EvolvePhpComparatorFixture.php';

return new Benchmark\EvolvePHP\EvolvePhpComparatorFixture();
36 changes: 36 additions & 0 deletions benchmarks/comparators/evolvephp/composer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"name": "evolvephp/benchmark-evolvephp",
"description": "EvolvePHP comparator fixture",
"type": "project",
"license": "BSD-3-Clause",
"minimum-stability": "dev",
"prefer-stable": true,
"repositories": [
{
"type": "path",
"url": "../../../packages/*",
"options": {
"versions": {
"evolvephp/contracts": "2.0.x-dev",
"evolvephp/core": "2.0.x-dev",
"evolvephp/http": "2.0.x-dev",
"evolvephp/module": "2.0.x-dev",
"evolvephp/plugin": "2.0.x-dev",
"evolvephp/testing": "2.0.x-dev"
},
"reference": "config"
}
}
],
"require": {
"php": "^8.4",
"evolvephp/core": "^2.0@dev",
"evolvephp/http": "^2.0@dev",
"nyholm/psr7": "^1.8"
},
"autoload": {
"psr-4": {
"Benchmark\\EvolvePHP\\": "src/"
}
}
}
Loading