Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions benchmarks/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
/vendor/
/.phpbench/
/.phpunit.cache/
/results/local/
/results/tmp/
/profiles/
99 changes: 99 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# EvolvePHP 2 Benchmarks

EvolvePHP performance work starts from measurements instead of guesses. This benchmark harness provides correctness checks, environment fingerprinting, local baseline output, and measured evidence for later optimization work.

This harness is benchmark infrastructure only. It does not optimize production framework code and it does not make a fastest-framework claim. There is no current fastest-framework claim and no current top-three performance claim.

## Install

From the repository root:

```powershell
composer update --working-dir=benchmarks --no-interaction
```

Benchmark-only dependencies are installed inside `benchmarks/vendor/` and locked in `benchmarks/composer.lock`.

## Protocol

The primary comparison lane is PHP 8.4 with OPcache enabled, JIT disabled, production-like framework configuration, and no debugging or profiling extension altering timings. PHP 8.5 results may be useful, but they must not be combined with PHP 8.4 results as if they came from the same environment.

one-off stopwatch results are not performance evidence. Warmup is required. Multiple iterations are required. The environment fingerprint must match before two baselines are treated as comparable. Shared GitHub-hosted runners are useful for smoke validation, but they are not authoritative sources for absolute wall-clock regression budgets. Future blocking budgets must come from a controlled or proven low-noise environment.

## Commands

Validate the benchmark Composer root:

```powershell
composer validate --working-dir=benchmarks --strict
```

Run correctness tests:

```powershell
php benchmarks\vendor\bin\phpunit --configuration benchmarks\phpunit.xml.dist
```

Run syntax checks:

```powershell
php benchmarks\bin\check-syntax.php
```

Capture the environment fingerprint:

```powershell
php benchmarks\bin\capture-environment.php --output results/local/environment.json
```

Run the fast benchmark smoke:

```powershell
php benchmarks\bin\benchmark-smoke.php
```

Run internal Core scenarios:

```powershell
php benchmarks\vendor\bin\phpbench run --config=benchmarks\phpbench.json --group=core --report=aggregate
```

Run representative HTTP scenarios:

```powershell
php benchmarks\vendor\bin\phpbench run --config=benchmarks\phpbench.json --group=http --report=aggregate
```

Run a local full benchmark pass and store XML:

```powershell
php benchmarks\vendor\bin\phpbench run --config=benchmarks\phpbench.json --report=aggregate --dump-file=benchmarks\results\local\phpbench.xml
```

Normalize an XML result:

```powershell
php benchmarks\bin\normalize-results.php --input benchmarks\results\local\phpbench.xml --output benchmarks\results\local\normalized-results.json
```

## Scenarios

Internal scenarios cover application boot/component preparation, service resolution for application, execution and transient lifetimes, execution orchestration with no sink versus a no-op sink, reset participant overhead, and persistent-style sequential execution evidence.

HTTP scenarios cover real `Route`, `RouteCollection`, `RouteMatcher`, `RoutingRequestHandler`, middleware dispatch, and full `HttpKernel` execution. Route tables cover 10, 100, and 1000 routes with first, middle, and last hit positions, plus miss and 405 paths. Middleware depths cover 0, 1, 5, 10, and 20 layers.

The persistent-style run is repeated sequential execution evidence only. It is not FrankenPHP certification, RoadRunner certification, or any other runtime-adapter claim.

## Results

Local results are written under `benchmarks/results/local/` and are ignored. Treat local output as:

```text
LOCAL / NON-CANONICAL BASELINE
```

A result qualifies as a reference baseline only when the documented PHP 8.4 protocol is run on the selected controlled environment with the exact source SHA, dependency lock state, and environment fingerprint recorded for comparison.

Baseline comparison must reject casual comparisons when the environment fingerprint differs. The normalized result schema includes scenario identifiers, sample counts, timing statistics, percentiles when enough samples exist, relative standard deviation, throughput where derivable, memory fields, environment fingerprint, source SHA, and schema version.

Cross-framework comparison, optimization work, and regression budgets are intentionally outside this initial harness. They should be introduced only after baseline measurements are reproducible and the relevant comparison methodology is defined.
34 changes: 34 additions & 0 deletions benchmarks/benchmarks/Core/ApplicationBootBench.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<?php

declare(strict_types=1);

namespace Evolve\Benchmarks\PhpBench\Core;

use Evolve\Benchmarks\Support\BenchmarkFixtureFactory;
use PhpBench\Attributes as Bench;

#[Bench\Revs(10)]
#[Bench\Iterations(8)]
#[Bench\Warmup(2)]
final class ApplicationBootBench
{
#[Bench\Groups(['core', 'application-boot'])]
#[Bench\ParamProviders(['componentCounts'])]
public function benchOverallApplicationBoot(array $params): void
{
$fixture = BenchmarkFixtureFactory::applicationBootFixture($params['components']);
$fixture['kernel']->boot();
}

/**
* @return array<string, array{components: int}>
*/
public function componentCounts(): array
{
return [
'minimal' => ['components' => 0],
'small-graph' => ['components' => 5],
'larger-graph' => ['components' => 50],
];
}
}
136 changes: 136 additions & 0 deletions benchmarks/benchmarks/Core/ContainerResolutionBench.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
<?php

declare(strict_types=1);

namespace Evolve\Benchmarks\PhpBench\Core;

use Evolve\Benchmarks\Support\BenchmarkFixtureFactory;
use Evolve\Core\Execution\ExecutionScope;
use PhpBench\Attributes as Bench;
use Psr\Container\ContainerInterface;

#[Bench\Revs(100)]
#[Bench\Iterations(10)]
#[Bench\Warmup(2)]
final class ContainerResolutionBench
{
private ContainerInterface $container;

private ?ExecutionScope $executionScope = null;

private string $applicationServiceId = '';

private string $executionServiceId = '';

private string $transientServiceId = '';

#[Bench\BeforeMethods(['setUpApplicationFirstResolution'])]
#[Bench\Revs(1)]
#[Bench\Warmup(0)]
#[Bench\Groups(['core', 'container'])]
#[Bench\ParamProviders(['serviceCounts'])]
public function benchApplicationFirstResolution(array $params): void
{
$this->container->get($this->applicationServiceId);
}

#[Bench\BeforeMethods(['setUpApplicationCachedResolution'])]
#[Bench\Groups(['core', 'container'])]
#[Bench\ParamProviders(['serviceCounts'])]
public function benchApplicationCachedResolution(array $params): void
{
$this->container->get($this->applicationServiceId);
}

#[Bench\BeforeMethods(['setUpExecutionFirstResolution'])]
#[Bench\AfterMethods(['tearDownExecutionScope'])]
#[Bench\Revs(1)]
#[Bench\Warmup(0)]
#[Bench\Groups(['core', 'container'])]
#[Bench\ParamProviders(['serviceCounts'])]
public function benchExecutionFirstResolution(array $params): void
{
$this->executionScope->get($this->executionServiceId);
}

#[Bench\BeforeMethods(['setUpExecutionCachedResolution'])]
#[Bench\AfterMethods(['tearDownExecutionScope'])]
#[Bench\Groups(['core', 'container'])]
#[Bench\ParamProviders(['serviceCounts'])]
public function benchExecutionCachedResolution(array $params): void
{
$this->executionScope->get($this->executionServiceId);
}

#[Bench\BeforeMethods(['setUpTransientResolution'])]
#[Bench\Groups(['core', 'container'])]
#[Bench\ParamProviders(['serviceCounts'])]
public function benchTransientRepeatedResolution(array $params): void
{
$this->container->get($this->transientServiceId);
}

public function setUpApplicationFirstResolution(array $params): void
{
$fixture = BenchmarkFixtureFactory::containerFixture($params['services']);
$this->container = $fixture['container'];
$this->applicationServiceId = 'bench.application.' . ($params['services'] - 1);
}

public function setUpApplicationCachedResolution(array $params): void
{
$this->setUpApplicationFirstResolution($params);
$this->container->get($this->applicationServiceId);
}

public function setUpExecutionFirstResolution(array $params): void
{
$fixture = BenchmarkFixtureFactory::containerFixture($params['services']);
$this->container = $fixture['container'];
$this->executionScope = $this->container->createExecutionScope();
$this->executionServiceId = 'bench.execution.' . ($params['services'] - 1);
}

public function setUpExecutionCachedResolution(array $params): void
{
$this->setUpExecutionFirstResolution($params);
$this->executionScope->get($this->executionServiceId);
}

public function setUpTransientResolution(array $params): void
{
$fixture = BenchmarkFixtureFactory::containerFixture($params['services']);
$this->container = $fixture['container'];
$this->transientServiceId = 'bench.transient.' . ($params['services'] - 1);
}

public function tearDownExecutionScope(): void
{
if ($this->executionScope !== null) {
$this->executionScope->close();
$this->executionScope = null;
}
}

public function resolvePreparedApplicationCachedService(): object
{
return $this->container->get($this->applicationServiceId);
}

public function applicationServiceId(): string
{
return $this->applicationServiceId;
}

/**
* @return array<string, array{services: int}>
*/
public function serviceCounts(): array
{
return [
'10-services' => ['services' => 10],
'100-services' => ['services' => 100],
'1000-services' => ['services' => 1000],
];
}
}
48 changes: 48 additions & 0 deletions benchmarks/benchmarks/Core/ExecutionOrchestratorBench.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<?php

declare(strict_types=1);

namespace Evolve\Benchmarks\PhpBench\Core;

use Evolve\Benchmarks\Support\BenchmarkFixtureFactory;
use Evolve\Core\Execution\ExecutionKind;
use PhpBench\Attributes as Bench;

#[Bench\Revs(50)]
#[Bench\Iterations(10)]
#[Bench\Warmup(2)]
final class ExecutionOrchestratorBench
{
#[Bench\Groups(['core', 'execution', 'instrumentation'])]
public function benchSuccessfulExecutionNoSinkZeroResetParticipants(): void
{
$fixture = BenchmarkFixtureFactory::executionOrchestratorFixture(resetParticipants: 0);
$fixture['orchestrator']->execute(ExecutionKind::HttpRequest, $fixture['operation']);
}

#[Bench\Groups(['core', 'execution', 'instrumentation'])]
public function benchSuccessfulExecutionNoOpSinkZeroResetParticipants(): void
{
$fixture = BenchmarkFixtureFactory::executionOrchestratorFixture(resetParticipants: 0, withObservationSink: true);
$fixture['orchestrator']->execute(ExecutionKind::HttpRequest, $fixture['operation']);
}

#[Bench\Groups(['core', 'execution', 'reset'])]
#[Bench\ParamProviders(['resetParticipantCounts'])]
public function benchSuccessfulExecutionWithResetParticipants(array $params): void
{
$fixture = BenchmarkFixtureFactory::executionOrchestratorFixture(resetParticipants: $params['reset_participants']);
$fixture['orchestrator']->execute(ExecutionKind::HttpRequest, $fixture['operation']);
}

/**
* @return array<string, array{reset_participants: int}>
*/
public function resetParticipantCounts(): array
{
return [
'one-reset-participant' => ['reset_participants' => 1],
'ten-reset-participants' => ['reset_participants' => 10],
];
}
}
32 changes: 32 additions & 0 deletions benchmarks/benchmarks/Core/PersistentSequentialExecutionBench.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<?php

declare(strict_types=1);

namespace Evolve\Benchmarks\PhpBench\Core;

use Evolve\Benchmarks\Support\BenchmarkFixtureFactory;
use PhpBench\Attributes as Bench;

#[Bench\Revs(1)]
#[Bench\Iterations(5)]
#[Bench\Warmup(1)]
final class PersistentSequentialExecutionBench
{
#[Bench\Groups(['core', 'persistent-style-memory'])]
#[Bench\ParamProviders(['iterationCounts'])]
public function benchPersistentStyleSequentialExecutionEvidence(array $params): void
{
BenchmarkFixtureFactory::persistentSequentialExecutionEvidence($params['iterations'], $params['checkpoint_every']);
}

/**
* @return array<string, array{iterations: int, checkpoint_every: int}>
*/
public function iterationCounts(): array
{
return [
'1000-executions' => ['iterations' => 1000, 'checkpoint_every' => 250],
'10000-executions' => ['iterations' => 10000, 'checkpoint_every' => 2500],
];
}
}
Loading