Skip to content

Improve Data hot-path performance with automatic execution recipes - #31

Closed
binaryfire wants to merge 6 commits into
0.4from
feature/data-lean-tier
Closed

Improve Data hot-path performance with automatic execution recipes#31
binaryfire wants to merge 6 commits into
0.4from
feature/data-lean-tier

Conversation

@binaryfire

Copy link
Copy Markdown
Member

Summary

This change makes common hypervel/data construction and transformation paths substantially faster without adding a second data-object API or changing the public contract.

Data classes now receive immutable execution recipes when their declarations can be handled safely by fixed operations. The runtime selects those recipes automatically. Classes using custom casts, normalizers, lifecycle hooks, validation, lazy values, partials, or unsupported union shapes continue through the existing general engine.

Motivation

The Data package supports validation, property mapping, nested objects and collections, lazy values, partial transformations, HTTP resources, request casting, and Eloquent persistence. Its general engine preserves all of those behaviors, but simple objects were paying much of that orchestration cost even when their declarations needed only fixed scalar, enum, date, or nested Data conversion.

The goal here is to make work proportional to the features a declaration uses. Applications should not need to choose a fast mode or a separate lightweight object type.

Design

Class metadata now records one of three transformation states: bulk copy, a fixed property recipe, or the general property loop. Construction similarly records a fixed recipe only for declarations whose behavior is known at metadata-build time.

The implementation:

  • preflights the complete input node before performing conversions, so a later fallback cannot run a nested factory, hook, or constructor twice;
  • shares built-in, enum, and date conversion through one ValueCaster, keeping lean and general behavior identical;
  • lets nested values choose the lean or general path independently while sharing the root operation memo;
  • caches one immutable default creation context per used Data class while still returning a fresh public factory;
  • preserves named factories, late-static factory() overrides, validation decisions, partials, transformation overrides, resource finalization, and persistence behavior;
  • keeps recipe metadata bounded to used classes and declared properties, with no payloads, runtime objects, closures, discovery, generated files, or eviction policy;
  • builds the PHPDoc parser lazily and skips annotation work when native types prove iterable metadata cannot apply;
  • resolves the fixed creator and transformer graph after application providers boot and before production workers fork, while skipping that warm-up during unit tests.

No public API changes are introduced. from(), factory(), collect(), transform(), all(), and toArray() retain their signatures and extension boundaries.

Performance

Measurements used PHP 8.4.23 on Linux x86-64 with CLI OPcache and JIT disabled. Results are the median across three alternating fresh-process runs against the current 0.4 branch. Each standard run used 2,000 operations, seven measured samples, and 100 warm-up operations.

Scenario 0.4 p50 This PR p50 Change 0.4 p95 This PR p95 Change
Flat construction 5.957 us 3.867 us -35.1% 7.306 us 4.050 us -44.6%
Nested construction 32.183 us 6.267 us -80.5% 32.985 us 6.484 us -80.3%
Deep/wide construction 82.337 us 11.302 us -86.3% 87.461 us 11.645 us -86.7%
Eager 1,000-item collection 4.074 ms 2.980 ms -26.9% 5.145 ms 3.171 ms -38.4%
Lazy 1,000-item traversal 3.934 ms 2.387 ms -39.3% 4.026 ms 2.469 ms -38.7%
Simple transformation 3.014 us 2.001 us -33.6% 3.225 us 2.075 us -35.7%
Nested transformation 5.831 us 4.660 us -20.1% 6.622 us 4.840 us -26.9%
Plain five-property transformation 1.829 us 1.809 us -1.0% 2.021 us 1.893 us -6.4%
Validate 5,000 nested items 406.324 ms 406.698 ms +0.1% 424.479 ms 426.319 ms +0.4%

AutoLazy, customized collection factories, lazy partials, Eloquent relation loading, and other fallback-heavy scenarios remained within run-to-run noise.

The tradeoffs are deliberately bounded:

  • metadata analysis adds about 16 us once per used class;
  • across 500 representative five-property classes, retained recipe metadata adds about 235 KB and cached default contexts add about 212 KB;
  • production application startup performs about 15 ms of fixed service initialization before worker fork, moving that cost out of the first request;
  • alternating Testbench runs showed no unit-test startup regression.

The committed benchmark harness reports p50, p95, throughput, query counts, memory, environment details, and checksums. A separate historical comparison fixture remains isolated under tests/Benchmarks/Data and is loaded only by its developer benchmark command.

Verification

  • Full composer fix, including formatting, static analysis, parallel tests, Testbench, and dogfood checks.
  • Focused construction, transformation, metadata, annotation, enum, FormRequest, resource, collection, lazy, and persistence coverage.
  • Lean/general equivalence tests for supported conversions, mappings, defaults, constructors, exceptions, named factories, hooks, and mixed nested graphs.
  • Fallback tests for customized and ambiguous declarations.
  • Repeated before/after performance, startup, and retained-memory measurements.

Summary by CodeRabbit

  • New Features

    • Added faster automatic construction and transformation for eligible data objects.
    • Improved handling of nested data, mapped fields, enums, dates, defaults, nulls, and optional values.
    • Added bulk-copy transformation for compatible data.
    • Added lazy metadata parsing and improved first-use performance.
  • Bug Fixes

    • Improved consistency between optimized and general construction and transformation paths.
    • Preserved fallback behavior for unsupported or complex data shapes.
  • Documentation

    • Added implementation details and expanded benchmark guidance.

Extract the built-in, backed-enum, and date conversion rules into one internal ValueCaster used by both ordinary casts and lean construction recipes.

This keeps coercion behavior, date formats and timezones, concrete date targets, and existing exception contracts in one authoritative implementation without retaining cast instances in worker metadata.
Compile immutable per-class construction and transformation recipes from existing Data metadata, then select them automatically for supported runtime shapes while preserving the general engine as the single fallback.

Reuse immutable default creation contexts, preflight complete nodes before conversion, share nested operation state, retain named-factory and hook boundaries, and keep bulk-copy transformation for plain objects. Unsupported declarations and ambiguous conversion families continue through the established path before construction.

Defer PHPDoc parser setup when native types prove iterable metadata cannot apply. Add focused equivalence, fallback, metadata, lazy, partial, mapping, constructor, and transformation coverage so lean execution stays behaviorally identical to the general path.
Resolve DataCreator and DataTransformer from an application booted callback after all providers have configured their dependencies. Production workers inherit the initialized immutable service graph instead of making the first request pay that fixed setup cost.

Keep unit-test applications on demand so repeated Testbench boots remain fast, and cover both testing and non-testing application lifecycles with stable instance assertions.
Expand the developer benchmark matrix across construction, validation, transformation, collections, resources, persistence, metadata, and first-use boundaries.

Add a dedicated historical DataObject comparison harness under the test namespace so supported shapes can be measured against the removed mapper without restoring it as framework API. Document both commands and keep raw reports opt-in and outside the repository.
Record the measured baseline, immutable recipe design, one-engine fallback rules, lifecycle constraints, integration boundaries, test matrix, performance acceptance criteria, and rejected alternatives.

The plan also preserves the required post-checkpoint merge and benchmark work so the current 0.4 enum and morph behavior is reconciled before final integration.
Bring the latest 0.4 framework changes into the automatic Data execution branch, including the first-party FormRequest casting redesign, shared enum coercion helpers, and the queue, database, validation, and lifecycle fixes already accepted on 0.4.

Reconcile the Data fast path with the shared enum_from semantics so lean and general construction accept integer-backed numeric strings consistently. Preserve enum_try_from morph selection and add a focused regression that proves the compiled operation, target enum, and concrete result.

Finish the execution terminology sweep, retain the measured performance and memory conclusions in the active plan, and record the completed verification state. The merged tree passes composer fix and focused Data, enum, FormRequest, and capability coverage; repeated benchmarks retain the material construction and transformation gains without a systematic fallback-path regression.
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 1a5a5c84-87f2-475a-86d9-ace46bec1a06

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds metadata-compiled construction and transformation recipes for eligible Data declarations while preserving the general engine as the fallback.

  • Shares scalar, enum, and date conversion primitives between optimized and general paths.
  • Adds immutable default creation-context reuse and deferred nested recipe selection.
  • Introduces fixed transformation recipes while retaining bulk-copy, partial, persistence, and custom-transformer boundaries.
  • Lazily initializes PHPDoc parsing and screens declarations whose native types cannot use iterable metadata.
  • Warms fixed Data services after production application boot and expands equivalence and benchmark coverage.

Confidence Score: 5/5

The PR appears safe to merge, with optimized paths narrowly gated and unsupported behavior continuing through the established general engine.

No actionable correctness, security, or repository-rule violations remained after tracing recipe compilation and execution through construction, transformation, annotation parsing, service boot, and benchmark subprocess handling.

Important Files Changed

Filename Overview
src/data/src/Support/Creation/DataCreator.php Adds preflighted recipe construction, default-context caching, shared nested operation state, and single-pass named-factory fallback without an identified contract regression.
src/data/src/Support/Factories/DataClassFactory.php Compiles bounded creation/transformation recipes and conservatively screens PHPDoc parsing while retaining unsupported declarations on the general path.
src/data/src/Support/Factories/DataPropertyFactory.php Classifies fixed property operations according to the existing conversion priority and excludes ambiguous transformation targets.
src/data/src/Support/Transformation/DataTransformer.php Adds guarded fixed-recipe transformation alongside the existing bulk and general paths while preserving finalization and nested partial handling.
src/data/src/Support/Creation/ValueCaster.php Centralizes built-in, enum, and date conversion behavior previously implemented by the individual cast adapters.
src/data/src/Support/Creation/CreationContextFactory.php Reuses immutable Create contexts and invalidates them consistently across every fluent factory mutation.
src/data/src/Support/Annotations/DataIterableAnnotationReader.php Defers parser construction until an eligible PHPDoc comment is actually parsed.
src/data/src/DataServiceProvider.php Resolves the fixed creator and transformer graph after production application boot while leaving unit-test startup lazy.
tests/Benchmarks/Data/compare-data-object.php Adds an isolated historical comparison harness whose subprocess command uses escaped, fixed inputs.
tests/Data/Support/Creation/DataCreatorTest.php Adds extensive lean/general equivalence, fallback, factory, conversion, lazy replay, and side-effect-order coverage.
tests/Data/Support/Transformation/DataTransformerTest.php Verifies fixed-recipe equivalence and guards around bulk copy, custom transformations, iterables, partials, and persistence.

Reviews (1): Last reviewed commit: "Merge 0.4 into Data lean execution" | Re-trigger Greptile

@binaryfire binaryfire closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant