Repository navigation
feat: add protocol-v2 multivariate flag evaluation - #744
Conversation
|
Performance review fixes pushed:
Validation: 285 tests passed; typecheck, workspace build, and lint passed. Added repeatable Vitest benchmarks and tests that assert Set lookup usage and no repeated percentage reads. Local benchmarks: prepared v2 membership checks stayed around 0.7 microseconds/check across 10, 1,000, and 100,000 candidates. At 100,000 candidates, the original PR implementation measured about 77 microseconds/check versus 0.68 microseconds prepared (roughly 113x faster, excluding preparation). Prepared v2 was also about 1.2x faster than prepared legacy in the included membership benchmark. Results are local microbenchmarks, not production throughput guarantees. |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the collection-based evaluator change.
No open issues 🎉
What changed in this PR
Introduces a collection-based protocol-v2 evaluator for multivariate flags, providing the foundation for separate SDK migrations while keeping the existing Node SDK on published evaluator v1.
Changes:
- Replaces v1 value-rule APIs with prepared batch and single-flag evaluation, generic variant values, and structured diagnostics.
- Adds recursive flag/segment references, collection validation, and conservative handling of invalid references without dependency-result caching.
- Adds regression tests, benchmarks, migration documentation, and release isolation for the legacy Node SDK.
| File | Description |
|---|---|
yarn.lock |
Records the published v1 alias separately from the v2 workspace. |
packages/node-sdk/src/types.ts |
Imports evaluator types through the v1 alias. |
packages/node-sdk/src/client.ts |
Uses the v1 alias for runtime evaluation. |
packages/node-sdk/package.json |
Pins the legacy evaluator through an npm alias. |
packages/flag-evaluation/test/variants.test.ts |
Tests variant selection, distributions, types, and preparation. |
packages/flag-evaluation/test/variants.bench.ts |
Benchmarks prepared membership and distribution evaluation. |
packages/flag-evaluation/test/release.test.ts |
Checks v1/v2 dependency isolation. |
packages/flag-evaluation/test/references.test.ts |
Tests recursive references, validation, and diagnostic boundaries. |
packages/flag-evaluation/test/index.test.ts |
Migrates operator and context tests to the v2 API. |
packages/flag-evaluation/test/helpers.ts |
Adds a single-flag fixture helper. |
packages/flag-evaluation/test/collection.test.ts |
Tests batch/single parity, isolation, and uncached dependencies. |
packages/flag-evaluation/src/index.ts |
Implements the collection-based multivariate evaluator. |
packages/flag-evaluation/README.md |
Documents v2 usage, reference semantics, and migration. |
packages/flag-evaluation/package.json |
Sets the v2 prerelease version and adds prepack compilation. |
.changeset/pin-legacy-node-evaluator.md |
Declares the Node SDK dependency-isolation patch. |
.changeset/multivariate-evaluation.md |
Declares the evaluator major release and API changes. |
Review effort: Balanced
Summary
newEvaluator({ flags, segments })prepares a complete collection and returnsevaluateAll(context)andevaluateFlag(context, flagKey)methods. Remove the v1 value-rule API and legacy serialization helpers.{ context, flags }with normalized context once and independent per-flag results/diagnostics. One unresolved flag does not stop the remaining outputs. Single-flag evaluation visits only its selected root and dependencies. The methods map naturally to Node SDKgetFlags()/getFlagsForBootstrap()andgetFlag()respectively; SDK wrappers, overrides and fallbacks remain outside the evaluator.resolvedresult, distinguishing selectedundefinedfrom failure and requiring success metadata.Main synchronization
Merged
mainatabb17bd2f801a8b299ef02eca87b2d9ca774572athrough merge commit486b2099e0d03f5e86566ab31660b0cb95b20ea8. The reviewed collection evaluator source is unchanged. Main's SDK fixes, dependency updates and Vitest 4.1.11 migration are retained; an additional invalid-operator regression covers v2 fail-closed handling. No history was rewritten. The separate SDK branches were not changed and need their own synchronization with this updated base.API
Reference filters:
A matching selected variant is authoritative even if earlier rules produced diagnostics. A diagnostic-bearing nonmatch is deliberately more conservative than explicitly matching the selected fallback variant. Segments do not introduce independent-rule boundaries.
Validation
Scope
Still a draft; nothing published or deployed. This layers the collection-based alternative discussed in #780 into v2 rather than adding
ruleEvaluation: "independent"metadata to v1. #780 and its companion web PR are untouched.Publisher/compiler changes and matching server-side consumers are separate in reflagcom/web#3660. SDK adoption of
/flagsand variant-valued public APIs are separate in #790 and draft #791. Existing v1 consumers must not receive v2 reference nodes.Read-only inspection of web and its Advanced flags work (#3660) confirms the follow-up needs a distinct reference-preserving v2 lowering, reachable compiled segments, complete collection transport, and snapshot-level evaluator ownership. Boolean v2 rules must be compiled from normalized sources rather than copied from the already-inlined legacy rules. Keep legacy
/featuresand config compilation unchanged, and retain the complete graph while selecting/authorizing output roots.