Repository navigation
Conversation
…lled A flag whose `filters.version` is 2 holds an ordered list of rules: targeted releases, percentage rollouts and experiments. The SDK cannot evaluate one locally. This adds the evaluator as a private module that nothing imports, so no behavior changes. - `posthog/_feature_flags_v2.py` has a closed parser and the evaluator. An unknown config version, rule type, property type, operator or field rejects the whole document. The result is a value with its reason code and rule, or inconclusive, unsupported or an error. It is never a guess. - The tests replay the boolean, typed value and experiment evaluation corpora and the `sha1_60_v1` hash vectors of posthog-sdk-test-harness 1.31.1 (contract 10.0.0): 254 cases run, and 3 rows that describe the remote request boundary are skipped. - The vendored contract files move from harness release 1.30.0 to 1.31.1. The files that were already vendored are identical in both releases.
posthog-python Compliance ReportDate: 2026-11-13T14:49:27.130825+00:00 ✅ All Tests Passed!271/271 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_V4 Tests✅ 150/150 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
… does not detect
The regex engine of the flags service refuses a pattern above its size limits, such as `\w{210}`, and the service then reports an invalid pattern. Python evaluates such a pattern. The module docstring now says so.
The test counted one hour back from the real clock in Europe/Oslo. In the hour after a spring-forward transition that wall clock does not exist, and the filter then matches nothing. The test now uses UTC and whole seconds.
SDK performance: posthog-pythonBenchmarks completed. No changes flagged for review. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate. View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
💡 Motivation and Context
A flag whose
filters.versionis 2 holds an ordered list of rules: targeted releases, percentage rollouts and experiments. The SDK has no evaluator for that format. If it met such a flag today, the version 1 code would find nogroupsand returnFalse.This PR adds the evaluator as a private module,
posthog/_feature_flags_v2.py. Nothing imports it, so no behavior changes. There is no new public name, the definitions request is unchanged, andclient.py,feature_flags.py,request.pyandtypes.pyare not touched. Later PRs select the evaluator byfilters.versionand then request the definitions that carry these flags.Falseor to the flag default on its own. An inconclusive rule ends the evaluation, because an earlier rule that matches is terminal.sha1_60_v1hash vectors join the typed value corpus. All vendored files now come from posthog-sdk-test-harness 1.31.1 (contract 10.0.0). The files that were already vendored are identical in 1.30.0 and 1.31.1, so only the pin inSOURCE.jsonand its test moved.No published sdk-spec covers local evaluation of this format yet. The harness contract is the cross-SDK reference here.
💚 How did you test it?
posthog/test/test_feature_flags_v2.pyreplays the public corpora in-process. Every replayed case must give the exact value, reason code, rule ID, rule type, rule index and variant key, or the exact non-evaluated outcome.v2_boolean_evaluation.jsonv2_value_evaluation.jsonv2_experiment_evaluation.jsonThe 3 skipped rows have
status: omitted. They describe the remote request boundary, where the flags service leaves a flag out of its response before it evaluates anything:v2_boolean.eligibility.inactive,v2_boolean.eligibility.deletedandv2_boolean.eligibility.request_filtered. A test pins the case counts per corpus and family and the skipped IDs, so a case cannot drop out silently.Also in the test file: the 31 hash vector cases (digest arithmetic, thresholds, variant boundaries, seed parity) and 250 hand-written cases for what the corpora do not cover.
Run locally on Python 3.14, and the two flag contract test files also on 3.11:
ruff format --check .,ruff check ., mypy with the baseline,make public_api_check(the snapshot is unchanged) andpython -W error -c "import posthog".test_metrics_autocapture.py(forked child processes on macOS). The same 3 fail onmainwithout this change.I also compared the module with the flags service outside this PR. I compiled the service's property matcher, config parser and evaluator, unchanged, from PostHog/posthog
cd4e9f5into a scratch binary and ran both sides on generated input:📝 Checklist
If releasing new changes
sampo addto generate a changeset fileNo changeset: nothing a user can observe changes, so this PR should not cause a release.
Details for readers of the code
Outcomes
_EvaluatedNonemeans no value), the reason code and, where they apply, the rule ID, rule type, rule index and variant key.status: success_InconclusiveUnresolvedFlagReason.status: error,missing_context_Unsupportedkindismalformedorunsupported.status: parse_error_EvaluationErrorkindisinvalid_propertyorinvalid_regex.status: error, those two kindsThe evaluated outcome has the fields that the
/flags?v=4reader keeps on a flag record, so a later PR can build the same record from it. An inactive flag returns no value with the reasonflag_disabled.Hashing
sha1_60_v1takes the first 60 bits ofSHA1(prefix + identifier + salt)and divides by0xFFFFFFFFFFFFFFF, with both operands as binary64. A rollout usesseed + "."as the prefix, a variant adds the saltvariant, and a holdout uses its own seed as the whole prefix. 100 percent includes without hashing. An empty identifier never enters a rollout or a holdout. The identifier is cut to 200 Unicode scalar values. Variant boundaries add left to right, and the last variant wins past the last boundary.The white-box corpus rows need to replace the hash.
_evaluate_with_hashis that seam. It is private, like everything else in the module.Property matching
The
exactand text operators and regular expressions call the helpers of the version 1 evaluator, which already follow the flags service. Numbers, semantic versions and dates are matched in the new module, because version 1 reads them differently:is_not,is_not_setand the negative text operators match there. A partial map makes a filter on a missing key inconclusive.-7dare read on the wall clock of the project's timezone. A wall clock in a spring-forward gap matches nothing, and one that repeats means its first instant.Where the evaluator declines
It returns inconclusive, or rejects the document, where Python cannot reproduce the flags service:
rewould not run like the service's regex engine. This is the existing version 1 rule.YYYY-MM-DDor a date and time without an offset. The service reads more forms on a best-effort basis. A date filter in another form makes the document unsupported.One difference is known and not detected. The service's regex engine refuses a pattern above its size limits and the service then reports an invalid pattern. At the commit linked above it compiles
\w{209}and refuses\w{210}. Python has no such limit and evaluates the pattern. The version 1 evaluator has the same difference.Limits of reading decoded JSON
The service checks three things in the JSON text that a decoded document no longer shows: a repeated object key, a nonzero number that rounds to zero, and a percentage written with more than two decimals that rounds to a two-decimal value.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
🤖 Generated with Claude Code