Skip to content

feat: emit LaunchDarkly context identity on AI SDK feature_flag spans - #60

Draft
ccschmitz-launchdarkly wants to merge 4 commits into
mainfrom
AIC-3230-context-identity
Draft

feat: emit LaunchDarkly context identity on AI SDK feature_flag spans#60
ccschmitz-launchdarkly wants to merge 4 commits into
mainfrom
AIC-3230-context-identity

Conversation

@ccschmitz-launchdarkly

Copy link
Copy Markdown
Contributor

What

Python counterpart of the js-ai-sdk change (filed alongside this). Carries the identity of the context an AI Config was evaluated for onto the run's root span, so observability can filter a config's traces down to a single context. This PR also carries the TELEMETRY-CONTRACT.md update for both SDKs, since that document is canonical for both.

Three attributes, none of them a new LaunchDarkly convention:

Where Attribute Value
feature_flag event feature_flag.context.id canonical key — an OTel semconv attribute (RC since v1.29.0), same value go-server-sdk/ldotel emits
feature_flag event feature_flag.contextKeys JSON object of per-kind keys — matches the observability browser SDK
root span context.contextKeys.<kind> one attribute per kind — matches the browser SDK and the product-analytics pipeline

Why the identity is emitted in two shapes

feature_flag.context.id is a composite for a multi-kind context (kind:key:kind:key, sorted), so it cannot answer "filter this config's traces to this one user" — which is exactly the question AI Config Monitoring's group-by asks, since that groups per kind. And feature_flag.contextKeys lands in a ClickHouse String column holding JSON, so filtering it per kind would be substring matching. The per-kind span attributes are what give an exact match on a single kind.

Why keys only

Context keys are identifiers, and LaunchDarkly's other OTel integrations already expose them. Context attribute values are where the personal data lives, so none are read or emitted, and no capture option is added — there is nothing to gate. A test asserts that a context carrying name and email leaks neither.

How it works

No plumbing. execute_and_track / execute_and_stream already merge ldContext into the variables dict next to __ld, after the caller's own variables so it can't be clobbered. set_ld_span_attributes reads it there, and all six handler packages reach it through their spans.start_root_span, so provider coverage is uniform by construction.

Deliberately not on TrackData: that dict is the data payload of every $ld:ai:* track call, where context keys would duplicate the call's own context argument and bloat every analytics event.

New ld_context.py is a port of getCanonicalKey/getContextKeys from the observability browser SDK's LaunchDarkly integration, sharing fixtures with the TypeScript port so the two cannot drift. It deliberately does not go through ldclient.Context.fully_qualified_key: ldclient is an optional import here (see to_ld_context), and an attribute that silently vanishes for anyone using a custom client is worse than no attribute.

context_identity never raises — this runs on the emit path of every run, so a malformed context degrades to emitting nothing.

Two details worth a reviewer's eye

json.dumps needs explicit separators. Python defaults to ", " / ": "; JSON.stringify emits neither. Without separators=(",", ":") the same context would produce a different string in Python than in TypeScript, in a column consumers may match as text. There is a test pinning the exact bytes.

The vocabulary lock needed widening, in the same commit. tests/test_cross_handler_parity.py statically scans attribute literals against a committed list. Its regex was lowercase-only (feature_flag\.[a-z_.]+) and limited to the gen_ai/launchdarkly/feature_flag/ld namespaces, so it could see feature_flag.context.id but not feature_flag.contextKeys (camelCase) or context.contextKeys.<kind> (f-string, context namespace). Adding the keys to EXPECTED_VOCABULARY without widening the scanner fails one lock test; widening without adding fails the other — so both halves are in one commit. Verified that widening surfaces nothing else: there are no pre-existing set_attribute("context… literals and no uppercase letters in any existing "feature_flag.…" literal.

RecordedSpan.add_event also discarded event attributes (events was a list[str]), so nothing could assert on the feature_flag event's payload. It is now a name-keyed dict; all existing usages are membership checks and survive unchanged.

Testing

  • make test — 1135 passed, 11 skipped
  • make lint — clean
  • Both TestVocabularyLock tests pass
  • New: 20 cases in test_ld_context.py (shared fixtures with the TS port), 9 in test_span_attributes.py (nothing covered set_ld_span_attributes before), plus 2 cross-handler cases asserting all six handlers write the per-kind attributes and that tool spans still do not

Pre-existing failure, unrelated: make typecheck fails with Duplicate module named "conftest". This repo has 8 identically-named conftest.py files and no per-package mypy config; the failure reproduces on unmodified origin/main and this branch touches none of them. Every new and modified file passes mypy --strict individually. Worth its own ticket.

Still outstanding, tracked on the ticket: end-to-end verification in staging that the context.contextKeys.user=… filter returns the run.

Links

Python twin of js-ai-sdk's packages/client/src/context.ts, sharing its
fixtures so the two ports cannot drift. Deliberately does not use
ldclient.Context: ldclient is an optional import here, and an attribute
that vanishes for custom clients is worse than no attribute.
Python counterpart of the js-ai-sdk change. Adds
feature_flag.context.id and feature_flag.contextKeys to the feature_flag
event, plus one context.contextKeys.<kind> span attribute per kind.

Keys only. Context attribute values are not emitted.

AIC-3230
The scanner was lowercase-only and namespace-limited, so it could not see
feature_flag.contextKeys or the interpolated context.contextKeys.<kind>.
Widened both, and added the keys to EXPECTED_VOCABULARY in the same
commit, because fixing either half alone turns the suite red.

Also asserts every handler writes the per-kind attributes and that tool
spans still do not, so a context-scoped query still finds one span per
run.

AIC-3230
Documents feature_flag.context.id, feature_flag.contextKeys and the
per-kind context.contextKeys.<kind> span attributes, and why the identity
is emitted in two shapes rather than one.

AIC-3230
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant