Affected package
fi-instrumentation-otel (Python)
Package version
fi-instrumentation-otel==1.0.0 (note: pyproject.toml reports 1.0.0 but fi_instrumentation/instruments/version.py reports __version__ = "0.1.19" - these disagree; flagging in case it's relevant to triage)
What happened?
With TraceConfig(pii_redaction=True) (or FI_PII_REDACTION=true), the regex-based PII scanner in pii_redaction.py is applied unconditionally to every span attribute value in TraceConfig.mask(), with no key-value based masking rule in that method, which is scoped to a specific attribute key (INPUT_VALUE, message content, etc.).
Two consequences:
- "Trace correlation identifiers get corrupted."
session.id, user.id, and metadata are injected via get_attributes_from_context() and pass through the same mask() call as message content. Because the credit-card pattern "|\b(?:\d[ -]*?){13,19}\b" is just a bare 13-19 digit run with no Luhn check or grouping requirement, any UUID whose hex segments happen to contain a long enough digit run gets partially replaced with <CREDIT_CARD>. Measured over 100 random uuid.uuid4() values, ~2% were corrupted. This silently breaks the exact session/user correlation the tracing products exists to provide exists to provide, and it's unattributed from the UI (looks like the session ID was never consistent).
- "Non-PII numeric content gets over-redacted." Timestamps, float values, ISBNs, and other incidentals digit runs of 13+ characters are replaced with
<CREDIT_CARD> even though they're not remotely credit-card shaped (no-grouping, fails Luhn). This degrades trace readability and defeats debugging on any span containing numeric metadata (costs, durations, request IDs, etc.).
Expected: PII redaction should apply only to attributes that hold actual free-text content (input/output values, messages, content) - not to structured identifiers like session.id, user.id, or metadata - and the credit-card pattern should require actual card-like structure, not just "13-19 digits in a row.".
Minimal reproduction
from fi_instrumentation.instrumentation.config import TraceConfig
from fi_instrumentation.fi_types import SpanAttributes
cfg = TraceConfig(pii_redaction=True)
# 1. session.id corruption (this specific UUID reproduces it; ~2% of random
# UUIDs hit this due to landing on a 13+ digit run)
sid = "73630065-0794-4450-a1f9-8cc987a02b09"
print(cfg.mask(SpanAttributes.SESSION_ID, sid))
# -> "<CREDIT_CARD>-a1f9-8cc987a02b09" (expected: unchanged)
# 2. user.id corruption
uid = "1234567890123"
print(cfg.mask(SpanAttributes.USER_ID, uid))
# -> "<CREDIT_CARD>" (expected: unchanged)
# 3. metadata corruption (numeric cost/timestamp fields mangled)
md = '{"request_id": "20240115103045123", "const_usd": 3.14159}'
print(cfg.mask(SpanAttributes.METADATA, md))
# -> '{"request_id":"<CREDIT_CARD>", "cost_usd":3.<CREDIT_CARD>}'Error output / stack trace
N/A - This is a silent data-corruption bug, not an exception. No error is raised; the returned values are simply wrong.
Environment
Python 3.12.5
Windows 11
Anything else?
Root cause is in two places:
fi_instrumentation/instrumentation/pii_redaction.py - has no grouping requirement and no Luhn check, so it matches any sufficiently long digit run regardless of shape.fi_instrumentation/instrumentation/config.py, TraceConfig.mask() - the if self.pii_redaction: resolved = redact_pii_in_value(resolved) branch runs unconditionally on every attribute key, unlike the hide_* rules above it in the same method, which are all gated with key == SpanAttributes.X / spanAttributes.X in key checks.
Affected package
fi-instrumentation-otel (Python)
Package version
fi-instrumentation-otel==1.0.0 (note:
pyproject.tomlreports1.0.0butfi_instrumentation/instruments/version.pyreports__version__ = "0.1.19"- these disagree; flagging in case it's relevant to triage)What happened?
With
TraceConfig(pii_redaction=True)(orFI_PII_REDACTION=true), the regex-based PII scanner inpii_redaction.pyis applied unconditionally to every span attribute value inTraceConfig.mask(), with no key-value based masking rule in that method, which is scoped to a specific attribute key (INPUT_VALUE, message content, etc.).Two consequences:
session.id,user.id, andmetadataare injected viaget_attributes_from_context()and pass through the samemask()call as message content. Because the credit-card pattern "|\b(?:\d[ -]*?){13,19}\b" is just a bare 13-19 digit run with no Luhn check or grouping requirement, any UUID whose hex segments happen to contain a long enough digit run gets partially replaced with<CREDIT_CARD>. Measured over 100 randomuuid.uuid4()values, ~2% were corrupted. This silently breaks the exact session/user correlation the tracing products exists to provide exists to provide, and it's unattributed from the UI (looks like the session ID was never consistent).<CREDIT_CARD>even though they're not remotely credit-card shaped (no-grouping, fails Luhn). This degrades trace readability and defeats debugging on any span containing numeric metadata (costs, durations, request IDs, etc.).Expected: PII redaction should apply only to attributes that hold actual free-text content (input/output values, messages, content) - not to structured identifiers like
session.id,user.id, ormetadata- and the credit-card pattern should require actual card-like structure, not just "13-19 digits in a row.".Minimal reproduction
Error output / stack trace
N/A - This is a silent data-corruption bug, not an exception. No error is raised; the returned values are simply wrong.Environment
Python 3.12.5
Windows 11
Anything else?
Root cause is in two places:
fi_instrumentation/instrumentation/pii_redaction.py- has no grouping requirement and no Luhn check, so it matches any sufficiently long digit run regardless of shape.fi_instrumentation/instrumentation/config.py,TraceConfig.mask()- theif self.pii_redaction: resolved = redact_pii_in_value(resolved)branch runs unconditionally on every attribute key, unlike thehide_*rules above it in the same method, which are all gated withkey == SpanAttributes.X/spanAttributes.X in keychecks.