fix: decide separated action text by shape, and label identifiers by recognizer - #19
Merged
Merged
Conversation
…recognizer
Two defects, both reproduced against 1.0.3.
scrub_dict mangled a plain "text" value. "text" is in SCRUB_KEYS_HTML, so
_scrub_text_item passed is_separated=True for it, and scrub_text re-joined the
anonymized result with ACTION_TEXT_SEP whether or not the input had ever been
split. A sentence came back one character per hyphen:
scrub_dict({"text": "Email: john@example.com"}, scrubber)
{'text': '<-P-E-R-S-O-N->-:- -<-E-M-A-I-L-_-A-D-D-R-E-S-S->'}
The split and the join are now symmetric: is_separated permits key-sequence
handling, and _is_separated_form decides whether to apply it by looking at the
value. Action text is exactly one typed character per separator, so "j-o-h-n"
is reassembled and re-separated as before, while "follow-up", "555-123-4567"
and ordinary prose are scrubbed as prose. Which keys may carry action text is
now PrivacyConfig.SCRUB_KEYS_SEPARATED instead of a hard-coded tuple, and
scrub_dict/scrub_list_dicts take separated_keys to set it per call.
Entity labels named the wrong type, so policy keyed on them was unusable.
Presidio's anonymizer settles overlapping spans by span before score, so a
spaCy DATE_TIME at 0.85 covering "Card 4111111111111111" replaced a
Luhn-checked CREDIT_CARD at 1.0. Same for US_BANK_NUMBER, IP_ADDRESS, and a
member ID that came back as PERSON. Nothing leaked, but nothing could be
routed on either.
_prefer_deterministic_identifier_labels drops the statistical result where a
pattern recognizer scoring at least 0.4 claims the span. It drops it only when
the pattern matches cover every digit the statistical span held, so relabelling
can never uncover part of an identifier: Presidio matches only the leading
"A123" of "A123-456-789-012", and a wider span over the rest survives.
A card number that fails the Luhn check still gets no CREDIT_CARD label,
because no recognizer validates it. That is now a documented, tested limit
rather than a surprise, and the README example uses a valid number.
test_phi_recall.py asserted only that the identifier had disappeared, which is
why this went unnoticed. Every case now pins the entity type as well.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects, both reproduced against the published 1.0.3 on macOS.
1.
scrub_dictmangled a plaintextvaluetextis inSCRUB_KEYS_HTML, so_scrub_text_itempassedis_separated=Truefor it. Inside
scrub_textthe split and the join were not symmetric: the spliton
ACTION_TEXT_SEPis a no-op on a value that holds no separator, but the joinran anyway and put a separator between every character of the result.
The design question: opt-in per call, or keyed off the field name?
Neither. It is keyed off the value's shape, and the key name only grants
permission.
A per-call flag pushes the decision onto a caller who often cannot make it.
scrub_dictrecurses through an arbitrary nested structure, so one flag has tocover every value at every depth, and a GUI element tree legitimately mixes
recorded keystrokes with element labels under the same key name. The shape test,
by contrast, is exact for the encoding in question: action text is
ACTION_TEXT_SEP.join(chars), so every separated chunk is exactly onecharacter.
follow-upand555-123-4567fail that test;j-o-h-npasses it.Applying the test in the wrong direction costs nothing in either direction:
invisible in the split form is still found. This is the direction that would
leak, and it is preserved.
a false positive is a no-op.
separated_keysis still exposed onscrub_dict/scrub_list_dicts, backedby the new
PrivacyConfig.SCRUB_KEYS_SEPARATED, so a caller who wants it offentirely passes
separated_keys=[]and one who uses a different field namepasses their own. The hard-coded
("text", "canonical_text")tuple is gone.Callers checked
No repository in the workspace calls
scrub_dict,scrub_list_dictsorDictScrubberfrom this package at all.PresidioScrubbingProvider().scrub_text(text)inprivacy.py,hosted.py,sanitized_artifact.py,compiler/compile.py,console/halt_detail.py,scripts/check_bundle_phi.py— always the one-argument form, neveris_separated. Its tests inject fakes, so no test pins a placeholder name from the real analyzer.engine/scrubber.pycallsself._provider.scrub_text(text). Same one-argument form.pyproject.toml, calls nothing.legacy/openadapt/privacy/base.py. Separate code, not this package.So the action-text path this behaviour existed for is not live in any current
consumer, and the shape gate preserves it anyway for the one that revives it.
2. Entity labels named the wrong type
Root cause is not recognizer ordering or a threshold.
analyzer.analyze()returns all four candidates for the first case,
CREDIT_CARDamong them atscore 1.0:
Presidio's anonymizer resolves the overlap by span before score, so the
wider spaCy
DATE_TIMEwon and took the word "Card" with it. A statistical NERspan outranking a Luhn-validated pattern match is the inversion.
_prefer_deterministic_identifier_labelsdrops the statistical result where apattern recognizer scoring at least 0.4 overlaps it. The 0.4 floor keeps the
0.01–0.05 context-free digit-run guesses from taking a label away from NER.
Redaction coverage cannot shrink. The statistical result is dropped only when
the pattern matches cover every digit its span held. Presidio's driver
licence recognizer matches only the leading
A123ofA123-456-789-012, so awider span over the rest survives and the trailing digits stay redacted — there
is a unit test for exactly that.
One case is not a code defect
'Card: 4532-1234-5678-9012'still returns<DATE_TIME>. That number fails itsLuhn check, so Presidio's card recognizer invalidates it and
CREDIT_CARDneverenters the candidate list. The fixture was wrong, not the analyzer. The recall
suite now uses
4532-1234-5678-9014, and the old number has its own testpinning the documented limit.
Not fixed, reported
A123-456-789-012still scrubs to<US_DRIVER_LICENSE>-456-789-012. That is aspan problem in Presidio's upstream recognizer, not a label problem, and
tightening it needs a new pattern with its own false-positive budget. Separate
work.
Tests
tests/test_phi_recall.pyasserted only that the identifier had disappeared,which is why a card number reading
<DATE_TIME>passed for months. Every casenow pins the entity type too. Both new tests were watched failing first:
118 pass after the fix. No existing test was weakened; the recall gate still
requires 26 of 26.
Docs
#18 documented both defects honestly and merged an hour before this branch.
This PR is rebased on it and rewrites the two paragraphs the fix invalidates:
the
Card ... -> <DATE_TIME>example and the "one caveat in 1.0.3" note aboutthe
textkey. The Luhn limit replaces them, with a measured example of eachside.
CLAUDE.md's entity table also claimed4532-1234-5678-9012produces<CREDIT_CARD>; it now uses a valid number.Released
Shipped as v1.0.4 and verified against the wheel installed from PyPI:
The release workflow break reported earlier today (
type object 'Actor' has no attribute 'name_email_regex'insidepython-semantic-release) did notreproduce: the post-merge run tagged, released and published without
intervention.