Skip to content

fix: sanitize ONNX backend errors and restore the green formatting gate - #57

Open
mazzasaverio wants to merge 1 commit into
ma2za:mainfrom
mazzasaverio:fix/backend-error-and-scan-hardening
Open

fix: sanitize ONNX backend errors and restore the green formatting gate#57
mazzasaverio wants to merge 1 commit into
ma2za:mainfrom
mazzasaverio:fix/backend-error-and-scan-hardening

Conversation

@mazzasaverio

Copy link
Copy Markdown
Contributor

Three release-gate defects, each reproducible on main at 0.17.0.

CI is red on main

uv run ruff format --check . fails on main: the 0.16.0 OCR commit landed unformatted (single-quoted string, over-length call). CI runs that step on every push and the release workflow runs it too, so 0.16.0 and 0.17.0 both shipped through a red gate, and every new PR inherits the failure. The two files are reformatted here so the gate passes again.

$ uv run ruff format --check .
2 files would be reformatted, 124 files already formatted

uv.lock also still recorded version = "0.16.0"; the 0.17.0 release commit only touched pyproject.toml. It is refreshed here.

The ONNX backend leaked third-party exception text

raise BackendExecutionError(f"ONNX PII inference failed: {e}") from e

The release gate requires that exceptions do not expose matched values, and #40 established the same invariant for adapters. Tokenizer and runtime failures routinely quote the input they choked on. The message is now fixed text and the cause is dropped, matching how every other adapter and backend in the tree raises. The two tests that asserted the leaked message now assert its absence, using a sabotaged runtime whose message quotes a name.

Detection scanned every character for format characters

_detect_block called unicodedata.category per character and built two N-sized lists on every block, even when nothing was stripped, which is the overwhelmingly common case.

No ASCII character has category Cf, so pure ASCII text now takes a single C-level str.isascii() scan and keeps its source offsets untouched; non-ASCII text pays for the rebuild only when a format character is actually present. Measured on a 64 KiB block, the old scan was 13% of end-to-end processing time.

Offset behaviour is unchanged and now pinned by a parametrized test covering plain ASCII, non-ASCII without controls, and zero-width and bidirectional controls inside and around a detected span.

Verification

263 passed, 1 skipped     # Tesseract not installed locally
ruff format --check .     All checks passed
ruff check .              All checks passed
mypy src tests            Success

🤖 Generated with Claude Code

https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf

Three release-gate defects, all reproducible on main.

The ONNX backend interpolated the originating exception message into
BackendExecutionError. The release gate requires that exceptions do not
expose matched values, and PR ma2za#40 established the same invariant for
adapters; tokenizer and runtime messages can quote the input they failed
on. The message is now fixed text and the cause is dropped, matching how
every other adapter and backend in the tree raises.

Detection also scanned every character with unicodedata.category on every
block, building two N-sized lists even when nothing was stripped. No ASCII
character has category Cf, so pure ASCII text now takes a single C-level
scan and keeps the source offsets untouched, and non-ASCII text only pays
for the rebuild when a format character is actually present. This is 13%
of end-to-end processing time on a 64 KiB block.

Finally, `uv run ruff format --check .` fails on main: the 0.16.0 OCR
commit landed unformatted, so CI has been red across 0.16.0 and 0.17.0
even though both shipped. The two files are reformatted here so the gate
passes again. uv.lock still recorded 0.16.0 and is refreshed to 0.17.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf
@mazzasaverio
mazzasaverio force-pushed the fix/backend-error-and-scan-hardening branch from 07e11d2 to 804a888 Compare August 30, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant