fix: sanitize ONNX backend errors and restore the green formatting gate - #57
Open
mazzasaverio wants to merge 1 commit into
Open
fix: sanitize ONNX backend errors and restore the green formatting gate#57mazzasaverio wants to merge 1 commit into
mazzasaverio wants to merge 1 commit into
Conversation
This was referenced Aug 29, 2026
Three release-gate defects, all reproducible on main. The ONNX backend interpolated the originating exception message into BackendExecutionError. The release gate requires that exceptions do not expose matched values, and PR ma2za#40 established the same invariant for adapters; tokenizer and runtime messages can quote the input they failed on. The message is now fixed text and the cause is dropped, matching how every other adapter and backend in the tree raises. Detection also scanned every character with unicodedata.category on every block, building two N-sized lists even when nothing was stripped. No ASCII character has category Cf, so pure ASCII text now takes a single C-level scan and keeps the source offsets untouched, and non-ASCII text only pays for the rebuild when a format character is actually present. This is 13% of end-to-end processing time on a 64 KiB block. Finally, `uv run ruff format --check .` fails on main: the 0.16.0 OCR commit landed unformatted, so CI has been red across 0.16.0 and 0.17.0 even though both shipped. The two files are reformatted here so the gate passes again. uv.lock still recorded 0.16.0 and is refreshed to 0.17.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf
mazzasaverio
force-pushed
the
fix/backend-error-and-scan-hardening
branch
from
August 30, 2026 05:33
07e11d2 to
804a888
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three release-gate defects, each reproducible on
mainat 0.17.0.CI is red on
mainuv run ruff format --check .fails onmain: the 0.16.0 OCR commit landed unformatted (single-quoted string, over-length call). CI runs that step on every push and the release workflow runs it too, so 0.16.0 and 0.17.0 both shipped through a red gate, and every new PR inherits the failure. The two files are reformatted here so the gate passes again.uv.lockalso still recordedversion = "0.16.0"; the 0.17.0 release commit only touchedpyproject.toml. It is refreshed here.The ONNX backend leaked third-party exception text
The release gate requires that exceptions do not expose matched values, and #40 established the same invariant for adapters. Tokenizer and runtime failures routinely quote the input they choked on. The message is now fixed text and the cause is dropped, matching how every other adapter and backend in the tree raises. The two tests that asserted the leaked message now assert its absence, using a sabotaged runtime whose message quotes a name.
Detection scanned every character for format characters
_detect_blockcalledunicodedata.categoryper character and built two N-sized lists on every block, even when nothing was stripped, which is the overwhelmingly common case.No ASCII character has category
Cf, so pure ASCII text now takes a single C-levelstr.isascii()scan and keeps its source offsets untouched; non-ASCII text pays for the rebuild only when a format character is actually present. Measured on a 64 KiB block, the old scan was 13% of end-to-end processing time.Offset behaviour is unchanged and now pinned by a parametrized test covering plain ASCII, non-ASCII without controls, and zero-width and bidirectional controls inside and around a detected span.
Verification
🤖 Generated with Claude Code
https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf