Repository navigation
fix: report a timeout or budget limit instead of folding it into "no match" - #8
Merged
Merged
Conversation
…match"
A match abandoned at a limit was indistinguishable from a subject that
genuinely does not match: every match path collapsed on
if err != nil || !matched { return 0, 0, false }
so ErrTimeout and ErrBudget both read as a clean non-match. For a Regexp
used as a validator or a denylist that is fail-open — a crafted subject
reads as "the guard did not fire" and passes the check. Ruby's
Regexp::TimeoutError exists to prevent exactly this, and could never be
raised through this API because the information did not leave the engine.
Two things change.
1. ErrTimeout and ErrBudget are exported, and four `…Err` methods report
them: MatchBoundsErr, MatchBoundsAtErr, FindStringSubmatchIndexErr and
FindStringSubmatchIndexAtErr. They are the four internal funnels every
other match method already went through, so this is the complete set of
places a limit can be reached — not a sample. The standard-library-shaped
methods keep their signatures and still fold both reasons into "no
match"; their doc comments now say so and point at the `…Err` twin, and
a test pins that shape so it is a recorded decision rather than an
accident.
ErrBudget needs no timeout to be configured to be reached: on `(a+)+\1b`
against "a"*26 the default budget is exhausted in ~0.83s, so the
fail-open was reachable at the default settings, not only when a caller
had asked for a limit.
2. The anchored paths carried no deadline at all. MatchBoundsAt and
FindStringSubmatchIndexAt called vm.MatchAt, the budget-only entry point,
so WithTimeout was silently ignored on every anchored match however small
the limit. They now call vm.MatchTimeoutAt with re.deadline(), which
already existed. Measured through the Ruby layer above this one,
String#rindex on a 26-byte subject with a 0.05s limit took 3.593s before
and 0.128s after.
No behaviour changes on the lazy-NFA subset, which cannot reach either
limit; a test asserts err is nil there, so a caller cannot be handed a
spurious refusal. The non-Err methods are unchanged for every input that
does not reach a limit.
Paired with go-ruby-regexp/regexp and go-embedded-ruby/ruby#776, which
cannot report Regexp::TimeoutError without this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README's only timeout row named `WithTimeout`/`Timeout` and nothing about what a reached limit reports. `ErrTimeout` and `ErrBudget` appeared nowhere in it, although they are now the package's safety contract: a caller who reads only this table takes away the old model, in which a limit is silently a non-match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
Every match path in
regexp.gocollapsed on the same line:ErrTimeoutandErrBudgetwere folded into a clean non-match. For aRegexpused as a validator, a denylist or any other guard that is fail-open: a
crafted subject reads as "the guard did not fire" and passes the check. Ruby's
Regexp::TimeoutErrorexists to prevent exactly this, and the Ruby layers abovethis package could never raise it, because the information did not leave the
engine.
This did not need a timeout to be configured.
ErrBudgetis reached underthe default
DefaultBudgeton a catastrophically backtracking pattern:(a+)+\1b"a"*20err == nil(search completed)"a"*22err == nil(search completed)"a"*24ErrBudget, swallowed into "no match""a"*26ErrBudget, swallowed into "no match""a"*28ErrBudget, swallowed into "no match"So the fail-open was reachable at the default settings, not only when a
caller had asked for a limit.
Change 1 — the limits are reportable
ErrTimeoutandErrBudgetare exported, and four…Errmethods report them:MatchBoundsErr(s)MatchString,Match,MatchBounds,FindStringIndex,FindStringMatchBoundsAtErr(s, pos)MatchBoundsAtFindStringSubmatchIndexErr(s)FindStringSubmatchIndex,FindStringSubmatch,FindAll*,ReplaceAll*FindStringSubmatchIndexAtErr(s, pos)FindStringSubmatchIndexAtThese are the four internal funnels every other match method already went
through, so this is the complete set of places a limit can be reached — not a
sample. Nothing new can reach a limit without going through one of them.
The standard-library-shaped methods keep their signatures and still fold both
reasons into "no match". Their doc comments now say so and name the
…Errtwin,and
TestNonErrFormsStillFoldTheLimitpins that shape across all twelve of them,so the divergence from
regexp.Regexpis a recorded decision rather than anaccident waiting to be "fixed" by someone adding a thirteenth.
Change 2 — the anchored paths carried no deadline at all
MatchBoundsAtandFindStringSubmatchIndexAtcalledvm.MatchAt— thebudget-only entry point — so
WithTimeoutwas silently ignored on everyanchored match, however small the limit. They now call
vm.MatchTimeoutAtwithre.deadline(); that function already existed, it was simply never wired up.Measured through the Ruby layers above this one,
String#rindex(which probesone anchored start position after another) on a 26-byte subject with a 0.05 s
limit:
"a"*26.rindex(/(a+)+\1b/)withtimeout: 0.05What does not change
TestLazyNFASubsetReportsNoErrorasserts
err == nilthere, so a caller cannot be handed a spurious refusal —including for
(a+)+b, the classic one-line ReDoS, which is linear here.method,
…Error not.FindAll*andReplaceAll*keep ending their walk at a reached limit withthe results found so far, exactly as a genuine non-match ends it. That family
has no channel to report the difference; the comments in
allMatchesandreplaceAllnow say so and point at the…Errprimitives. This is theone remaining fail-open surface in the package and it is deliberate: the
alternative is seventeen
…Errtwins, of which the 100 % coverage gate wouldmake thirteen dead weight. The Ruby layer does not use that family — it drives
its own scan loop over the primitives.
Cost
Both additions sit on every match: one type assertion on the VM's class-level
default in the Ruby layer's matcher, and one extra error return per call. Measured through the
repo's own Ruby-level regexp benchmarks (
internal/vm/bench_test.go),origin/mainvs this branch, interleaved main/fix over 5 rounds at
-benchtime=2son anotherwise idle machine, from pre-built test binaries:
origin/mainmedianInlineRegexpMatchInlineRegexpScanAllocations are identical — 10,982 allocs/op on
Matchand 34,331 onScanon both sides, with byte counts agreeing to within a few bytes of 424 KB and
1.977 MB. The deltas are an order of magnitude below the run-to-run spread within
each side, so the honest statement is no measurable cost, in either direction.
The zero-limit path allocates nothing new by construction: with both limits unset
the type assertion fails and
matcherreturns the receiver's engine Regexpunwrapped, exactly as before, and the VM never reads the clock.
Verification
go test -racewith the CI's-coverpkg: all green, coverage 100.0 % (thegate this repo enforces).
gofmt -lclean onregexp.go,replace.goand the new test file. (gofmt -l .reports 18 files on this repo, all of them pre-existing test files untouched by
this change —
git stash && gofmt -l . | wc -lgives the same 18 onorigin/main.)GOOS=wasip1/GOOS=js(GOARCH=wasm) builds: both green.Paired PRs
This is one of three. None of them is useful alone:
Regexp::TimeoutErrorfrom them.Merge order is this PR, then the wrapper, then rbgo; each downstream
go.modcurrently points at the upstream branch commit and needs re-pointing at the
tagged release as each merges.
🤖 Generated with Claude Code