Skip to content

fix(ir): keep CanonicalWords a fixed point on caseless capitals - #366

Open
OmarAlJarrah wants to merge 1 commit into
mainfrom
fix/ir-canonicalwords-caseless-run
Open

fix(ir): keep CanonicalWords a fixed point on caseless capitals#366
OmarAlJarrah wants to merge 1 commit into
mainfrom
fix/ir-canonicalwords-caseless-run

Conversation

@OmarAlJarrah

Copy link
Copy Markdown
Member

Summary

ir.CanonicalWords is documented as a fixed point, and
FuzzCanonicalWords_Properties/assertIdempotent asserts it. Feeding a
canonical back in could still change it:

CanonicalWords("ℤℤA") == "ℤℤa"
CanonicalWords("ℤℤa") == "ℤ_ℤa"

The acronym-tail rule in wordBoundary splits at the last capital of a run when
the next rune is lowercase. "ℤℤA" has no lowercase rune at all, so the first
pass produces one word and lowercases it — and lowercasing is exactly what
supplies the missing lowercase letter: has no lowercase form and survives as
a capital, while A becomes a. The second pass now sees two capitals with a
lowercase letter following, fires the tail rule, and yields "ℤ_ℤa".

This is #187's mechanism at a position #187 did not reach — there the caseless
capital followed the cased one, here it precedes it — and the consequence is the
same one #187 recorded: the segmentation depends on the casing of the source
spelling rather than on its words, and Naming.Canonical is ABI precisely so an
emitter never has to know which spelling produced it.

ir/naming.go's own comment stated the invariant this broke, and stated it as
the reason the two rules were safe together:

Both stay idempotent: after one pass the only capitals left are the ones
lowercasing does not change, and the tail rule needs a lowercase letter
following, which such a run does not produce on its own.

The run does not produce it. The rune after the run does, by being lowercased.

The fix, and why it is this one

The tail rule now requires that lowercasing change at least one of the two
capitals either side of the split
. Requiring it of both is too strong in either
direction, and each direction breaks a row canonicalCases already pins:

spelling asking case of the left rune asking case of the right rune asking it of the pair
ℤServer ℤserver ℤ_server ℤ_server
HTTPℤerver http_ℤerver httpℤerver http_ℤerver
ℤℤa ℤℤa ℤℤa ℤℤa

Asking it of the pair keeps every pinned row and makes the grammar a fixed point
for a reason that can be stated rather than observed: one pass lowercases every
rune that carries case, so both case rules — each of which now requires one at
the boundary — have nothing left to fire on, and the letter/digit rule is
case-independent and has already been applied wherever it applies.

Three spellings change, all of them a caseless run splitting itself:
"ℤℤa"ℤℤa (was ℤ_ℤa), "aℤℤb"aℤℤb (was aℤ_ℤb), and the
"ℤℤA" above. None is pinned by a table row today; all three are now.

Test plan

  • Three new rows in canonicalCases pinning ℤℤA, ℤℤa and ℤℤAb, the last
    holding that a caseless run still does end at a capital that carries case.
  • "ℤℤA" added to adversarialRunes with the reason it is there: the seed
    beside it, "ℤℤa", was already present and passed, because the input that
    broke the property was the uppercase spelling one mutation away from it.
  • Watched red: with the pair condition removed,
    TestCanonicalWords_Conformance/so_the_lowercase_spelling_segments_the_same_way
    fails expected: "ℤℤa", actual: "ℤ_ℤa", and the fuzz seed fails with
    not a fixed point / input: "ℤℤA" / once: "ℤℤa" / twice: "ℤ_ℤa".
  • Searched for another counterexample: -fuzz for 90s, 5,303,142 executions,
    228 interesting inputs, no failure.

No golden moved: no name in the corpus contains a letter with no lowercase form.

Full gate green: gofmt, go vet ./..., golangci-lint run (0 issues),
go build ./..., ./scripts/check-coverage.sh (all 4947 statements covered).

Closes #336

CanonicalWords is documented as a fixed point and the fuzz target asserts
it, but feeding a canonical back in could still change it:

    CanonicalWords("ℤℤA") == "ℤℤa"
    CanonicalWords("ℤℤa") == "ℤ_ℤa"

The acronym-tail rule splits at the last capital of a run when the next
rune is lowercase. "ℤℤA" has no lowercase rune at all, so the first pass
produces one word and lowercases it -- and lowercasing is what supplies
the missing lowercase letter, since ℤ has no lowercase form and survives
as a capital while A becomes a. The second pass sees two capitals with a
lowercase following, fires the tail rule, and yields "ℤ_ℤa".

This is #187's mechanism at a position #187 did not reach: there the
caseless capital followed the cased one, here it precedes it. The
consequence is the same -- the segmentation depends on the casing of the
source spelling rather than on its words, and Canonical is ABI precisely
so an emitter never has to know which spelling produced it.

The tail rule now requires that lowercasing change at least one of the
two capitals either side of the split. Requiring it of both would be too
strong in either direction: asking it of the left rune loses ℤ_server,
asking it of the right loses http_ℤerver, and canonicalCases pins both.
Asking it of the pair keeps every pinned row, and makes the grammar a
fixed point for a reason that can be stated: one pass lowercases every
rune that carries case, so neither case rule has anything left to fire
on, and the letter/digit rule is case-independent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ir: CanonicalWords is not idempotent when lowercasing creates an acronym tail

1 participant