Skip to content

fix(post_rules): a trailing particle attaches to the family under a declared family-first order (#365) - #466

Closed
derek73 wants to merge 3 commits into
masterfrom
fix/365-trailing-particle-attaches
Closed

fix(post_rules): a trailing particle attaches to the family under a declared family-first order (#365)#466
derek73 wants to merge 3 commits into
masterfrom
fix/365-trailing-particle-attaches

Conversation

@derek73

@derek73 derek73 commented Aug 30, 2026

Copy link
Copy Markdown
Owner

A trailing surname particle stops being stranded as a standalone middle or given name under a declared family-first order.

Closes #365

The defect

The two family-first orders disagreed about the same input. They permute which role a trailing never-given particle falls into — GIVEN under one, MIDDLE under the other — and only the given position was inspected:

Mesnil Garcia de   FAMILY_FIRST             family 'Mesnil'  given 'Garcia'  middle 'de'
Mesnil Garcia de   FAMILY_FIRST_GIVEN_LAST  family 'Mesnil Garcia de'  given ''

Both now give family de Mesnil, given Garcia.

It is P6's rule, not P1's

The issue proposed adding MIDDLE as a third P1 site, and its own revision comment proposed dropping the other fold. Both are worse: the first loses Garcia because a particle trailed it, the second reports de as the given name, which P1 exists to prevent.

rules.md#P6 already states this rule for exactly this position — a particle ending the name has nothing to link forward to and must belong to the family written beside it — and scoped it to the comma form. Its decision record left the gap as an open parenthesis: "arguably a missing comma under a declared family-first order." This decides that. A declared family-first order names the family as a comma does, so Jong Anke de now reads as Jong, Anke de does.

The design is the issue's own third reading, mechanism included — family-first as an implied comma, with FOLDED_TAG doing the reorder. What this PR adds is the guards.

Three conditions, each the comma form read across

Each was a defect found in review before it was a condition. The comma and comma-less forms now agree row for row:

Jong, de         → family 'Jong de'     |  Jong de          FF → family 'Jong de'
Mesnil, Juan de  → family 'de Mesnil'   |  Mesnil Garcia de FF → family 'de Mesnil'

A name word must remain besides the family. Without P6's own words-to-spare test the new site was wider than the rule it narrows: Jong de attached where Jong, de declines, so two writings of one one-given-word listing disagreed about word order — and the no-given-word case decisions.md deliberately leaves open was half-decided in passing.

The family must have a base of its own. A name word merely ahead of the run is not enough. van Berg Jan de under FAMILY_FIRST lost its given name outright — 'Berg' before, empty after, family de van Berg Jan — because the leftover is partitioned into UNITS and _units chains a leading particle over every word after it, collapsing [van][Berg][Jan] into one unit that takes FAMILY. 288 of a 45,648-parse battery. P1 masks it for every never-given opening, which is why fixtures that all open with de missed it.

No suffix word stranded at the end. assign leaves a trailing Jr. in a name position when a particle follows it; take the particle away and the leftover is not the same name one word shorter, because the re-layout does not re-run assign's trailing peel. Berg Jan Jr. de reported given Jr., and every battery name where the two orders agreed before and disagreed after was this shape — the rule re-opening the divergence it exists to close. It declines rather than mirroring assign's peel by hand, which is the #424/#425 failure.

Never-given vocabulary only

The comma path takes ambiguous particles too; this one cannot. P6's own Accepted clause decides it: the comma-less FAMILY_FIRST_GIVEN_LAST form is the one format that reads the unaccented Vietnamese spelling correctly, and an ambiguous-inclusive attachment takes that format away — measured, Nguyen Thi Van would give family Van Nguyen, given Thi, against P6's pinned given Van. The words-to-spare guard does not save it, since a given word does remain.

The cost is the Dutch flagship word itself. van is ambiguous, so Beethoven Ludwig van keeps middle van where Beethoven, Ludwig van gives family van Beethoven. The two forms agree for the never-given half of the tussenvoegsel set and not for the rest. The rule says so, as an Accepted clause.

Reach

12 of 6600 parses — the 1100-name corpus × three name_order values × middle_as_family off and on — over four names, none under the default order. Two of the four are names this PR's own example lines add to the corpus, which is why the corpus is 1100 and not master's 1094.

The gate cannot see any of it. compare.py parses every name as HumanName(name) and sweeps no policy at all, so no non-default name_order behavior has ever been compared across versions. Output is unchanged at all three baselines apart from the added corpus names, none of which diffs. The counts carry a recompute recipe in decisions.md#P6 instead, under the AGENTS.md provenance rule — the comparator named as the parent commit reading the same corpus files.

The issue's headline number does not measure this shape: "464 inputs where the two orders differ" is what permuting given and middle does.

Review

Four agents. Two regressions, one missed instance of the bug being fixed, one design gap — plus a fourth finding neither the suite nor the corpus could reach: Mesnil Garcia de, Dr. still exhibited #365 verbatim. state.order is not None already excludes every comma that named a family, so testing structure is not FAMILY_COMMA as well excluded only the comma that fixed nothing. Two reviewers found that independently with different inputs.

A guard became dead code. The base and words-to-spare conditions made P1's lead site and P6's new site mutually exclusive — P1's lead fires only on a leading lone never-given particle, which by definition has no base. The FOLDED_TAG exclusion added to P1's redistribution is now unreachable (0 differences over 61,854 generated parses) and is deleted, with the exclusivity pinned instead.

Two comment defects worth naming, both of which would have misled a reader more than the code:

  • The site's headline example claimed Beethoven Ludwig van reads like its comma form. False, and it contradicted a machine-checked example line twelve lines below it in the same commit. A survivor of the rejected ambiguous-inclusive draft.
  • "Attaching first leaves that site nothing to fire on — 6 of 1094 names × 3 orders" was true of the corpus before this commit added van der to it via a new example line. corpus_rules.jsonl regenerates from rules.md, so a rule that adds examples adds corpus names and then counts them. Every count now names its corpus.

Also corrected: the order-agreement claim held only at two leftover pieces (Mesnil Garcia Carlos de disagrees at three); P6's P5 clause was unqualified where it is now comma-only; the stage header was behind the code; and the #359 release bullet described behavior this PR's bullet contradicts.

Verification

  • 6155 → 6178 passed; mypy, ruff and 232 sphinx doctests clean.
  • Gate green at all three baselines: 1100 names, 229 / 194 / 102 intentional, unexplained: 0.
  • Eleven conditions and branches mutation-verified, each killed by its own test — including the eight the test reviewer had killed silently on the first draft.
  • Reach measured against a checkout of the parent commit reading the same corpus files, not against any construction of the rule.

🤖 Generated with Claude Code

…-first order (#365)

The two family-first orders disagreed about the same input. They
permute which role a trailing never-given particle falls into --
GIVEN under one, MIDDLE under the other -- and only the given
position was inspected, so "Mesnil Garcia de" read family 'Mesnil',
given 'Garcia', middle 'de' under FAMILY_FIRST and family
'Mesnil Garcia de' with no given name at all under
FAMILY_FIRST_GIVEN_LAST.

The issue proposed adding MIDDLE as a third P1 site, and its own
later revision proposed dropping the other fold. Both are worse: the
first loses a given name because a particle trailed it, the second
reports 'de' AS the given name, which P1 exists to prevent. The
disagreement is not P1's. P6 already states this rule for exactly
this position -- a particle ending the name has nothing to link
forward to and must belong to the family written beside it -- and
scoped it to the comma form, leaving "arguably a missing comma under
a declared family-first order" as an open parenthesis in its own
decision record. That is what this decides: a declared family-first
order names the family as a comma does, so the attachment fires
there too, and "Jong Anke de" now reads as "Jong, Anke de" does.

Never-given vocabulary only, where the comma path takes ambiguous
particles too. P6's own Accepted clause is what decides it: the
comma-less FAMILY_FIRST_GIVEN_LAST form is the ONE format that reads
the unaccented Vietnamese spelling correctly, and taking ambiguous
particles here takes that format away.

Three details measurement found rather than design:
- the words-to-spare test is a BASE, not a name piece -- the wider
  reading reorders inside an all-particle name ('van der' -> 'der van')
- P6 now lands at two code sites, the no-comma one before P1 (whose
  given site would take the same run with the other reading) and the
  comma one after it (it reads the roles P1 settles)
- P1's family-first lead branch handed the attached particle straight
  back to the given slot, so it now skips FOLDED_TAG tokens

14 of 6564 parses move (1094 names x 3 orders x middle_as_family off
and on), over four names, none under the default order. The gate
cannot see any of it -- compare.py parses with default policy and
sweeps no policy at all -- so the count carries a recompute recipe in
decisions.md#P6 instead. Gate green at all three baselines.
_is_name_word carried `rules.md#R2's "a particle needs a base to
attach to"`. The citation test recognizes `rules.md#R2:` and not the
possessive form, so the quote was never compared to anything. Writing
it in the checked form made the test fail twice over: a docstring's
continuation lines do not start with `#`, so the extractor sees only
the first line and finds no closed quote; and the excerpt is from an
Accepted clause, where _statements() stops at a rule's first example.

Both are properties of the doc format rather than defects, so the
docstring says what it means in its own words and records why it
cites nothing. The rule's words are in P6's body, which is where this
predicate's caller cites.
Two regressions, one missed instance of the bug being fixed, and a
design gap that made the new site wider than the rule it extends.

(i) A NAME WORD MUST REMAIN BESIDES THE FAMILY. Without P6's own
words-to-spare test the no-comma site was WIDER than the comma site
it claims to narrow: `Jong de` attached and gave family 'de Jong'
where `Jong, de` declines and gives 'Jong de'. Two writings of one
one-given-word listing disagreed about word order, and the no-given-
word case that decisions.md leaves deliberately open was half-decided
in passing. With the test the two forms agree row for row, `Mesnil de`
stays P1's fold, and P1's example line for it is unchanged by this PR.

(ii) THE FAMILY MUST HAVE A BASE, not merely a name word ahead of the
run. `van Berg Jan de` under FAMILY_FIRST lost its given name outright
-- 'Berg' before, empty after -- because the leftover is partitioned
into units and _units chains a leading particle over every word after
it, collapsing [van][Berg][Jan] to one unit that takes FAMILY. 288 of
a 45,648-parse battery lost a given name that way. P1 masks it for
every never-given opening, which is why fixtures all opening with `de`
missed it.

(iii) NO SUFFIX WORD STRANDED AT THE END. assign leaves a trailing
`Jr.` in a name position when a particle follows it, and the re-layout
does not re-run assign's trailing peel, so `Berg Jan Jr. de` reported
given 'Jr.'. Every battery name where the two orders agreed before and
disagreed after was this shape -- the rule re-opening the divergence
it exists to close. It declines rather than mirroring assign's peel by
hand, which is the #424/#425 failure.

(iv) NO STRUCTURE TEST. `state.order is not None` already excludes
every comma that named a family, so testing the structure too excluded
only the comma that fixed nothing -- where #365's symptom survived
verbatim (`Mesnil Garcia de, Dr.`).

Reach falls from 14 parses to 12 of 6600 over four names, none under
the default order. Conditions (i) and (ii) also made the two particle
sites mutually exclusive, so the FOLDED_TAG exclusion added to P1's
redistribution is unreachable -- 0 differences over 61,854 parses --
and is deleted, with the exclusivity pinned instead.

Comments and docs corrected with it: the site's headline example
claimed `Beethoven Ludwig van` reads like its comma form, which is
false and contradicts an example line twelve lines away (`van` is
ambiguous vocabulary); "attaching first leaves that site nothing to
fire on" counted a corpus this commit then changed; the order-agreement
claim held only at two leftover pieces; P6's P5 clause and the stage
header were behind the code; and the #359 release bullet described
behavior this bullet contradicts. Every count now names its corpus.

Eleven conditions and branches mutation-verified, each killed by its
own test.
@derek73 derek73 added this to the v2.2 milestone Aug 30, 2026
@derek73 derek73 added the bug label Aug 30, 2026
@derek73 derek73 self-assigned this Aug 30, 2026
@codecov

codecov Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.66%. Comparing base (caaed90) to head (c0c4175).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #466      +/-   ##
==========================================
+ Coverage   98.65%   98.66%   +0.01%     
==========================================
  Files          45       45              
  Lines        3194     3219      +25     
==========================================
+ Hits         3151     3176      +25     
  Misses         43       43              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73

derek73 commented Aug 30, 2026

Copy link
Copy Markdown
Owner Author

Closing unmerged. The review that hardened this PR also undermined its premise.

The rule this extends is a relaxation: a particle can only join backward when something has said the family comes first, and without a comma the parser is inserting one on no evidence but the particle's position. Jong Anke de was being read as Jong, Anke de by guess.

Three review rounds found three misparses from that guess reaching too far — a lost given name (van Berg Jan de), a post-nominal promoted into the given slot (Berg Jan Jr. de), and a missed instance of the original bug (Mesnil Garcia de, Dr.). Each was answered with another guard, four in the end. That ratio is the evidence: the feature was being fenced rather than fixed.

Two premises this PR argued from are also wrong, and both were mine:

  • "A never-given particle cannot be a name at all." False. NON_GIVEN_NAME_PARTICLES supplies a default reading, not a prohibition — measured, all 33 of them read as the given name where position forces it ("de" and "de Jr."). This was the sole argument I gave against the strict resolution, and it does not stand.
  • 1.4.0 parity. Irrelevant here: 1.4.0 has no name_order, so nothing in the family-first orders has a v1 answer to be measured against. The comparisons in this branch's release bullet were beside the point.

#365 stays open and will be resolved the strict way instead: a particle is read where the declared order puts it, and joins backward only after an explicit comma. Superseded by the design work tracked in the follow-up.

The branch fix/365-trailing-particle-attaches is kept for the measurements and the review record.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mesnil Garcia de strands the particle under FAMILY_FIRST but folds it under FAMILY_FIRST_GIVEN_LAST

1 participant