fix(post_rules): a trailing particle attaches to the family under a declared family-first order (#365) - #466
fix(post_rules): a trailing particle attaches to the family under a declared family-first order (#365)#466derek73 wants to merge 3 commits into
Conversation
…-first order (#365) The two family-first orders disagreed about the same input. They permute which role a trailing never-given particle falls into -- GIVEN under one, MIDDLE under the other -- and only the given position was inspected, so "Mesnil Garcia de" read family 'Mesnil', given 'Garcia', middle 'de' under FAMILY_FIRST and family 'Mesnil Garcia de' with no given name at all under FAMILY_FIRST_GIVEN_LAST. The issue proposed adding MIDDLE as a third P1 site, and its own later revision proposed dropping the other fold. Both are worse: the first loses a given name because a particle trailed it, the second reports 'de' AS the given name, which P1 exists to prevent. The disagreement is not P1's. P6 already states this rule for exactly this position -- a particle ending the name has nothing to link forward to and must belong to the family written beside it -- and scoped it to the comma form, leaving "arguably a missing comma under a declared family-first order" as an open parenthesis in its own decision record. That is what this decides: a declared family-first order names the family as a comma does, so the attachment fires there too, and "Jong Anke de" now reads as "Jong, Anke de" does. Never-given vocabulary only, where the comma path takes ambiguous particles too. P6's own Accepted clause is what decides it: the comma-less FAMILY_FIRST_GIVEN_LAST form is the ONE format that reads the unaccented Vietnamese spelling correctly, and taking ambiguous particles here takes that format away. Three details measurement found rather than design: - the words-to-spare test is a BASE, not a name piece -- the wider reading reorders inside an all-particle name ('van der' -> 'der van') - P6 now lands at two code sites, the no-comma one before P1 (whose given site would take the same run with the other reading) and the comma one after it (it reads the roles P1 settles) - P1's family-first lead branch handed the attached particle straight back to the given slot, so it now skips FOLDED_TAG tokens 14 of 6564 parses move (1094 names x 3 orders x middle_as_family off and on), over four names, none under the default order. The gate cannot see any of it -- compare.py parses with default policy and sweeps no policy at all -- so the count carries a recompute recipe in decisions.md#P6 instead. Gate green at all three baselines.
_is_name_word carried `rules.md#R2's "a particle needs a base to attach to"`. The citation test recognizes `rules.md#R2:` and not the possessive form, so the quote was never compared to anything. Writing it in the checked form made the test fail twice over: a docstring's continuation lines do not start with `#`, so the extractor sees only the first line and finds no closed quote; and the excerpt is from an Accepted clause, where _statements() stops at a rule's first example. Both are properties of the doc format rather than defects, so the docstring says what it means in its own words and records why it cites nothing. The rule's words are in P6's body, which is where this predicate's caller cites.
Two regressions, one missed instance of the bug being fixed, and a design gap that made the new site wider than the rule it extends. (i) A NAME WORD MUST REMAIN BESIDES THE FAMILY. Without P6's own words-to-spare test the no-comma site was WIDER than the comma site it claims to narrow: `Jong de` attached and gave family 'de Jong' where `Jong, de` declines and gives 'Jong de'. Two writings of one one-given-word listing disagreed about word order, and the no-given- word case that decisions.md leaves deliberately open was half-decided in passing. With the test the two forms agree row for row, `Mesnil de` stays P1's fold, and P1's example line for it is unchanged by this PR. (ii) THE FAMILY MUST HAVE A BASE, not merely a name word ahead of the run. `van Berg Jan de` under FAMILY_FIRST lost its given name outright -- 'Berg' before, empty after -- because the leftover is partitioned into units and _units chains a leading particle over every word after it, collapsing [van][Berg][Jan] to one unit that takes FAMILY. 288 of a 45,648-parse battery lost a given name that way. P1 masks it for every never-given opening, which is why fixtures all opening with `de` missed it. (iii) NO SUFFIX WORD STRANDED AT THE END. assign leaves a trailing `Jr.` in a name position when a particle follows it, and the re-layout does not re-run assign's trailing peel, so `Berg Jan Jr. de` reported given 'Jr.'. Every battery name where the two orders agreed before and disagreed after was this shape -- the rule re-opening the divergence it exists to close. It declines rather than mirroring assign's peel by hand, which is the #424/#425 failure. (iv) NO STRUCTURE TEST. `state.order is not None` already excludes every comma that named a family, so testing the structure too excluded only the comma that fixed nothing -- where #365's symptom survived verbatim (`Mesnil Garcia de, Dr.`). Reach falls from 14 parses to 12 of 6600 over four names, none under the default order. Conditions (i) and (ii) also made the two particle sites mutually exclusive, so the FOLDED_TAG exclusion added to P1's redistribution is unreachable -- 0 differences over 61,854 parses -- and is deleted, with the exclusivity pinned instead. Comments and docs corrected with it: the site's headline example claimed `Beethoven Ludwig van` reads like its comma form, which is false and contradicts an example line twelve lines away (`van` is ambiguous vocabulary); "attaching first leaves that site nothing to fire on" counted a corpus this commit then changed; the order-agreement claim held only at two leftover pieces; P6's P5 clause and the stage header were behind the code; and the #359 release bullet described behavior this bullet contradicts. Every count now names its corpus. Eleven conditions and branches mutation-verified, each killed by its own test.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #466 +/- ##
==========================================
+ Coverage 98.65% 98.66% +0.01%
==========================================
Files 45 45
Lines 3194 3219 +25
==========================================
+ Hits 3151 3176 +25
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Closing unmerged. The review that hardened this PR also undermined its premise. The rule this extends is a relaxation: a particle can only join backward when something has said the family comes first, and without a comma the parser is inserting one on no evidence but the particle's position. Three review rounds found three misparses from that guess reaching too far — a lost given name ( Two premises this PR argued from are also wrong, and both were mine:
#365 stays open and will be resolved the strict way instead: a particle is read where the declared order puts it, and joins backward only after an explicit comma. Superseded by the design work tracked in the follow-up. The branch |
A trailing surname particle stops being stranded as a standalone middle or given name under a declared family-first order.
Closes #365
The defect
The two family-first orders disagreed about the same input. They permute which role a trailing never-given particle falls into — GIVEN under one, MIDDLE under the other — and only the given position was inspected:
Both now give family
de Mesnil, givenGarcia.It is P6's rule, not P1's
The issue proposed adding MIDDLE as a third P1 site, and its own revision comment proposed dropping the other fold. Both are worse: the first loses
Garciabecause a particle trailed it, the second reportsdeas the given name, which P1 exists to prevent.rules.md#P6already states this rule for exactly this position — a particle ending the name has nothing to link forward to and must belong to the family written beside it — and scoped it to the comma form. Its decision record left the gap as an open parenthesis: "arguably a missing comma under a declared family-first order." This decides that. A declared family-first order names the family as a comma does, soJong Anke denow reads asJong, Anke dedoes.The design is the issue's own third reading, mechanism included — family-first as an implied comma, with
FOLDED_TAGdoing the reorder. What this PR adds is the guards.Three conditions, each the comma form read across
Each was a defect found in review before it was a condition. The comma and comma-less forms now agree row for row:
A name word must remain besides the family. Without P6's own words-to-spare test the new site was wider than the rule it narrows:
Jong deattached whereJong, dedeclines, so two writings of one one-given-word listing disagreed about word order — and the no-given-word casedecisions.mddeliberately leaves open was half-decided in passing.The family must have a base of its own. A name word merely ahead of the run is not enough.
van Berg Jan deunderFAMILY_FIRSTlost its given name outright —'Berg'before, empty after, familyde van Berg Jan— because the leftover is partitioned into UNITS and_unitschains a leading particle over every word after it, collapsing[van][Berg][Jan]into one unit that takes FAMILY. 288 of a 45,648-parse battery. P1 masks it for every never-given opening, which is why fixtures that all open withdemissed it.No suffix word stranded at the end.
assignleaves a trailingJr.in a name position when a particle follows it; take the particle away and the leftover is not the same name one word shorter, because the re-layout does not re-run assign's trailing peel.Berg Jan Jr. dereported givenJr., and every battery name where the two orders agreed before and disagreed after was this shape — the rule re-opening the divergence it exists to close. It declines rather than mirroring assign's peel by hand, which is the #424/#425 failure.Never-given vocabulary only
The comma path takes ambiguous particles too; this one cannot. P6's own Accepted clause decides it: the comma-less
FAMILY_FIRST_GIVEN_LASTform is the one format that reads the unaccented Vietnamese spelling correctly, and an ambiguous-inclusive attachment takes that format away — measured,Nguyen Thi Vanwould give familyVan Nguyen, givenThi, against P6's pinned givenVan. The words-to-spare guard does not save it, since a given word does remain.The cost is the Dutch flagship word itself.
vanis ambiguous, soBeethoven Ludwig vankeeps middlevanwhereBeethoven, Ludwig vangives familyvan Beethoven. The two forms agree for the never-given half of the tussenvoegsel set and not for the rest. The rule says so, as an Accepted clause.Reach
12 of 6600 parses — the 1100-name corpus × three
name_ordervalues ×middle_as_familyoff and on — over four names, none under the default order. Two of the four are names this PR's own example lines add to the corpus, which is why the corpus is 1100 and not master's 1094.The gate cannot see any of it.
compare.pyparses every name asHumanName(name)and sweeps no policy at all, so no non-defaultname_orderbehavior has ever been compared across versions. Output is unchanged at all three baselines apart from the added corpus names, none of which diffs. The counts carry a recompute recipe indecisions.md#P6instead, under theAGENTS.mdprovenance rule — the comparator named as the parent commit reading the same corpus files.The issue's headline number does not measure this shape: "464 inputs where the two orders differ" is what permuting given and middle does.
Review
Four agents. Two regressions, one missed instance of the bug being fixed, one design gap — plus a fourth finding neither the suite nor the corpus could reach:
Mesnil Garcia de, Dr.still exhibited #365 verbatim.state.order is not Nonealready excludes every comma that named a family, so testingstructure is not FAMILY_COMMAas well excluded only the comma that fixed nothing. Two reviewers found that independently with different inputs.A guard became dead code. The base and words-to-spare conditions made P1's lead site and P6's new site mutually exclusive — P1's lead fires only on a leading lone never-given particle, which by definition has no base. The
FOLDED_TAGexclusion added to P1's redistribution is now unreachable (0 differences over 61,854 generated parses) and is deleted, with the exclusivity pinned instead.Two comment defects worth naming, both of which would have misled a reader more than the code:
Beethoven Ludwig vanreads like its comma form. False, and it contradicted a machine-checked example line twelve lines below it in the same commit. A survivor of the rejected ambiguous-inclusive draft.van derto it via a new example line.corpus_rules.jsonlregenerates from rules.md, so a rule that adds examples adds corpus names and then counts them. Every count now names its corpus.Also corrected: the order-agreement claim held only at two leftover pieces (
Mesnil Garcia Carlos dedisagrees at three); P6's P5 clause was unqualified where it is now comma-only; the stage header was behind the code; and the#359release bullet described behavior this PR's bullet contradicts.Verification
unexplained: 0.🤖 Generated with Claude Code