Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Three committed contributor docs carry the parser's normative rules and their re

**Counting claims.** A bare count in prose is either an assertion or a liability, keyed by who observes its staleness: asserted counts (a test holds the number) fail CI at change time — the useful kind; dated snapshots ("51 sites at spec time") cannot go stale; standing present-tense prose counts are the forbidden class — promote to an assertion, add a date, or state the invariant and let a test count. After changing how many times something runs, sweep for counts, not for the thing's name.

**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary or be verified against rules.md examples, never written from memory. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline; free prose is for ledger mechanics only (owned by tools/differential/README.md).
**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The gate compares the seven role fields and `_ambiguities`, so `capitalized()`, `initials()` and any future render view are invisible to it (decisions.md#R4, #R3) and the first two sources cannot reach a claim about one: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline; free prose is for ledger mechanics only (owned by tools/differential/README.md).

**Working on docs/design/ has its own AGENTS.md.** `docs/design/AGENTS.md` carries the landing-a-design distillation checklist, the primary-source review rule, the dated-count convention, and the ten review axes. Claude Code loads it automatically when a session reads or edits anything under docs/design/; if your tool does not do nested discovery, read it yourself before touching those files or reviewing a change to them.

Expand Down
12 changes: 12 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -722,6 +722,18 @@ Declined:

- 2026-08-18 — the GROUPING half of #404: a particle run that joins nothing does not chain, so "Jong van der" would split into middle 'van' plus family 'der'. Measured and rejected, though NOT for the reason first recorded here. The first draft said the split makes the family "no longer all-particle so the base fix stops firing" — false, and `der` and `la` are both shipped particles, so a family of either IS all-particle and the rule fires on it ("Juan Smith der" gives base 'der'). What the split actually costs is the SCOPE of the base and a stray particle relocated: grouping can decline to merge but cannot keep the words apart, because roles re-assemble them and two adjacent same-role pieces are one part at the field level. "Juan Smith van der" becomes middle 'Smith van', family 'der' — a base of 'der' rather than 'van der', and a middle name nobody wrote. Keeping the run whole in one part is what gives the base its full extent. The split reading needs the leftover distribution to know these are separate units, which is mechanisms.md#UNIT-PARTITION's problem.

### R3 — initials honor the fold

R3's earlier history is under `decisions.md#R2`, which this entry does not repeat: the #461 backout and the unsettled given-group conjunction question are recorded there because they were decided alongside R2's all-particle clause. This entry is R3's own, and R3's `history:` points here.

- 2026-08-30 #408 — DONE: `initials()` orders folded words first, as every other view of the same parse already did. O3's fold and P6's attachment both TAG rather than move a token (mechanisms.md#FOLDED_TAG — spans cannot reorder), and `_types._text_for` partitions on that tag for every role it renders, so the `family` field reads `van der` where the tokens sit `der ... van`. `initials()` walked `tokens_for(role)` in written order and never asked, so the two views disagreed about one parse: `parse("der, y van")` gave family `van der` and initials `y. d. v.`, and it gives `y. v. d.` now. This is mechanisms.md#RENDER-HONORS-THE-PARSE's second shape, filed as its worked instance, so the design is not re-argued here — only what the measurement added to it.
WHAT THE MEASUREMENT ADDED, and it reframes the issue from a consistency fix to a parity restoration. v1 had this feature as `CONSTANTS.middle_name_as_last` (`_config_shim` maps it to `Policy.middle_as_family`) and had NO general `name_order`, so only the two default-order policies have a v1 reference at all. Measured on the released 1.4.0 wheel over the 1094-name deduped corpus, from a script run outside the worktree that asserts `nameparser.__version__` first: at the default policy 1 name moves and agrees with 1.4.0 neither before nor after; under `middle_as_family` 71 move, of which **54 come into exact 1.4.0 agreement and NOT ONE moves away from it**. Worked pair: `Doe, Dr. John A.` gives `J. A. D.` at 1.4.0, gave `J. D. A.` here, and gives `J. A. D.` again; `Brundridge, Contessa A` likewise `C. A. B.` / `C. B. A.` / `C. A. B.`. The facade had been giving the folded answer all along — `HumanName.initials()` reads `first_list`/`middle_list`/`last_list`, and `_facade._list_for` prepends the carriers exactly as `_text_for` does — so the core view was simultaneously out of step with the field beside it, with the facade above it, and with 1.4.0 behind it. A view that stopped honoring a record had been reproducing a v1 bug v1 did not have.
THE 17 THAT AGREE WITH 1.4.0 NEITHER WAY, recorded because the obvious explanation is wrong — and the FIRST REPLACEMENT for it was wrong the same way, corrected by this branch's docs review before merge. It is not that their parse already differs: only 2 of the 17 parse differently from 1.4.0 under this policy, `Smith, John V.` (#432's dotted numeral) and `남궁민수 지훈` (the unspaced-hangul surname split and the script order, rules.md#W1). The other 15 have SEVEN BYTE-IDENTICAL FIELDS at 1.4.0 and here — `der, y van` INCLUDED, which an earlier draft of this bullet filed with the two above by carrying over a measurement taken at the OTHER policy, where its parse does differ. Of those 15, 14 differ from 1.4.0 only in v1's initials GRANULARITY — v1 initials one group per `*_list` element, so a run its parse joined into one element (`Velasquez y Garcia`, `abdul salam`) yields `V G.` where 2.x, one token per word, yields `V. G.` — and strip the periods and spaces and all 14 come into 1.4.0's letter ORDER after this change, none having been in it before (`JVGQ` → `JQVG`). The 15th is `der, y van`, and it is worth naming rather than lumping: its fields match 1.4.0 exactly and its initials do not, because 1.4.0 contributes NOTHING for a family that is all particles (`van der` → `y.`) where 2.x contributes its words. That is R3's own adjudicated clause and decisions.md#R2's territory — R3 carries `"Juan van der" → initials="J. v. d."` against 1.4.0's `J.` — so this one name is a divergence this document has already decided in favor of, not an unexplained residue. So the parity claim is stronger than "54 of 71 and the rest unexplained": 68 of the 71 come into 1.4.0's ordering, 54 of them exactly, one more is R2's decided divergence, and 2 have a parse difference that predates this change.
BLAST RADIUS by policy, over the same corpus × three `name_order` values × `middle_as_family` off/on (6564 parses, 660 move). Default order: 1 / 71. FAMILY_FIRST: 1 / 294. FAMILY_FIRST_GIVEN_LAST: 1 / 292. STATE THE POPULATION BESIDE THE COUNT, per this file's own rule: the shape an order change can be observed in is a contributing folded token standing AFTER a contributing unfolded one of the same role, and at the default order the corpus holds exactly 1 such name without `middle_as_family` and 72 with it. So the movers are 1 of 1 and 71 of 72 — a near-total hit rate over a corpus that carries one instance of P6's half, not a blast radius near zero. (The one non-mover is `Chang, Andy C I`, where the folded `C` and the family `Chang` share a letter and the reorder is invisible in initials.) Nine corpus names carry the tag at all at the default order; the other eight have a single contributing family token, which makes the partition the identity. The `middle_as_family` half is where the reach is, which follows from the producers: O3 folds every middle word, while P6's attachment moves initials only where the attached run is READMITTED by the unjoined mark — a family that is all particles — so it reaches one corpus name in each order and the same one. RECOMPUTE, and the COMPARATOR is the half that has to be stated, because the obvious phrasing of it measures nothing: parse the four `tools/differential/corpus*.jsonl` files deduped under each of the six policies and compare `initials()` against THE PRE-CHANGE RENDERING — the same contributing tokens left in WRITTEN order. What this sentence said until 2026-08-30 was "against a folded-first partition of `tokens_for(role)`", which is what `initials()` now IS: that recipe compares the shipped implementation with itself and yields 0 differing parses, the inert-measurement shape (mechanisms.md's Verification shapes section) written into the very sentence meant to prevent it, and the only recompute path the repository offered for docs/release_log.rst's counts as well. CONTRIBUTING is the other half of the recipe and it comes FIRST: apply this view's own skip filter — outside the given group a token tagged `particle` or `conjunction` contributes nothing unless it also carries UNJOINED_TAG — and compare the two orders only over what survives it. Leave the filter out and every skipped word reads as a difference instead: 1546 rather than 660 against written order, and 1018 rather than 0 against the folded partition. Re-measured this way on 2026-08-30 and reproducing the counts above: 660 of 6564 parses, 1 / 71 at the default order, 1 / 294 and 1 / 292 under the two family-first orders.
THE VERIFICATION LIMIT, and it is decisions.md#R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules.md#R3`'s example line and `tests/v2/test_render.py`, both mutation-checked.
WHY R3 GAINED A SENTENCE rather than leaning on R1. R1 already says folded family words "render before the rest of the family wherever they stood in the string" — but its subject is "Every FIELD", and `initials()` is not a field: R3 calls it "this view", and it takes a format spec and two separators rather than being read as an attribute. So R1 does not reach it by its own words, and #408 is precisely what leaving that to inference costs. R3's new clause states the order in R3's own voice, carries the O3 example line (`"Hassan, Mohamad Ahmad Ali"` under `middle_as_family` → `initials="M. A. A. H."`, reusing R1's own input so `corpus_rules.jsonl` is unchanged at 241 names), and R1 and R3 now name each other in `interacts:` — the field says which order, the view says it follows the field. P6's half gets no example line, for the same reason R1 carries none: the only input that witnesses it is `der, y van`, whose PARSE is itself contested, so pinning its initials normatively would pin a value another open question can move. It is pinned in the unit test instead, which is where today's-behavior pins belong.
THE PARTITION IS PER ROLE, not scoped to FAMILY, and this is a deliberate choice rather than a copied line. Both producers re-role to FAMILY, so FAMILY is the only role a parse can put the tag on today and the GIVEN and MIDDLE arms are unreachable. They are there because `_text_for` partitions for every role it renders and the two views must not diverge again — the same uniformity `_post_rules`' own UNJOINED_TAG loop takes for its three roles, "for uniformity with the rule, not because all three are observable". A producer that ever folded into another part would otherwise reopen #408 there with nothing to fail. `test_initials_folds_in_every_role_it_renders` pins it from a hand-built name, since no input string can. Mutation-checked four ways — scoping the partition to FAMILY, and skipping it for GIVEN, for MIDDLE, for FAMILY — and each fails that test. It took THREE drafts of the fixture to make that true, and the two misses are the same miss one role apart. The first carried two GIVEN tokens and ONE MIDDLE: a one-element partition is the identity, so the MIDDLE arm was asserted and unpinned, and skipping the partition for MIDDLE passed the entire suite. The second fixed MIDDLE and carried NO FAMILY token at all, in a test named for every role it renders — so skipping the partition for FAMILY passed this test, and was caught only by its siblings and by R3's example line, neither of which is about per-role application. A zero-element group is the identity too, and reads even less like a gap than a one-element one, which is why the second miss survived a review that had just named the first. Two tokens per role is what closes it, and the general form is carried as mechanisms.md#TWO-ELEMENT-GROUPS rather than left here: a test written to pin a partition, a sort or a dedup needs at least two elements in every group it claims to cover, or the claim rides on a no-op.

### R4 — case repair reads the unjoined mark

- 2026-08-29 #407 — DECIDED: case repair consults the mark #404 introduced, and consults it for the WHOLE PART rather than for a particle standing alone. Until now three views read it and the fourth did not, which is how `parse("ANH DO")` came to report family_base 'DO' and initials 'A. D.' while `capitalized()` returned 'Anh do' — the same word treated as an ordinary name word by three readers and as a particle by the fourth. The scope question is the only real choice here, and standing-alone loses on the first Vietnamese name that tests it: `anh van do` has family `van do`, two particle-vocabulary words and neither of them alone, so a rule keyed on aloneness would capitalize `Anh Do` and leave `Anh van do`, making the same surname read differently for having a given-name word in front of it. rules.md#R2 states the test as "Every word", not "standing alone", and this is the case that shows why it is stated that way. One predicate, four views.
Expand Down
Loading