Skip to content

feat!: derive leaf parameter names from the leaf's own type (#372) - #378

Closed
seabbs-bot wants to merge 7 commits into
feat/227-single-table-slice-1from
feat/372-type-level-leaf-param-names
Closed

feat!: derive leaf parameter names from the leaf's own type (#372)#378
seabbs-bot wants to merge 7 commits into
feat/227-single-table-slice-1from
feat/372-type-level-leaf-param-names

Conversation

@seabbs-bot

Copy link
Copy Markdown
Collaborator

Any Distributions.jl type works as a leaf with zero registration and zero custom code, and the codec pays nothing for it. Replaces the closed #345 and the rejected value-search branch.

Stacked on #343 (feat/227-single-table-slice-1, which already carries #362). GitHub will retarget to main as those land.

Mechanism

A @generated two-argument helper _derived_param_names(::Type{D}, ::Type{P}), where D = typeof(free_leaf(leaf)) and P = typeof(params(free_leaf(leaf))). Taking arity from the params tuple's type (fieldcount(P)) rather than length(params(d)) is what keeps the whole thing in the type domain: both arguments are Const at every call site, so the generated body is a literal tuple of Symbols.

Only Base type reflection runs at generation time — nothing user-extensible — so this does not repeat the world-age wall that #189 hit by dispatching on extension leaf methods inside a generated body.

@assume_effects :foldable was measured and also passed, but @generated was chosen: folding is a heuristic subject to cost limits that can differ across Julia versions and across an AD package's abstract interpreter, and if it ever failed, the transliteration body (Dict lookup, IOBuffer, string iteration) would run at runtime inside a differentiated function — exactly the Mooncake string-iteration hazard. With @generated there is no runtime body to shield, and the Mooncake extension is untouched.

Measured cost: nothing

Against the six trees of the S2 layout-parity item, each variant in a fresh session:

curated table (base) type-level
@inferred unflatten all six pass all six pass
@inferred flatten all six pass all six pass
allocations 0 B on five trees, 384 B pooled identical
leaf_param_names(Gamma(2.0, 1.0)) 0 B, inferred 0 B, inferred

For contrast, the rejected value-search branch cost 1920 B per unflatten and 784 B per param_names. The 384 B on the pooled tree is pre-existing on the base branch and is asserted as such, not as zero.

AD verified by running, not by argument: logpdf(reconstruct(tree, x), y) differentiates identically under ForwardDiff and Mooncake reverse, with no extension changes.

Assertions strengthened, not removed

The rejected branch passed its gates by deleting the @inferred assertions. This one moves the other way: @inferred 3 → 5 across the suite, @test_throws unchanged at 124 (every removal was a rename), total @test 1586 → 1638, and codec_gen.jl now asserts inference in both codec directions plus an explicit allocation count.

Coverage, measured on the resolved Distributions.jl

65 families derive real names. Exactly four fall back positionally, each for a structural reason the type rule can see: Categorical (params is (p,) but field 1 is support::Base.OneTo), InverseGamma (field 1 is an inner Gamma{T} against a Float64 param), OrderStatistic (4 params over 3 fields), and UnivariateGMM. Kolmogorov has no parameters and derives ().

Seven families have no params method at all (Chernoff, Dirac, the Edgeworth family, KSDist, KSOneSided). These already throw from leaf_param_names on the base branch, so that is not a regression, and the hasmethod reflection the old branch added is unnecessary.

No Unicode dependency: Julia normalises identifiers to NFC in the parser, and every fieldname of every constructible exported univariate family was checked to be NFC already.

The identity guard may only throw, never choose

This is subtle and worth reading. The per-slot guard getfield(d, fs[i]) === params(d)[i] buys zero extra coverage — it agrees with the type rule on every measured family. What it catches is a conforming downstream type whose field order differs from its params order with matching types; a constructed Swapped example derives the wrong labels and the guard catches it.

But the same guard passes on Swapped(1.0, 1.0) — the guard is itself value-dependent. So if it were allowed to switch the answer to positional, names would again depend on values, which is precisely what #372 rejects. Names are computed from the type unconditionally; the guard is a diagnostic that may only throw.

Breaking

Gamma and Weibull (:shape, :scale)(:alpha, :theta), Exponential (:scale,)(:theta,), Uniform (:lower, :upper)(:a, :b). Normal and LogNormal unchanged. The other 61 families move from positional param_1 to real names.

The dotted flat coordinates move with them: onset_admit.shapeonset_admit.alpha, and a pooled stratum's <stratum>.shape.z<stratum>.alpha.z. A chain or posterior stored against a 0.1.x template cannot be read back onto a 0.2.0 one. This is the only break in 0.2.0 that silently invalidates stored artefacts.

The rename was done by hand, per site, in three separate commits (src docstrings, test suite, Literate tutorials) — not by sed, which is how the earlier branch damaged 18 lines including two @testitem names.

Closes #372. Refs #332.

This was opened by a bot. Please ping @seabbs for any questions.

`param_names`/`leaf_param_names` no longer read a curated six-family
table. `param_names(leaf)` now derives names generation-time, via a
`@generated` two-type-argument helper keyed on `typeof(leaf)` and
`typeof(params(leaf))`: the first N fieldnames of the leaf's type,
transliterated (Greek letters to English), where N is params' arity.
Falls back to positional `:param_1, :param_2, ...` when the fields
don't line up 1:1 with params in order (too few fields, a fieldtype
mismatch, or a name collision after transliteration). A wrapper reads
through `inner_dist` with no method of its own.

This gives real names to 57 further Distributions.jl families with
zero registration (steer 1), while an explicit `param_names` override
still wins by dispatch specificity for a leaf whose fields don't align
with its params (steer 2).

A runtime identity guard (`_check_leaf_param_alignment`) catches the
one thing the type-level rule cannot see: a contract-conforming leaf
whose field ORDER differs from its params order with matching field
types. It runs in `Uncertain`'s inner constructor (values already in
hand there) and in `TestUtils`'s conformance check over every real
leaf of a tree, restricted to Real-valued slots, and never from the
gradient path (`param_names`/`leaf_param_names`/codec), so it stays
Mooncake-safe. It is a diagnostic only -- names never depend on values.

No new public or exported name. Documented in the leaf-protocol
developer page with a worked example and the alignment contract.
Update codec_gen.jl's leaf keyword/NamedTuple literals to the derived
names (Gamma/Weibull shape,scale -> alpha,theta). Restore is a
misnomer here -- the @inferred assertions in the "S2 layout parity"
item were already present on this branch -- but add a new allocation
assertion pinned to measured values: 0 B on five trees, 384 B on the
pooled tree (pre-existing on the base branch, asserted against its
measured value rather than 0 so it can't be mistaken for a regression
introduced here). Neither @inferred assertion was weakened or removed.
Gamma/Weibull derive (:alpha, :theta) now, not the old curated
(:shape, :scale); update the affected docstring examples in Pool.jl,
uncertain_macro.jl and varying.jl by hand, leaving every
"location-scale" (statistical term of art) and "loc, scale" (the
population's own hyperparameters, a different pair of names) mention
untouched.
Gamma/Weibull now derive (:alpha, :theta) instead of the curated
(:shape, :scale); rename by hand, file by file, reading every site
against the trap list before touching it:

- local variables that are not parameter names left alone (e.g. a
  covariate `inc_shape`, the classifier vocabulary's own :shape/:scale
  entries)
- @testitem names describing tree/moment/record "shape" (structural)
  left alone; the one @testitem about a Gamma value
  ("at shape == 1.0", ad/scenarios.jl) left alone per its own history
- the DI-style flattened negative fixtures
  (`:onset_admit_shape`/`_shape`) renamed to `_alpha` for consistency,
  a conscious choice since their exact spelling doesn't affect what
  they test
- ADFixtures' one real `uncertain(...; shape = ...)` kwarg site
  renamed; its historical Mooncake xlogy comment left as prose

Every touched file's tests were run and pass after the rename.
composing-distributions.jl, strata-uncertainty.jl and
event-skeletons.jl each construct/inspect a Gamma leaf by its now-real
derived name. Ran the full docs build (SKIP_NOTEBOOKS=true
julia --project=docs docs/make.jl) to confirm the Literate execution
of every edited block still succeeds.
…tted

Project.toml, benchmark/Project.toml, docs/Project.toml,
test/Project.toml and test/ad/Project.toml were applied unstaged (per
PR#359, chore/drop-convolved-sources-pin) to make this worktree's
environment instantiate against ConvolvedDistributions 0.4.0. They
were already staged in the index before this branch's first commit
and got swept into it by mistake. Restore all five to their
origin/feat/227-single-table-slice-1 content; anyone picking this
branch up must reapply PR#359's patch locally and keep it uncommitted.
)

Add the missing `using ComposedDistributions, Distributions` to the
MomentLeaf @example block in the leaf-protocol developer docs -- it is
the first `leaf-protocol`-named example block on the page and used
`ContinuousUnivariateDistribution` unqualified, so the full docs build
(`task docs-fast`) failed with an UndefVarError.

File #377 to track the default_prior misclassification regression for
InverseGaussian.mu, SkewNormal.alpha and NormalInverseGaussian.beta
(a real build_priors correctness bug), and point the NEWS.md entry and
the pinning testitem at it instead of only the parent #372.
@github-actions

Copy link
Copy Markdown
Contributor

Try this Pull Request!

Option 1: Julia Package Manager

Open Julia and type:

import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/EpiAware/ComposedDistributions.jl", rev="feat/372-type-level-leaf-param-names")
using ComposedDistributions

Option 2: Local Checkout

If you have the repo locally:

git checkout feat/372-type-level-leaf-param-names
julia --project=. -e "using Pkg; Pkg.instantiate()"

@seabbs-bot

Copy link
Copy Markdown
Collaborator Author

Superseded by #381, which folds this and seven other PRs into one branch off main so the 0.2.0 contract reviews as a single diff. Its gates: test-fast 2379/2379, test-quality 255/255, full docs build clean.

This branch is preserved and the PR body is kept for its rationale — the detail here is the record of why, which #381 summarises rather than repeats. Closing so it does not compete for review.

This comment was posted by a bot. Please ping @seabbs for any questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant