Skip to content

feat!: remove default_prior, param_priors and build_priors — priors are not this package's job - #373

Closed
seabbs-bot wants to merge 5 commits into
mainfrom
feat/free-uncertain-drop-prior-machinery
Closed

feat!: remove default_prior, param_priors and build_priors — priors are not this package's job#373
seabbs-bot wants to merge 5 commits into
mainfrom
feat/free-uncertain-drop-prior-machinery

Conversation

@seabbs-bot

Copy link
Copy Markdown
Collaborator

Priors leave this package for DistributionsInference. Structure, supports and which parameters are free stay here.

What changes

default_prior, param_priors and build_priors are removed with no replacement. A parameter's name does not determine its family — an InverseGaussian's mu is positive, a SkewNormal's shape is signed — so support-and-name-keyed prior guessing was wrong in exactly the cases where it mattered.

Bare uncertain(tree) still means estimate-everything. It now marks each parameter no_prior() rather than inventing a distribution for it. To attach priors: take composed_to_table(tree), fill the prior column, update(tree, tbl).

rand on an Uncertain leaf still carrying an unresolved no_prior() throws, naming the parameter, instead of silently drawing from a template. logpdf on such a tree throws where it previously returned the template's value. The fitting path is unaffectedreconstruct collapses every spec first.

Resolve.branch_prob_prior accepts no_prior(), and bare uncertain(tree) no longer injects Dirichlet(ones(K)) on your behalf. A user-supplied Dirichlet behaves exactly as before.

Gates

test-fast 1692/1692, test-quality 233/233, clean full docs/make.jl, formatter clean. All three documented rand-refusal behaviours reproduced by direct execution. Project.toml is unchanged from main.

Draft, for two reasons

One test is missing and it is the important one. Uncertain's eager NoPrior check and Base.rand(::AbstractRNG, ::NoPrior) are both new here, both work today, and both would survive their own deletion undetected by all 1925 tests. The guard is the safety property this PR exists to add, so it needs a red test pinning it before merge.

One design question is yours (Q4 in the batch report). The marker lives **in the prior column**, so prior !== nothing still means estimated and all six downstream consumers keep working unchanged — DI's estimated_rows and required_parameters, the flat_dimension == count(!isnothing, prior) invariant, CentredPoolPrior, table round-trip. The alternative is a new estimated::Bool column, which is cleaner on principle — storing a non-prior in a column named prior is the CentredPoolPrior mistake again — but means rebasing a new column through #343's rewritten walk and its 16 golden fixtures.

If you keep the column marker, two conditions: re-document the column as holding a prior or a marker, and make DI raise an explicit error when a NoPrior reaches flat_priors, since today it would MethodError on logpdf.

Known hole

#366: a Resolve whose branch_prob_prior is no_prior() still draws silently from its fixed branch_probs — exactly the class of silence this PR exists to remove. Recommend fixing it here rather than shipping it documented.

Refs #332.

This was opened by a bot. Please ping @seabbs for any questions.

Priors are DistributionsInference.jl's job, not ComposedDistributions'.
Bare uncertain(tree) now marks every free parameter no_prior() instead
of guessing a support-derived default, including a Resolve node's
branch-probability simplex.

Review follow-up on this branch:
- Reworded the no_prior()/rand guarantee to scope it to an Uncertain
  leaf; a Resolve node's branch_prob_prior does not yet carry the same
  refuse-on-draw guard (filed as #366, referenced from NEWS.md and
  no_prior.jl).
- Dropped a duplicate ConvolvedDistributions [sources]/compat change
  that belongs to the separate, already in-flight #359.
- Fixed a docstring detachment bug Aqua's undocumented-names check
  caught: _is_spec_value had been inserted between Uncertain's
  @doc block and the struct it documents, silently stealing the
  docstring. Moved it above the docstring instead.
- Ran task format and confirmed task test-quality passes clean
  (233/233).
@github-actions

Copy link
Copy Markdown
Contributor

Try this Pull Request!

Option 1: Julia Package Manager

Open Julia and type:

import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/EpiAware/ComposedDistributions.jl", rev="feat/free-uncertain-drop-prior-machinery")
using ComposedDistributions

Option 2: Local Checkout

If you have the repo locally:

git checkout feat/free-uncertain-drop-prior-machinery
julia --project=. -e "using Pkg; Pkg.instantiate()"

Both Uncertain's eager unresolved-spec check in Base.rand and
Base.rand(::AbstractRNG, ::NoPrior) are new on this branch and both
worked undetected by all existing tests: neither guard's deletion
would have failed a single test.

Verified by deleting each guard in turn, watching these new tests
fail, then restoring it.
…_prior

A Resolve whose branch_prob_prior is marked no_prior() still draws
silently from its fixed branch_probs on every rand path (standalone,
outcome-pair, and nested in a tree) — exactly the class of silence
the leaf-level Uncertain guard exists to remove.

Red: fails against current Resolve.jl (no guard yet).
A Resolve whose branch_prob_prior was marked no_prior() drew silently
from its fixed branch_probs, ignoring the marked uncertainty -- the
same class of silence Uncertain's leaf-level rand guard already
refuses.

_check_branch_probs_resolved(c::Resolve) mirrors that guard's error
style and message shape exactly, naming the node and pointing at
uncertain(tree; branch_probs = prior, ...) / update(tree, table) /
update(tree, params). It runs in both rand entry points that read
branch_probs directly: _rand_outcome (the standalone named-record and
outcome-pair draws) and _one_of_marginal_rand (the in-tree flat-value
path via child_rand!).

A fixed node (branch_prob_prior === nothing) or one carrying a
resolved Dirichlet is unaffected: the pre-existing gap where rand does
not yet draw fresh branch probabilities from an attached Dirichlet
(noted in the issue) stays out of scope for this fix.
@seabbs-bot

Copy link
Copy Markdown
Collaborator Author

Two gaps from the batch report are now closed on this branch.

#366 is fixed (961f26c). A Resolve whose branch_prob_prior carried an unresolved marker drew silently from its fixed branch_probs. rand now refuses eagerly, naming the node, mirroring the leaf-level guard's error style and message shape. It runs in both entry points that read branch_probs directly — _rand_outcome for the standalone and outcome-pair draws, and _one_of_marginal_rand for the in-tree flat-value path via child_rand!. A fixed node, or one carrying a resolved Dirichlet, is unaffected. The separate pre-existing gap where rand does not draw fresh branch probabilities from an attached Dirichlet is scoped out and stays on #366.

The missing guard test is added (6d64fec). Uncertain's eager unresolved-spec check and Base.rand(::AbstractRNG, ::NoPrior) were both new here and both would have survived their own deletion undetected by every existing test. Verified the hard way: each guard was deleted in turn, the new tests were watched to fail, then it was restored.

Both followed red/green TDD — the #366 reproduction (275e342) is a real red commit that fails against the unguarded Resolve.jl.

Still draft, for one reason only. The open question is whether "estimated" is carried by the NoPrior marker in the prior column, as here, or by a separate estimated::Bool column with NoPrior deleted. The #366 fix touches src/composers/no_prior.jl, so it is coupled to that choice and would need a small rework if the column wins. The guard's behaviour — refuse to draw or score an unresolved estimated parameter, naming it — is correct under either representation.

Gates: test-fast was green before the final commits; test-quality has not been re-run locally since, because the machine is saturated with the other agents in this batch. CI will cover it once #359 lands and unblocks resolution.

This comment was posted by a bot. Please ping @seabbs for any questions.

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.83051% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/composers/introspection.jl 84.37% 5 Missing ⚠️
src/composers/no_prior.jl 83.33% 1 Missing ⚠️
Flag Coverage Δ
ad-enzyme-forward 0.00% <0.00%> (ø)
ad-enzyme-reverse 13.73% <10.34%> (+0.10%) ⬆️
ad-forwarddiff 12.14% <6.89%> (+0.09%) ⬆️
ad-mooncake-forward 0.00% <0.00%> (ø)
ad-mooncake-reverse 12.34% <6.89%> (+0.09%) ⬆️
ad-reversediff 12.14% <6.89%> (+0.09%) ⬆️
unit 87.03% <94.64%> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/composers/Choose.jl 70.45% <ø> (ø)
src/composers/Resolve.jl 81.85% <100.00%> (+2.48%) ⬆️
src/composers/Shared.jl 62.50% <ø> (ø)
src/composers/Uncertain.jl 93.91% <100.00%> (+0.21%) ⬆️
src/composers/codec_gen.jl 99.13% <100.00%> (ø)
src/composers/no_prior.jl 83.33% <83.33%> (ø)
src/composers/introspection.jl 84.18% <84.37%> (+0.34%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown
Contributor

📖 Documentation preview is ready!

View the docs for this PR at: https://EpiAware.github.io/ComposedDistributions.jl/previews/PR373/

This preview will be updated automatically when you push new commits.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark comparison vs base

Minimum time per call. Buckets are PR time as a % of base, so lower is faster (🟢 faster, ⚪ within 5%, 🔴 slower). Counts of benchmarks per bucket:

Group 🟢 <50% 🟢 50–75% 🟢 75–95% ⚪ 95–105% 🔴 105–125% 🔴 125–150% 🔴 >150%
Evaluation · · 2 14 1 · ·
ForwardDiff · · 1 6 2 · ·
ReverseDiff (tape) · · 1 7 1 · ·
Mooncake reverse · · · 7 2 · ·
Enzyme reverse · · 1 8 · · ·
Evaluation — 17 benchmarks (by time change)
Benchmark base PR time memory
Composition / Resolve / rand 561.0 ns 501.0 ns 🟢 0.89× ⚪ 1.0×
Composition / Compete / rand 570.0 ns 510.0 ns 🟢 0.89× ⚪ 1.0×
Composition / Sequential / logpdf 100.0 ns 110.0 ns 🔴 1.1× ⚪ 1.0×
Composition / Parallel / rand 1.19 μs 1.16 μs ⚪ 0.97× ⚪ 1.0×
Composition / Parallel / construct 1.36 μs 1.33 μs ⚪ 0.98× ⚪ 1.0×
Composition / Sequential / rand 1.19 μs 1.17 μs ⚪ 0.98× ⚪ 1.0×
Composition / Nested / rand 4.25 μs 4.29 μs ⚪ 1.01× ⚪ 1.0×
Composition / Sequential / construct 1.37 μs 1.38 μs ⚪ 1.01× ⚪ 1.0×
Composition / Nested / logpdf 1.58 μs 1.59 μs ⚪ 1.01× ⚪ 1.0×
Composition / Nested / compose 2.63 μs 2.62 μs ⚪ 1.0× ⚪ 1.0×
Composition / Choose / construct 30.0 ns 30.0 ns ⚪ 1.0×
Composition / Choose / logpdf 60.0 ns 60.0 ns ⚪ 1.0×
Composition / Compete / construct 30.0 ns 30.0 ns ⚪ 1.0×
Composition / Compete / logccdf 350.0 ns 350.0 ns ⚪ 1.0×
Composition / Parallel / logpdf 110.0 ns 110.0 ns ⚪ 1.0× ⚪ 1.0×
Composition / Resolve / construct 30.0 ns 30.0 ns ⚪ 1.0×
Composition / Resolve / logpdf 140.0 ns 140.0 ns ⚪ 1.0×
AD gradients — 36 benchmarks (by time change)
Benchmark base PR time memory
AD gradients / Truncated uncertain-leaf unflatten/update codec / Enzyme reverse 20.62 μs 17.06 μs 🟢 0.83× ⚪ 1.0×
AD gradients / Censored leaf marginal logpdf / ForwardDiff 10.48 μs 11.59 μs 🔴 1.11× ⚪ 1.0×
AD gradients / Truncated uncertain-leaf unflatten/update codec / ForwardDiff 4.22 μs 3.81 μs 🟢 0.9× ⚪ 1.0×
AD gradients / Sequential Gamma+LogNormal logpdf / ReverseDiff (tape) 28.84 μs 26.93 μs 🟢 0.93× ⚪ 1.0×
AD gradients / Censored leaf marginal logpdf / ReverseDiff (tape) 27.28 μs 29.0 μs 🔴 1.06× ⚪ 1.0×
AD gradients / Pool non-centred reconstruction logpdf / Mooncake reverse 21.05 μs 22.29 μs 🔴 1.06× ⚪ 1.0×
AD gradients / Resolve mixture marginal logpdf / ForwardDiff 6.74 μs 7.11 μs 🔴 1.05× ⚪ 1.0×
AD gradients / Choose selected-branch logpdf / Mooncake reverse 16.52 μs 17.43 μs 🔴 1.05× ⚪ 1.0×
AD gradients / Truncated uncertain-leaf unflatten/update codec / Mooncake reverse 158.12 μs 151.3 μs ⚪ 0.96× ⚪ 1.0×
AD gradients / Shared-tag unflatten/update codec / Mooncake reverse 89.39 μs 93.0 μs ⚪ 1.04× ⚪ 1.0×
AD gradients / Choose selected-branch logpdf / ReverseDiff (tape) 13.75 μs 13.27 μs ⚪ 0.97× ⚪ 1.0×
AD gradients / Shared-tag unflatten/update codec / ForwardDiff 1.81 μs 1.75 μs ⚪ 0.97× ⚪ 1.0×
AD gradients / Compete racing-hazard marginal logpdf / ForwardDiff 8.17 μs 7.91 μs ⚪ 0.97× ⚪ 1.0×
AD gradients / Pool non-centred reconstruction logpdf / ForwardDiff 981.0 ns 1.01 μs ⚪ 1.03× ⚪ 1.0×
AD gradients / Resolve stick-breaking branch-prob logpdf / ReverseDiff (tape) 55.11 μs 53.43 μs ⚪ 0.97× ⚪ 1.0×
AD gradients / Resolve stick-breaking branch-prob logpdf / ForwardDiff 6.7 μs 6.88 μs ⚪ 1.03× ⚪ 1.0×
AD gradients / Sequential Gamma+LogNormal logpdf / Mooncake reverse 294.99 μs 302.54 μs ⚪ 1.03× ⚪ 1.0×
AD gradients / Truncated uncertain-leaf unflatten/update codec / ReverseDiff (tape) 12.7 μs 13.01 μs ⚪ 1.02× ⚪ 1.0×
AD gradients / Resolve stick-breaking branch-prob logpdf / Enzyme reverse 21.21 μs 20.73 μs ⚪ 0.98× ⚪ 1.0×
AD gradients / Shared-tag unflatten/update codec / ReverseDiff (tape) 11.75 μs 12.01 μs ⚪ 1.02× ⚪ 1.0×
AD gradients / Pool non-centred reconstruction logpdf / ReverseDiff (tape) 25.79 μs 26.35 μs ⚪ 1.02× ⚪ 1.0×
AD gradients / Shared-tag unflatten/update codec / Enzyme reverse 7.66 μs 7.8 μs ⚪ 1.02× ⚪ 1.0×
AD gradients / Resolve mixture marginal logpdf / ReverseDiff (tape) 63.36 μs 62.24 μs ⚪ 0.98× ⚪ 1.0×
AD gradients / Choose selected-branch logpdf / ForwardDiff 691.0 ns 701.0 ns ⚪ 1.01× ⚪ 1.0×
AD gradients / Sequential Gamma+LogNormal logpdf / Enzyme reverse 103.08 μs 101.72 μs ⚪ 0.99× ⚪ 1.0×
AD gradients / Resolve mixture marginal logpdf / Enzyme reverse 21.24 μs 21.01 μs ⚪ 0.99× ⚪ 1.0×
AD gradients / Choose selected-branch logpdf / Enzyme reverse 3.11 μs 3.14 μs ⚪ 1.01× ⚪ 1.0×
AD gradients / Compete racing-hazard marginal logpdf / Mooncake reverse 51.72 μs 52.14 μs ⚪ 1.01× ⚪ 1.0×
AD gradients / Sequential Gamma+LogNormal logpdf / ForwardDiff 10.46 μs 10.38 μs ⚪ 0.99× ⚪ 1.0×
AD gradients / Compete racing-hazard marginal logpdf / Enzyme reverse 10.38 μs 10.32 μs ⚪ 0.99× ⚪ 1.0×
AD gradients / Pool non-centred reconstruction logpdf / Enzyme reverse 1.91 μs 1.9 μs ⚪ 0.99× ⚪ 1.0×
AD gradients / Resolve stick-breaking branch-prob logpdf / Mooncake reverse 87.72 μs 88.15 μs ⚪ 1.0× ⚪ 1.0×
AD gradients / Censored leaf marginal logpdf / Mooncake reverse 328.54 μs 327.03 μs ⚪ 1.0× ⚪ 1.0×
AD gradients / Compete racing-hazard marginal logpdf / ReverseDiff (tape) 47.06 μs 46.91 μs ⚪ 1.0× ⚪ 1.0×
AD gradients / Resolve mixture marginal logpdf / Mooncake reverse 86.46 μs 86.21 μs ⚪ 1.0× ⚪ 1.0×
AD gradients / Censored leaf marginal logpdf / Enzyme reverse 122.44 μs 122.18 μs ⚪ 1.0× ⚪ 1.0×

seabbs-bot added a commit that referenced this pull request Aug 14, 2026
…n/0.2.0-contract (#381)

Folds the prior-removal work into the leaf/node contract branch so the
breaking 0.2.0 surface lands as one PR. Kept #373's NoPrior-in-prior-column
design (confirmed) layered on #381's composed_to_table/full-inventory
rewrite; removed build_priors/default_prior/param_priors and their
dedicated test coverage throughout.
@seabbs-bot

Copy link
Copy Markdown
Collaborator Author

Folded into #381 (2026-08-14), which now carries this branch's prior-removal work merged on top of the leaf/node contract rewrite. The open estimated::Bool question is resolved there — no new column, no_prior() stays a spec value on the prior column — and the merge caught two stacking-only bugs (the Resolve guard method built against the old codec calling convention, and a naive params_tablecomposed_to_table rename in _estimate_all that would have mis-marked non-parameter rows) plus a handful of stale shape/scale references from before #378's Gamma rename. test-fast, test-quality and a full docs build all pass clean on the merged tree, including this PR's guard tests and the #366 fix.

Closing in favour of #381; this branch stays for the detailed rationale.

This was closed by a bot. Please ping @seabbs for any questions.

@seabbs-bot seabbs-bot closed this Aug 14, 2026
seabbs-bot added a commit that referenced this pull request Aug 14, 2026
These sat outside every conflict marker (git's 3-way merge never flagged
them), so the earlier merge-conflict sweep didn't catch them. Caught by
running task test-fast against the actual committed tree instead of the
locally-fixed-but-uncommitted working copy CI would not see.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants