feat: full node/attribute/param table with Tables.jl-forwarding tree - #343
feat: full node/attribute/param table with Tables.jl-forwarding tree#343seabbs-bot wants to merge 13 commits into
Conversation
Codecov Report❌ Patch coverage is
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Try this Pull Request!Option 1: Julia Package ManagerOpen Julia and type: import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/EpiAware/ComposedDistributions.jl", rev="feat/227-single-table-slice-1")
using ComposedDistributionsOption 2: Local CheckoutIf you have the repo locally: git checkout feat/227-single-table-slice-1
julia --project=. -e "using Pkg; Pkg.instantiate()" |
|
📖 Documentation preview is ready! View the docs for this PR at: https://EpiAware.github.io/ComposedDistributions.jl/previews/PR343/ This preview will be updated automatically when you push new commits. |
Benchmark comparison vs baseMinimum time per call. Buckets are PR time as a % of base, so lower is faster (🟢 faster, ⚪ within 5%, 🔴 slower). Counts of benchmarks per bucket:
Evaluation — 17 benchmarks (by time change)
AD gradients — 36 benchmarks (by time change)
|
Introduce composed_to_table(d), the full node/attribute/param inventory of a composed tree, alongside the existing params_table(d). Both are produced by one pre-order walk threading a sink object (_ParamSink or _FullSink) instead of five positional column vectors, so params_table stays the exact role == :param projection with no extra traversal or allocation on its existing AD-adjacent call sites (e.g. centred_pool_rows). Every composer node and leaf (wrapper) layer gets a :node row and any node_attributes :attribute rows; a leaf's wrapper stack (Truncated, Censored, Shared, Varying, Uncertain) is listed via the new leaf_layers hook, one row per layer at the leaf's real path, emitted before the shared-tag dedup so every shared occurrence is visible structurally even though its :param rows are still inventoried once under the tag. A Resolve's own branch_probs rows stay emitted after its children's rows, matching the codec's type-level walk order. node_kind, node_children (renamed from _node_children, aliased for source compatibility), node_attributes and leaf_layers are the new public node-emission hooks a downstream node/leaf-wrapper type overrides. Golden projection-parity, row-invariant and structural-recovery tests cover Sequential/Parallel/Resolve/Compete/Choose, shared and pooled leaves, Truncated/Censored/Varying/Uncertain wrappers, a Convolved composite, and a third-party leaf wrapper that only extends free_leaf/rewrap_leaf (an opaque node row, by design).
A composed distribution is now a Tables.jl source in its own right: Tables.columns/columnnames/getcolumn/schema/rows all forward to its full composed_to_table, so DataFrame(tree) yields the full table (DataFrame(params_table(tree)) stays the parameter-only one). No Base.getproperty override, so field access on every node keeps working. Because a composed distribution is now Tables.istable, update(a, b) with b a tree would otherwise silently reach update's table arm and bulk-write b's rows into a. Add an explicit update(::AbstractComposedDistribution, ::AbstractComposedDistribution) guard that throws, naming composed_to_table/params_table as the explicit way to copy another tree's rows. update(d, table) now filters a role-carrying table (composed_to_table or a DataFrame of one) to its role == :param rows first, so passing a tree or its full table straight to update only ever writes parameters; a table with no role column (the existing params_table shape) is unaffected.
Mention composed_to_table alongside params_table in the introspection contract section of the developer interface-contracts page, and add an Unreleased NEWS.md entry summarising the new full table, the tree's Tables.jl forwarding, and role-aware update. Every new public docstring (composed_to_table, node_kind, node_children, node_attributes, leaf_layers) already carries # Arguments and a runnable @example, verified against a full docs build.
`_walk_rows!`'s leaf branch built and iterated `leaf_layers(leaf)` unconditionally, so `params_table`'s AD-hot path paid for the full layer walk even though `_ParamSink`'s node/attribute pushes are no-ops. Add `_emit_layers!`, dispatched on the sink: a no-op for `_ParamSink`, the layer loop for `_FullSink`, so the params-only path never touches `leaf_layers` at all.
…e table surface BREAKING CHANGE: params_table(d) is removed. composed_to_table(d) is now the only table-reading verb: it returns the full node/attribute/parameter inventory, and the parameter-only view is a filter over it (`filter(row -> row.role == :param, Tables.rows(composed_to_table(d)))`). This folds into the unregistered 0.2.0 breaking window rather than a deprecation cycle. - build_priors and update now both accept a composed_to_table-shaped table directly, filtering to :param rows internally the same way, so build_priors(composed_to_table(tree)) and update(tree, composed_to_table(tree)) work without hand-filtering. - centred_pool_rows (Pool.jl) and required_parameters (varying.jl) migrate to the _ParamSink walk directly, the same zero-extra-work path params_table used internally, so neither regresses to building the full table on their AD-adjacent call sites. - ParamsTable's show header drops the now-dead params_table/composed_to_table branch. - Tests recast the golden parity fixture as an explicit role == :param filter over composed_to_table, verified against an internal _param_rows helper that still runs the parameter-only walk independently (proving the filter reproduces the historical params_table output). Other tests move to composed_to_table directly where row count is not significant, or to _param_rows where a test's assertions depend on the historical five-column shape or exact row count. - Docs and tutorials teach composed_to_table plus the role filter as the parameter-view idiom.
guards and tests Reviewer follow-ups on the #227 params_table removal: - extend the leaf hot-path testitem to exercise required_parameters and centred_pool_rows directly, not just _param_rows - rename the composed_to_table wrapper type from ParamsTable to ComposedTable (breaking, same unregistered 0.2.0 window) - add the missing # Arguments section to composed_to_table's docstring - pin and document the empty-path/root-row convention that required_parameters and centred_pool_rows fall back to for a bare leaf - give build_priors the same up-front Tables.istable/edge/param guard update already has, so a DI-shaped table is refused loudly - dedupe the restated row-structure sentence in NEWS.md and fix an unrewrapped line
5f9a8db to
838654d
Compare
Bumps the github-actions group with 9 updates: | Package | From | To | | --- | --- | --- | | [EpiAware/.github/.github/workflows/tagbot.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/ad.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/cancel-on-close.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/coverage.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/docs-preview-cleanup.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/documentation.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/format-check.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/release-nudge.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | | [EpiAware/.github/.github/workflows/tests.yml](https://github.com/epiaware/.github) | `11263ddd07f3d88c54a6245828a81d10a83fcb29` | `d024dc6927cc3523292a159dc0d4a45193c9c65d` | Updates `EpiAware/.github/.github/workflows/tagbot.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/ad.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/cancel-on-close.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/coverage.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/docs-preview-cleanup.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/documentation.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/format-check.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/release-nudge.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) Updates `EpiAware/.github/.github/workflows/tests.yml` from 11263ddd07f3d88c54a6245828a81d10a83fcb29 to d024dc6927cc3523292a159dc0d4a45193c9c65d - [Release notes](https://github.com/epiaware/.github/releases) - [Commits](EpiAware/.github@11263dd...d024dc6) --- updated-dependencies: - dependency-name: EpiAware/.github/.github/workflows/tagbot.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/ad.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/cancel-on-close.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/coverage.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/docs-preview-cleanup.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/documentation.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/format-check.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/release-nudge.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions - dependency-name: EpiAware/.github/.github/workflows/tests.yml dependency-version: d024dc6927cc3523292a159dc0d4a45193c9c65d dependency-type: direct:production dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
…b-actions-6efa85718e ci(deps): bump the github-actions group with 9 updates
The reworked node-emission surface derives a leaf's wrapper layers by folding `inner_dist` rather than shipping its own `leaf_layers` hook, so this branch stacks on #362.
The single-table walk shipped four public hooks. Three of them asked a downstream type for something the package can work out for itself, so only one survives. - `node_kind` is now the internal `_node_kind`. It reads the type name off the value, which is correct for every type that can appear in a tree, so there is nothing for a downstream type to supply. - `leaf_layers` is gone. A leaf's wrapper layers are the fold of the `inner_dist` peel a wrapper already defines to take part in the leaf protocol, so the internal `_leaf_layers` folds it instead of asking for a second, redundant method. `Truncated`, `Shared`, `Varying`, `Uncertain` and `Censored` lose their per-type layer methods; `Censored` gains the `inner_dist` method it was missing, which also retires its four read-through forwards. - `node_children` is left as the internal `_node_children` it was on main. Promoting it here would collide with the independent promotion in the node-surface work, which documents it as the accessor a downstream node defines for `has_varying`/`has_uncertain`. This walk consumes it under its existing internal name instead, so exactly one definition ships. `node_attributes` earns its place: a node's fixed, non-parameter structure (a `Choose`'s selector, a truncation bound, a shared tag) cannot be derived from the type or from the peel. A downstream leaf wrapper now defines two methods for the table (`inner_dist`, `node_attributes`) instead of four (`node_kind`, `leaf_layers`, `node_attributes` plus the peel), and a downstream composer node defines one (`node_attributes`) instead of three.
Two repairs the reworked branch needs to run its own gates green. - `_pool_hyper_rows!` pushed a parameter name whose type the compiler cannot pin down for a population template of non-concrete type, so the typed row push did not resolve statically. A leaf's parameter names are `Symbol`s by contract; state that at the loop, which the row push already required. - Three codec tests still called `params_table`, removed earlier on this branch, and errored on load. They assert the parameter-row projection's order, so they now run it through `_param_rows`.
`node_attributes` is public but not exported, so an `@ref` from a page whose module context is `Main` cannot resolve it. The neighbouring leaf-wrapper section already names such hooks in plain backticks.
|
Reworked per @seabbs's steer: adding a node may require methods, but they must be as few and as clean as possible, public but not exported, with a stated interface and developer docs. Retargeted onto #362. Dropping The hook set, cut from four to one
The numbers
Two pre-existing defects fixed, both worth knowing aboutThis branch's
Gates
Expected conflictThe two developer pages updated here are the ones #365 deletes into This comment was posted by a bot. Please ping @seabbs for any questions. |
|
Superseded by #381, which folds this and seven other PRs into one branch off This branch is preserved and the PR body is kept for its rationale — the detail here is the record of why, which #381 summarises rather than repeats. Closing so it does not compete for review. This comment was posted by a bot. Please ping @seabbs for any questions. |
Slice 1 of #227: a single pre-order walk now feeds two projections.
Golden order-sensitive projection-parity tests over 16 fixtures, row invariants, structural-recovery tests (Sequential vs Parallel now distinguishable, truncation bounds visible, shared occurrences get node rows), Tables-interface and role-aware-update tests. Quality 244/244, full non-AD suite 2060/2060, full docs build clean. In-memory round trip is the documented target (Varying maps and composite solvers are live objects).
This was opened by a bot. Please ping @seabbs for any questions.
Closes #227.