Skip to content

assignments_to_experiments: fix lost switches, dropped changes and late evaluation of condition changes - #574

Merged
FFroehlich merged 8 commits into
mainfrom
claude/assignments-to-experiments-fixes
Oct 26, 2026
Merged

FFroehlich merged 8 commits into
mainfrom
claude/assignments-to-experiments-fixes

Conversation

@FFroehlich

@FFroehlich FFroehlich commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Follows #573. Fixes in assignments_to_experiments, found while reviewing #573's refactor and by differential testing against simulation. Each one changes the converted problem, and each has a regression test that fails without the fix. None was introduced by the refactor.

Fixes

  • A switch exactly at the start of a later period was lost when that period did not change any symbol of the assignment. With k1 := piecewise(0, time < 10, 1) and periods at 0 and 10, k1 stayed 0 after t = 10. Likewise time <= 0 after preequilibration. A later period now also sets the value when it differs from the value at the end of the previous period.
  • Condition changes of symbols still used elsewhere were dropped. With k1 := dose * piecewise(0, time < 10, 1) and a reaction that also uses dose, the condition's dose = 2 was removed, so the reaction silently used the model value. Changes are now only dropped if nothing else (the model's math or conversion factors, an observable, a condition change) refers to the symbol.
  • A condition change was re-evaluated wherever a switch used it, with the values at the switch. PEtab applies a condition once, at the start of its period, and all its changes simultaneously, with the values from before the period. With d1 = b1 + 0.5 at t = 2 and b1 = 2 at t = 3 (or b1 = 2 in the same condition), a switch to d1 at t = 5 used 2.5 instead of 1.7. Changes are now evaluated at the start of their period; one whose value refers to something not known there, e.g. a species, is rejected. As a side effect, switching times set to expressions such as t1 = t2 + 1 now convert instead of being rejected.
  • Time-dependent initial assignments were converted into switches, but SBML evaluates an initial assignment once, at the start of the simulation, so k1 = piecewise(0, time < 10, 1) never switches. They now stay in the model, as the docstring's scope ("assignment rules") already said.
  • At preequilibration, a value referring to model parameters outside the parameter table failed the lint (CheckInitialChangeSymbols), since preequilibration starts the experiment. Those parameters now take their model values, as at the first period.
  • An assignment to a species or compartment crashed with an AttributeError; it now raises NotImplementedError.
  • A problem without time-dependent assignments was still changed: an experiment that only preequilibrates got an empty period at t = 0, and periods were re-sorted. It is now returned as it was, after the lint.
  • A ramp's slope parameter could clash with PEtab IDs, e.g. an observable parameter k1_slope, since it was only checked against SBML IDs. It is now named with Problem.get_unused_id.
  • A switching time that a condition set from a parameter fixed by the parameter table was rejected, e.g. t1 = p + 1, although t1 := p + 1 in the assignment itself converts with p's nominal value. Fixed parameters now take their nominal values in condition changes too; switching times that depend on estimated parameters are still rejected.
  • New condition values used the symbols parsed from SBML, without the assumptions of PEtab's symbols (petab_symbol, real), so they compared unequal to the same expressions parsed by PEtab, e.g. after reading the condition table back. They now use PEtab's symbols.

Test changes: test_assignments_to_experiments and _unused_condition_id used an initial assignment (k1 = ...) to mean a switching rule and now use an assignment rule (k1 := ...). _validate relied on an estimated parameter with a time-dependent initial assignment; it now uses an existing experiment default_experiment, to which the measurements without an experiment ID would also be assigned, which the final lint catches.

Module

As suggested in review, assignments_to_experiments and its helpers now live in their own module, petab.v2.assignments_to_experiments (moved unchanged; still exported from petab.v2.converters), and their tests in tests/v2/test_assignments_to_experiments.py. SbmlModel.get_referenced_ids() and ObservableTable.free_symbols are new, also from review.

Checked

  • pytest tests/v2 tests/v1/test_petab.py tests/v1/test_sbml.py: 266 passed; pre-commit clean.
  • The benchmark problems Brannmark_JBC2010, Fujita_SciSignal2010, Isensee_JCB2018 and Raimundez_PCB2020 (and Oliveira_NatCommun2021's error) convert identically before and after.
  • Differential testing against simulation: random problems with dose sums, several and nested pieces, ramps, every inequality form, switching times set by conditions (often exactly at period starts), symbols shared with reactions, two converted assignments in one model, and condition values that are expressions in other parameters. Each experiment of the original and the converted problem is turned into SBML events with ExperimentsToSbmlConverter, simulated in BasiCO, and compared (rtol 1e-6). On this PR's head, 2093 problems match and none is rejected; in 7 more, COPASI fails on the original model, and the converted one matches it up to there. The same generator finds 59 mismatches in 300 problems on main. Not covered: preequilibration, experiments that start at t ≠ 0, and first periods with expression values (only parameter table symbols are allowed there).
  • A second pass with an independent reference simulator (SciPy, integrating piecewise between all switching times and period starts; it agrees with BasiCO on 94 cross-checked simulations), covering what the first could not: experiments that start at t = -5, 2 or 5, preequilibration (with the documented semantics: converted assignments hold their value at time 0), parameter table entries whose estimated or fixed nominal values differ from the model's (in values, switching times and condition values), condition values that refer to parameter table symbols at the start and to a species in unrelated changes. Each problem also checks that the input stays unchanged, that conversion is deterministic and idempotent, and that the converted condition and experiment tables survive a round trip through their DataFrames. On this PR's head, 1415 of 1500 problems match and pass all checks; the other 85 are rejected because a switching time depends on an estimated parameter, as intended. On main, the same pass finds 21 mismatches and 16 lint failures of the converted problem in 200 problems.
  • Where BasiCO disagrees with the reference simulator on an original problem (a switch of the assignment rule exactly at the time of a period's event, which COPASI mis-simulates; moving the switch by 0.001 removes the disagreement), the reference simulator agrees with the converted problem.

🤖 Generated with Claude Code

@FFroehlich
FFroehlich added this pull request to stack #575 October 25, 2026 16:51
@codecov-commenter

codecov-commenter commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.61702% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.06%. Comparing base (597c0a6) to head (3e6327d).

Files with missing lines Patch % Lines
petab/v2/assignments_to_experiments.py 93.05% 7 Missing and 8 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #574      +/-   ##
==========================================
+ Coverage   78.97%   79.06%   +0.09%     
==========================================
  Files          69       70       +1     
  Lines        8096     8155      +59     
  Branches     1450     1461      +11     
==========================================
+ Hits         6394     6448      +54     
- Misses       1215     1218       +3     
- Partials      487      489       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from claude/assignments-to-experiments-refactor to main October 25, 2026 17:15
FFroehlich and others added 4 commits October 25, 2026 17:15
…ope gaps

- A switch exactly at the start of a later period was lost when that period
  did not change any symbol of the assignment, e.g. `piecewise(0, time < 10,
  1)` with a period at t = 10, or `time <= 0` after preequilibration. A later
  period now also sets the value when it differs from the value at the end
  of the previous period.
- The condition changes of every symbol of a converted assignment were
  dropped, even when the model still used the symbol elsewhere, e.g. a dose
  that also enters a reaction. They are now only dropped if nothing else
  (the model, an observable, a condition change) refers to the symbol.
- Time-dependent initial assignments were converted into switches, but SBML
  evaluates them once, at the start of the simulation. They now stay in the
  model.
- At preequilibration, which starts the experiment, a value that referred
  to model parameters outside the parameter table failed the lint. Those
  parameters now take their model values, as at the first period.
- An assignment to a species or compartment crashed with an AttributeError;
  it now raises NotImplementedError.
- A problem without time-dependent assignments was still changed: an
  experiment that only preequilibrates got an empty period at t = 0.

`test_assignments_to_experiments` and `_unused_condition_id` used an
initial assignment to mean a switching rule and now use an assignment rule. `_validate` relied on an
estimated parameter with a time-dependent initial assignment; it now uses a
ramp whose generated slope parameter collides with an observable parameter.

The converted benchmark problems (Brannmark_JBC2010, Fujita_SciSignal2010,
Isensee_JCB2018, Raimundez_PCB2020) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nconverted problems alone, pick an unused slope ID

- A parameter used as a model or species conversion factor counted as
  unused, so a condition change of it was dropped when the parameter also
  appeared in a converted assignment. Conversion factors now count as
  references.
- A problem without time-dependent assignments was still changed: its
  experiments' periods were re-sorted. It is now returned as it was, after
  the lint.
- A ramp's slope parameter was only named so as not to clash with SBML IDs,
  and clashed with, e.g., an observable parameter `<target>_slope`. It is now
  named with `Problem.get_unused_id`.

`test_assignments_to_experiments_validate` relied on that slope clash to
fail the lint; it now uses an existing experiment `default_experiment`,
which the measurements without an experiment ID are assigned to as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…start of its period

The converter kept a condition change's value as an expression and
substituted it wherever a switch used it, so the value was re-evaluated at
the switch with the values at that time. A condition change is evaluated
once, at the start of its period, and all changes of a period
simultaneously, with the values from before the period. E.g., with `d1 = b1
+ 0.5` at t = 2 and `b1 = 2` at t = 3, a switch to `d1` at t = 5 used 2.5
instead of 1.7; likewise with `b1 = 2` in the same condition as `d1`.

A change's value is now evaluated at the start of its period: symbols set
by an earlier condition take that value, other constant model parameters
their model values, and parameter table symbols stay. A change of a symbol
of the assignment whose value still refers to anything else, e.g. a
species, is rejected, since its value is not known there.

Found by differential testing against simulations of the original and the
converted problem.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FFroehlich added a commit that referenced this pull request Oct 25, 2026
Follows #559. A refactor of `assignments_to_experiments` that does not
change its behaviour. The fixes for bugs found while reviewing and
stress-testing it are in #574, stacked on this PR.

## Slimming down `assignments_to_experiments`

The function had grown to 340 lines, most of them one loop body passing
around the parsed assignment, its switches, its value before every
switch, the switching times' fixed values and the slope parameter of a
ramp. It is now about 75 lines (plus its docstring) that run the steps
in order:

- `_add_start_periods`: give every experiment the period that starts it,
and find the periods that may be dropped;
- for each time-dependent assignment rule, a `_PiecewiseAssignment`:
- its constructor checks that the assignment is supported and works out
the value before every switch, the switching times' fixed values and
whether the value ramps;
- `replace_in_model()` replaces the rule in the model (the slope
parameter and rate rule for a ramp, the target's value);
- `changes(experiment)` returns the condition changes that replace it in
an experiment. The old module-level `_changes` and `_value_from` became
its methods.
- `_drop_changes` / `_add_conditions`: replace the condition changes the
new conditions cover;
- drop empty periods, add the default experiment, lint.

The `redundant` / `after_preequilibration` double negative became one
decision per period: one branch for the first period, one for the later
ones.

On the review comment on #557 about not turning this into a class:
`assignments_to_experiments` stays a function. `_PiecewiseAssignment`
only holds the values of one assignment that the old loop body passed
between its parts. If you would rather not have it, it can become a few
free functions that take those values.

No change in behaviour: on all 40 conversions in the tests, the old and
new implementations give the same experiment table, condition table,
measurement table, SBML, errors and warnings. The benchmark problems
Brannmark_JBC2010, Fujita_SciSignal2010, Isensee_JCB2018,
Raimundez_PCB2020 (and Oliveira_NatCommun2021's error) also convert
identically.

## Checked

- `pytest tests/v2 tests/v1/test_petab.py`: 244 passed; pre-commit
clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Fabian Fröhlich <frohlif@login006.nemo.thecrick.org>
@FFroehlich
FFroehlich force-pushed the claude/assignments-to-experiments-fixes branch from d60a383 to 0bd974e Compare October 25, 2026 17:15
FFroehlich and others added 2 commits October 25, 2026 17:31
…fixed parameter table values in condition changes

- The values of the new condition changes used the symbols parsed from
  SBML, without the assumptions of the symbols PEtab creates
  (`petab_symbol`, real), so they compared unequal to the same
  expressions parsed by PEtab, e.g. after reading the condition table
  back. They now use PEtab's symbols.
- A condition change evaluated at the start of its period kept parameter
  table symbols, also those of fixed parameters. A switching time set by a
  condition to, e.g., `p + 1` with `p` fixed by the parameter table was
  therefore rejected, although `t1 := p + 1` in the assignment itself is
  converted with `p`'s nominal value. Fixed parameters now take their
  nominal values there as well; estimated ones stay, and switching times
  that depend on them are still rejected.

Both found by differential testing with a reference simulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FFroehlich

Copy link
Copy Markdown
Collaborator Author

did not review code in detail, but has now survived multiple rounds of fuzzing/adversarial review+testing.

@FFroehlich
FFroehlich marked this pull request as ready for review October 25, 2026 19:02
@FFroehlich
FFroehlich requested a review from a team as a code owner October 25, 2026 19:02

@dweindl dweindl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By now, I'd say that assignments_to_experiments deserves a separate module and separate test module. Fine to export it from petab.v2.converters.

Comment thread petab/v2/converters.py Outdated
Comment thread petab/v2/converters.py Outdated
FFroehlich and others added 2 commits October 26, 2026 07:59
`assignments_to_experiments` collected the IDs that the SBML model refers
to, and the symbols in the observable and noise formulas, itself. These
are now `SbmlModel.get_referenced_ids()` (IDs in any math of the model,
and conversion factors) and `ObservableTable.free_symbols`, in line with
`ConditionTable.free_symbols`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`assignments_to_experiments` and its helpers move unchanged from
`petab.v2.converters` to `petab.v2.assignments_to_experiments`, and its
tests from `tests/v2/test_conversion.py` to
`tests/v2/test_assignments_to_experiments.py`. `petab.v2.converters` still
exports `assignments_to_experiments` and `DEFAULT_EXPERIMENT_ID`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FFroehlich
FFroehlich merged commit 19df050 into main Oct 26, 2026
13 checks passed
@FFroehlich
FFroehlich deleted the claude/assignments-to-experiments-fixes branch October 26, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants