Skip to content

03: Update HRV and respiration windows incrementally - #105

Draft
mikenrafter wants to merge 26 commits into
OpenStrap:mainfrom
mikenrafter:split/area-3-hrv-respiration
Draft

mikenrafter wants to merge 26 commits into
OpenStrap:mainfrom
mikenrafter:split/area-3-hrv-respiration

Conversation

@mikenrafter

@mikenrafter mikenrafter commented Nov 3, 2026 •

Copy link
Copy Markdown

Review updates

Inherited the review fixes from #102–#104, including honest invalid-input refusals, explicit RR gap labels, streaming checkpoint validation, and regressing clock refusal across all six HRV/RSA/CVHR entry points without shifting recorded segment times. Also refuses mismatched supplied time vectors and non-finite cosinor moments. Latest Dart 3.11.4 analysis is clean and the full suite passed 1,844 tests with 6 existing capture-dependent skips. All exact output-change fixtures remain unchanged. This cumulative draft will be rebased onto upstream main and marked ready after its predecessors merge.

Depends on 02 / #104.

Depends on 01 / #103.

Depends on 00 / #102.

This is draft 03 of the split of #99. It targets main, so its diff is cumulative and includes the preceding branches. Review this area in its final commit; merge the preceding PRs first, then rebase this branch before merging.

HRV and respiration updates repeatedly evaluate completed windows, while seam errors can count a difference or breathing segment twice. This adds summaries and ordered streams for time/frequency HRV, RSA, CVHR and PRSA, with the shared corrected-NN tests that exercise their boundaries together.

What changes

  • Add time-domain HRV composition and mergeable jitter evidence, then use their arithmetic in the batch wrapper.
  • Add production frequency-HRV, RSA and CVHR streams with provisional-tail evaluation and versioned checkpoints.
  • Add PRSA and capped raw-RR sandwich aggregators.
  • Retain the separate research accumulators, their oracles and joint Welch/property tests in this area.

Task Force 1996 supports time-domain HRV; Welch 1967 and Lomb/Scargle support spectral calculations; Hirsch and Bishop 1981 support respiratory coupling; Bauer et al. 2006 supports PRSA and Hayano 2011 supports CVHR, with the original source citations retained. Internal jitter exemptions, the RR cap and screening thresholds remain marked ESTIMATE. This does not turn 1 Hz pulse timing into ECG-grade HRV. The batch wrappers use the parent's output arithmetic; regrouped floating sums can change rounding and near-threshold decisions. The RSA floor guard is retained, and old incompatible RSA checkpoints are rejected. Adoption must respect the output/versioning notes from the parent and area 2.

Algebraic breakdown

For mergeable entries, the common tests cover both identities, split, associativity, singleton and sliced lifts, finish purity and JSON continuation. These are observational laws at fixed parameters, not bitwise associativity of floating-point state.

Algorithm/structure State and empty Combine; lift; finish Laws and tolerance
Beats, RawRr Aligned NN/time or raw-RR/time columns; zero rows Ordered sequence concatenation; lift copies slice; finish exposes columns. Sequence monoids, not commutative. Range/join/codec and malformed-length tests; exact rows with explicit NaN handling.
HRV _DiffStats/_DiffState/_DiffSummary Centered difference moments, squared-difference and pNN counters; zeros Merge moment/counter contributions; Lookback(2) lifts eligible adjacent pairs including seams; finish RMSSD/pNN components. Approximate ordered monoid once wrapped. hrv_time_aggregator_test.dart, HRV support and checkpoint checks: fields 1e-9 absolute and relative, counts/absence exact.
HRV _LagStats/_LagState/_LagSummary Centered lag-pair covariance and associated sums/counts; zeros Merge centered co-moments; Lookback(3) lifts adjacent differences without crossing a gap; finish ACF1. Approximate ordered monoid. Same laws; near-constant 1e-8 perturbation case allows ACF1 absolute error below 1e-6; 1e-4 perturbation split case requires below 1e-9. Those exceptions are explicit tests, not a universal gate bound.
hrvTimeAggregator, _FiniteTimes and checkpoint wrappers Product of diff/lag summaries, level RunningMoments, five-minute grouped moments and jitter state; child identities Ordered child combine; lift Beats; finish HRV fields and publication gates. Approximate ordered monoid under fixed origin/gap policy. hrvTimeOneSlice uses the same numeric core and builds evidence lazily. Full law/parity cases and malformed checkpoint tests. HRV standard fields 1e-9 absolute/relative; metadata/counts exact on fixtures. Nonfinite time input is refused consistently.
JitterEvidenceAggregator, JitterState, _Piece, _Phase, _Win, _WinRun Difference sums, magnitude distributions, run/head/tail context, phase alternatives and per-window evidence; empty maps/counts/pieces Merge interiors and repair boundary runs/phases; lift NN differences and their timing; finish jitter/RSA evidence. Ordered approximate summary, not a constant-size commutative statistic. Boundary runs and exact magnitude evidence can grow. HRV jitter/exemption cases exercise chunking and restart through the finished HRV result, using the same tolerances and exact publication decisions on tested fixtures. No standalone universal associativity error bound is claimed.
HrvFreqStream, _HrvBandState Settled NN/time plus per-band next window and completed segment contributions; fresh stream No combine. Lift fold; finish snapshot(tail) with batch arithmetic over closed segments and provisional tail. Ordered checkpointed fold. welch_stream_test.dart and welch_stream_laws_test.dart: chunk invariance and JSON text exact, snapshots bit-for-bit against batch including notes/confidence, empty fold identity, refusal atomicity, conservation and purity. Associative state merging is not exposed.
RsaRespStream, _RsaSegmentResult Closed RSA subwindow results, ordered open beats and rate/NN evidence; empty No combine. Fold settled beats; snapshot adds provisional subwindows and recomputes the batch consensus/rate gates. Ordered fold, not a monoid. Same exact Welch laws; rsa_floor_stream_test.dart pins the below-band floor refusal and checkpoint versions. Completed-window reuse does not justify changing the adaptive frequency ceiling.
CvhrStream, _CvhrContribution, night/distribution results Closed stretch contributions plus current gap-bounded NN stretch; empty No combine. Fold by timestamp gaps; snapshot runs the open-stretch detector and finishes nightly distribution. Ordered fold. CVHR tests and joint stream laws require exact batch results, JSON continuation, empty fold, snapshot purity and conservation. A long uninterrupted open stretch still costs work at snapshot.
_PrsaWindows, _PrsaState, prsaAggregator Acceleration/deceleration profile sums and anchor counts; zero profiles/counts Add profiles/counts; Lookback(2*l+1) lifts eligible centered anchor windows; finish profile means and DC/AC. Ordered monoid through boundary context; floating sums approximate. prsa_aggregator_test.dart: common laws, conversion and restart; fields 1e-9 absolute/relative, confidence 1e-12; discrete metadata exact. Timestamp-aware PRSA remains a batch path.
RrSandwich, RrSandwichState Raw bounded head/tail, input length and downstream middle state; zero rows plus inner identity Re-correct adjacent raw seam, settle newly independent NN into inner; lift raw RR slices; finish corrected boundaries plus middle through inner. Ordered monoid claimed for the capped/locality contract, conditional on lawful inner. Uncapped arbitrarily long runs do not have guaranteed bounded-state exactness. rr_sandwich_test.dart: exact NN/source-time/class parity with capped correctRr, association/identity/checkpoint/bounded-state cases. HRV inner 1e-9; PRSA fields 1e-9 absolute/relative and confidence 1e-12. Passing tests do not prove every adversarial soft-run influence bound. No sandwich benchmark.
NnColumn Inner state; inner empty Delegate combine, lift NN column from Beats, finish inner. Preserves the inner algebra. Sandwich PRSA composition tests use inner tolerance; no new estimator.
Research HrvTimeAcc, HrvTimelineState, NightShapeState Scalar HRV/jitter accumulator and keyed window accumulators; empty Ordered settled-NN fold; evaluate with provisional tail and batch gates. No state combine. test/incremental/hrv_resp_nn_test.dart and boundary tests compare to production batch/oracles; scalar HRV fields use the explicit 1e-9 relative comparisons where called, envelopes and tested decisions exact.
Research DayHrvCurveState, DaytimeHrvState Per-bin/per-window states and day/mask context; fresh state Fold new settled beats; evaluate keyed curves/daytime summary with provisional tail. Ordered folds, no merge API. Day-curve acceptance tests arrive in area 6 with the cross-area driver. No untested associative merge claim.
Research NocturnalRmssdState, SessionRmssdState, research IrregularScreenState Retained window records, adjacency and scalar evidence, or a session corrector plus windows; empty Ordered fold; evaluate eligible window medians/session result or screen verdict. Not monoids: window eligibility and medians depend on retained context. hrv_windows_test.dart and boundary/thin cases check exact published window outputs, flags and nulls; confidence uses each test's comparator.
Research _SegCounts, RsaWelchState, RespWindowsState, _Band, HrvFreqState, DayRespCurveState Completed segment counts/results, open window state and keyed rate/spectral summaries; empty Ordered fold and tail evaluation; finish via batch band integration/median/consensus. No combine operation or commutativity claim. Corrected-NN, boundary and thin-input tests compare exact published results where asserted; numeric error recorders report differences without inventing a global bound. Day-curve driver tests arrive in area 6.
edge_oracles.dart, stream test support and probes Reference input windows and seeded recipes Batch reference/projection or measurement only; no monoid interface. Travel here because the joint NN tests import them. Some oracles share production helpers; parity is not an independent proof of the physiological estimator.

Where the speedup comes from

Every sync repeats completed HRV and respiration work, so this PR retains those results and adds cheaper CVHR median calculations, while CVHR still scans its open stretch. This PR: 52.058 of the 109.065 s saved per night (47.73%). Overall after the full stack: 13.98×.

Workload / calculation #102 batch s Retained s This PR saves s
night / HRV time 0.034481 0.020068 0.014413
night / HRV frequency 19.799112 1.823393 17.975719
night / RSA 15.033006 1.004195 14.028811
night / CVHR 8.916577 0.754348 8.162229
night / PRSA 0.008815 0.000594 0.008221
night / RSA bins 12.751969 0.883364 11.868605
day / HR dip 0.339279 0.134907 0.204372

See the sync-work explanation in #103 for the timeline, overall table and Edge adoption requirement; these are AOT medians from at least five full schedules on generated data, with this PR credited only beyond #102 and its inherited gains included in the overall saving.

How each structure saves work

Time HRV reuses level/difference/lag summaries; scalar moment combination is O(1), while grouped bins and jitter boundary evidence add their own costs. Frequency/RSA streams compute completed Welch windows once, so repeated syncs revisit new windows and the open tail. Median/consensus finishing can still scan the completed result list. CVHR preserves closed stretch contributions but can recompute a long current stretch. PRSA combines O(profile length) sums plus O(window width) seam context. The capped sandwich corrects boundary context instead of all settled raw RR. The research day/window states avoid replaying completed bins; serialization and result-list construction remain proportional to retained state/output.

Fresh timings are in "Where the speedup comes from" above. Older numbers in the docs predate this split.

Tests

Dart 3.11.4, resolved matcher 0.12.20. dart analyze: no issues. Full timeout 3000 nice -n 10 dart test -j 2 --reporter json: 1,812 passed, 6 skipped, 0 failed. Counts include the inherited suite. Elapsed test time 162.350 seconds.

Logs: areas-logs/area-3.analyze.log, areas-logs/area-3.test.jsonl, exit status files and start/finish timestamps. Skips are optional external fixtures/corpora. No assertion was removed and no tolerance was widened. Dart 3.5.0 itself was not run; final test blobs retain the existing NaN-aware matcher fix.

Suggested review order

One commit: f118248836ec HRV and respiration updates rescan completed windows. Review the state/empty/combine or append transitions first, the finish/publication rules next, then the area reference/law/checkpoint tests and tools.

Change size and stacking

Logic under lib/: +2,988/-368, 3,356 changed lines. Tests, tools and docs: +7,712/-0, 7,712 changed lines. Other root documentation: +0/-0.

Stacks on split/area-2-rr-rhythm at 0725b6932e4e. This is branch 3 of 6, one compiled and fully tested commit per branch. Apply the stack in numerical order; the final branch must match origin/split/aggregators exactly.

Questions for the reviewer

  • Is the capped sandwich's tested locality contract acceptable while the adversarial soft-run bound remains unproved?
  • Should the research prototype classes remain separate from the production streams until their consumers converge?

Lipponen & Tarvainen (2019), Peltola (2012). The 32-beat cap is an internal ESTIMATE rule, not a published threshold. Include synthetic input recipes and old/new measurements.
Lomb (1976), Scargle (1982), Press & Rybicki (1989). Shift the spectral origin to the first sample before forming trigonometric arguments; pin the unrounded power on synthetic epoch seconds.
Preserve Task Force (1996) and Welch (1967) formulas for finite inputs. Timestamp validity is an implementation guard, preventing endless segment walks and invalid integer conversions.
Nelson et al. (1979), Methods for cosinor-rhythmometry; Welford (1962) centered co-moments. A relative determinant gate replaces the absolute pivot test; preserve upstream significance policy.
Morton, Fitz-Clarke & Banister (1990); Neumaier (1974) compensated summation. Preserve weighting and anchors; pin changed last digits.
Banister et al. (1975); Morton et al. (1990); Neumaier (1974). Preserve EWMA formulas and bounded-prefix priming while matching the arithmetic of the later incremental path.
Karvonen (1957) supports reserve zones only. Duration credit is an internal ESTIMATE rule with low confidence; preserve cadence validation, median capping and tail credit.
Witting (1990); van Someren (1999); Welford (1962). Preserve batch profile sums and first-winner ties; centered variance changes last digits in IS and IV.
Task Force (1996); Welford (1962). Port centered NN, lag and bin moments into batch code and retain shared spectral jitter policy. Rejection and breathing-line exemptions remain internal ESTIMATE rules with low confidence.
Record synthetic old/new observations and freeze twelve batch-only cases before aggregator adoption. Pin unchanged batch timestamp-gap exclusion and circular near-tie selection as controls. Methods and internal-policy limits are documented with the results.
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @mikenrafter, this pull request is too large to review

It changes 29,856 lines. Sourcery can review pull requests with up to 20,000 changed lines.

@mikenrafter

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant