Repository navigation
03: Update HRV and respiration windows incrementally - #105
Draft
mikenrafter wants to merge 26 commits into
Draft
mikenrafter wants to merge 26 commits into
mikenrafter wants to merge 26 commits into
Conversation
Lipponen & Tarvainen (2019), Peltola (2012). The 32-beat cap is an internal ESTIMATE rule, not a published threshold. Include synthetic input recipes and old/new measurements.
Lomb (1976), Scargle (1982), Press & Rybicki (1989). Shift the spectral origin to the first sample before forming trigonometric arguments; pin the unrounded power on synthetic epoch seconds.
Preserve Task Force (1996) and Welch (1967) formulas for finite inputs. Timestamp validity is an implementation guard, preventing endless segment walks and invalid integer conversions.
Nelson et al. (1979), Methods for cosinor-rhythmometry; Welford (1962) centered co-moments. A relative determinant gate replaces the absolute pivot test; preserve upstream significance policy.
Morton, Fitz-Clarke & Banister (1990); Neumaier (1974) compensated summation. Preserve weighting and anchors; pin changed last digits.
Banister et al. (1975); Morton et al. (1990); Neumaier (1974). Preserve EWMA formulas and bounded-prefix priming while matching the arithmetic of the later incremental path.
Karvonen (1957) supports reserve zones only. Duration credit is an internal ESTIMATE rule with low confidence; preserve cadence validation, median capping and tail credit.
Witting (1990); van Someren (1999); Welford (1962). Preserve batch profile sums and first-winner ties; centered variance changes last digits in IS and IV.
Task Force (1996); Welford (1962). Port centered NN, lag and bin moments into batch code and retain shared spectral jitter policy. Rejection and breathing-line exemptions remain internal ESTIMATE rules with low confidence.
Record synthetic old/new observations and freeze twelve batch-only cases before aggregator adoption. Pin unchanged batch timestamp-gap exclusion and circular near-tie selection as controls. Methods and internal-policy limits are documented with the results.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
Sorry @mikenrafter, this pull request is too large to review
It changes 29,856 lines. Sourcery can review pull requests with up to 20,000 changed lines.
Author
|
@coderabbitai review |
Action not completed
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Review updates
Inherited the review fixes from #102–#104, including honest invalid-input refusals, explicit RR gap labels, streaming checkpoint validation, and regressing clock refusal across all six HRV/RSA/CVHR entry points without shifting recorded segment times. Also refuses mismatched supplied time vectors and non-finite cosinor moments. Latest Dart 3.11.4 analysis is clean and the full suite passed 1,844 tests with 6 existing capture-dependent skips. All exact output-change fixtures remain unchanged. This cumulative draft will be rebased onto upstream main and marked ready after its predecessors merge.
Depends on 02 / #104.
Depends on 01 / #103.
Depends on 00 / #102.
This is draft 03 of the split of #99. It targets
main, so its diff is cumulative and includes the preceding branches. Review this area in its final commit; merge the preceding PRs first, then rebase this branch before merging.HRV and respiration updates repeatedly evaluate completed windows, while seam errors can count a difference or breathing segment twice. This adds summaries and ordered streams for time/frequency HRV, RSA, CVHR and PRSA, with the shared corrected-NN tests that exercise their boundaries together.
What changes
Task Force 1996 supports time-domain HRV; Welch 1967 and Lomb/Scargle support spectral calculations; Hirsch and Bishop 1981 support respiratory coupling; Bauer et al. 2006 supports PRSA and Hayano 2011 supports CVHR, with the original source citations retained. Internal jitter exemptions, the RR cap and screening thresholds remain marked ESTIMATE. This does not turn 1 Hz pulse timing into ECG-grade HRV. The batch wrappers use the parent's output arithmetic; regrouped floating sums can change rounding and near-threshold decisions. The RSA floor guard is retained, and old incompatible RSA checkpoints are rejected. Adoption must respect the output/versioning notes from the parent and area 2.
Algebraic breakdown
For mergeable entries, the common tests cover both identities, split, associativity, singleton and sliced lifts, finish purity and JSON continuation. These are observational laws at fixed parameters, not bitwise associativity of floating-point state.
Beats,RawRr_DiffStats/_DiffState/_DiffSummaryLookback(2)lifts eligible adjacent pairs including seams; finish RMSSD/pNN components. Approximate ordered monoid once wrapped.hrv_time_aggregator_test.dart, HRV support and checkpoint checks: fields1e-9absolute and relative, counts/absence exact._LagStats/_LagState/_LagSummaryLookback(3)lifts adjacent differences without crossing a gap; finish ACF1. Approximate ordered monoid.1e-8perturbation case allows ACF1 absolute error below1e-6;1e-4perturbation split case requires below1e-9. Those exceptions are explicit tests, not a universal gate bound.hrvTimeAggregator,_FiniteTimesand checkpoint wrappersRunningMoments, five-minute grouped moments and jitter state; child identitiesBeats; finish HRV fields and publication gates. Approximate ordered monoid under fixed origin/gap policy.hrvTimeOneSliceuses the same numeric core and builds evidence lazily.1e-9absolute/relative; metadata/counts exact on fixtures. Nonfinite time input is refused consistently.JitterEvidenceAggregator,JitterState,_Piece,_Phase,_Win,_WinRunHrvFreqStream,_HrvBandStatefold; finishsnapshot(tail)with batch arithmetic over closed segments and provisional tail. Ordered checkpointed fold.welch_stream_test.dartandwelch_stream_laws_test.dart: chunk invariance and JSON text exact, snapshots bit-for-bit against batch including notes/confidence, empty fold identity, refusal atomicity, conservation and purity. Associative state merging is not exposed.RsaRespStream,_RsaSegmentResultrsa_floor_stream_test.dartpins the below-band floor refusal and checkpoint versions. Completed-window reuse does not justify changing the adaptive frequency ceiling.CvhrStream,_CvhrContribution, night/distribution results_PrsaWindows,_PrsaState,prsaAggregatorLookback(2*l+1)lifts eligible centered anchor windows; finish profile means and DC/AC. Ordered monoid through boundary context; floating sums approximate.prsa_aggregator_test.dart: common laws, conversion and restart; fields1e-9absolute/relative, confidence1e-12; discrete metadata exact. Timestamp-aware PRSA remains a batch path.RrSandwich,RrSandwichStaterr_sandwich_test.dart: exact NN/source-time/class parity with cappedcorrectRr, association/identity/checkpoint/bounded-state cases. HRV inner1e-9; PRSA fields1e-9absolute/relative and confidence1e-12. Passing tests do not prove every adversarial soft-run influence bound. No sandwich benchmark.NnColumnBeats, finish inner. Preserves the inner algebra.HrvTimeAcc,HrvTimelineState,NightShapeStatefold; evaluate with provisional tail and batch gates. No state combine.test/incremental/hrv_resp_nn_test.dartand boundary tests compare to production batch/oracles; scalar HRV fields use the explicit1e-9relative comparisons where called, envelopes and tested decisions exact.DayHrvCurveState,DaytimeHrvStateNocturnalRmssdState,SessionRmssdState, researchIrregularScreenStatehrv_windows_test.dartand boundary/thin cases check exact published window outputs, flags and nulls; confidence uses each test's comparator._SegCounts,RsaWelchState,RespWindowsState,_Band,HrvFreqState,DayRespCurveStateedge_oracles.dart, stream test support and probesWhere the speedup comes from
Every sync repeats completed HRV and respiration work, so this PR retains those results and adds cheaper CVHR median calculations, while CVHR still scans its open stretch. This PR: 52.058 of the 109.065 s saved per night (47.73%). Overall after the full stack: 13.98×.
See the sync-work explanation in #103 for the timeline, overall table and Edge adoption requirement; these are AOT medians from at least five full schedules on generated data, with this PR credited only beyond #102 and its inherited gains included in the overall saving.
How each structure saves work
Time HRV reuses level/difference/lag summaries; scalar moment combination is O(1), while grouped bins and jitter boundary evidence add their own costs. Frequency/RSA streams compute completed Welch windows once, so repeated syncs revisit new windows and the open tail. Median/consensus finishing can still scan the completed result list. CVHR preserves closed stretch contributions but can recompute a long current stretch. PRSA combines O(profile length) sums plus O(window width) seam context. The capped sandwich corrects boundary context instead of all settled raw RR. The research day/window states avoid replaying completed bins; serialization and result-list construction remain proportional to retained state/output.
Fresh timings are in "Where the speedup comes from" above. Older numbers in the docs predate this split.
Tests
Dart 3.11.4, resolved matcher 0.12.20.
dart analyze: no issues. Fulltimeout 3000 nice -n 10 dart test -j 2 --reporter json: 1,812 passed, 6 skipped, 0 failed. Counts include the inherited suite. Elapsed test time 162.350 seconds.Logs:
areas-logs/area-3.analyze.log,areas-logs/area-3.test.jsonl, exit status files and start/finish timestamps. Skips are optional external fixtures/corpora. No assertion was removed and no tolerance was widened. Dart 3.5.0 itself was not run; final test blobs retain the existing NaN-aware matcher fix.Suggested review order
One commit:
f118248836ecHRV and respiration updates rescan completed windows. Review the state/empty/combine or append transitions first, the finish/publication rules next, then the area reference/law/checkpoint tests and tools.Change size and stacking
Logic under
lib/: +2,988/-368, 3,356 changed lines. Tests, tools and docs: +7,712/-0, 7,712 changed lines. Other root documentation: +0/-0.Stacks on
split/area-2-rr-rhythmat0725b6932e4e. This is branch 3 of 6, one compiled and fully tested commit per branch. Apply the stack in numerical order; the final branch must matchorigin/split/aggregatorsexactly.Questions for the reviewer