Harmonic-plus-noise: additive filtered-noise branch (DDSP-style)#7
Draft
jmxpearson wants to merge 46 commits into
Draft
Harmonic-plus-noise: additive filtered-noise branch (DDSP-style)#7jmxpearson wants to merge 46 commits into
jmxpearson wants to merge 46 commits into
Conversation
Alternative to the in-ODE OU forcing: keep the oscillator PURELY deterministic (RK4) and add, OUTSIDE the tract, a parallel filtered-noise source summed with the tract output: white noise -> sigma AM gate g(t) -> learned per-frame magnitude filter -> add The oscillator carries the tonal/harmonic content, the noise branch the broadband/aperiodic floor; the two can't fight (no ODE coupling), so no collapse and no stochastic Heun. Motivated by an inference test where deterministic-harmonic + sigma-gated filtered noise beat both the harmonic-only baseline and the noise-inside-ODE model on RMS-matched MRSTFT (2.35 vs 2.86/2.44). - model.Ouroboros: use_noise_branch builds the sigma AM head (shared with the OU mode) plus a new noisefilt Mamba head emitting a per-timestep magnitude response over noise_bands bands; get_sigma now keys off the head existing; new get_noise_filter. - spectral_rollout.filtered_noise_branch: white noise -> STFT -> per-frame filter (bands interpolated to rFFT bins, phase kept) -> iSTFT -> sigma AM; added to xg after the tract in spectral_rollout_step, gated by the same noise_gain ramp. Oscillator stays deterministic. - config/CLI (--use-noise-branch / --noise-bands / --noise-nfft / --noise-hop), freeze group, save/load persistence. Verified: forward finite, gradients reach both the sigma and noise-filter heads, noise_gain=0 is deterministic (dW-independent), save/load round-trips the heads. Off by default; the OU path is untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…oly_autonomous) So autonomy_score / the monitor render the FULL harmonic+noise output (noise added outside the tract, gated by noise_gain), matching training -- not just the deterministic harmonic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
autonomy_score built drives_n by subscripting every drive value, but the drives dict carries 'sigma'=None when the model has a sigma head but enable_noise_forcing is off (harmonic-plus- noise mode) -> TypeError. Populate g_gate whenever get_sigma returns non-None (head exists, not just OU mode), and make the drives_n comprehension None-safe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
… of 65-band FIR Replace the DDSP-style 65-band per-frame magnitude filter with the noise branch's own low-order rational filter -- a Tract instance (noise_tract_n_sec 2nd-order pole/zero sections, same parameterization as the vocal tract), applied to sigma-gated white noise. The low filter order can shape a broadband spectral envelope but CANNOT synthesize sharp harmonic peaks, so it can't model the tonal structure outright -- forcing the oscillator to carry it. Motivated by the freeze-noise test where the 65-band filter modelled the org545 syllables so completely that the oscillator was abandoned (alpha -> 0, source dead). Drops noise_bands/noise_nfft/noise_hop + the noisefilt Mamba head and get_noise_filter; adds noise_tract_n_sec (default 3) + --noise-tract-n-sec. Verified: forward finite, gradients reach the sigma head AND the noise_tract pole/zero params, noise_gain=0 deterministic, save/load round-trips, eval path finite. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…stead of duplicate drives
The drives were plotted in BOTH lower panels (left+right). Replace the LEFT one with the additive
filtered-noise term -- the noise_gain*filtered_noise the model adds to the tract output to make the
final waveform -- y-locked to the target scale like the other left-column waveforms, so its
amplitude is directly comparable to target/reconstruction/source. Drives stay on the right panel;
non-noise models fall back to drives on the left (legacy view).
integrate_poly_autonomous now captures the added noise term and returns it via the drives dict
('noise' key); autonomy_score threads it through the trajectories. Verified: noise term present,
correct length, monitor renders NEW_CKPT with no error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
The monitor SIGSTOPs the GPU trainer during each score so the autonomy subprocess isn't starved -- necessary for GPU scoring, but in CPU mode (MONITOR_DEVICE=cpu) scoring never touches the GPU, so freezing GPU training is pure waste (~80s/checkpoint, and it pauses through the whole backlog when the sidecar falls behind). Guard: skip the SIGSTOP/SIGCONT entirely when MONITOR_DEVICE=cpu, so the CPU sidecar runs fully decoupled and training is never interrupted. Picked up on the running sidecar's next poll (it re-invokes the monitor per poll). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
A comb (1 - r*e^{-jw*tau}) imposes periodic spectral teeth -- exactly the harmonic-like structure
the low-order noise filter must NOT be able to synthesize. Leaving it in gave the noise a way to
fake periodic peaks and partly defeat the handicap. Add a use_comb switch to Tract (default True,
so the vocal tract is unchanged); build the noise branch's Tract with use_comb=False, so it's a
pure broadband pole/zero envelope with no periodic structure. r_raw/tau_raw aren't even created
when disabled. Verified: noise_tract has no comb params, vocal tract keeps its comb, gradients
reach the noise pole/zero params, save/load round-trips.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…default off) Sigma was the only head not low-passed. Add sigma_lowpass_ms (default 0.0 = off, unchanged sharp gate); when >0, get_sigma smooths g(t) with the same zero-phase Gaussian as the drives, so the noise amplitude envelope can't snap abruptly (a candidate cause of the occasional nan_skips, and sensible since the noise isn't meant to model syllable onsets). The Gaussian's positive unit-sum weights keep the relu'd gate nonnegative. Threaded through model_cv, save/load, and the entry CLI (--sigma-lowpass-ms). Verified: off by default leaves g unchanged; >0 smooths it (roughness 0.085->~0) while staying nonneg; gradients flow through the low-pass; sigma_lowpass_ms round-trips. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Sigma/g is ~1000x smaller than gamma (e.g. sigma ~1e-3 vs gamma ~1.0), so sharing gamma/alpha's y-axis squashed it to a flat line -- looked like a dead/constant gate when it's just quiet. Give sigma its own twin axis (offset spine, clears omega^2's twin) so its structure is visible. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…king legacy sibling path) detect_batches_per_epoch only looked at <parent>/<runname>_train.log, but the launch script writes <run_dir>/train.log -- so the log was never found and it fell back to 750 (the old batch-16 default). That mis-stamped the monitor's val_* scalars at epoch*750 while the trainer stamps at epoch*188 (batch 64), stretching the val x-axis ~4x. Check <run_dir>/train.log first, then the legacy sibling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…), not N*bpe checkpoint_N is saved at the END of epoch N, i.e. trainer global step (N+1)*bpe-1, but the monitor stamped its Val/* scalars + spectrograms at N*bpe -- one epoch (bpe steps) too early, so every point sat one epoch left of the train-loss curve for the same model. Use (N+1)*bpe-1 (exact for from-scratch runs). Was previously documented as a 'good enough' shortcut.
…the sidecar uses) Previous commit fixed only the fallback branch, but run_autonomy_on_checkpoint always sets MONITOR_STEP_OVERRIDE=ckpt_epoch*bpe, so the TB step took the override branch and stayed one epoch early. The noise-gain calc already adds +bpe to reach end-of-epoch; do the same for the TB step: step_override + bpe - 1 = (ckpt_epoch+1)*bpe-1. Now Val/spectrogram points land on the trainer's end-of-epoch scalar step.
…lpha-lowpass-ms alpha (kernel weight [0,0] = pressure-analog DC drive) is normally smoothed at drive_lowpass_ms (1ms) with the other drives. Add alpha_lowpass_ms (default 0 = off): when >0, alpha gets an extra low-pass at that (longer) timescale in _lowpass_weights, so the driving pressure varies slowly (syllable-scale) -- matching the physical syringeal pressure wave. Only the [0,0] term is affected; all other kernel weights unchanged. Verified: alpha roughness collapses at 15ms while other weights are identical; off by default; save/load round-trips. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…-> NaN) The deterministic RK4 path evaluated the v^P polynomial on UNCLAMPED substep state and relied only on the end-of-step state clamp -- fine for gamma>0 (bounded amplitude) but a strongly anti-damped (gamma<0) oscillator's intermediate substep velocity overflowed fp32 mid-step -> NaN -> every batch nan-skipped (a no-noise slow-alpha run diverged and stalled at 100% nan-skip). Add the same kernel-input soft-tanh clamp the Heun core already has (xx_c=BX*tanh(xx/BX), vv_c=BXP*tanh(vv/BXP) before the polynomial; linear terms keep the raw state) to both the eager _rk4_core_factory and the graphstep _GraphedRK4Step. Verified: the diverged checkpoint now yields a finite loss+grads on both backends (was NaN), eager==graphstep. Small backward-memory cost (a couple (B,) tanh tensors/substep). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…d of duplicate drives) The lower-left panel is reserved for the filtered-noise waveform; when there's no noise branch it was falling back to re-plotting the drives (duplicating the right panel). For noise-off models leave it blank with a 'no noise branch' annotation instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…near data-init Adds a gauge anchor on the tract GAIN K=softplus(tract.K_raw): (K/K0 - 1)^2 toward its data-init value K0, mirroring the envelope anchor. Closes the (K, source) -> (c*K, source/c) gauge that the envelope anchor leaves open -- pins the absolute output gain WITHOUT flattening the envelope (which mean((e-1)^2) does). K is a scalar so the term is tiny with a bounded gradient (dL/dK_raw = 2(K/K0-1)/K0 * sigmoid(K_raw)). - model.py: register persistent buffer Tract.K_anchor_target = softplus(K_raw) at init. - model_cv.py: after the data-driven K_raw init, update K_anchor_target so the anchor pins K near the data-matched start, not softplus(0). - spectral_rollout.py: compute L_k_anchor from model.tract, add lam_tract_k_anchor*L_k_anchor to total, return 'k_anchor'. - train.py: thread param, log Loss/k_anchor + LossW/k_anchor, print K_anchor_target; load_state_dict strict=False guarded to tolerate the new buffer's absence in pre-existing checkpoints (any OTHER key mismatch still raises). - entry script: --lam-tract-k-anchor flag + manifest. Verified: old checkpoints (no buffer) still load; penalty is 0 at K=K0, >0 when perturbed with finite grad to K_raw, 0 when lam=0. Not yet used by any run (default 0). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Adds mean((max_t e - 1)^2): a quadratic penalty on the departure of the envelope's PEAK from 1. Unlike --lam-env-anchor (pins e(t) to a per-sample target -> penalizes the whole shape) and unlike a K-only anchor (leaves the envelope free to roam its level up, observed diverging to ~14x), this fixes ONLY the absolute scale of e(t) while leaving its time-variance (syllable shaping) unpenalized. Gradient flows through each sample's argmax timestep (max-pool subgradient). Threaded through spectral_rollout_step (returns 'env_max'), train.py (param + Loss/env_max + LossW logging + config print), model_cv.py, and the entry script (--lam-env-max-anchor + manifest). Verified: 0 at peak=1 / lam=0, >0 with finite grad when the peak departs from 1. Default 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Replace the linear-frequency STFT spectrograms in the diagnostic panels with mel-spaced ones (librosa mel filterbank, already a codebase dep via visualization/model_vis.py; n_mels=128, fmax=16kHz, n_fft=1024/hop=256). n_fft=1024 gives enough low-frequency resolution that the bottom mel bands aren't empty (an earlier n_fft=512 manual filterbank striped). Keeps the existing shared normalization: dB relative to the TARGET's peak MEL POWER so all rows stay directly comparable in amplitude. Removes the now-unused linear _stft_mag helper. Y-axis relabeled 'mel (Hz)' with mel-spaced Hz ticks; title notes 'mel'. Verified render on a live checkpoint: no striping, low/mid vocal structure resolved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…ls the LF rumble Adds a deterministic low-frequency 'rumble' source: a parallel Mamba head (rumble_mamba + zero-init rumble_net) that DIRECTLY emits a waveform (it does NOT drive an oscillator/ODE), band-limited to < --rumble-lowpass-hz via a fixed soft-mask rFFT low-pass, and ADDED to the tract+noise output OUTSIDE the tract. Gives the model a cheap dedicated channel for the sub-cutoff recording floor so the oscillator isn't forced to spend capacity on it -- but the oscillator is left FULL-RANGE (not high-passed), so it can still produce LF when a vocalization has genuine low-frequency content. - model.py: Ouroboros(use_rumble_branch, rumble_lowpass_hz=250); get_rumble() + _lowpass_fft() (raised-cosine rolloff to 1.5x cutoff, differentiable). Zero-init head -> starts silent. - spectral_rollout.py: rumble_branch(); added to the forward alongside the noise branch. - eval.py: rumble added to the autonomous reconstruction + captured (_rumble_cap), threaded into all 3 return_drives dicts as 'rumble'. - train.py/model_cv.py/entry: persistence + config threading + --use-rumble-branch/--rumble-lowpass-hz. - monitor: per-epoch mel-PSD panel (psd/mel) -- target / tract out (=auto-noise-rumble) / noise / rumble for the 3 TB vocs. Verified: low-pass band-limits (energy ratio above 1.5x cutoff ~1e-8); forward+backward finite with grad reaching the zero-init head; end-to-end monitor render produces spectrograms + psd/mel with drives['rumble'] populated; save/load round-trips the config. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Add the rumble branch (drives['rumble']) as its own spec row in the diagnostic panels, between source and the drives/noise row: target / auto / source / rumble / (noise+drives). Same mel-spec + target-referenced dB treatment as the other rows; band-limiting is visible as energy confined to the bottom (< cutoff) mel bands. Only shown when the rumble branch is present. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
(1) --mel-spec: compute the MRSTFT magnitude loss on MEL-warped spectra (librosa mel filterbank applied to |STFT| before the SC + log-mag terms; --mel-n-mels, fmax=Nyquist). Emphasizes low/mid vocal structure the way a mel spectrogram does. (2) --osc-warmup-epochs N: gate the deterministic (oscillator->env->tract) output by osc_gain, held at 0 for the first N epochs then ramped 0->1 over the next epoch. So the rumble+noise branches fit the SPECTRAL loss FIRST and the oscillator is brought in afterward (for the harmonics), avoiding the collapse where an easy branch grabs the loss early and knocks the oscillator out of the pitch basin. At osc_gain=0 the spectral loss sends no gradient to the oscillator/tract/envelope (the TF anchor still idles the drives in a learnable basin). Threaded through the training forward, train.py ramp, eval autonomous reconstruction, autonomy_score, and the monitor (per-epoch osc_gain from the ramp schedule, like noise_gain; title shows osc_gain). Verified: mel loss finite + differs from linear + grads flow; osc_gain=1 gives full oscillator gradient (~89), osc_gain=0 drops the spectral-path gradient to 0 (residual is the TF anchor); end-to-end monitor render + score with the osc path works. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
During oscillator warmup (osc_gain=0) the source row now reads ~0 (flat waveform, empty spectrogram), reflecting that the oscillator isn't contributing to the output yet -- consistent with the auto row (= rumble+noise only) and the osc_gain title tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
The rumble low-pass passes 0 Hz and the MRSTFT loss ignores the 0-freq bin, so the rumble accumulated an un-penalized DC offset (a pure additive offset on the output). Subtract its per-sample mean in rumble_branch (both training + eval use it), matching the oscillator source's DC-drop. NOTE: this zeroes the 0-freq term but does NOT remove the STFT edge frames' broadband spike -- that comes from the nonzero signal value at the window boundary, not the DC, and is a cosmetic STFT boundary effect. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…ss (raw target) autonomy_score + integrate_poly_autonomous used correct() (data - butter_lowpass, a high-pass) on BOTH the target and the reconstruction, while the training loss fits the RAW target (only the 0-freq STFT bin dropped) with a mean-subtracted source. Align them: target = raw seg; source detrend = mean removal (matching xg - xg.mean()); ref_rms from raw audio. Now the metric + spectrograms + PSD reflect exactly what the loss optimizes, with the LF modeled by the rumble branch rather than filtered out. NOTE (surfaced by this change): on the RAW org545 target the val metric jumps (autonomy -1.7 -> +0.9 at warmup, osc off) because the raw target is dominated by a broadband noise floor that the noise branch matches ~perfectly + an LF lobe the rumble matches -- i.e. the metric now rewards matching the recording noise, not the (buried) vocal harmonics. Was the point of the high-pass/denoising. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
The time-varying noise gate g(t) was tracking syllabic content (sigma peaked on loud syllables; temporal CV up to 0.8), so it soaks up vocalization energy -- fatal for a 'subtract the noise floor' scheme. --sigma-constant makes the sigma Mamba head mean-pool over time and predict ONE number per segment, forcing g to model a stationary floor level. Threaded through the constructor, save/load, model_cv, and the CLI + manifest. Verified: g is exactly time-invariant (std 0) when set. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
The constant sigma under the mean/log-mel loss parked ~20-50x above the true floor (loudness-weighted average, not floor). --noise-floor-fit replaces the BROADBAND (>= --floor-cutoff-hz) spectral target with a per-freq FLOOR = the --floor-pctile (default 15th) percentile of |STFT(target)| pooled over batch AND time. Batch pooling is key -- a 46ms rollout may be all-syllable, but across 64 segments the quiet inter-syllable frames reveal the floor (minimum statistics). LF bins keep the real time-resolved target so the rumble fits the LF signal. Threaded through mrstft_loss -> spectral_rollout_step -> train -> model_cv -> CLI + manifest. Verified: floor loss finite, grads flow, target switches to floor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…ed cutoff) The broadband noise carried LF energy, so the single shared sigma gate was propped up by the LF fit (rumble band uses the real loud target) -- preventing sigma from converging to the true broadband floor. Now, when the rumble branch is on, the noise is high-passed ABOVE rumble_lowpass_hz (high-pass = x - lowpass(x) at the SAME cutoff), so rumble (lowpass) and noise (highpass) cross over cleanly and ONE parameter (--rumble-lowpass-hz) controls both. LF is owned entirely by the rumble Mamba head; sigma only ever scales the broadband floor. Verified: noise LF/total 0.0116 -> 0.0000 with rumble on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…le-lowpass-hz --floor-cutoff-hz now defaults to -1 = auto (1.5*rumble_lowpass_hz, the rumble's effective upper edge), so ONE user parameter (--rumble-lowpass-hz) controls the whole LF/noise split: rumble low-pass, noise high-pass, and floor-fit boundary. Explicit --floor-cutoff-hz still overrides. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…d percentile) The batch-pooled per-freq percentile floor converged the noise to a SHARED target -- right for vocs whose floor matched the batch average, off-by-a-constant for others (a target problem, per-voc floors vary). New target, per sample: select the QUIET TIME FRAMES (broadband power -- NOT total power, which the LF contaminates -- in the [0.4x,1x]*floor_pctile percentile band) and average their spectra -> a coherent per-sample floor SPECTRUM. Unlike a per-frequency percentile (which stitches a different time frame per bin and sits ~4-6 dB below the mean), the quiet-frame mean is only ~1 dB biased; a floor_correction factor (~1.2, cf MAD->sigma 1.4826) de-biases the residual minimum-statistics pull. The shared noise_tract learns the shape. Broadband bins get this floor; LF keeps the real target. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Add docs/noise_floor_fit.md documenting the noise-floor-fit choices (per-sample quiet-frame floor target, single LF/noise crossover, why quiet-frame-spectrum over per-freq-percentile, the total-power vs per-bin de-bias equivalence) and the principled Gamma-K derivation of --floor-correction with its <1% validation vs simulation. Cross-referenced from the floor block in mrstft_loss. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…-pool The mean-pool reduction for --sigma-constant is loudness-weighted, so it can't estimate a per-voc floor from a loud-syllable voc -- and with per-voc floor targets it collapsed the shared sigma to 0 (noise branch died) by ep2 trying to satisfy the low-floor vocs. The sigma_mamba already reads the full waveform + its reversal (x_in=[flip(z),z], bidirectional); the fix is the REDUCTION: take the 15th percentile over time of the per-time gate. The floor IS the quiet-moment level, so this lets the SSM park syllable loudness in the high percentiles and read the floor off the low ones, and lets different vocs land on different floors. See docs/noise_floor_fit.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…ercentile) The hard 15th-percentile readout gave gradient to a single time sample -> sparse updates that overshot and oscillated (syllD 109->52->0.95->13.8, syllE 46->16->0->6.9) and snapped to 0 when the head zeroed part of the gate. Replace with the mean of the lowest ~20% of the per-time gate: dense gradient over the whole quiet tail, smooth, and it can't collapse to 0 unless the entire tail is 0. Still reads the floor (quiet-moment level), still per-voc. See docs/noise_floor_fit.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
… reduction in gradient) Every scalar reduction of the gate in the forward pass failed: mean-pool collapsed sigma to 0, hard-percentile oscillated (109->0.95->13.8), soft-low-k still fragile -- because the reduction sat in the gradient path and the frames were selected by g(t)'s own moving values. New scheme: keep g(t) TIME-VARYING and mask the BROADBAND spectral loss to the target's QUIET frames (broadband power in [0.4p,p] percentile). There g(t)*noise_tract is regressed onto the real quiet-frame spectra (the floor); loud frames get no broadband gradient so syllables can't drag g up; LF keeps the loss at all frames (rumble). Frames are selected by the TARGET (stable per-voc), not g(t). The constant per-voc readout g0 = mean(g(t)) over the quiet frames x floor_correction is taken POST-HOC, out of the gradient. get_sigma no longer reduces under --sigma-constant. Next: a learned g0 head. docs updated separately. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…s) + masked loss Per user: strip the noise-floor readout to its simplest form. Mamba sees the whole waveform, mean-pools to a single scalar gain on the filtered noise; the spectral loss is masked to the quiet frames, so the gain is trained directly to the floor. No time-varying g(t), no post-hoc readout, no second head -- the scalar IS the readout. Softplus (not relu) so it can approach 0 smoothly for clean vocs rather than dead-relu collapsing (the failure mode of the earlier mean-pool+relu under the unmasked floor-replace loss). The masked-loss regression is what makes this a fair test of whether the Mamba can just learn the quiet-part gain. Time-varying version preserved at 9b58d79 as fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…td (time-domain rumble MSE) Two fixes for the noise-floor-fit stage: (1) --noise-fit-only: the oscillator is frozen+off during this stage, so skip the drives / TF anchor / RK4 rollout entirely and run only the feedforward rumble + filtered noise. No O(H) rollout cost, so the window can be LONG (use --context-len ~0.3s): a 46ms window has no quiet frames for a dense vocalizer (syllD's 46ms quiet-frame floor is 67x its true full-voc floor), so g was fitting an elevated target; a long window contains quiet gaps and the per-segment quiet-frame floor becomes the true floor. (2) --lam-rumble-td: time-domain MSE of the rumble vs the raw LF waveform (lowpass at rumble_lowpass_hz, DC-subtracted). The rumble is deterministic so it CAN phase-align; the magnitude loss can't (phase random -> raw-rumble ADDS power, 4-5x, instead of cancelling). Rumble-only; drop once rumble frozen -> then raw-rumble subtracts cleanly. Verified: noise-fit-only 13k-sample window in 0.43s, no oscillator grad; rumble MSE fires on LF signals with grad to rumble_net. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Fold in everything the runs established: the masked-regression floor loss (scalar softplus gain trained by masking the broadband loss to target-selected quiet frames -- after mean-pool collapse / percentile oscillation showed no in-forward reduction survives the gradient); the long-window --noise-fit-only stage (skip oscillator -> 300ms window so dense vocs' segments contain quiet gaps -> true floor, vs 67x elevated at 46ms); the rumble time-domain MSE (deterministic rumble CAN phase-align; enables clean time-domain subtraction, magnitude-only would ADD LF 4-5x); the residual recipe (rumble time-domain + noise power-domain); [5,15] beats [10,25] as the floor estimator; universal floor shape -> one shared filter; and the K-on-quiet-frames caveat for the self-calibrating correction. Replaces the now-resolved mean-pool 'open issue'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…d (was unapplied + hard-coded) Two bugs the user caught: (1) when the floor loss switched from replacing the target to MASKING it, the floor_correction went dead -- the gain was trained to the raw quiet-frame low-tail with no de-bias, so sparse vocs undershot below their floor. (2) it was a hard-coded scalar (1.2/1.3). Fix: in the floor block, lift the broadband quiet-frame TARGET by C(K, band), where C is the closed-form minimum-statistics correction _floor_C(K, p_lo, p_hi) computed from the [0.4p,p] band and K = mean^2/var of the broadband level over its lower half (~noise; the loud syllables that collapse K are excluded), estimated per batch. --floor-correction default -1 = auto; >0 overrides. Verified C tracks the band (1.21/1.29/1.30 for [10,25]/[6,15]/[5,15] at K=30) and grads flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…t -> 2.3x over-correction) The auto C(K,band) estimated K by pooling the broadband level bbp across the whole batch, so the cross-voc LEVEL differences (recordings vary in floor level) dominated the variance and collapsed K far below its true value -> a huge cmag and a 2.3x over-correction at ep0. Estimate K PER SAMPLE instead (mean^2/var of each voc's own below-median bbp), then per-sample C(K,band). Now K~92 -> cmag~1.15 (near the true magnitude-sum pure-noise DOF ~150-300 -> ~1.10), a sane modest de-bias. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…mble low-pass The single shared crossover (noise HP = rumble LP = 250) leaves a residual bump in 250-375Hz: the rumble rolls off from 250 while the raw still has LF signal there. Add --noise-highpass-hz (default 0 = use rumble_lowpass_hz, unchanged): set noise-HP 250 < rumble-LP 300 to DECOUPLE, giving a 250-300 OVERLAP where the rumble carries the deterministic (phase-matched) LF signal and the noise carries the floor. Threaded through the model, filtered_noise_branch, save/load, model_cv, CLI + manifest. Verified the noise gains power in 250-300 when decoupled. To use next run after corr2 finishes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
Two materially-stale sections fixed: (1) the bias correction is no longer 'not yet wired'/hard-coded -- it is computed live as C(K,band) from the band and a PER-SAMPLE K (mean^2/var of each voc's own below-median broadband level; batch-pooling collapses K), applied by lifting the quiet-frame target before both loss terms; document the two bugs this fixed (correction went dead on the mask switch; hard-coded 1.2/1.3) and the magnitude-sum DOF caveat. (2) new --noise-highpass-hz decouples the noise HP from the rumble LP (rumble-LP 300 / noise-HP 250) to fill the 250-375 crossover residual bump (measured 0.51 -> 0.05 at 300-375, floors unchanged). Update flags + branches + validation notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…round freeze) Stage 2 of the sequential separation trains the oscillator on the residual by freezing the whole learned BACKGROUND (noise gate + noise_tract + rumble head) and resuming a noise-fit checkpoint. freeze_noise_epochs previously covered only sigma+noise_tract; add rumble_mamba+rumble_net so one flag freezes both branches. Set freeze_noise_epochs >= n_epochs to keep the background frozen for the whole oscillator run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…C) rollout The diagnostic panels reused the cold-start autonomy_score trajectories -- silence-lead-in vocs whose IC (audio[0]) is silence -- so the 'source' row was a non-ignited cold-start rollout, not comparable to the target panel and not what training optimizes. Add a second rollout for the PANELS on the same vocs with the ~45ms/2000-sample silence lead-in stripped (audio[0] = onset = the segment data IC training supplies), cold_start=False; keep the cold-start call only for the Val/autonomy deployment METRIC. Drives stay open-loop from the target and the state autonomous -- identical dynamics to teacher_forced_rollout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…ng like a kernel weight) Stage-2 diagnosis: the oscillator's gamma drive runs away negative (bias 0->-0.07, weight collapsing), so the anti-damped state diverges to the +-3 soft clamp -- there's nothing anchoring gamma. Add a penalty lam_gamma * lam**1 * sum_t gamma(t)^2: gamma is the degree-1 linear-damping coefficient (the x'^1 term), so this is the SAME degree-graded-L2 form the kernel weights get, pulling it toward neutral damping so it can't drift into the diverging regime. On its OWN scale (not lam_reg): gamma needs a far larger coefficient to bite than the kernel tolerates without losing its amplitude-bounding nonlinearity. Omega is deliberately NOT regularized (that would drop the natural frequency out of the harmonic band). Threaded through spectral_rollout_step/train/model_cv/CLI + Loss/gamma_reg TB scalar. Default 0 = off (no behavior change). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…h), drop separate lam_gamma Per design call: regularize gamma EXACTLY as a kernel term rather than on a separate scale. gamma is the degree-1 linear-damping coefficient (x'^1), so add lam**1 * sum_t gamma^2 directly into L_reg -> scaled by the same lam_reg and graded identically to the polynomial coefficients. Removes the lam_gamma param/CLI/ threading added in the prior commit. Omega is NOT regularized (would pull the natural frequency down). gamma_reg still logged (Loss/gamma_reg) as the gamma-only slice of L_reg for monitoring. Raising lam_reg now tightens kernel + gamma together (toward the linear, neutral-damping, on-frequency regime). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
…oll CPU cost) The prior data-IC-panels patch kept the cold-start autonomy_score (for the Val/autonomy metric) AND added a second data-IC autonomy_score (for the panels) -> two full-voc integrations per poll, ~2x slower on CPU. Collapse to ONE data-IC call (silence lead-in stripped, cold_start=False) feeding both the Val/* scalars and the panels. Trades away the separate cold-start (ignition) autonomy number; Val/autonomy is now the data-IC autonomy, which matches what the panels show and what training optimizes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
… raw The oscillator is meant to reconstruct the residual after the frozen rumble+noise; the TF acceleration anchor was still teacher-forcing to the RAW audio, so it fought the rumble's LF content. Subtract the (frozen, detached) rumble waveform and its 1st/2nd derivatives -- same 9-pt finite-diff stencil and tau/dt rescaling as the data derivatives -- from the teacher-forced state (x, z2), the target accel (d2x), and recompute the polynomial kernel at the residual state. Gated on use_rumble_branch; no-op otherwise. Verified finite L_tf + drive grads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TbP4euX42eLH7CXGQbwaQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Harmonic-plus-noise: additive filtered-noise branch
Reworks the noise model from inside the ODE (the OU forcing on
v̇, which collapses the limit cycle — γ flips positive as the optimizer offloads generation onto the noise) to a parallel additive branch (DDSP-style), so the deterministic oscillator and the noise can't fight.Architecture (only when
--use-noise-branch):The oscillator carries the tonal/harmonic content; the noise branch supplies the broadband/aperiodic floor. No ODE coupling ⇒ no collapse, no stochastic Heun, no NaN clamps needed for the source.
Why: an inference test (deterministic ckpt_4 + sigma-gated filtered noise) beat both the harmonic-only baseline and the noise-inside-ODE model on RMS-matched multi-res STFT: 2.35 vs 2.86 (harmonic only) / 2.44 (noise-in-ODE), with an oracle ceiling of 1.70 for a fully time-varying filter.
Changes
model.Ouroboros:use_noise_branchbuilds the sigma AM head (shared with the OU mode) + a newnoisefiltMamba head → per-timestep magnitude response overnoise_bands.get_sigmakeys off the head existing; newget_noise_filter.spectral_rollout.filtered_noise_branch: white noise → STFT → per-frame filter (bands interpolated to rFFT bins, phase kept) → iSTFT → sigma AM; added toxgafter the tract, gated by the samenoise_gainramp.Verified: forward finite; gradients reach both the sigma and noise-filter heads;
noise_gain=0is deterministic (dW-independent); save/load round-trips the heads. Off by default; the OU path is untouched.Not yet done: an actual training run (next step); the OU/
enable_noise_forcingpath is left in place for now rather than deleted.🤖 Generated with Claude Code