Skip to content

track.start shifts the muxed audio but not the analysis audio-reactive components read #190

Description

@LeadcodeDev

Symptom

An audio track with start: 73 plays the file from the beginning at t=73 in the video. A waveform or audio_spectrum at t=73 draws the file's content at 73 seconds, not at 0. Picture and sound disagree by exactly start.

Cause

Two independent time bases:

  • crates/rustmotion/src/encode/audio.rs places a track at track.start * TARGET_SAMPLE_RATE on the scenario timeline and copies the file from its own sample 0.
  • crates/rustmotion/src/encode/audio_analysis.rs analyses the raw file, indexed from its own frame 0, and the painters call amplitude_at(ctx.time) with scenario time. track.start appears nowhere in that file.

So the mapping is 1:1 only when start == 0. end has the same issue in reverse: the analysis keeps running past the point where the track has been cut.

Practical consequence

Gating a soundtrack to one scene is a natural thing to want, and the obvious way to write it — start/end — silently desynchronises the visualisation. The workaround is volume_keyframes, which leaves the time mapping intact but is not what the fields suggest.

Proposed fix

  1. Offset the lookup by the track's placement: the painters resolve a track (by style.audio-reactive.track, or the first cached entry), so the same resolution can carry start, and read amplitude_at(ctx.time - track.start).
  2. Return 0 outside [start, end) so a visualisation goes flat exactly when the track is silent, instead of drawing an envelope for audio nobody hears.
  3. volume_keyframes and the fades are a separate question — the analysis is of the source, not of the mix. Worth deciding explicitly and documenting either way, since feat(studio): play the scenario's audio while previewing #182 made the studio play the mixed output, so a viewer now hears the envelope while the waveform draws the raw one.

A test with a track at start: 1.0 whose first second is silence, asserting the visualisation is flat over [0,1) and active after, pins it.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions