Repository navigation
Music-reactive visuals (MAD-772) - #54
Merged
Merged
Conversation
A client beat engine turns the playback position into CSS variables (--beat, --downbeat, --swell, --energy, bands) on the elements that use them. The backdrop breathes once a bar, the now-playing art lifts and blooms, the mini-player's glass catches light on the one, and the equalizer follows the beat. Each song gets a backdrop scene (mesh, horizon, ripples, aurora, embers) drawn in its palette. A dev-only beat lab sets the grid by tap tempo until the server sends beat maps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new analysis package works out each song's beat map from its audio:
spectral-flux onsets, tempo from the onset curve's autocorrelation with a
prior around 128 BPM, beats by dynamic programming (Ellis), the downbeat
from where the kicks land, sections from the novelty of a bar-by-bar
self-similarity matrix (Foote), a 12-band spectrum envelope at ~21.5 Hz,
and whole-song features (energy, brightness, dynamics). Pure Go; ffmpeg
only decodes.
Songs are analysed in the background as they're queued and kept per
track in beat_maps (misses too, retried after a week; unused maps swept
after 180 days). GET /rooms/{roomId}/queue/{itemId}/beatmap serves the
map, analysing on the spot if needed, for members, guests and displays.
A song playing from a stand-in service is analysed from that copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…778) The beat engine now takes its beat from the server's beat map: the song's own beats and bars, its section energy and loudness, and a real 12-band spectrum for the equalizer and the horizon backdrop. A grid tapped in the beat lab still wins, for songs the analysis got wrong; songs without a map fall back to a default tempo. useBeatSync feeds the engine from the app shell and the big screen. The beat lab is no longer dev-only: it opens from now playing and shows where the beat comes from, an intensity slider (calm to max), the backdrop scene and, tucked away, tap-to-fix. Settings are kept per device. Auto backdrops now suit the song: flowing scenes for songs with no beat or quiet ones, horizon and ripples for loud driving ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…AD-776..778, MAD-782) The beat lab is back as a floating button on every page (in now playing's header while that's open) and now edits everything about the visuals on this device: an overall switch and intensity, the backdrop scene, each effect's strength or off (backdrop breathing, scene, drift, album art, glass shine, equalizer, lyrics glow, waveform), this device's sound delay (set by hand or measured by tapping along against the beat map), and fixing a song's beat by tapping. Reset everything restores the defaults. The lyric being sung glows on the beat; the seek bar shows the song's waveform from its beat map; flashes never exceed three a second; the app's backdrop scene stops drawing while now playing covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A big visualizer for the TV and the app, in the album's colors and driven by the song's beat map: a WebGL palette fluid (domain-warped noise; bass pushes the flow, the bar swells it, the beat lights its folds, energy sets the speed; resolution adapts so slow TV sticks keep up), a spectrum ring around the art, a starfield that surges on the bar, and the backdrop scenes at full strength. It picks a show to suit each song, or the one you choose. Big screens follow a new room setting, screens: look (stage, visualizer, or auto: the stage, with the visualizer for songs with no words to sing), the show, and intensity. Whoever runs the room sets it from the Big screen dialog; paired screens pick changes up live. In the app, the visualizer opens full screen from now playing or the beat lab; arrows change the show, space plays and pauses, Esc closes, and the controls fade while the mouse rests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fluid drew nothing in development: strict mode mounts it twice, and the first unmount let go of the canvas's WebGL context, which the second mount then couldn't use. Each mount now gets a canvas of its own. The shader also asks for full precision where there is any, since phones run mediump at half precision. The starfield becomes a tunnel of the song's own spectrum: a ring cast in its shape every 30 ms flies at you, with a bright ring on each bar. On the visualizer, ripples become rain (ripples all over, a burst on each bar, the treble flicking tiny ones in between) and embers become sparks (hundreds of glowing embers, a fountain on each bar). The app's backdrop keeps its calmer ripples and embers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every song change passes through the loading state, sometimes more than once, and the engine treated each as a pause: the backdrop, glow and scene faded out and back in, which with now playing open filled the screen. Now the backdrop holds for a second and a half of not playing before it fades, while only the beat's pulses stop at once. The scene and visualizer show stay put until the new song's beat map lands, rather than switching on the song and again on its map; no default tempo is guessed in between; and the waveform keeps its space as a flat line until the map arrives, so the player doesn't jump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A whole-screen black flash on some song changes looks like the browser dropping layers under GPU pressure: two full-screen backdrops (the app's, still composited behind now playing, and now playing's own), each crossfading a hugely blurred album cover, and, when the scene changed too, a second blurred canvas mounting for the crossfade. Scenes now crossfade on the one canvas they already have, and the app's backdrop stops rendering once now playing has fully covered it, coming back as soon as it starts to close. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An opt-in full-screen wash of the song's palette that flashes on each downbeat. Every phone reads the same synced playback clock and beat map, so a room of them flashes together and changes color bar by bar. Flashes are once a bar, with the engine's capped beat pulse faint underneath; reduced motion or visuals off hold a steady glow. Opens from now playing's header and the beat lab; keeps the screen awake. The wake lock moves out of the TV stage into hooks/use-wake-lock.ts, and beat frames now carry the beat's place in its bar. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heir shape Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A speaker that went quiet made three songs time out, which stopped the room, and a stopped room had no play button. Now a song that never starts on a speaker that said nothing about it pauses instead of skipping (the speaker acknowledges each song it loads), and the room shows a play button whenever it's stopped with songs waiting. Pressing play with no speaker makes this device the speaker and starts the room in one tap; "Play on this phone" starts a paused room too. Tap a queued song's artwork to play it now. It takes the skip permission; in a room that votes on skips, you ask the room and everyone gets a card to agree, passing on the same share as a skip vote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opening the big screen on the device that was the speaker left the signed-in app, which released the speaker and paused the room, and the TV had no way to play again. Now leaving the app or the big screen hands the speaker over for a few seconds instead of letting go, and the next screen keeps it playing. The big screen gets play/pause and skip. Play with no speaker makes the screen the speaker and starts the room in one press, and "Play the audio here" starts a paused room. A paired screen with audio on may play, pause and skip as whoever paired it. The mouse pointer shows while it moves, so the buttons can be clicked on a desktop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pressing play on a queued song with no speaker said "No speaker yet". Now the tap makes this device the speaker and plays that song. Claiming takes an optional playItemId, so the server starts the chosen song directly instead of starting the next in fair order and then skipping it (which would have logged a skip against someone else's song). In a room that votes, the room starts as usual and the request goes out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On shorter screens a two-line title made the now-playing column taller than the screen, and centering it clipped the top of the artwork and the times at the bottom. The artwork now shrinks to the height that's left, so the title, who queued it and the times always fit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 9: music-reactive visuals.
Testing
npm run typecheck,npm run lint,npm test(178 passing)🤖 Generated with Claude Code