From 00f147b30e92d61ec657c5b5855f540201e7b66f Mon Sep 17 00:00:00 2001 From: Kirari Sawamura Date: Sun, 27 Sep 2026 22:15:17 +0900 Subject: [PATCH] docs: refresh README presentation and onboarding --- README.md | 295 +++++++++++------------------------- docs/assets/chroma-hero.svg | 41 +++++ docs/guide.md | 282 ++++++++++++++++++++++++++++++++++ docs/readme-design.md | 48 ++++++ 4 files changed, 460 insertions(+), 206 deletions(-) create mode 100644 docs/assets/chroma-hero.svg create mode 100644 docs/guide.md create mode 100644 docs/readme-design.md diff --git a/README.md b/README.md index eaaed80..10f282d 100644 --- a/README.md +++ b/README.md @@ -1,276 +1,159 @@ -# Codex Micro Chroma - -Codex Micro Chroma is a local, audio-reactive lighting daemon for macOS, written in Rust. It extracts a representative colour from the artwork or thumbnail shown in macOS Now Playing and uses it to light the outer LED ring of the Work Louder Codex Micro. +
-It is not tied to Apple Music, Spotify, or any other service. Any music player, video player, or browser that publishes its current media and artwork through macOS Now Playing can use the same integration. +# Codex Micro Chroma -Lighting effects and brightness respond dynamically to the audio's musical dynamics, frequency balance, timbre, transients, rhythmic pulse, and stereo width. Everything is processed on your Mac: there is no OpenAI or Codex model usage, no Spotify or Apple Music API, no API key, no OAuth flow, and no external server. +**Your artwork. Your audio. Your desk, in colour.** -## How it works +Turn your Work Louder Codex Micro's LED ring into an extension of what you're playing. +Artwork sets the colour. Sound shapes the motion. Everything runs on your Mac. -```text -macOS Now Playing (Music / Spotify / browser / other players) - -> macOS MediaRemote - -> ad-hoc signed embedded per-player session helper - -> active-session arbitration from macOS-owned state - -> local image decoding and representative-colour extraction - -> artwork colour +[![Release](https://img.shields.io/github/v/release/hauntedfail/Codex-Micro-Chroma?style=flat-square&color=f2a577)](https://github.com/hauntedfail/Codex-Micro-Chroma/releases/latest) [![CI](https://github.com/hauntedfail/Codex-Micro-Chroma/actions/workflows/ci.yml/badge.svg)](https://github.com/hauntedfail/Codex-Micro-Chroma/actions/workflows/ci.yml) [![macOS](https://img.shields.io/badge/macOS-14.2%2B-8583ed?style=flat-square)](#requirements) [![MIT licence](https://img.shields.io/badge/licence-MIT-a6d9bb?style=flat-square)](LICENSE) -macOS system output - -> public Core Audio Process Tap - -> local Rust DSP (loudness / bands / centroid / flatness / flux / onset / - stereo width / pulse / tempo confidence) - -> LightingComposer (effect state machine / smoothing / hysteresis) - -> persistent hidapi session - -> Codex Micro v.oai.rgbcfg RPC -``` +[Get started](#get-started) · [Effects](#lighting-effects) · [Compatibility](#compatibility) · [Technical guide](docs/guide.md) -MediaRemote is a private Apple framework. Its behaviour may change after a macOS update, and this architecture is not suitable for App Store distribution. If a player does not publish artwork to Now Playing, Codex Micro Chroma cannot derive a colour from it. +![Concept illustration: artwork colour and an audio waveform combine into a glowing LED ring. This is an illustration, not a hardware recording.](docs/assets/chroma-hero.svg) -When several applications publish Now Playing sessions simultaneously, Codex Micro Chroma enumerates the current OS sessions on every refresh and considers only sessions that macOS reports as playing with a resolved playback state. A per-session scoped-playing error leaves only that session unresolved and ineligible for selection. A completed metadata failure still publishes the authoritative playing session identity without stale track fields or artwork, so previous lighting is cleared while macOS continues reporting that session as active. A globally timed-out refresh is treated as a broken helper generation: the helper emits a reset control message, Rust clears the last Now Playing state, and the helper replaces its own process image before publishing a fresh ready message. The session with the newest macOS `lastPlayingDate` wins; the OS-elected Now Playing session breaks equal or unavailable-date ties, followed by a stable identifier for deterministic output. No separate playback-order history is persisted by this application. If the selected session stops, its lighting is cleared and the next still-playing session is selected from that same OS snapshot. If none remains, lighting stays off. The helper gathers identity, scoped playing state, election, and `lastPlayingDate` for the complete MediaRemote session list in fixed eight-client batches while retaining only the top eight ranked playing records. Clients without scoped playing support use a no-artwork metadata fallback in that same bounded batch pipeline to resolve `playbackRate`. A second phase fetches metadata and artwork only for the final top eight, and only those selected candidates are published. If more than eight sessions are simultaneously playing, lower-ranked sessions wait for a later snapshot after higher-ranked sessions stop or move down. +
-MediaRemote payloads are treated as untrusted local input. The helper bounds copied text fields before UTF-8 allocation, per-item raw artwork, aggregate retained artwork cache bytes, and aggregate serialized artwork. Rust bounds each helper JSON line, rejects oversized encoded or decoded artwork before image decoding, and decodes images with explicit dimension and allocation limits. Oversized or invalid artwork is omitted gracefully while the authoritative playback candidate is still processed. +Reading this from a release archive? [Open the illustrated README and full guide on GitHub](https://github.com/hauntedfail/Codex-Micro-Chroma#readme). -## Requirements +## From Now Playing to your desk -- macOS 14.2 or later for reactive mode; `--mode static` does not require a Process Tap -- A music player, video player, or browser that publishes artwork to macOS Now Playing -- A connected Work Louder Codex Micro -- Rust 1.88 or later -- Xcode Command Line Tools +- **A colour for every track.** Picks a representative colour from the artwork or video thumbnail in macOS Now Playing, then adjusts it for the LEDs. +- **Light that moves with the audio.** Loudness, bass, rhythm, and stereo width shape the brightness, speed, and choice of effect. +- **Use the player you already have.** Reads macOS Now Playing, with no music-service account to connect. Artwork availability depends on the player; see [compatibility](#compatibility). +- **Keep it local.** No runtime network connection, API keys, OAuth, or AI models. Raw audio stays in memory; diagnostic logs remain on your Mac. +- **Make it part of your setup.** Run in the terminal or install a background worker that starts at login. Choose a fixed effect with static mode when you only want artwork colours. -## Compatibility and current status +## Get started -The integration is service-agnostic, but artwork compatibility ultimately depends on each player publishing an image to macOS Now Playing. The browser/Helium path has been verified. Apple Music and Spotify artwork and colour output still require validation on the target Mac, as do long-running reconnect behaviour and final effect calibration on physical Codex Micro hardware. +### Requirements -Reactive analysis listens to the complete mixed macOS system output. If several applications play audio at once, the effects respond to that combined output while the base colour comes from the active session selected by the arbitration above. +You need **macOS 14.2 or later**, **one connected Work Louder Codex Micro**, and a player that publishes artwork to macOS Now Playing. The release binary supports Apple Silicon and Intel; Rust is only needed to build from source. -Exactly one HID interface matching the Codex Micro is required. No matching device, more than one matching interface, or missing Input Monitoring access leaves `run` waiting for the controller and causes one-shot commands such as `probe` or `set` to report an error. +### 1. Download -## Download +Get the universal macOS archive and its matching `.sha256` file from **[GitHub Releases](https://github.com/hauntedfail/Codex-Micro-Chroma/releases/latest)**. The latest published release is **v0.1.0**. -The current release is **v0.1.0**. Download the universal macOS archive and its SHA-256 checksum from GitHub Releases: +
+Download and verify from the terminal ```bash VERSION=0.1.0 BASE_URL="https://github.com/hauntedfail/Codex-Micro-Chroma/releases/download/v${VERSION}" ARCHIVE="codex-micro-chroma-v${VERSION}-macos-universal.tar.gz" -curl -LO "${BASE_URL}/${ARCHIVE}" -curl -LO "${BASE_URL}/${ARCHIVE}.sha256" -shasum -a 256 -c "${ARCHIVE}.sha256" -tar -xzf "${ARCHIVE}" -cd "codex-micro-chroma-v${VERSION}-macos-universal" +curl -fLO "${BASE_URL}/${ARCHIVE}" && +curl -fLO "${BASE_URL}/${ARCHIVE}.sha256" && +shasum -a 256 -c "${ARCHIVE}.sha256" && +tar -xzf "${ARCHIVE}" && +cd "codex-micro-chroma-v${VERSION}-macos-universal" && ./codex-micro-chroma --version ``` -The release binary supports both Apple Silicon and Intel Macs. It is ad-hoc signed but is not Developer ID signed or notarised, so macOS may ask you to confirm the first launch in **System Settings > Privacy & Security**. - -To install the downloaded binary as a per-user LaunchAgent: - -```bash -./codex-micro-chroma install -``` - -## Build from source +
-Build the release binary from the project root: +The binary is ad-hoc signed, **not Developer ID signed or notarised**. macOS may require first-launch approval in **System Settings → Privacy & Security**. -```bash -cargo build --release -``` +### 2. Connect and check -Check the local MediaRemote and HID connections, inspect the current artwork colour, and verify system-audio analysis: +Connect your Codex Micro and play something with Now Playing artwork. From the extracted release directory, run: ```bash -./target/release/codex-micro-chroma probe -./target/release/codex-micro-chroma status -./target/release/codex-micro-chroma audio-probe --seconds 15 +./codex-micro-chroma probe +./codex-micro-chroma status ``` -## Required user interaction and permissions - -macOS privacy permissions cannot be granted by the installer or from the command line. You must approve them in System Settings. Permissions are associated with the executable that requests access, so a command run from Terminal and the installed background worker may need separate approval. - -1. Connect one Codex Micro to the Mac. -2. Run `./target/release/codex-micro-chroma probe`. -3. If HID access is reported as `not permitted`, open **System Settings > Privacy & Security > Input Monitoring**, add and enable the terminal application you are using, then run `probe` again. -4. Start some media so that macOS Now Playing contains a current item and artwork, then run `status`. -5. Run `audio-probe --seconds 15` or start `run` in reactive mode. When macOS asks, approve **Screen & System Audio Recording**, or **System Audio Recording Only**, depending on your macOS version. - -The embedded `NSAudioCaptureUsageDescription` explains that system audio is analysed locally to animate the LED ring. Audio is converted into feature values in memory; it is never recorded, saved, or sent over the network. +`probe` checks the device and MediaRemote connection. `status` shows the current media and detected artwork colour. -`run` retries HID access every two seconds until the device and Input Monitoring permission become available. In reactive mode it also retries the Process Tap every two seconds until System Audio Recording permission is available, so you do not normally need to restart it after granting access. `audio-probe`, `probe`, `status`, `set`, and `off` are one-shot commands; correct the permission or device problem and run the command again. +Approve the relevant permissions in **System Settings → Privacy & Security**: -Use `--mode static` if you do not want to grant system-audio access. Static mode still needs HID access and Now Playing artwork, but it does not start a Core Audio Process Tap. +| Permission | Used for | +| --- | --- | +| **Input Monitoring** | Access to the Codex Micro over USB HID. If `probe` reports `not permitted`, enable your terminal app and retry. | +| **Screen & System Audio Recording** or **System Audio Recording Only** | Local audio analysis in reactive mode. macOS requests this when audio capture starts. | -### Start automatically at login - -Install and start the per-user LaunchAgent: +### 3. Turn on the lights ```bash -./target/release/codex-micro-chroma install -``` - -The command copies the release binary into Application Support, gives it an ad-hoc signature with the fixed identifier `com.local.codex-micro-chroma`, and registers an Aqua-session LaunchAgent. The installed worker is located at: - -```text -~/Library/Application Support/CodexMicroChroma/codex-micro-chroma +./codex-micro-chroma run ``` -If the system-audio permission prompt appears immediately after installation, grant it. If you dismiss it, add and enable the worker shown above in **System Settings > Privacy & Security > Screen & System Audio Recording**, then run `install` again. - -The LaunchAgent cannot approve its own privacy prompts. If the background worker is waiting, manually add and enable this installed executable for both permissions that are required on your Mac: - -- **Input Monitoring**, for Codex Micro HID access -- **Screen & System Audio Recording** or **System Audio Recording Only**, for reactive audio analysis - -After changing a permission, check `~/Library/Logs/CodexMicroChroma/worker-error.log`. The worker normally resumes through its built-in retry loop. If macOS does not apply the change to the running worker, run `install` again to restart the LaunchAgent. - -An ad-hoc signature's designated requirement includes the binary's CDHash. After rebuilding and reinstalling, macOS may therefore ask you to approve Input Monitoring or System Audio Recording again. If necessary, switch the installed worker's permission off and on. A distribution build signed with a Developer ID or local code-signing certificate can avoid this repeated approval. - -Worker and track logs are stored in `~/Library/Logs/CodexMicroChroma/`. - -To stop the LaunchAgent and remove the installed copy: +The ring follows the artwork colour while effects respond to the audio. Stop with **Ctrl-C** to clear the lighting. Prefer an artwork colour with a fixed effect and no system-audio capture? ```bash -~/Library/Application\ Support/CodexMicroChroma/codex-micro-chroma uninstall +./codex-micro-chroma run --mode static --effect breath ``` -`uninstall` removes the LaunchAgent plist and installed executable, but preserves diagnostic logs. - -## Usage - -### Follow the current media - -Start the default reactive mode: - -```bash -./target/release/codex-micro-chroma run -``` - -- When the player or media changes, the new artwork is analysed automatically. -- A thumbnail fingerprint is used as the content identity when no title is available, and artwork changes are detected even when the title remains the same. -- White backgrounds and transparent pixels are excluded before the dominant colour is adjusted to be brighter and more vivid on the LEDs. -- The previous media colour is cleared while new artwork is pending. -- Relative loudness controls brightness, combined musical motion controls speed, and spatial width and change control the device's `magic` parameter. -- Brief onset, flux, pulse, and bass events are held with attack-and-release envelopes so that they can pass the 650 ms candidate dwell. Hysteresis preserves close-scoring candidates, allowing a beat to trigger a dynamic effect such as `snake` even if it has disappeared by the next audio frame. -- Effects have a two-second minimum hold to prevent flicker. If the same pattern remains the strongest candidate for more than 12 seconds, a semantically close runner-up may take over at a phrase boundary to avoid becoming stuck on a single effect such as `breath`. `rainbow` has a 12-second cooldown. -- Speed and `magic` use effect-specific profiles: `solid` remains still, `snake` follows the beat, `gradient` reflects stereo width, `rainbow` marks a climax, and the two breathing effects use different depths. A brief rest does not switch the ring off. -- A persistent HID connection is reused and updated every 100 ms by default. If the device or Input Monitoring permission is unavailable at start-up, the worker waits rather than exiting. It automatically reconnects once after a disconnection, response timeout, or other transport failure. -- If Process Tap frames stop for three seconds during playback, or remain completely zero-filled for 15 seconds, the tap and aggregate device are rebuilt with a 30-second recovery cooldown. -- Control-C or SIGTERM stops the process and switches the LEDs off. - -The default values can be set explicitly: - -```bash -./target/release/codex-micro-chroma run \ - --mode reactive \ - --brightness 1.0 \ - --poll-ms 250 \ - --device-ms 100 -``` - -To follow artwork while keeping one fixed effect: - -```bash -./target/release/codex-micro-chroma run \ - --mode static \ - --effect breath \ - --brightness 1.0 \ - --speed 0.85 \ - --magic 0.0 \ - --refresh-ms 750 -``` - -### Test the lighting directly - -Set a colour without reading Now Playing, then switch the lighting off. The default effect for both `set` and `run` is `breath`. - -```bash -./target/release/codex-micro-chroma set --color '#33AAFF' -./target/release/codex-micro-chroma set --color '#33AAFF' --effect snake -./target/release/codex-micro-chroma off -``` - -`off` clears both the key lighting and ambient-ring lighting controlled by this tool. - -### Inspect audio features - -`audio-probe` does not write to the LEDs. It prints `AudioFeatureFrame` values calculated from the Process Tap as JSON Lines, which is useful for permission checks and algorithm tuning. - -```bash -./target/release/codex-micro-chroma audio-probe --seconds 15 -``` +For login startup, stop the foreground process and run `./codex-micro-chroma install`. This starts a background worker in the default reactive mode. The **installed worker may need its own privacy approvals**; follow the [background setup guide](docs/guide.md#start-automatically-at-login). ## Lighting effects -All currently known Codex Micro effects can be selected. +Reactive mode chooses effects from the audio's character, with smoothing and minimum hold times to avoid rapid switching. These are selection tendencies, not a fixed mapping for every track. -| CLI name | Device value | Role in reactive mode | -| --- | ---: | --- | -| `off` | 0 | Sustained silence or stopped playback | -| `solid` | 1 | Speech or direct, centre-panned sound | -| `snake` | 2 | Strong bass, pulse, or rhythmic periodicity | -| `rainbow` | 3 | A climax combining high level, strong flux, and pronounced onsets | -| `breath` | 4 | Smooth, sustained, tonal material | -| `gradient` | 5 | Wide stereo material | -| `shallow-breath` | 6 | Quiet passages, introductions, and outros | +| What you're hearing | Effect | +| --- | --- | +| Strong bass and a steady pulse | `snake` | +| Smooth, sustained tones | `breath` | +| Wide stereo sound | `gradient` | +| Loud, rapidly changing passages with strong onsets | `rainbow` | +| Quiet passages, introductions, and outros | `shallow-breath` | +| Speech or centred sound | `solid` | +| Sustained silence or stopped playback | `off` | -The shared `--brightness`, `--speed`, and `--magic` parameters each accept values from 0 to 1. Their visible behaviour and useful combinations depend on the device firmware's implementation of each effect. +Try a colour and effect directly, without a media player. Stop any running worker first so it does not overwrite your selection: -## Per-track effect logs - -`run` follows the Now Playing `elapsed_time`, `duration`, and `playback_rate` values and automatically writes one JSON Lines log per track: - -```text -~/Library/Logs/CodexMicroChroma/tracks/-.jsonl +```bash +./codex-micro-chroma set --color '#F2A577' --effect snake +./codex-micro-chroma off ``` -- `track_start`: media metadata, source application, duration, and observed starting position -- `effect_transition`: position, effect, colour, brightness, speed, `magic`, and all audio features at that moment -- `track_summary`: time and percentage per effect, transition count, observed time, and completion reason +`off` clears both the key lighting and ambient ring controlled by this tool. See the [usage reference](docs/guide.md#usage) for brightness, speed, timing, and audio diagnostics. -`complete_track` is `true` only when observation began within the first two seconds, ended within the final two seconds, and covered at least 85% of the media's duration. Accounting pauses with playback and resumes in the same log when the same media continues. +## Compatibility -If a repeat or crossfade causes the position to jump backwards to within the first five seconds, the current pass is finalised with `position_restarted` and the next pass is written to a separate log. The previous pass still counts as complete if at least 85% of its duration was observed. Records are flushed on every transition, so a live log can be inspected with `tail -f`. +**Early-stage hardware project.** The integration reads macOS Now Playing rather than individual service APIs, but each player must publish usable artwork. -MediaRemote elapsed time is anchored to a local monotonic clock when each event arrives. This prevents time spent paused from being added to the media position if MediaRemote returns an old timestamp after a long pause. +| Path | Validation status | +| --- | --- | +| Browser / Helium | Verified in the project's existing testing | +| Apple Music / Spotify | Artwork and colour output still need validation on the target Mac | +| Other music players, video players, and browsers | Depend on Now Playing artwork; not individually verified | +| Extended sessions and hardware effect tuning | Long-running reconnect behaviour and final calibration still need validation | -## Privacy and security +When multiple apps play at once, the artwork colour comes from the selected Now Playing session. **Reactive effects analyse the full mixed system output**, so they can respond to audio from other apps too. -- No OpenAI, Codex, Spotify, Apple Music, or other external API is used. -- No network connection is made at runtime. -- System Integrity Protection is not changed. -- Root access, `sudo`, and code injection are not used. -- Process Tap PCM is not written to a file. -- Playback is observed but never controlled. -- HID access opens exactly one interface matching the Codex Micro VID, PID, and usage page. -- An inter-process lock serialises concurrent LED writes. -- The installed worker uses a fixed identifier and an ad-hoc signature, not a Developer ID certificate. +Now Playing access uses Apple's private **MediaRemote** framework. macOS updates may affect it, and this approach is not suitable for App Store distribution. See [how it works](docs/guide.md#how-it-works) for session selection and input limits. -## Implementation notes +## Need a hand? -`CATapDescription` is an Objective-C API, so only Process Tap and aggregate-device creation and destruction are isolated in a small Objective-C bridge. After the bridge delivers Float32 PCM, the FFT, feature extraction, relative normalisation, effect scoring, state transitions, and HID control are implemented in Rust. +| Symptom or task | Where to start | +| --- | --- | +| Device access fails or the worker waits | Connect one device and check [permissions](docs/guide.md#required-user-interaction-and-permissions). | +| No artwork colour | Run `status`; check whether the current player publishes artwork. | +| Colour works, but audio response does not | Run `audio-probe --seconds 15` and check system-audio permission. | +| Terminal works, login startup does not | Approve the [installed worker](docs/guide.md#start-automatically-at-login), then inspect `worker-error.log`. | +| Stop and remove the login worker | Use the [uninstall instructions](docs/guide.md#start-automatically-at-login); diagnostic logs are preserved. | -The audio callback sends fixed-size packets to a bounded channel with `try_send`; FFT work and locking never run on Core Audio's real-time thread. +## Build and contribute -## Development +With **Rust 1.88+** and **Xcode Command Line Tools** installed: ```bash -cargo fmt --all --check -cargo clippy --all-targets --all-features -- -D warnings -cargo test --all-targets --all-features +git clone https://github.com/hauntedfail/Codex-Micro-Chroma.git +cd Codex-Micro-Chroma +cargo build --release --locked +./target/release/codex-micro-chroma --help ``` -## Publishing a release +Use `./target/release/codex-micro-chroma` in place of `./codex-micro-chroma` in the examples above. -The `Release macOS binary` GitHub Actions workflow builds an ad-hoc-signed universal binary, creates a checksum, and publishes both files in a new GitHub Release. The requested version must match the `version` in `Cargo.toml`. +Have a Codex Micro? [Report a player test or a bug](https://github.com/hauntedfail/Codex-Micro-Chroma/issues), or share a short clip of your lighting setup. Include your macOS version, player/browser, Chroma version, and what you expected to see. Review diagnostic logs before sharing: they can contain media titles and source applications. -To publish manually, open **Actions > Release macOS binary > Run workflow** and enter a version without the leading `v`, such as `0.1.0`. Alternatively, push a matching tag such as `v0.1.0`. +Code contributions are welcome. See the guide for [development checks](docs/guide.md#development), [per-track logs](docs/guide.md#per-track-effect-logs), and [release publishing](docs/guide.md#publishing-a-release). -An existing release or manually supplied existing tag is never overwritten. Update `Cargo.toml` and the version shown in the Download section before publishing the next version. +If this belongs on your desk, **give the project a star** so you can find it again. ## Licence -Codex Micro Chroma is available under the [MIT Licence](LICENSE). +[MIT](LICENSE). “Codex Micro” refers to the Work Louder hardware; no OpenAI or Codex model is used. diff --git a/docs/assets/chroma-hero.svg b/docs/assets/chroma-hero.svg new file mode 100644 index 0000000..b1f57ac --- /dev/null +++ b/docs/assets/chroma-hero.svg @@ -0,0 +1,41 @@ +<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="480" viewBox="0 0 1200 480" role="img" aria-labelledby="title desc"> + <title id="title">Artwork becomes colour. Audio becomes motion. + An abstract orange and violet artwork flows through a waveform into a glowing ring. Concept illustration of Codex Micro Chroma, not a hardware recording. + + + + + + + + + + + CODEX MICRO CHROMA + ARTWORK × AUDIO + + + + + + + + + + + + + + + + + COLOUR + IN MOTION + + Pick up the colour. + Follow the sound. + Light up your desk. + + CONCEPT ILLUSTRATION · NOT A HARDWARE RECORDING + + diff --git a/docs/guide.md b/docs/guide.md new file mode 100644 index 0000000..b0196f3 --- /dev/null +++ b/docs/guide.md @@ -0,0 +1,282 @@ +# Codex Micro Chroma: technical guide + +[Back to the project overview](../README.md) + +Setup, permissions, command examples, and implementation details. Commands using +`./target/release/` assume a source build from the repository root; for a downloaded +release, use `./codex-micro-chroma` from the extracted directory instead. + +Codex Micro Chroma is a local, audio-reactive lighting daemon for macOS, written in Rust. It extracts a representative colour from the artwork or thumbnail shown in macOS Now Playing and uses it to light the outer LED ring of the Work Louder Codex Micro. + +It is not tied to Apple Music, Spotify, or any other service. Any music player, video player, or browser that publishes its current media and artwork through macOS Now Playing can use the same integration. + +Lighting effects and brightness respond dynamically to the audio's musical dynamics, frequency balance, timbre, transients, rhythmic pulse, and stereo width. Everything is processed on your Mac: there is no OpenAI or Codex model usage, no Spotify or Apple Music API, no API key, no OAuth flow, and no external server. + +## How it works + +```text +macOS Now Playing (Music / Spotify / browser / other players) + -> macOS MediaRemote + -> ad-hoc signed embedded per-player session helper + -> active-session arbitration from macOS-owned state + -> local image decoding and representative-colour extraction + -> artwork colour + +macOS system output + -> public Core Audio Process Tap + -> local Rust DSP (loudness / bands / centroid / flatness / flux / onset / + stereo width / pulse / tempo confidence) + -> LightingComposer (effect state machine / smoothing / hysteresis) + -> persistent hidapi session + -> Codex Micro v.oai.rgbcfg RPC +``` + +MediaRemote is a private Apple framework. Its behaviour may change after a macOS update, and this architecture is not suitable for App Store distribution. If a player does not publish artwork to Now Playing, Codex Micro Chroma cannot derive a colour from it. + +When several applications publish Now Playing sessions simultaneously, Codex Micro Chroma enumerates the current OS sessions on every refresh and considers only sessions that macOS reports as playing with a resolved playback state. A per-session scoped-playing error leaves only that session unresolved and ineligible for selection. A completed metadata failure still publishes the authoritative playing session identity without stale track fields or artwork, so previous lighting is cleared while macOS continues reporting that session as active. A globally timed-out refresh is treated as a broken helper generation: the helper emits a reset control message, Rust clears the last Now Playing state, and the helper replaces its own process image before publishing a fresh ready message. The session with the newest macOS `lastPlayingDate` wins; the OS-elected Now Playing session breaks equal or unavailable-date ties, followed by a stable identifier for deterministic output. No separate playback-order history is persisted by this application. If the selected session stops, its lighting is cleared and the next still-playing session is selected from that same OS snapshot. If none remains, lighting stays off. The helper gathers identity, scoped playing state, election, and `lastPlayingDate` for the complete MediaRemote session list in fixed eight-client batches while retaining only the top eight ranked playing records. Clients without scoped playing support use a no-artwork metadata fallback in that same bounded batch pipeline to resolve `playbackRate`. A second phase fetches metadata and artwork only for the final top eight, and only those selected candidates are published. If more than eight sessions are simultaneously playing, lower-ranked sessions wait for a later snapshot after higher-ranked sessions stop or move down. + +MediaRemote payloads are treated as untrusted local input. The helper bounds copied text fields before UTF-8 allocation, per-item raw artwork, aggregate retained artwork cache bytes, and aggregate serialized artwork. Rust bounds each helper JSON line, rejects oversized encoded or decoded artwork before image decoding, and decodes images with explicit dimension and allocation limits. Oversized or invalid artwork is omitted gracefully while the authoritative playback candidate is still processed. + +## Requirements + +- macOS 14.2 or later; `--mode static` does not require a Process Tap or system-audio permission +- A music player, video player, or browser that publishes artwork to macOS Now Playing +- A connected Work Louder Codex Micro +- Rust 1.88 or later (source builds only) +- Xcode Command Line Tools (source builds only) + +## Compatibility and current status + +The integration is service-agnostic, but artwork compatibility ultimately depends on each player publishing an image to macOS Now Playing. The browser/Helium path has been verified. Apple Music and Spotify artwork and colour output still require validation on the target Mac, as do long-running reconnect behaviour and final effect calibration on physical Codex Micro hardware. + +Reactive analysis listens to the complete mixed macOS system output. If several applications play audio at once, the effects respond to that combined output while the base colour comes from the active session selected by the arbitration above. + +Exactly one HID interface matching the Codex Micro is required. No matching device, more than one matching interface, or missing Input Monitoring access leaves `run` waiting for the controller and causes one-shot commands such as `probe` or `set` to report an error. + +## Download + +The current release is **v0.1.0**. Download the universal macOS archive and its SHA-256 checksum from GitHub Releases: + +```bash +VERSION=0.1.0 +BASE_URL="https://github.com/hauntedfail/Codex-Micro-Chroma/releases/download/v${VERSION}" +ARCHIVE="codex-micro-chroma-v${VERSION}-macos-universal.tar.gz" + +curl -fLO "${BASE_URL}/${ARCHIVE}" && +curl -fLO "${BASE_URL}/${ARCHIVE}.sha256" && +shasum -a 256 -c "${ARCHIVE}.sha256" && +tar -xzf "${ARCHIVE}" && +cd "codex-micro-chroma-v${VERSION}-macos-universal" && +./codex-micro-chroma --version +``` + +The release binary supports both Apple Silicon and Intel Macs. It is ad-hoc signed but is not Developer ID signed or notarised, so macOS may ask you to confirm the first launch in **System Settings > Privacy & Security**. + +To install the downloaded binary as a per-user LaunchAgent: + +```bash +./codex-micro-chroma install +``` + +## Build from source + +Build the release binary from the project root: + +```bash +cargo build --release +``` + +Check the local MediaRemote and HID connections, inspect the current artwork colour, and verify system-audio analysis: + +```bash +./target/release/codex-micro-chroma probe +./target/release/codex-micro-chroma status +./target/release/codex-micro-chroma audio-probe --seconds 15 +``` + +## Required user interaction and permissions + +macOS privacy permissions cannot be granted by the installer or from the command line. You must approve them in System Settings. Permissions are associated with the executable that requests access, so a command run from Terminal and the installed background worker may need separate approval. + +1. Connect one Codex Micro to the Mac. +2. Run `./target/release/codex-micro-chroma probe`. +3. If HID access is reported as `not permitted`, open **System Settings > Privacy & Security > Input Monitoring**, add and enable the terminal application you are using, then run `probe` again. +4. Start some media so that macOS Now Playing contains a current item and artwork, then run `status`. +5. Run `audio-probe --seconds 15` or start `run` in reactive mode. When macOS asks, approve **Screen & System Audio Recording**, or **System Audio Recording Only**, depending on your macOS version. + +The embedded `NSAudioCaptureUsageDescription` explains that system audio is analysed locally to animate the LED ring. Audio is converted into feature values in memory; it is never recorded, saved, or sent over the network. + +`run` retries HID access every two seconds until the device and Input Monitoring permission become available. In reactive mode it also retries the Process Tap every two seconds until System Audio Recording permission is available, so you do not normally need to restart it after granting access. `audio-probe`, `probe`, `status`, `set`, and `off` are one-shot commands; correct the permission or device problem and run the command again. + +Use `--mode static` if you do not want to grant system-audio access. Static mode still needs HID access and Now Playing artwork, but it does not start a Core Audio Process Tap. + +### Start automatically at login + +Install and start the per-user LaunchAgent: + +```bash +./target/release/codex-micro-chroma install +``` + +The command copies the release binary into Application Support, gives it an ad-hoc signature with the fixed identifier `com.local.codex-micro-chroma`, and registers an Aqua-session LaunchAgent. The installed worker is located at: + +```text +~/Library/Application Support/CodexMicroChroma/codex-micro-chroma +``` + +If the system-audio permission prompt appears immediately after installation, grant it. If you dismiss it, add and enable the worker shown above in **System Settings > Privacy & Security > Screen & System Audio Recording**, then run `install` again. + +The LaunchAgent cannot approve its own privacy prompts. If the background worker is waiting, manually add and enable this installed executable for both permissions that are required on your Mac: + +- **Input Monitoring**, for Codex Micro HID access +- **Screen & System Audio Recording** or **System Audio Recording Only**, for reactive audio analysis + +After changing a permission, check `~/Library/Logs/CodexMicroChroma/worker-error.log`. The worker normally resumes through its built-in retry loop. If macOS does not apply the change to the running worker, run `install` again to restart the LaunchAgent. + +An ad-hoc signature's designated requirement includes the binary's CDHash. After rebuilding and reinstalling, macOS may therefore ask you to approve Input Monitoring or System Audio Recording again. If necessary, switch the installed worker's permission off and on. A distribution build signed with a Developer ID or local code-signing certificate can avoid this repeated approval. + +Worker and track logs are stored in `~/Library/Logs/CodexMicroChroma/`. + +To stop the LaunchAgent and remove the installed copy: + +```bash +~/Library/Application\ Support/CodexMicroChroma/codex-micro-chroma uninstall +``` + +`uninstall` removes the LaunchAgent plist and installed executable, but preserves diagnostic logs. + +## Usage + +### Follow the current media + +Start the default reactive mode: + +```bash +./target/release/codex-micro-chroma run +``` + +- When the player or media changes, the new artwork is analysed automatically. +- A thumbnail fingerprint is used as the content identity when no title is available, and artwork changes are detected even when the title remains the same. +- White backgrounds and transparent pixels are excluded before the dominant colour is adjusted to be brighter and more vivid on the LEDs. +- The previous media colour is cleared while new artwork is pending. +- Relative loudness controls brightness, combined musical motion controls speed, and spatial width and change control the device's `magic` parameter. +- Brief onset, flux, pulse, and bass events are held with attack-and-release envelopes so that they can pass the 650 ms candidate dwell. Hysteresis preserves close-scoring candidates, allowing a beat to trigger a dynamic effect such as `snake` even if it has disappeared by the next audio frame. +- Effects have a two-second minimum hold to prevent flicker. If the same pattern remains the strongest candidate for more than 12 seconds, a semantically close runner-up may take over at a phrase boundary to avoid becoming stuck on a single effect such as `breath`. `rainbow` has a 12-second cooldown. +- Speed and `magic` use effect-specific profiles: `solid` remains still, `snake` follows the beat, `gradient` reflects stereo width, `rainbow` marks a climax, and the two breathing effects use different depths. A brief rest does not switch the ring off. +- A persistent HID connection is reused and updated every 100 ms by default. If the device or Input Monitoring permission is unavailable at start-up, the worker waits rather than exiting. It automatically reconnects once after a disconnection, response timeout, or other transport failure. +- If Process Tap frames stop for three seconds during playback, or remain completely zero-filled for 15 seconds, the tap and aggregate device are rebuilt with a 30-second recovery cooldown. +- Control-C or SIGTERM stops the process and switches the LEDs off. + +The default values can be set explicitly: + +```bash +./target/release/codex-micro-chroma run \ + --mode reactive \ + --brightness 1.0 \ + --poll-ms 250 \ + --device-ms 100 +``` + +To follow artwork while keeping one fixed effect: + +```bash +./target/release/codex-micro-chroma run \ + --mode static \ + --effect breath \ + --brightness 1.0 \ + --speed 0.85 \ + --magic 0.0 \ + --refresh-ms 750 +``` + +### Test the lighting directly + +Set a colour without reading Now Playing, then switch the lighting off. The default effect for both `set` and `run` is `breath`. + +```bash +./target/release/codex-micro-chroma set --color '#33AAFF' +./target/release/codex-micro-chroma set --color '#33AAFF' --effect snake +./target/release/codex-micro-chroma off +``` + +`off` clears both the key lighting and ambient-ring lighting controlled by this tool. + +### Inspect audio features + +`audio-probe` does not write to the LEDs. It prints `AudioFeatureFrame` values calculated from the Process Tap as JSON Lines, which is useful for permission checks and algorithm tuning. + +```bash +./target/release/codex-micro-chroma audio-probe --seconds 15 +``` + +## Lighting effects + +All currently known Codex Micro effects can be selected. + +| CLI name | Device value | Role in reactive mode | +| --- | ---: | --- | +| `off` | 0 | Sustained silence or stopped playback | +| `solid` | 1 | Speech or direct, centre-panned sound | +| `snake` | 2 | Strong bass, pulse, or rhythmic periodicity | +| `rainbow` | 3 | A climax combining high level, strong flux, and pronounced onsets | +| `breath` | 4 | Smooth, sustained, tonal material | +| `gradient` | 5 | Wide stereo material | +| `shallow-breath` | 6 | Quiet passages, introductions, and outros | + +The shared `--brightness`, `--speed`, and `--magic` parameters each accept values from 0 to 1. Their visible behaviour and useful combinations depend on the device firmware's implementation of each effect. + +## Per-track effect logs + +`run` follows the Now Playing `elapsed_time`, `duration`, and `playback_rate` values and automatically writes one JSON Lines log per track: + +```text +~/Library/Logs/CodexMicroChroma/tracks/-.jsonl +``` + +- `track_start`: media metadata, source application, duration, and observed starting position +- `effect_transition`: position, effect, colour, brightness, speed, `magic`, and all audio features at that moment +- `track_summary`: time and percentage per effect, transition count, observed time, and completion reason + +`complete_track` is `true` only when observation began within the first two seconds, ended within the final two seconds, and covered at least 85% of the media's duration. Accounting pauses with playback and resumes in the same log when the same media continues. + +If a repeat or crossfade causes the position to jump backwards to within the first five seconds, the current pass is finalised with `position_restarted` and the next pass is written to a separate log. The previous pass still counts as complete if at least 85% of its duration was observed. Records are flushed on every transition, so a live log can be inspected with `tail -f`. + +MediaRemote elapsed time is anchored to a local monotonic clock when each event arrives. This prevents time spent paused from being added to the media position if MediaRemote returns an old timestamp after a long pause. + +## Privacy and security + +- No OpenAI, Codex, Spotify, Apple Music, or other external API is used. +- No network connection is made at runtime. +- System Integrity Protection is not changed. +- Root access, `sudo`, and code injection are not used. +- Process Tap PCM is not written to a file. +- Playback is observed but never controlled. +- HID access opens exactly one interface matching the Codex Micro VID, PID, and usage page. +- An inter-process lock serialises concurrent LED writes. +- The installed worker uses a fixed identifier and an ad-hoc signature, not a Developer ID certificate. + +## Implementation notes + +`CATapDescription` is an Objective-C API, so only Process Tap and aggregate-device creation and destruction are isolated in a small Objective-C bridge. After the bridge delivers Float32 PCM, the FFT, feature extraction, relative normalisation, effect scoring, state transitions, and HID control are implemented in Rust. + +The audio callback sends fixed-size packets to a bounded channel with `try_send`; FFT work and locking never run on Core Audio's real-time thread. + +## Development + +```bash +cargo fmt --all --check +cargo clippy --all-targets --all-features -- -D warnings +cargo test --all-targets --all-features +``` + +## Publishing a release + +The `Release macOS binary` GitHub Actions workflow builds an ad-hoc-signed universal binary, creates a checksum, and publishes both files in a new GitHub Release. The requested version must match the `version` in `Cargo.toml`. + +To publish manually, open **Actions > Release macOS binary > Run workflow** and enter a version without the leading `v`, such as `0.1.0`. Alternatively, push a matching tag such as `v0.1.0`. + +An existing release or manually supplied existing tag is never overwritten. Update `Cargo.toml` before publishing the next version. Once the release is available, update the download version in both [the README](../README.md#get-started) and this guide. + +## Licence + +Codex Micro Chroma is available under the [MIT Licence](../LICENSE). diff --git a/docs/readme-design.md b/docs/readme-design.md new file mode 100644 index 0000000..cbdab76 --- /dev/null +++ b/docs/readme-design.md @@ -0,0 +1,48 @@ +# README design notes + +Research checked on 27 September 2026. These projects are references for presentation, +not evidence that a README layout causes stars. Audience size, distribution, project +age, and the usefulness of the product also affect adoption. + +## References and decisions + +| Project | What its README does | Application here | +| --- | --- | --- | +| [Glow](https://github.com/charmbracelet/glow#readme) | Opens with a short identity and visual presentation, then explains the tool and how to install it. | Give the lighting project a recognisable visual identity and explain the outcome before its internals. | +| [uv](https://github.com/astral-sh/uv#readme) | Puts its purpose, a benchmark graphic, highlights, and installation near the top; links to deeper documentation. | Make the core value visible immediately and move implementation detail to a technical guide. A concept diagram communicates the pipeline but does not provide the proof of a benchmark or recording. | +| [Starship](https://github.com/starship/starship#readme) | Provides prominent navigation, compact benefits, and installation steps with explicit prerequisites. | Put download, effects, compatibility, and guide links at the top; distinguish runtime requirements from build tools. | +| [CAVA](https://github.com/karlstav/cava#readme) | Gives an audio visualiser a demo link before its lengthy platform setup reference. | Prioritise seeing the lighting result. Use an explicitly labelled illustration until real hardware footage is available. | + +## What changed + +- Lead with artwork colour and audio-driven movement, rather than daemon architecture. +- Add an original, self-contained SVG. It uses no player logos, borrowed album artwork, + fake hardware photographs, animation, scripts, or external fonts. +- Keep badges to release, CI, platform, and licence; avoid a wall of counters. +- Separate a short download → check → run path from detailed permissions and DSP notes. +- Retain all previous technical material in [the guide](guide.md). +- Keep player validation gaps, mixed-system-audio behaviour, private MediaRemote use, + and signing limitations visible. Do not claim universal player support. +- Verify that the recommended release exists. At review time, v0.1.0 is published; + the v0.2.0 version-bump PR is still open. +- Give readers specific ways to contribute: test a player, report a bug, or share a clip. + Include one modest star invitation after the useful content. + +## Best next visual + +A short real-device clip would provide evidence the illustration cannot. Frame the +Codex Micro and current artwork together, show a track change, then show the ring +reacting to a change in the music. Label the macOS version, Chroma version, and player. +Keep exposures consistent and avoid synchronising an edited animation to imply measured +latency. Use media you have permission to share. + +Place the clip or a compact GIF with a still fallback immediately below the introduction. +Keep the illustration only if it still helps explain the two inputs. No demonstration +footage was available in the repository during this edit. + +## Evaluating the change + +Compare repository visitors, release downloads, player reports, and stars over similar +time windows, noting any launches or social posts. A change in stars alone cannot +establish that the README caused it. This edit improves presentation and onboarding; +it does not establish product adoption or hardware validation.