Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .cursor/rules/gl-post-process.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Reuse or add `*_active()` helpers in `post_fx.py` that check `render.post_fx.ena

When enabled, compositor FBOs use `GL_RGBA16F` ([cleave/gl_color_format.py](cleave/gl_color_format.py)). `GlPostProcess` must match via `moderngl_external_dtype` / `moderngl_internal_dtype` (`f2` half-float, `f1` for 8-bit internal). **Never `u1` for 8-bit internal buffers** (breaks copy/bloom/grit). Skip output clamp when wired for `RGBA16F` (`hdr` uniform). 8-bit black-key stacking washes bright layers before tone curves.

`finish_content_frame` order: HDR display shoulder (fixed constants in `post_fx.py`; not gated on post-FX `enabled`) -> user composite highlight rolloff -> user composite chroma boost -> frame fade -> overlay -> present (8-bit clamp). Per-layer rolloff/chroma_boost keep frozen compositor source textures for paused live tuning.
`finish_content_frame` order: HDR display shoulder (fixed constants in `post_fx.py`; not gated on post-FX `enabled`) -> visual-limiter busyness sample ([cleave/viz/visual_limiter.py](cleave/viz/visual_limiter.py)) -> user composite highlight rolloff -> user composite chroma boost -> frame fade -> overlay -> present (8-bit clamp). Per-layer rolloff/chroma_boost keep frozen compositor source textures for paused live tuning.

## Known pitfalls (GlPostProcess)

Expand Down
6 changes: 3 additions & 3 deletions .cursor/rules/live-tuning-ui.mdc

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .cursor/rules/preset-scan-rotation-set.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Plan: [docs/legacy-plans/presets-scan-plan.md](docs/legacy-plans/presets-scan-pl
- Rotation set (`preset_switching_rotation_set`): `directory` | `user_defined` (used when mode is `projectm` or `timeline`).
- **directory** live rotation: immediate `*.milk` siblings in `playlist.current_dir`, **non-recursive** (`recurse=False`), then `sort()`.
- **user_defined** live rotation: `preset_switching_presets` via `add_presets(..., allow_duplicates=True)` (no sort).
- Project scan includes the anchor directory for **every** layer (even `preset_switching: none` or `enabled: false`), plus `preset_switching_presets` when `preset_switching_rotation_set` is `user_defined`.
- Project scan includes the anchor directory for **every** layer (even `preset_switching: none` or `enabled: false`), plus `preset_switching_presets` when `preset_switching_rotation_set` is `user_defined`, plus non-recursive `*.milk` in each `preset_root/roles/<role>/` pool (`bed`, `pulse`, `lead`, `accent`). Role pools are part of the live rotation surface: timeline cues with a `role` cast from those directories via `role_rotations` in [cleave/viz/preset_switching.py](cleave/viz/preset_switching.py), so scan must enumerate the same files.
- Bulk mode uses `--presets-dir` with optional `--recursive`; project mode does not recurse into subfolders.

## When rotation set expands
Expand Down
4 changes: 2 additions & 2 deletions .cursor/rules/project-context.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ alwaysApply: true

# Cleave project context

- BAU iterative development. Editor: `python -m cleave play` via [cleave/cli.py](cleave/cli.py) calling `cleave.viz.launch()`; [cleave.py](cleave.py) is an alias; implementation in [cleave/viz/](cleave/viz/) (up to eight Milkdrop/libprojectM layers, default four; add/remove in live tuning, 1280x720 default resolution, live at display frame rate, offline render fps via `render.fps`, stem PCM). Black-key stack in [cleave/gl_compositor.py](cleave/gl_compositor.py). GPU post-FX (bloom, grit, highlight rolloff, chroma boost): [cleave/gl_post_process.py](cleave/gl_post_process.py). Live tuning: session in [cleave/viz/session.py](cleave/viz/session.py), input in [cleave/viz/controls.py](cleave/viz/controls.py), view state in [cleave/viz/tuning_view_state.py](cleave/viz/tuning_view_state.py), panel draw in [cleave/viz/tuning_panel_draw.py](cleave/viz/tuning_panel_draw.py); layout in [cleave/viz/row_layout.py](cleave/viz/row_layout.py); section tree in [cleave/viz/row_sections.py](cleave/viz/row_sections.py); panel field manifest in [cleave/viz/row_fields.py](cleave/viz/row_fields.py); shared live/offline frame finish in [cleave/viz/frame_finish.py](cleave/viz/frame_finish.py). projectM user time is a monotonic render clock ([cleave/viz/projectm_frame_clock.py](cleave/viz/projectm_frame_clock.py)), not song playhead. Timeline: per-track lanes (`TimelineLane` / `SlotCue` in [cleave/timeline.py](cleave/timeline.py); `timeline.lanes` in YAML; strip in [cleave/viz/timeline_overlay.py](cleave/viz/timeline_overlay.py)). Render credits overlay in [cleave/viz/render_overlay.py](cleave/viz/render_overlay.py) (`render.overlay` in YAML): live preview in play, burned in by [cleave/viz/render.py](cleave/viz/render.py) offline. Cleave effects: [cleave/effects/](cleave/effects/) (dispatch via [cleave/effects/handlers.py](cleave/effects/handlers.py)). Paths: [cleave/paths.py](cleave/paths.py) (repo-root data dir by default, `projects/<slug>/`; `CLEAVE_DATA` override). Config: parse and defaults in [cleave/config_schema.py](cleave/config_schema.py); repo-root [cleave-viz.yaml](cleave-viz.yaml) copied into projects; `unnamed-N.yaml` snapshots via [cleave/config_snapshot.py](cleave/config_snapshot.py). Shared easing: [cleave/easing.py](cleave/easing.py). Architecture conventions: [.cursor/rules/architecture-principles.mdc](.cursor/rules/architecture-principles.mdc).
- BAU iterative development. Editor: `python -m cleave play` via [cleave/cli.py](cleave/cli.py) calling `cleave.viz.launch()`; [cleave.py](cleave.py) is an alias; implementation in [cleave/viz/](cleave/viz/) (up to eight Milkdrop/libprojectM layers, default four; add/remove in live tuning, 1280x720 default resolution, live at display frame rate, offline render fps via `render.fps`, stem PCM). Black-key stack in [cleave/gl_compositor.py](cleave/gl_compositor.py). GPU post-FX (bloom, grit, highlight rolloff, chroma boost): [cleave/gl_post_process.py](cleave/gl_post_process.py). Live tuning: session in [cleave/viz/session.py](cleave/viz/session.py), input in [cleave/viz/controls.py](cleave/viz/controls.py), view state in [cleave/viz/tuning_view_state.py](cleave/viz/tuning_view_state.py), panel draw in [cleave/viz/tuning_panel_draw.py](cleave/viz/tuning_panel_draw.py); layout in [cleave/viz/row_layout.py](cleave/viz/row_layout.py); section tree in [cleave/viz/row_sections.py](cleave/viz/row_sections.py); panel field manifest in [cleave/viz/row_fields.py](cleave/viz/row_fields.py); shared live/offline frame finish in [cleave/viz/frame_finish.py](cleave/viz/frame_finish.py). projectM user time is a monotonic render clock ([cleave/viz/projectm_frame_clock.py](cleave/viz/projectm_frame_clock.py)), not song playhead. Timeline: per-track lanes (`TimelineLane` / `SlotCue(t, level)` in [cleave/timeline.py](cleave/timeline.py); `timeline.lanes` in YAML; strip in [cleave/viz/timeline_overlay.py](cleave/viz/timeline_overlay.py)); opt-in stem conductor for generative presets in [cleave/timeline_presets/conductor.py](cleave/timeline_presets/conductor.py). Render credits overlay in [cleave/viz/render_overlay.py](cleave/viz/render_overlay.py) (`render.overlay` in YAML): live preview in play, burned in by [cleave/viz/render.py](cleave/viz/render.py) offline. Cleave effects: [cleave/effects/](cleave/effects/) (dispatch via [cleave/effects/handlers.py](cleave/effects/handlers.py)). Paths: [cleave/paths.py](cleave/paths.py) (repo-root data dir by default, `projects/<slug>/`; `CLEAVE_DATA` override). Config: parse and defaults in [cleave/config_schema.py](cleave/config_schema.py); repo-root [cleave-viz.yaml](cleave-viz.yaml) copied into projects; `unnamed-N.yaml` snapshots via [cleave/config_snapshot.py](cleave/config_snapshot.py). Shared easing: [cleave/easing.py](cleave/easing.py). Architecture conventions: [.cursor/rules/architecture-principles.mdc](.cursor/rules/architecture-principles.mdc).
- Must-do list: [docs/todos.md](docs/todos.md). Aspirational: [docs/roadmap.md](docs/roadmap.md).
- See [README.md](README.md) for usage.
- Project layout: `projects/<slug>/` under repo root (or `CLEAVE_DATA` override) with copied mix audio, `project.yaml`, `signals.json`, `stems/` (four stem wavs), and optional configs. `separate` copies the source file, writes the manifest, runs Demucs, and writes `signals.json`; the editor reads the mix from `project.yaml`. CLI: `python -m cleave separate|play` (or `python cleave.py`, same entry point).
- Project layout: `projects/<slug>/` under repo root (or `CLEAVE_DATA` override) with copied mix audio, `project.yaml`, `signals.json` (version 4), `stems/` (four stem wavs), and optional configs. `separate` copies the source file, writes the manifest, runs Demucs, and writes `signals.json`; the editor reads the mix from `project.yaml`. CLI: `python -m cleave separate|play` (or `python cleave.py`, same entry point).
- Stem PCM ([cleave/stem_pcm.py](cleave/stem_pcm.py), [cleave/pcm_io.py](cleave/pcm_io.py)) preserves native mono or stereo from WAV files; [cleave/projectm.py](cleave/projectm.py) feeds libprojectM with matching channel layout. Mix playback and solo use stereo SDL output. Analysis for `signals.json` ([cleave/extract.py](cleave/extract.py)) stays mono.
- Overlay layout and typography: [.cursor/rules/live-tuning-ui.mdc](.cursor/rules/live-tuning-ui.mdc).
- Preset curation: `favourites/` and `blacklist/` under `paths.preset_root`; file ops in [cleave/preset_curation.py](cleave/preset_curation.py), modal orchestration in [cleave/viz/preset_curation_controls.py](cleave/viz/preset_curation_controls.py); **f** / **b** / **r** on preset file rows (`PRESET_FILE_ROW_KINDS` in [cleave/viz/row_semantics.py](cleave/viz/row_semantics.py)); dedup via [cleave/viz/user_presets.py](cleave/viz/user_presets.py) `resolve_user_preset_dest`.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ The project directory stores all files required in a self-contained bundle...

* `project.yaml` - project metadata
* `cleave-viz.yaml` - editor & final render configuration. Not everything in here is surfaced in the editor UI just yet
* `signals.json` - audio analysis data used by `cleave effects`
* `signals.json` - audio analysis data (schema version 4) used by `cleave effects` and the opt-in timeline preset conductor; re-run `separate` on existing projects after a schema bump so envelopes stay current
* `mysong.wav` - original source audio is copied into the project directory
* `stems/` - separated audio stems
* `renders/` - final output renders
Expand Down
7 changes: 5 additions & 2 deletions cleave/analyse.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def run_analyse(
downbeat_times = [float(t) for t in downbeats]

output: dict = {
"version": 3,
"version": 4,
"sample_rate_hz": int(TARGET_HZ),
"duration_sec": duration_sec,
"beat_detection_stem": beat_detection_stem,
Expand All @@ -91,7 +91,10 @@ def run_analyse(
),
},
"other": {
"spectral_centroid": resample_to_100hz(*other, duration_sec).tolist(),
"spectral_centroid": resample_to_100hz(
*other["spectral_centroid"], duration_sec
).tolist(),
"rms": resample_to_100hz(*other["rms"], duration_sec).tolist(),
},
"full_mix": {
"onset_strength": resample_to_100hz(*mix_onset, duration_sec).tolist(),
Expand Down
25 changes: 24 additions & 1 deletion cleave/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
_YAML_DUMP_WIDTH = 2**31 - 1

from cleave.blend_modes import BlendMode
from cleave.cue_roles import CueRole
from cleave.effects.constants import clamp_effect_pct
from cleave.extract import StemSource
from cleave.config_schema import (
Expand All @@ -24,6 +25,8 @@
BEAT_SENSITIVITY_MIN,
DEFAULT_BEAT_SENSITIVITY,
DEFAULT_LAYER_Z_ORDER,
DEFAULT_CAST_ROLES_DEFAULT_ROLE,
DEFAULT_CAST_ROLES_TIMELINE_BEHAVIOUR,
DEFAULT_PRESET_SWITCHING,
DEFAULT_PRESET_SWITCHING_ROTATION_SET,
DEFAULT_PRESET_SWITCHING_SHUFFLE,
Expand All @@ -36,6 +39,7 @@
DEFAULT_EASTER_EGG,
DEFAULT_PRESET_START_CLEAN,
DEFAULT_PRESET_ROOT,
CastRolesTimelineBehaviour,
PresetSwitchingMode,
PresetSwitchingRotationSet,
DEFAULT_RENDER_OVERLAY_BACKGROUND_COLOUR,
Expand Down Expand Up @@ -64,6 +68,9 @@
DEFAULT_TIMELINE_FADE_IN,
DEFAULT_TIMELINE_FADE_OUT,
DEFAULT_TIMELINE_PLACEMENT_SNAP,
DEFAULT_VISUAL_LIMITER_ENABLED,
DEFAULT_VISUAL_LIMITER_THRESHOLD,
DEFAULT_VISUAL_LIMITER_RELEASE,
TimelinePlacementSnap,
DEFAULT_HDR_COMPOSITING,
DEFAULT_RENDER_FPS,
Expand Down Expand Up @@ -109,6 +116,7 @@
)
from cleave.timeline import TimelineLane
from cleave.timeline_presets.characters import DEFAULT_TIMELINE_PRESET_KIND
from cleave.timeline_presets.conductor import DEFAULT_TIMELINE_PRESET_CONDUCTOR
from cleave.timeline_presets.crescendo import CrescendoTarget
from cleave.timeline_presets.density import (
DEFAULT_TIMELINE_PRESET_DENSITY,
Expand Down Expand Up @@ -136,6 +144,10 @@ class LayerConfig:
locked: bool = False
preset_switching: PresetSwitchingMode = DEFAULT_PRESET_SWITCHING
preset_switching_rotation_set: PresetSwitchingRotationSet = DEFAULT_PRESET_SWITCHING_ROTATION_SET
cast_roles_timeline_behaviour: CastRolesTimelineBehaviour = (
DEFAULT_CAST_ROLES_TIMELINE_BEHAVIOUR
)
cast_roles_default_role: CueRole = DEFAULT_CAST_ROLES_DEFAULT_ROLE
preset_switching_shuffle: bool = DEFAULT_PRESET_SWITCHING_SHUFFLE
preset_switching_shuffle_salt: int = DEFAULT_PRESET_SWITCHING_SHUFFLE_SALT
preset_duration: float = DEFAULT_PRESET_DURATION
Expand Down Expand Up @@ -268,11 +280,21 @@ class TimelineFadesConfig:

@dataclass(frozen=True)
class TimelinePresetConfig:
"""Staged character / crescendo / density for the timeline preset action."""
"""Staged character / crescendo / density / conductor for the timeline preset action."""

character: str = DEFAULT_TIMELINE_PRESET_KIND
crescendo: CrescendoTarget | None = None
density: TimelinePresetDensity = DEFAULT_TIMELINE_PRESET_DENSITY
conductor: bool = DEFAULT_TIMELINE_PRESET_CONDUCTOR


@dataclass(frozen=True)
class TimelineLimiterConfig:
"""Live visual limiter knobs under ``timeline.limiter``."""

enabled: bool = DEFAULT_VISUAL_LIMITER_ENABLED
threshold: float = DEFAULT_VISUAL_LIMITER_THRESHOLD
release: float = DEFAULT_VISUAL_LIMITER_RELEASE


@dataclass(frozen=True)
Expand All @@ -283,6 +305,7 @@ class TimelineConfig:
fades: TimelineFadesConfig = field(default_factory=TimelineFadesConfig)
placement_snap: TimelinePlacementSnap = DEFAULT_TIMELINE_PLACEMENT_SNAP
preset: TimelinePresetConfig = field(default_factory=TimelinePresetConfig)
limiter: TimelineLimiterConfig = field(default_factory=TimelineLimiterConfig)


@dataclass
Expand Down
Loading
Loading