Patternflow Firmware

Arduino-based firmware for the ESP32-S3 powering Patternflow. One image serves every board generation — the pin map is identical on v2.x, v3.0 and v3.9.

The firmware handles the ESP32-S3 DMA driver for the HUB75 LED matrix, reads four rotary encoders to control generative patterns, and supports Arduino OTA for wireless updates.

⚠️ Panel compatibility. This firmware drives the panel directly from the ESP32-S3, so the panel's driver IC decides whether it lights up: classic shift-register parts work, S-PWM "video wall" parts stay completely dark, and "HUB75E" on a listing proves nothing. The list of chips, and how to check before buying, is docs/panel-compatibility.md; the driver is selected below via PANEL_PROFILE.

Reading order

This file is long because it is the firmware's reference. Most people need one part of it:

you want to	read
put Patternflow on a new board and get it on Wi-Fi	not this file: `BUILD_GUIDE.md` §8, or the Play guide's Flash chapter. The short version is First flash and Wi-Fi
build and flash the firmware locally	Setup — the board package, PlatformIO, the libraries
write a feature or cut an edition	`docs/EDITIONS.md` first, then Adding features — the constraints that matter here
write a pattern	`PATTERN_GUIDE.md` (nothing here is needed); the hands-on route is `CUSTOM_PATTERNS.md`
know what a pattern can call	Foundation modules
know how `.pfm` modules load and what they cost	Loadable pattern modules
tune brightness, calibration, refresh	Configuration
know what the knobs and buttons do	Controls, Sleep mode
drive the panel from other software	the contracts in `docs/`: HTTP, OSC, MIDI, MQTT, audio
edit a console page	`patternflow/console/README.md`

Building your own firmware

One image ships on the board, and it is not the only one you can run. Features are modules in this tree; a build is a choice of which ones compile in; a published build is an edition, installed from the shelf in one click, with your patterns and settings intact.

docs/EDITIONS.md is where that starts: what a feature is, how to write one, how to cut an edition, and the promises every build keeps so switching stays safe. Read it before writing a feature.

First flash and Wi-Fi

Nothing on this page is needed to put the released firmware on a board: the browser does it. The full walk-through is BUILD_GUIDE.md §8 and, on a board you can turn, patternflow.work/guide/play#flash. In order:

The DevKit out of the board, on its left USB-C port (USB, the native one), with a cable that carries data. Desktop Chrome or Edge. The port is for flashing and data only — the panel is powered through J4, the screw terminal, and nothing else.
Download mode, every first flash: hold BOOT, tap RST, let go of BOOT. The port then shows as USB JTAG/serial debug unit.
Flash Patternflow (the Pattern tab on patternflow.work, or the guide), pick that port, Install Patternflow, confirm. About two minutes. From download mode the flasher cannot tell a board already runs Patternflow, so it erases the whole flash, patterns and saved Wi-Fi included — nothing to lose on a new build; a running board updates over Wi-Fi instead (see Wireless update from the browser).
The Wi-Fi step usually does not appear after an install from download mode, because the board does not restart into Patternflow by itself (core_improv.h can only answer once it runs). Expected, not a failure: press RST once, press Flash Patternflow again, pick the port, and the dialog opens on Patternflow's own menu. Connect to Wi-Fi; after about 10 s of "Scanning" (the firmware sends no list) type the network's name exactly and its password. 2.4 GHz only.
Or skip the flasher for Wi-Fi. Seat the DevKit and power the panel: about 15 s after it finds no network it knows (PF_HOTSPOT_AUTO_AFTER_MS), it raises its own, patternflow-xxxx (the last two MAC bytes), password patternflow (core_hotspot.h). Hold K2 and the NETWORK screen shows HOTSPOT and the name. Not in the phone's list yet? It only exists from about 15 s after power-on: wait, refresh the list, and match the exact name on K2. Join it (the phone says "no internet", which is true; stay on it), open http://192.168.4.1/, and pick your network on the Wi-Fi page.

Once it is on your network, hold K2 for its address; the console is at patternflow.local (on Android, type the IP). A new board runs Origin only — the Basics pack installs from the decks page over Wi-Fi.

Setup

Required board package

ESP32 by Espressif Systems — 2.0.x, not the latest 3.x (see below)

⚠️ The core version decides how much RAM your patterns get. Core 3.x ships ESP-IDF 5.5, which occupies about 71 KB more internal RAM before the sketch even starts than core 2.x / IDF 4.4 — the gap is already there at the first heap reading in setup(), before Wi-Fi is up, and no sdkconfig setting reverses it (the Wi-Fi buffer flags are identical between the two prebuilts). Measured on the same board with the same source, free internal heap once the network services are up:

free largest block

core 3.3.8 (IDF 5.5) 15,320 B 7,668 B

core 2.0.17 (IDF 4.4) 98,708 B 90,100 B

When this was measured that largest-block number was the hard ceiling on a loadable module: a .pfm's .text had to land in one contiguous internal executable block, so on core 3.x a 12 KB pattern was simply refused and on core 2.x it loaded. That ceiling is gone - a module's code now runs from PSRAM (src/core_module_memory.h says how) - but the headroom is still the reason for this choice: it is what the console, lwIP and every feature run on. See Internal RAM is the budget.

	free	largest block
core 3.3.8 (IDF 5.5)	15,320 B	7,668 B
core 2.0.17 (IDF 4.4)	98,708 B	90,100 B

Build with PlatformIO (recommended)

platformio.ini in patternflow/ pins core 2.x and is what the measurements above come from. It also builds in ~90 s against the IDE's several minutes.

cd firmware/patternflow
pio run                       # → .pio/build/firmware/firmware.bin
pio run -t upload             # or flash it yourself at 0x10000

toolchain/sync_ino_to_src.py copies the sketch into pio_src/ and clones the libraries below into lib/ on first run, so there is nothing to install by hand. On Windows, build from a path with no non-ASCII characters, or point PLATFORMIO_BUILD_DIR at one — the xtensa linker cannot write outputs underneath a path it cannot encode.

The Arduino IDE path still works and is documented below; it is what the older guides describe. Just install core 2.0.x rather than the latest.

Arduino IDE board settings

Board: ESP32S3 Dev Module
PSRAM: OPI PSRAM
Flash Size: 16MB
Partition Scheme: 16M Flash with PSRAM-aware partition
USB CDC On Boot: Enabled
Upload Mode: UART0 / Hardware CDC

Why CDC On Boot matters. It decides whether Serial is the native USB peripheral or UART0 — and therefore which of the two USB-C ports answers the browser flasher's Wi-Fi setup (Improv speaks over Serial). Enabled puts it on the left/native USB port, which is what the released firmware does and what §8.1 of the build guide tells people to use. Building with it disabled produces firmware that flashes fine but whose Wi-Fi setup only appears on the other socket.

Required libraries

PlatformIO clones these for you (lib_deps in patternflow/platformio.ini is the authoritative list). For the Arduino IDE, install them via the Library Manager:

ESP32-HUB75-MatrixPanel-DMA (for driving the matrix)
Adafruit GFX Library (dependency)
WebSockets by Markus Sattler / Links2004 (audio-react, Audio edition)
NimBLE-Arduino (the ble feature — in the tree, in no edition)
AppleMIDI by lathoub (network MIDI, Audio edition)

The Arduino IDE builds the default firmware. It does not build the editions: their feature set is chosen by firmware/bundles/<name>/features_local.h, which only bundles/build.sh wires in. Use PlatformIO for anything that is not the plain device.

The HUB75 library is also vendored under patternflow/src/hub75/, and that copy is the one that compiles. UPSTREAM.md there names the upstream tag and commit it was copied from, and each upstream commit taken since. Every line Patternflow changed is in patternflow.patch beside it, and VENDORED.md says why. After an edit under src/hub75/, rebuild the patch with the command in UPSTREAM.md instead of editing it. The installed/cloned copy only needs to satisfy headers, so its exact version does not matter.

OSC uses the ESP32 Arduino core's built-in WiFi and WiFiUdp libraries, so it does not require an extra OSC library. The MQTT client (pubsubclient) and the HUB75 driver are vendored under patternflow/src/.

Project layout

firmware/
├── patternflow/                 # The sketch (PlatformIO project; open patternflow.ino)
│   ├── patternflow.ino          # Main sketch: input routing, mode dispatch — THE CORE
│   ├── config.h                 # Hardware configuration (pin mappings, limits, LED calibration)
│   ├── net_config.h             # Wi-Fi / Improv / OTA / self-update defaults (core only)
│   ├── pattern_registry.h       # Presets compiled in (Origin) + .pfm modules discovered on FATFS
│   ├── platformio.ini           # The build. bundles/build.sh drives it.
│   ├── presets/                 # Curated presets (35). Only Origin is compiled in; the
│   │                            # rest ship as the Basics pack (web/public/packs/)
│   ├── abi/                     # Host ⇄ module contract for loadable patterns
│   │   ├── pf_abi.h             # Frozen C ABI: PFHostAPI, PFPatternModule, PF_ABI_VERSION
│   │   ├── pf_params.h          # The absolute parameter bus
│   │   ├── pf_module.h          # Module SDK — the ONE header a .pfm pattern includes
│   │   └── pf_libm.h            # ...and what makes floorf/fmodf & co. exact inline code in it
│   ├── patternflow_secrets.example.h  # Template (copy to patternflow_secrets.h)
│   ├── src/                     # Foundation — the core, feature-free (see docs/EDITIONS.md)
│   │   ├── core_display.h       # HUB75 driver init + refresh-rate config
│   │   ├── core_encoders.h      # Encoder ISRs + InputFrame contract
│   │   ├── core_canvas.h        # 128×64 RGB888 framebuffer + gamma/WB/sat
│   │   ├── core_math.h          # PFMath — shared verbatim with modules
│   │   ├── core_color.h         # PFColor — shared verbatim with modules
│   │   ├── core_noise.h         # PFNoise — shared verbatim with modules
│   │   ├── core_mem.h           # PSRAM-first allocator (PFMem)
│   │   ├── core_tables.h        # Panel-space polar tables (host-owned, 32 KB each)
│   │   ├── core_module_loader.h # ELF loader/relocator for .pfm modules
│   │   ├── core_power.h         # Total power clamp (measures demand, caps brightness)
│   │   ├── core_sleep.h         # Sleep mode: panel off, board idle, still on the network
│   │   ├── core_wifi.h          # Multi-network Wi-Fi (up to 5 saved, tried in order)
│   │   ├── core_improv.h        # Improv-Serial provisioning from the browser flasher
│   │   ├── core_hotspot.h       # The panel's own Wi-Fi network when no saved one answers
│   │   ├── core_http.h          # The one port-80 WebServer every page registers on
│   │   ├── core_net_task.h      # The network task on Core 0: Wi-Fi, HTTP, Improv, OTA
│   │   ├── core_loop_sync.h     # Hands a handler's frame-touching work to the loop task
│   │   ├── core_bus.h           # Absolute knob bus (0..1000 wire scale)
│   │   ├── core_web_update.h    # Browser self-update (/update)
│   │   ├── core_patterns_http.h # Module manager (/patterns + /api/patterns)
│   │   ├── core_status_http.h   # Diagnostics (/status + /api/status)
│   │   ├── core_wifi_http.h     # Wi-Fi manager (/wifi + /api/wifi)
│   │   ├── *_index.h            # PROGMEM HTML for the pages above (generated from console/)
│   │   ├── hub75/, webserver/, pubsubclient/   # Vendored libraries (see their VENDORED.md)
│   │   └── fonts/
│   ├── features/                # Everything that is not the device: one directory each
│   │   ├── pf_feature.h         # The hook interface a feature implements
│   │   ├── pf_features.h        # The dispatcher
│   │   ├── features.h           # The default list (empty) and the escape hatch
│   │   ├── osc/  audio/  audio_in/  midi/  ble/  mqtt/  show/  weather/  clock/
│   │   └── README.md            # What a feature is and the hooks it can answer
│   ├── console/                 # The device's web pages as plain HTML — edit these
│   └── toolchain/               # PlatformIO extra_scripts (must stay sketch-relative)
├── bundles/                     # Editions: two files each naming a feature set
│   ├── audio/  performance/  clock/  midi/   # features_local.h + overrides.h (+ env)
│   ├── build.sh                 # build the default, a named edition, or `all`
│   └── shelf.sh                 # stage a publishable image for the site
├── modules/                     # The module toolchain's working folder, not an inbox — see modules/README.md
├── encoder_test/                # Standalone encoder diagnostic sketch
└── toolchain/                   # Repo-level tooling: build a module or a pack, the console pages, the CI checks — see toolchain/README.md

src/ holds the foundation that patterns build on; features/ holds everything the device can do beyond being a panel with four knobs, and the core never names any of it — that rule, and why, is docs/EDITIONS.md. Patterns and the main sketch reference the helpers via #include "src/core_*.h".

The foundation files are stateless utilities — no global state to coordinate, safe to include from any pattern.

Foundation modules

Patterns should not duplicate trig tables, color converters, or noise functions. The foundation modules provide them once, shared across every pattern.

`core_canvas.h` — PFCanvas

The single point of contact with the LED driver. Patterns write pixels into the canvas; the canvas pushes the frame to the HUB75 panel.

PFCanvas::setPixel(x, y, r, g, b);   // inside the pixel loop
PFCanvas::present();                  // last line of draw()

Patterns must not call dma_display->drawPixelRGB888() directly. Global brightness, gamma, white balance, saturation, and any future post-processing live in present() — patterns that bypass the canvas miss those.

present() runs three post-processing steps before pushing pixels:

Saturation boost — pulls each pixel away from its Rec.601 luma in 8.8 fixed-point. Gray pixels are mathematically unchanged; saturated colors land closer to where the JS preview puts them. LED panels look washed out vs. a calibrated monitor, so a mild boost (default 1.10×) compensates.
Per-channel gamma + white balance — three 256-entry LUTs (one per channel) with the WB gain pre-multiplied into the gamma curve. HUB75 panels are linear PWM with unbalanced LED primaries (red is brighter per duty, blue is dimmer), so a single global gamma can't cover both correction needs. The pre-folded LUT means the inner loop pays the same cost as a single lookup.
DMA push — the vendored driver's blitRGB888() composes both scan halves and two adjacent columns per word, preserving the DMA control bits. Saturation and LUT work above are fused into that pass.

The current word-aligned blit measured 6.86 → 5.90 ms on the bench's 128×64, 8-bit, 260 Hz configuration, with identical output. It needs no extra framebuffer. See the performance and recovery bench report for the controlled comparison and its limits.

All five calibration values are tunable from config.h — see "LED panel calibration" below.

Runtime notes (3.10): activation worker, memory admission, network maintenance, MIDI transport

The follow-up runtime research measures whole-loop delays, upload/load overlap, simulation warm-up and memory policy. It separates tested improvements from experimental results and proposed architecture work; its temporary profiler is not part of a normal build.

The resulting common-runtime implementation adds a small permanent runtime snapshot to /api/status. Its whole-loop maximum includes work outside the rendered-frame timer; ?resetTiming=1 returns the previous snapshot and begins a new window. This is scheduling evidence, not a physical encoder-to-photon measurement.

Pattern activation uses one reusable Core-0 worker with a 12,288-byte stack reserved at boot. Keeping that stack resident costs RAM between selections, but avoids repeatedly finding a large contiguous block in a fragmented heap. If it cannot be created, a later activation can retry; it never runs arbitrary module setup on the render loop as a fallback. Boot restoration still uses the synchronous path before interactive rendering begins. An allocation refusal can be temporary while HTTP/UDP buffers are in use. The worker retries only that failure class, at most five times with 1.5 seconds of total backoff. Corrupt files and unresolved symbols fail immediately; the frame never waits on the retry delay. moduleMemory.loaderRetries distinguishes recovered admission attempts from a final loadError.

Storage mutations use conditional frame-boundary attempts. An unfinished loader leaves the request pending while frames continue. Once quiescent, activation is held during file changes, and later selections retain their module path across catalog reordering. Catalog restoration starts an asynchronous load. The single-core HTTP fallback replies busy when it cannot complete the attempt inline. An open upload body prevents idle recovery from starting a module against partially written storage; abort removes that partial file and schedules restoration.

src/core_module_memory.h applies a 24,576-byte internal byte-addressable service reserve to module allocations, including PSRAM-to-internal fallbacks. Admission checks the relevant largest block and rechecks service RAM after an allocation. Executable IRAM is not presumed interchangeable with byte RAM. This protects against module allocations consuming the reserve; unrelated system/feature allocations can still reduce free memory. Data goes to PSRAM first since 2026-10, small sections included (they used to stay internal while the budget allowed): that is what lets a module be parked holding no internal RAM. Measured A-B-B-A on two boards, most modules' frame time did not move (-1.1..+0.8 %); small per-pixel state arrays now read through the PSRAM cache cost Burgers +2.6 % and Wave Cascade +2.2 %. PF_MODULE_DATA_PSRAM_FIRST 0 puts the old placement back. Module dynamic allocations keep their 16-slot, unload-as-a-group lifetime and add a configurable 4 MiB limit (PF_MODULE_RUNTIME_MAX_BYTES); moduleMemory reports usage and refusals.

During loading, the loop waits out an 8 ms minimum period and running-mode previews refresh at most every 64 ms unless their target/canvas changes. Input and feature hooks still run each loop. Normal rendering has no new cap. These policies apply to existing and future modules without editing pattern code or changing the ABI.

The network worker's Wi-Fi/name maintenance hook is also serviced while the HTTP parser waits, a conditional loop request is pending, or a page-body socket is backpressured. Nonblocking sends retain the existing 20 s stall / 120 s total budgets. No HTTP re-entry or extra task is involved. Filesystem operations and other synchronous library calls can still delay maintenance; netMaintenance.maxGapMs exposes those gaps.

The Audio edition's MIDI feature additionally owns a separate 4,096-byte Core-0 transport worker. A display frame can be long enough to overflow the SDK's six-datagram UDP mailbox when four independent CCs arrive together. The worker parses and sends RTP-MIDI between frames, while fixed queues hand events to/from the frame task. Their 1,536 bytes of payload storage live in PSRAM; queue indices and synchronization remain internal. The worker never changes the parameter bus or a pattern directly. If storage or worker creation fails, frame polling remains available and midi.rxDetail.worker reports false. This implementation lives entirely in features/midi/.

See the implementation and MIDI follow-up report for measured before/after results and remaining limitations.

`core_math.h` — PFMath

PFMath::buildSinLUT();                       // call from setup() — idempotent
PFMath::fastSin(angle);                      // ~5x faster than sinf in pixel loops
PFMath::fastCos(angle);
PFMath::fract(x);                            // x - floor(x), without calling floorf
PFMath::ifloor(x);                           // floor as an int; floorf() is a libm CALL here
PFMath::floorF(x);                           // floor as a float, same reason
PFMath::clamp(x, lo, hi);                    // fminf/fmaxf are libm calls too
PFMath::clamp01(x);
PFMath::jsMod(x, m);                         // JavaScript's x % m (sign of x), no fmodf
PFMath::lerp(a, b, t);
PFMath::approxLength(x, y);                  // 6.8% accurate sqrt(x*x + y*y)

The sin LUT is 4 KB (1024 entries, ~0.35° resolution) and shared. Do not build your own.

floorf, fminf and fmaxf are function calls on this target, not instructions - GCC emits a call into libm at every use, at every optimization level. That is what ifloor, floorF, fract, clamp and clamp01 exist for: they truncate and correct, or compare, which the FPU does in one instruction each. Removing a single floorf from a per-pixel path measured smaller object code as well as faster, because the call sequence costs more than the inline does.

That is still how it is for code compiled into the firmware image (Origin, the features). In a module the SDK now does it for a pattern that does not: since 2026-10, abi/pf_libm.h puts inline code behind floorf, ceilf, truncf, roundf, fminf, fmaxf and fmodf - and behind floor(x), std::floor(x), fmod(a, b) and the like on floats - that returns, bit for bit, what the firmware's libm returns (toolchain/check_libm.py runs every float there is through it). A header nobody edits stops paying 185-840 ns for each of them (as timed; truncf was not). The PFMath forms are still the least code where they fit, with one exception worth knowing: jsMod(x, m) divides, which is a call unless m is a power of two, and in a module fmodf(x, m) with m a constant no longer does.

approxLength is an octagonal sqrt approximation — 6.8% error (measured by toolchain/check_math.py, which pins it; the 0.375 coefficient is the shift-friendly one, not the minimax one), no sqrtf in the pixel loop. Use it only when distance is a secondary signal. If distance IS the visual structure of the pattern (radial ripples, concentric rings, vortex centers, anything that uses 1/dist for amplification), use real sqrtf instead — the octagonal contour shows up as visible polygonal artifacts in those cases.

`core_color.h` — PFColor

PFColor::hsvToRgb(h, s, v, r, g, b);                // h is 0..1 (not degrees)
PFColor::ColorStop ramp[] = { {0.0f, 0,0,0}, ... };
PFColor::sampleRamp(ramp, count, t, r, g, b);       // POSTERISED - flat bands, no divide
PFColor::sampleRampLerp(ramp, count, t, r, g, b);   // gradient - one divide per sample

`core_noise.h` — PFNoise

PFNoise::perlin2D(x, y);                     // NOT normalised: runs to about +/-1.51
PFNoise::valueNoise2D(x, y);                 // 0..1
PFNoise::cellHash(gx, gy);                   // 0..1
PFNoise::cellHash(gx, gy, seed);             // seed decorrelates, measured
PFNoise::fractal2D(x, y, octaves, roughness);

The 512-byte permutation table is shared. Do not duplicate it.

`core_ota.h` — PatternflowOta

ArduinoOTA wrapper for wireless flashing. The main sketch only needs:

PatternflowOta::begin();   // in setup() — connects Wi-Fi if not already up
PatternflowOta::handle();  // first line of loop() — UDP poll, ~free when idle

Shares Wi-Fi credentials and connection with OSC (if both are enabled, the connection is reused). When PF_OTA_ENABLED is 0 everything compiles to a no-op. See the OTA Updates section below for the user-facing workflow.

`core_improv.h` — PatternflowImprov

Self-contained Improv-Serial implementation so the browser flasher (ESP Web Tools, behind the website's Flash Patternflow button) can set Wi-Fi over USB once the firmware is running — no rebuild, no patternflow_secrets.h. The main sketch only needs:

PatternflowImprov::begin();   // in setup() — announces it speaks Improv
PatternflowImprov::handle();  // in loop()  — drains Serial, drives provisioning

The flasher sends the SSID/password over serial; core_wifi.h stores them in NVS (separate pf_wifi namespace) and uses them in preference to the built-in PF_WIFI_SSID/PASS placeholders on every boot. Shares the debug USB Serial — the host parser scans for the IMPROV header and ignores println() noise. Compiled in only when Wi-Fi is actually used (one of OTA/OSC/audio enabled) and PF_IMPROV_ENABLED is 1; otherwise a no-op. Set #define PF_IMPROV_ENABLED 0 in patternflow_secrets.h to drop it.

The flasher can only ask for Wi-Fi once Patternflow is running and answering on that port. After an install from download mode (BOOT + RST) the board usually does not restart into Patternflow by itself, so the flasher's dialog ends at Installation complete! with no Wi-Fi step; pressing RST and opening the flasher again brings up Patternflow's own menu with Connect to Wi-Fi. The flasher waits up to 20 s for that answer after an install (new_install_improv_wait_time in web/public/flash/manifest.json), and a join is given up on after 30 s.

`core_hotspot.h` — PatternflowHotspot

The panel as a network of its own, so Wi-Fi can be set up without the flasher, and a panel can be used where there is no network at all. Core, not a feature: every image since 3.10.5 has it. In the default auto mode, once the station has had no link for PF_HOTSPOT_AUTO_AFTER_MS (15 s) the panel raises a WPA2 access point named patternflow-xxxx (the hostname plus the last two bytes of its MAC, patternflow-a1b2), password PF_HOTSPOT_PASS (patternflow) until it is changed on /wifi, and the whole console answers at http://192.168.4.1/. The NETWORK screen (hold K2) shows HOTSPOT and the name while it is what there is. On the Wi-Fi page the network-name field offers the networks the hotspot's channel scan saw, so a phone user picks one instead of typing a case-sensitive name; the panel then tries it at once and reports on the page and on its LEDs whether it joined. The hotspot answers every DNS name with the panel and fails the phone's internet probe at once, so the phone says "no internet" — true — and stays connected. Modes off / auto / always are kept in NVS and set on /wifi or POST /api/hotspot (docs/rest-api.md). The header comment records what the bench decided and why, line by line.

Patterns

The pattern list has two tiers:

Presets — compiled into firmware.bin. Today that is Origin alone, pattern 1, the failsafe; the rest of the curated set in presets/ ships as the Basics pack. A preset works whether or not the filesystem mounts, and switching to one is instant.
Modules — .pfm files on the FATFS partition, discovered at boot and appended after the presets. This is how new patterns normally arrive now: a ~3–22 KB file installed over Wi-Fi at http://patternflow.local/patterns, no reflash, no reboot. See Loadable pattern modules.

Both tiers share the same pattern shape — a namespace with:

NAME
KNOB_LABELS
setup()
update(float dt, const InputFrame& input)
draw() — draws via PFCanvas::setPixel(...) and ends with PFCanvas::present();

and the same foundation (PFCanvas/PFMath/PFColor/PFNoise), so a pattern written for one tier ports to the other mechanically. The everyday way to make one is the Pattern Lab — PATTERN_GUIDE.md — which generates conforming C++ from a JavaScript pattern and installs it over Wi-Fi; CUSTOM_PATTERNS.md is the hands-on route.

To add a preset (rare — curated set): copy _TEMPLATE.h into presets/preset_<name>.h and add one PATTERN_ENTRY(...) line in pattern_registry.h. To add a module (the usual way): see the next section.

Loadable pattern modules (.pfm)

Patterns no longer have to be compiled into the firmware image. A pattern can be built as a relocatable Xtensa ELF module, copied to the device over Wi-Fi, and it appears in the list immediately.

firmware/modules/<slug>/pattern.cpp     ← freestanding pattern source
        │  python firmware/toolchain/build_module.py firmware/modules/<slug>
        ▼
<slug>.pfm  (+ <slug>.json sidecar)     ← 3–22 KB relocatable ELF
        │  drag onto http://patternflow.local/patterns   (several at once)
        ▼
FATFS /patterns/<slug>.pfm  →  loaded on selection in ~5–11 ms

The design comes from Simone Majocchi (@SimonePDA), who proved the whole idea in a working fork: a frozen C ABI between host and module, a linker script that collapses each module to four sections, and an on-device relocator.

How it works, briefly. The firmware exposes a versioned function table (abi/pf_abi.h — framebuffer, present, alloc, log, millis, polar tables, libm hooks). A module is compiled freestanding against abi/pf_module.h only — never against Arduino or the HUB75 driver — and partially linked with toolchain/module.ld so all code lands in one contiguous .text with literals inside it. At load time core_module_loader.h reads the ELF from FATFS, places it (PSRAM first since 2026-10, its code run through the instruction-bus alias), applies R_XTENSA_32 relocations, resolves libm/libgcc against the host's own symbols, runs .init_array (so C++ global constructors work), syncs the instruction cache, and calls the module's entry point. One module runs at a time. Switching away parks it: a module that has run stays loaded in PSRAM, and picking it again resumes it in the frame it is picked — no read, no relocation, no second setup() — the way a compiled-in preset resumes (115–197 µs, measured on two boards). That is when no other load is in flight and no upload, delete or format is holding pattern switches; picked during either, it comes in once that is over — resumed if it is still parked, loaded if an install has emptied the table. Up to 16 are kept (PF_MODULE_RESIDENT_MAX in src/core_module_memory.h; src/core_module_resident.h has the rules), only modules that hold no internal RAM, and they are evicted least recently used first when a new load or a running pattern needs the PSRAM — until a pattern's allocation leaves at least 512 KB free, PF_MODULE_RESIDENT_HEADROOM; a feature's allocation evicts nothing — and all at once when patterns are installed or deleted. A module loaded before its file was replaced — by the console or by a feature — is never resumed from the old file.

Converting an existing pattern header:

python firmware/toolchain/port_preset.py firmware/patternflow/presets/preset_origin.h
python firmware/toolchain/build_module.py firmware/modules/origin
# outputs land in firmware/patternflow/data/patterns/ — upload from /patterns

The port is mechanical (drop firmware includes, add #include "pf_module.h", append PF_REGISTER_PATTERN(Namespace)); the body is copied verbatim because the module SDK provides the same PFCanvas/PFMath/PFColor/PFNoise surface — the math headers are literally the same files, included with -DPF_MODULE_BUILD.

Install paths, easiest first:

A community deck or pattern page → "Send over Wi-Fi" (or a deck's "Install to my board"): opens /patterns?src=<modules-url>, and the page fetches and installs every file itself. Nothing downloaded.
Drag any number of .pfm/.json files onto /patterns — a visible queue uploads them one by one with progress, retries, and per-file results.
curl -F "module=@slug.pfm" http://patternflow.local/api/patterns for scripts.

Costs and limits, measured on hardware (v3.2, 2026-07, 128×64, esp32 core 3.3.8 build — a core 2.x build has more heap and renders faster; see How to measure this):

Property	Measured
Switch latency (7.5 KB module)	6.0 ms = read 4.4 + relocate 0.6 + setup 1.0
Switch latency (22 KB module)	10.9 ms — still under one 60 fps frame
Runtime speed vs the same source compiled in	2.7 % slower — Origin draws a frame in 9.56 ms compiled in (`-Os`) and 9.81 ms as a module (`-O2`); one board, A-B-B-A in one boot, 2026-10-04. The 2026-07 figure of ~20 % (53.4 → 43.5 fps), which this row blamed on relocatable code (`-mlongcalls`), is not in today's build.
Build	`-O2` since 2026-10-04 (`build_module.py`; `--opt s` builds at `-Os` again). On the Basics presets, same board, same boot, `-Os` → `-O2`: 0510 11.9 → 10.3 ms a frame, 0512 23.2 → 22.6, 0515-4 15.0 → 11.2, 0520 23.4 → 19.7, 0531 13.7 → 12.4, 0601 15.0 → 13.6 — 3 to 34 % more frames a second, for a third more code in those builds (67 → 90 KB over the pack). The same change makes `floorf`/`fmodf` and their kin inline and exact in a module (`abi/pf_libm.h`): presets as they were written before PFMath, with those calls per pixel, went 0510 13.3 → 10.3 ms, 0515-4 23.7 → 17.5, 0520 23.1 → 20.3, 0601 24.5 → 20.2 at `-O2` with a first version of that header (a first version of the header; with the header as committed, all 33 Basics modules are faster than their August builds on the same board, 1.11 to 2.36 times the frames, 1.52 at the median). With both, the pack is 104 KB of code, and 145 community patterns built both ways are 1.36 times their old code at the median. Two costs. Room: from 3.10.5 that code is in PSRAM and its size does not decide whether a module loads; before 3.10.5 it has to fit in internal RAM (about 7.8 KB of budget on an Audio build), so a larger pattern that loaded as the old build can be refused as the new one - that firmware needs updating. The last bit: the seven functions are exact, but `-O2` and inline code move where GCC fuses a pattern's own `a*b+c`, so its arithmetic can differ in the last bit from the build before, which shows only where the pattern amplifies it (a float hash, state carried between frames, rarely a `floorf` band edge; `abi/pf_libm.h` has the cases). A module built this way imports nothing an older loader lacks.
Comfortable module size	~32 KB of statics on a core 2.x build — measured: 0–64 KB of static tables all render at the same 62 fps. They spend PSRAM since 2026-10, not internal heap: statics go to PSRAM first, which costs most modules nothing measurable and a few with small per-pixel state arrays 2–3 % (Burgers +2.6 %, Wave Cascade +2.2 %; `PF_MODULE_DATA_PSRAM_FIRST 0` puts small statics back in internal RAM). (The old "≲ 8 KB, then `.rodata` spills to PSRAM and ~14 fps" figure was the core 3.x build's 7.7 KB largest block talking; see Required board package.)
Panel size	Baked in at build (`-DPF_PANEL_W/H`); the loader rejects a mismatch
ABI	`PF_ABI_VERSION` must match; bump it on ANY layout change in `pf_abi.h` and rebuild every module

If a module fails to load, nothing else is affected — the presets and other modules keep working, the panel shows a PATTERN FAILED screen with the reason, and /api/status reports it as loadError. Check loadError first for any "this pattern is broken" report — it turns a vague complaint into a specific one.

unresolved symbol: X used to be the usual cause, and it is worth knowing what it cost before it was closed: the table carried tanhf but not coshf, and since sech(x) = 1/cosh(x) is the closed form of a soliton, an entire genre of wave patterns failed to load in a way that read as "too heavy for the board". As of v3.5.2 the table was audited in bulk — every shipped module's demand plus a synthetic probe of the whole libc/libm surface (hyperbolics, erf, the rintf family, 64-bit integer helpers, qsort/strto*, string/ctype, an atexit shim) — and build_module.py now checks every .pfm it links against the loader's own table, so a pattern the device would refuse fails on the build machine with the missing names spelled out. If a new name ever does surface, the fix is still one PF_HOST_SYMBOL line in resolveHostSymbol() — add it there and the build checker learns it automatically, since it parses that header.

Pattern names are UTF-8. "Dynamic Moiré" and "Poincaré Sphere" are real entries in the library, and an accent used to break three layers at once: the toolchain scripts printing a name on a cp949 console, a loader guard that tested names for printable ASCII and called anything else a relocation bug, and the panel's 5×7 ASCII font. Names stay UTF-8 in the ELF and in every JSON API; only the panel folds them (asciiFold() in patternflow.ino). Any new code that touches a name must assume UTF-8.

How many can you install

Installing costs nothing at runtime. The registry allocates its arrays at full capacity on boot, in PSRAM, whether or not the modules exist — so five installed modules and 128 installed modules use exactly the same RAM. Only the resident module costs internal RAM, and unloading returns all of it (measured: 4,548 B free with a module resident → 11,692 B on a preset). Since 2026-10 a module's code and data both go to PSRAM first, so a module that loaded normally costs no internal RAM at all, and a module kept loaded after being left (parked, see above) only ever holds PSRAM.

Limit	Value	Binding?
Installed modules	128 (`MAX_MODULE_PATTERNS`)	The only real cap, and it is a UX choice — 136 B of PSRAM per slot, 17 KB total
Storage	~1,500 modules (10.2 MB partition, 5.9 KB median)	No — 12× the count cap
Per-module RAM	~32 KB of statics comfortable on a core 2.x build (0–64 KB measured at identical fps; the old ≲ 8 KB figure was the core 3.x heap talking)	PSRAM since 2026-10. Frame rate mostly does not move; a few modules whose small per-pixel arrays used to sit in internal RAM run 2–3 % slower (Burgers +2.6 %, Wave Cascade +2.2 %) — `PF_MODULE_DATA_PSRAM_FIRST 0` trades that back for internal heap
Concurrent modules	1 running, up to 16 parked (`PF_MODULE_RESIDENT_MAX`)	By design — one pattern is selected at a time; parked ones hold PSRAM only and give it back to a new load or a running pattern that needs it

A large module only costs internal RAM while it is the selected pattern, if at all; it has no effect on the presets or on any other module. On the core 3.x build a resident module dropped heapLargest to ~3 KB — which is what made opening a console page pause the pattern; on the core 2.x release build the same module leaves ~65 KB and the console stays comfortable (the pause-on-open mechanism remains, see the constraints section below).

Measured across the real 42-pattern community library: median .pfm 5,924 B, largest 17,512 B, smallest 4,612 B, 281 KB for all 42 together.

How to measure this, before reading it

heapLargest is not a property of a build. It decays as the device serves HTTP, in steps of exactly 2,048 B — polling a build eighteen times walks it down ~4 KB — so a firmware sampled often reads worse than one sampled rarely, and two numbers taken over different windows do not compare.

Take it this way, or do not quote it:

flash → POST /api/wifi/reboot → wait ~80 s → one read of /api/status

Two names that mean different things, and have already been mixed up here:

Flash. The toolchain's figure is program storage (1,412,457 = 44.9 % of the 3,145,728 B partition). The .bin on disk is ~360 B larger — header, padding, hash. Say which one you mean.
RAM. The build's 92,920 is static RAM used, 28.4 % of 327,680. Free internal heap is a different number and only the running device knows it.

And the rule the rest of this section exists to serve: a heap number travels with its build, its procedure and its date, or it is not evidence. The figures below are v3.5.2 on a core 2.x build and are quoted as history — the September 2026 runtime report has the 3.10 numbers, taken under the protocol above. In 2026-08 the 86,004 below was copied into five documents as a measurement of something else entirely.

Current state (v3.5.2, measured on hardware, core 2.x build)


Frame time, preset (Origin)	~16,270 µs = 61.5 fps
Internal heap, preset resident	96,324 B free, largest block 86,004 B
Internal heap, module resident (Muybridge 35mm, 29 KB `.pfm`)	76,668 B free, largest block 65,524 B — 48.5 fps
PSRAM	8.33 MB idle
Flash	1,148,960 B = 37 % of the app partition
Globals	84,660 B = 26 %, 243 KB left for locals
Pattern storage	131,072 / 10,235,904 B with 13 modules installed
Leak check	48 consecutive page loads: −8 B net, largest block unchanged (v3.1.0 measurement; not re-run since)

The v3.1.0 / core 3.x numbers this table used to carry — 53.4 fps, 11.7 KB free, 7.7 KB largest block — are what the constraints section below calls the other build. Anyone comparing a measurement against this table should first check which core their firmware was built with.

Adding features — the constraints that matter

Read this before adding anything to the firmware. Every item here was learned the expensive way, with measurements to back it.

Internal RAM is the budget. Everything else is roomy.

The ESP32-S3 has 512 KB of internal SRAM, but what a feature can actually claim is far smaller:

≈320 KB   Arduino data region (of 512 KB SRAM; rest is cache/ROM/RTOS)
 −92 KB   this firmware's globals (24 KB canvas, fonts, module state…)
 =213 KB  free heap at boot                          ← measured, core 2.x
 −150 KB  HUB75 DMA framebuffers (see below)
 −49 KB   Wi-Fi socket buffers, HTTP, OSC, mDNS…
 ≈ 15 KB  steady-state free internal heap

That last line is a build choice, not a hardware limit. The figures above are the core 3.x picture. Core 2.x gives back roughly 71 KB that IDF 5 takes before setup() even runs, so the same firmware idles at ~98 KB free, 90 KB largest block — see Required board package. Build on core 2.x and the numbers below stop being the constraint they were. Build on core 3.x and they are real, so know which one you measured on before concluding a feature is too expensive.

Below roughly 10 KB free, the web console starts failing in a maddening way: every endpoint returns its status line and then hangs, while Wi-Fi, OSC and OTA look perfectly healthy. A 7 KB static array added during development took down four unrelated pages at once. So:

Static buffers are guilty until proven innocent. If data is written at boot and read occasionally — lists, tables, names — allocate it from PSRAM (heap_caps_calloc(1, size, MALLOC_CAP_SPIRAM)). There are 8 MB of PSRAM sitting idle; the pattern registry lives there for exactly this reason. Reserve internal RAM for hot loops and DMA.
Check /status (or /api/status) after your change: it reports free internal heap, largest block, and PSRAM live from the device.

Why HUB75 owns 150 KB and it cannot move

HUB75 panels have no framebuffer of their own and only understand on/off per pixel. The driver therefore streams the whole frame continuously (the panel lights 2 rows at a time) and achieves 256 brightness levels by re-sending the frame as 8 binary-weighted bit-planes (BCM). That waveform is what DMA plays out autonomously:

128 cols × 32 row-pairs × 2 B = 8 KB per bit-plane
× 8 bit-planes                = 64 KB per frame buffer
× 2 (double buffering)        = 128 KB  (+ descriptors ≈ 150 KB)

It must be internal RAM — PSRAM's latency jitter glitches the 15 MHz stream. Shrinking it costs image quality directly (6-bit color = banding in exactly the gradients these patterns live on; single buffering = tearing). This is the standard tax every ESP32-driven HUB75 project pays; in exchange, refresh costs zero CPU and the render loop effectively owns a core.

The Arduino WebServer will hurt you. Known landmines:

Writing a client rather than a route? The endpoints, their JSON shapes and the rules that follow from this section are specified in docs/rest-api.md. Build against that file; this one explains why the server behaves as it does, which is what you need when adding to it.

The console shares one synchronous, single-client WebServer on port 80 (owned by core_audio_ws.h; other features attach routes to it). Learned so far, each confirmed by A/B on hardware:

A multipart POST with a URL query string gets an empty reply. Send flags as form fields instead (fd.append('last', …)), never ?last=1 on an upload.
Never do heavy work inside a request handler. A FATFS rescan + module reload inside the upload handler crashed the device mid-batch — and the crash corrupted the FAT. Handlers set a flag; tick() functions called from loop() do the work ~150 ms later (see core_patterns_http.h).
Never force-close connections from the server (client().stop()). Each server-initiated close parks a TCP pcb in TIME_WAIT; after a couple of upload batches every multi-chunk transfer died until reboot.
Rapid sequential uploads flake ~1 in 12 even when healthy (dead reply after the file was stored). Client pages pace files ~350 ms apart and retry each file twice; re-uploading is harmless (same bytes, same name).
Never auto-format the filesystem on a failed mount. A crash mid-write once corrupted the FAT and the "helpful" auto-format erased every installed module. Mount failure now means presets-only; formatting is an explicit button (POST /api/patterns/format).
A response larger than ~5.6 KB needs the heap a loaded module is holding. This one looked for a long time like a slow "leak" that degraded over hours; it is not. Measured on one device, one minute apart:
```
module resident    internal heap  4,932   /patterns truncated at 5,633 B
                                          after a 10 s stall
module unloaded    internal heap 11,852   /patterns whole (15,903 B), 0.43 s
```
5,633 B is four TCP segments — one bufferful. Under a starved heap the one-shot send() fills that and gives up, and the client waits out the timeout holding half a page. The HTML renders, its script is cut mid-statement and never runs, and the console looks blank while every API underneath answers fine.

Until 3.6.3 the console therefore paused the pattern: opening a page evicted the module and restored it after 25 s of silence. Since 3.6.3 the page sender streams PROGMEM in small slices under a 5-second budget, the pages are gzip-compressed, and the core-2 builds have the heap for it, so a page no longer evicts anything; status.consolePaused now means only that a pattern-install batch is in progress (docs/rest-api.md). The memory wall below is still real, and is why pages stay small.

This is a memory wall, not a pacing problem. It is tempting to read the 10 s stall as impatience — NetworkClient::write() really does send with MSG_DONTWAIT and give up after WIFI_CLIENT_MAX_WRITE_RETRY (10) non-writable selects of 1 s each, which is where the ten seconds comes from. Rewriting the send path to be patient does not help, because the number that matters is heapLargest: 3,060 bytes with a module resident. lwIP cannot allocate the pbufs, and no amount of waiting creates them.

Three fixes were tried on hardware and do not work; do not retry them:
- Spilling module data sections to PSRAM to spare internal RAM — reboots the device the moment a module is selected.
- Chunked transfer encoding — delivers less, not more (one 1 KB chunk, or nothing), with or without pacing between chunks.
- 512-byte slices with a wall-clock deadline instead of a retry count (core_http_send.h, reverted in bb8e52c) — delivers 3,072 B, worse than the 5,633 B it replaced, and still takes the full 10 s. It is ~3× faster than send_P on a healthy heap, which is exactly the trap: it measures beautifully in the condition that was never the problem.
An async server (ESPAsyncWebServer or esp-idf httpd) would lift the one-client and upload-pacing limits, but it shares this same lwIP heap — do not expect it to make a 16 KB page deliverable on a 4.5 KB heap. The only real levers on page delivery are using less internal RAM per module or shrinking the pages.

Adding a console page

The pages are plain HTML in patternflow/console/; edit them there, preview with console_serve.py, and console_pages.py build splices them into the *_index.h headers — the whole workflow is patternflow/console/README.md. A page that belongs to a feature travels with the feature (navPath/navLabel/navDesc in its descriptor); the core never names it. Two rules that the README does not repeat:

Syntax-check the page's JavaScript before flashing: extract the <script> body and run node --check on it. A single stray newline in a string once shipped a page whose script never ran — the page rendered but showed nothing, which reads as "the device is broken".
Never leave the render loop with nothing to draw. If a screen decides not to paint, set frameDrawn = false; flipping an unpainted buffer shows a torn leftover frame, which every tester reads as "the pattern is broken".

What is cheap and what is expensive here

Cheap (fits the platform well):

Wired OSC over USB-C — the S3's native USB does CDC serial out of the box; OSC 1.1 over SLIP is a standard TouchDesigner/Max speak. Reuses the existing OSC message parser with a different transport. Near-zero RAM, independent of Wi-Fi entirely. A very good first contribution.
More device pages, more OSC addresses, more module-SDK helpers (add the symbol to the host table + bump nothing, if it's a new function on PFHostAPI — that's an ABI bump).
Anything whose cold data can live in PSRAM.

Expensive (think twice, or push to "Remote compute"):

TLS clients/servers (tens of KB of internal RAM per connection).
Many concurrent WebSockets or HTTP clients (single-threaded server today).
Anything needing large always-resident internal-RAM buffers — that budget is 15 KB, total, shared with everyone.

Configuration (`config.h`)

All hardware-specific pins and limits are centralized in config.h.

Panel Selection: PANEL_PROFILE picks the driver-init path — PANEL_STANDARD (no init; the default and what the browser flasher ships), PANEL_HIGHREFRESH (FM6126A), PANEL_FM6124, PANEL_ICN2038S, PANEL_MBI5124, PANEL_DP3246. It expands to the HUB75_DRIVER value core_display.h passes to the library. Don't set it by part number — leave it at PANEL_STANDARD unless the panel comes up completely dark. The init sequence only writes a brightness register and an output-enable bit; many panels ship with those already usable, and the reference panel's FM6124 runs fine without it. A panel that genuinely needs a different profile needs one custom build. Full buyer's guide: docs/panel-compatibility.md.
Pin Mapping: Adjust the R1_PIN, ENC1_A etc. if you are not using the official Patternflow PCB.
Hardware Settings: INVERT_ENCODER can be toggled depending on whether you mounted your encoders on the front or back of the PCB. DEFAULT_BRIGHTNESS controls the initial matrix brightness.

LED panel calibration

LED panels render the same RGB triplets differently than a calibrated monitor — the LED primaries are at different wavelengths than sRGB phosphors, red LEDs are brighter per PWM duty than blue, and the panel reads washed-out next to a monitor showing the same frame. The defaults are the ones that ship in config.h (saturation tuned by eye on 2026-08-09 against the Calibration preset, on the maintainer's panel; white balance is identity since 2026-09-17, when the bit-plane weights were made binary and the earlier trim turned out to have been correcting the driver rather than the panel); every value is #ifndef-guarded, so a panel can override any of them from patternflow_secrets.h:

#define LED_GAMMA_R   1.0f    // the pow-LUT already handles perceptual response;
#define LED_GAMMA_G   1.0f    // extra gamma here only made the panel darker
#define LED_GAMMA_B   1.0f
#define LED_WB_R      1.000f  // identity: the reference panel reads neutral uncorrected.
#define LED_WB_G      1.000f  // Judge a trim on the GRAYS screen, not on white alone
#define LED_WB_B      1.000f
#define LED_SAT_BOOST 1.62f   // strong: matches design intent on real patterns

Tune live from /api/display (the /status page has the sliders), then land the converged numbers in your secrets file. The comments in config.h above each value record how they were arrived at.

Refresh rate (anti-flicker for video)

core_display.h sets the refresh floor, and the vendored driver lands the panel on exactly 300 Hz:

mxconfig.i2sspeed         = HUB75_I2S_CFG::HZ_15M;  // pixel clock — see below, this is 16 MHz
mxconfig.min_refresh_rate = 240;                      // refresh floor (shipped: exactly 300 Hz)
mxconfig.latch_blanking   = 2;                        // brightness uniformity

The library's clock enum names are wrong. HZ_15M is 16000000 and HZ_10M is 8000000 — HZ_8M and HZ_10M are the same value, and there is nothing between 8 and 16 MHz. Read the enum, not the name, and if you are hunting harmonics on a spectrum analyser the fundamental is at 16 MHz (so 96, 112, 128, 144, 160 MHz…), not 15.

A HUB75 panel lights one row pair at a time: each row is on for 1/32 of a refresh and dark for the rest — a short pulse, not a glow. A phone's rolling shutter exposes each sensor line for the same length of time but starting a little later than the one above, so each line collects a whole number of those pulses. If the exposure is an exact multiple of the refresh period, every line collects the same number and the picture is clean. If it is not, some lines get N pulses and some N + 1, and that is the banding you see on video — its contrast is about 1/N, and how much of the picture it covers depends on how far the exposure is from a whole number of periods. A faster refresh makes the bands fainter and more numerous; it does not make them go away.

Exposure times are not arbitrary, though. To dodge mains flicker a phone pins them to 1/50, 1/100, 1/25 (50 Hz countries) or 1/60, 1/30 (60 Hz countries), and 300 Hz is the lowest rate all of those are whole multiples of. The shipped chain would naturally run at 325.5 Hz — nearly the worst case for 1/50 (6.51 periods) and 1/60 (5.43) — so the driver ends every frame with 4,181 blank clocks, output off, to make it 16 MHz / 300 = 53,333 clocks exactly (PF_TARGET_REFRESH_HZ in src/hub75/, 0 to switch off; it never pads below min_refresh_rate). About 8% of the light, and no CPU: refresh runs on the ESP32-S3's DMA hardware in parallel with everything else.

What 300 Hz cannot fix: 1/120 and faster, and slow motion. At 1/120 an exposure is 2.5 periods — the worst case — and the rate that would cover it, 600 Hz, is out of reach of a 16 MHz clock at 8 bits. When filming: lock the shutter at 1/60 or 1/50 (a manual camera app, or pro mode), or turn the panel down so the phone lengthens its exposure; lock white balance and focus; switch HDR and night modes off.

The trade-off is brightness. A row is sent as a chain of bit-plane buffers, and the vendored driver picks the brightest chain that still clears min_refresh_rate (pfChainExtraPasses in src/hub75/, and VENDORED.md beside it for why the chain is not upstream's). Every scheme is exactly binary — colour depth is never what gets sacrificed — but a shorter chain keeps less of each row lit. Both settings move together — you cannot lower one without deciding about the other. On the reference panel (128×64, 8-bit):

`i2sspeed`	`min_refresh_rate`	Chain	Row lit	Actual refresh
16 MHz	240 (shipped)	scheme 0, 12 buffers, padded	61%	300 Hz exactly
8 MHz	240	scheme 2, 8 buffers	25% — dimmest	~244 Hz
8 MHz	200	scheme 1, 9 buffers	44%	~217 Hz
8 MHz	150	scheme 0, 12 buffers — same as shipped	66%	~163 Hz

So if you drop the clock, drop min_refresh_rate with it — halving the clock while leaving 240 in place is the worst of both worlds, giving up most of the panel's light to claw back a refresh rate you were not going to keep anyway. Banding on video of the panel is a matter of shutter against refresh, as above — at 8 MHz the frame is already longer than 1/300 s, nothing is padded, and the rates in the table are what you get. (The 8 MHz rows are computed from the chain lengths, not re-measured since the chain changed; the 16 MHz row is what the panel reports.)

Lowering the panel clock is also an EMI knob, and it has been measured. 8 MHz improves Wi-Fi latency on the reference unit (median 15 → 4 ms) and one contributor found it was the difference between a usable and an unusable radio on their board — but at any min_refresh_rate that preserves colour depth it bands on video, so it is not shipped. Full measurements, the parts that cannot be adopted (do not raise the Wi-Fi TX power — it is a conformance setting), and what to try first if your unit's Wi-Fi is bad: docs/investigations/2026-08-the-panel-clock-and-the-wifi-radio.md.

Controls

The four rotary encoders control pattern-specific parameters. Each pattern exposes its own labels through KNOB_LABELS.

For the original defaults:

Encoder 1: Hue
Encoder 2: Speed
Encoder 3: Mode/Preset
Encoder 4: Frequency

Longpress actions

Encoder 1 longpress (≥1s) — enter/exit global brightness mode. While active, K1 rotation adjusts panel brightness (5–255, ~5 per detent), the active pattern does not see K1 input, and a "BRIGHTNESS XX%" overlay shows the current level. Exits on a second longpress or after 5 seconds of idle. Value persists across reboots via NVS.
Encoder 2 longpress (≥1s) — enter/exit the NETWORK screen (portrait status view: the Wi-Fi state and the board's IP address — or, while the panel has no Wi-Fi and its own hotspot is up, HOTSPOT and the hotspot's name — plus a row for each of the first two features in the build that can be switched at runtime). The core does not know what those rows are: a feature appears by declaring a short name and a switch (features/README.md). On the default build there are none; on the Audio edition they are OSC and AUD (audio-react). Inside the screen, rows are toggled by knob rotation, not clicks — right = ON, left = OFF — so that holding K2 to leave can't flip anything on the way out:
- turn K2 → the first row on/off (persists in NVS, so the device reboots into the same state). Wi-Fi stays connected either way; the switch only enables/disables that feature's traffic.
- turn K3 → the second row on/off (also persisted).
- turn K1 → sleep (see Sleep mode below). Either direction, unlike the toggles above: the screen is about to go dark, so "right = on" has nothing to mean.
- turn K4 → hand off to the UPDATE screen, which arms the /update endpoint (the arming is the security model — see core_web_update.h).
Exits on a second K2 longpress, a K2 click, or after 8 seconds of idle.
Encoder 3 longpress (≥1s) — enter/exit the KNOB MAP screen: it shows which physical knob is which number (front view: K1 top-right, K2 top-left, K3 bottom-right, K4 bottom-left), and turning any knob lights its digit green so each one can be verified without leaving the screen. Knob input is swallowed while it's up, so the pattern underneath never sees it. Exits on a K3 click, a second K3 longpress, or after 8 seconds of idle.
Encoder 4 longpress (≥1s) — enter/exit pattern SELECT mode. In SELECT mode, K4 rotation moves the highlight through the list — three detents per pattern, so a hand does not overshoot — and the panel shows each pattern's thumbnail (the frame it last drew, kept from the last time it ran); the highlighted pattern loads once the knob has rested for a third of a second — on the other core, so the knob keeps answering while a heavy pattern's setup runs — then runs live behind the overlay. A pattern already in memory — a preset, or a module still parked from its last run — comes on as the knob reaches it and runs live straight away (unless the pattern being left cannot be kept, when it waits for the knob to rest like any other). A pattern that has never run shows nothing but its name until it has. Longpress again to confirm — a choice the knob was still resting on loads right then.

Thumbnail ownership follows the frame boundary too: the loop owns the PSRAM cache, and the existing network task reads/writes one immutable job at a time. Capturing a picture copies pixels; it opens no file. A completed read is adopted between frames, and deletion/format invalidates jobs by generation. The optional cache and its one 16 KB work buffer use PSRAM exclusively; allocation failure leaves the name-only preview. No extra task stack is reserved. This separation does not shorten a module's own setup(); the asynchronous loader still handles that wait. /api/status.thumbs separates capture cost from background I/O cost.

One detent, one step

Knob deltas are linear: a pattern sees exactly the number of detents that were turned, however fast the turn was.

There used to be a fast-spin multiplier here (×2 to ×5 as the gap between detents shrank), meant to let one knob sweep a wide range quickly. It was removed after testing against a linear build, because it made the knobs unpredictable on exactly the parameter that most needs landing on a value — Origin's Mode knob picks a discrete preset, and a quick turn skipped five at a time. OSC had already been routed around the multiplier for the same reason, which was the tell.

If a pattern's range feels slow to cross, raise that pattern's own step constant (d * 10 → d * 25). Linear and predictable beats a curve guessing at intent.

Short press (per-pattern, opt-in)

There is no global short-press handler. Each pattern decides what K1..K4 short press does for itself, by reading input.btnPressed[i] inside its update(). The built-in patterns (Origin, Wave Saw) use short press to reset the corresponding parameter to its default. A pattern that does not handle btnPressed — most of the curated presets — simply ignores short presses.

When you generate a new pattern from the Live Editor, the conversion prompt does not force a particular short-press convention — if you want one, either ask for it in the prompt ("K1 short press resets hue") or add the line by hand in update().

Sleep mode

src/core_sleep.h. The panel goes dark and the board idles, without anything being unplugged, and it stays on the network the whole time.

That last part is the design decision. esp_deep_sleep would take the draw to microamps, but it takes the radio with it, and then the only way back is a button — which is exactly what "switch the lights off from the sofa" needs not to be true. So sleep here means:


Panel	framebuffer blanked, brightness 0, HUB75 DMA transfer stopped
Wi-Fi	modem sleep — still associated, wakes on its DTIM beacon
CPU	80 MHz, the floor at which the radio still works
`loop()`	returns early and yields 20 ms; no render, no flip, no power clamp

Stopping the DMA transfer is what makes this worth more than blanking. The LEDs are most of the draw and blanking collects that, but with the transfer running the panel's driver ICs are clocked at 15 MHz forever, lit or not. core_power.h puts that idle floor at ~550 mA against ~1900 mA for a bright pattern.

The yield is not politeness either: without it the sleeping loop polls sockets flat out and the core never idles, so neither modem sleep nor the IDF's clock gating gets a chance.

Entering and leaving

Enter	Leave
the On / Sleep switch on the console home page (`/`)	the same switch
NETWORK screen (hold K2) → turn K1	any physical knob turn or button press
MQTT `<prefix>/sleep` ← `1` / `on` / `true` / `sleep` / `toggle`	same topic ← `0` / `off` / `false` / `wake`
`POST /api/sleep` ← `on=1` (also `on=toggle`)	`POST /api/sleep` ← `on=0`
	a firmware image starting to arrive (web update or `espota`)

"Physical" is meant literally: waking checks the raw encoder counters, not input.knobDeltas, because remote OSC/MQTT/audio deltas are merged into that field by the time the input frame exists. A show still streaming knob values at a sleeping panel must not switch the lights back on. Input is ignored for the first 800 ms so the second detent of the K1 turn that started the sleep doesn't end it.

Sleep is never entered while an image is being written to flash, and a device already asleep wakes when one starts arriving — whoever is flashing wants the UPDATE screen anyway.

The state is not persisted: a device unplugged while asleep boots awake, because a panel that stays dark after you plug it in reads as broken.

Waking gives the pattern back

The one thing that still evicts the resident module is a pattern-install batch (a console page has not since 3.6.3). So a wake — from any of the three sources, not just the web switch — asks for the pattern back via PatternflowPatternsHttp::requestReload(), the same path the console's Play Now button uses, and gets it whether or not something was holding it. The reload happens at the frame boundary from loop(), never inside an HTTP transaction.

The switch, and why `/api/sleep` is its own endpoint

The console home page carries an On / Sleep pair in the device card, styled like /mqtt's channel buttons — the console's existing way of picking between states. It is a view of the device's state: K1 and MQTT change the same thing, and the page's 3-second /api/status poll follows them within a beat.

POST /api/sleep only queues the transition (PatternflowSleep::request()), so its reply reports the state as it stands, not as it will be — stopping a DMA engine and reclocking the CPU belong in loop(), not inside an open response. The page therefore paints optimistically and suppresses the poll for 1.5 s, or the reply would snap the switch back under the cursor.

Sleep control deliberately does not live on /api/display, where the plumbing first landed. That endpoint is independently compile-out-able (-DPF_DISPLAY_HTTP_ENABLED=0), which would leave the switch dead in a build with an otherwise complete console — and sleep is a power state, not a display calibration. /api/display still reports sleep, read-only, because a dark panel is the first thing to rule out while tuning.

MQTT

<prefix>/sleep is obeyed in either role, unlike the knob and pattern topics — a panel set to Publisher is still a panel you want to be able to switch off. It is a broadcast for the same reason <prefix>/message is: one 1 on a shared prefix sleeps every panel on it. Give a panel its own prefix if that is not what you want, and send the command non-retained unless you really do want panels coming back asleep after every reconnect.

The device publishes <prefix>/sleep/state (0 or 1, non-retained) on every change and once per connection — enough for a Home Assistant switch.

mosquitto_pub -h <broker> -t patternflow/sleep -m 1
mosquitto_sub -h <broker> -t 'patternflow/sleep/state'

Unrecognised payloads are ignored rather than guessed at; a typo should not black out a panel.

Compile-time flags (`config.h`)

Flag	Default	Meaning
`PF_SLEEP_ENABLED`	`1`	compile the feature in at all
`PF_SLEEP_STOP_DMA`	`1`	stop the HUB75 transfer, not just blank. Set to 0 if the panel comes back black or garbled after waking — sleep then blanks only, which is safe on any panel and still saves the LED current
`PF_SLEEP_CPU_MHZ`	`80`	clock while asleep; `0` disables the switch
`PF_AWAKE_CPU_MHZ`	`240`	clock restored on wake

PF_SLEEP_STOP_DMA depends on resumeDMAoutput(), a Patternflow addition to the vendored HUB75 driver — upstream's stopDMAoutput() is a one-way trip. See src/hub75/VENDORED.md.

Remembering the selected pattern

The pattern you left running comes back after a sleep and after a full power cut. It is stored in NVS as a slug, not an index: installing or deleting one .pfm renumbers the whole list, so an index would come back as somebody else's pattern. Resolution on boot goes through findPatternByName() in pattern_registry.h, which matches display name and slug alike.

The write is debounced — it waits until SELECT mode is left and the choice has been still for 3 seconds — so spinning K4 through fifty patterns is one NVS write, not fifty. Brightness has used the same shape for longer.

A remembered pattern that is gone, or that no longer loads, falls back to Origin with a log line. The PATTERN FAILED screen is the right answer for a pattern somebody just picked and the wrong one for a boot: nobody asked for it in this session.

Audio-react, OSC, MIDI, MQTT

None of these is in the core. They are features, and each ships in an edition you install from the shelf: audio-react, OSC and network MIDI in Audio, MQTT in Performance. What a client sends is a written contract in docs/ — audio-ws-spec.md, osc-spec.md, midi-spec.md, mqtt-spec.md — and the guides are AUDIO_GUIDE.md (sound in, MIDI, OSC) and docs/midi-ableton.md. The ready-made clients are the audio extension and phone app in tools/ and the Max for Live bridge in integrations/ableton/.

From a pattern's point of view every one of them is the user turning the four encoders: audio bands, OSC deltas and MIDI CCs arrive as knobDeltas (or pin a lane on the absolute parameter bus), so an encoder-driven pattern reacts with no transport-specific code.

Two switches, for a local build:

Compile-time — whether a feature is linked in at all is the bundle: firmware/bundles/<edition>/features_local.h lists the features an edition carries (bundles/README.md). A feature's own tunables live in features/<name>/<name>_config.h (the audio WebSocket port and knob scale in features/audio/audio_config.h, the OSC ports in features/osc/osc_config.h, the broker defaults in features/mqtt/mqtt_config.h), every one #ifndef-guarded so a line in patternflow_secrets.h wins. PF_OSC_ENABLED 0 in the secrets file compiles OSC's traffic out of an edition that would otherwise carry it (its NETWORK row stays, and switches nothing).
Runtime — once compiled in, a feature with a runtime switch gets a row on the panel's NETWORK screen (hold K2; turn K2 for the first row, K3 for the second), the state saved in NVS. On the Audio edition those rows are OSC and AUD; MIDI is switched on its console page (/midi). OSC learns your computer's address from the first packet it receives — send /patternflow/ping (the Max for Live bridge's Connect button does) — see docs/osc-spec.md.

Wireless update from the browser

Once the device is on Wi-Fi, it can flash itself from any browser on the same network — no Arduino IDE, no USB cable, no TLS setup (#232).

Get a firmware .bin, or let the site fetch it for you: when a newer release exists the console says so and links to patternflow.work/update, which hands the image to the panel; an edition installs the same way from the shelf. A file of your own is a release's patternflow.ino.bin (or patternflow-<edition>.ino.bin), or a local build (./firmware/bundles/build.sh, or build.sh <edition> flash <host>, which also sends it) — the app image, not a merged full-flash image.
Open http://patternflow.local — the device's console — and pick Update in its header, or go straight to /update. If .local doesn't resolve (common on Android), use http://<ip>/update; the device shows its IP on the NETWORK screen (hold K2) and on the UPDATE screen.
Drop the .bin on the page. The panel shows flash progress; the device verifies, reboots, and comes back on the new firmware in about ten seconds.

Notes:

Uploads are accepted at any time by default (PF_WEBUPDATE_ALWAYS_ARMED 1): drop a .bin whenever, no trip to the device. The flip side, stated plainly: anyone on the same Wi-Fi can flash the device from a phone browser — the same exposure ArduinoOTA's no-password default already has. On shared, office, or exhibition Wi-Fi, build with #define PF_WEBUPDATE_ALWAYS_ARMED 0 in patternflow_secrets.h: uploads are then refused (403) unless the UPDATE screen is open on the device (hold K2 → NETWORK, turn K4) — a physical arming switch only someone at the device can flip; leaving the screen (K4 click, or the 10-minute idle timeout) disarms it again.
A failed or interrupted upload leaves the old firmware running — Update.h only switches the boot partition after a complete, verified image. Power loss during the flash write is the one case to avoid; the panel says so while flashing.
Like all wireless paths, this capability has to arrive over USB once (it ships in the stock release).
Set #define PF_WEBUPDATE_ENABLED 0 in patternflow_secrets.h to compile it out.

OTA Updates (For Developers)

The firmware includes ArduinoOTA for wireless flashing from the Arduino IDE — no USB cable, no port juggling.

One-time setup

Copy patternflow/patternflow_secrets.example.h to patternflow/patternflow_secrets.h and fill in your local Wi-Fi credentials:
```
#define PF_WIFI_SSID "your-wifi-name"
#define PF_WIFI_PASS "your-wifi-password"
```
(You don't need to enable PF_OSC_ENABLED — OTA brings up Wi-Fi on its own. One secrets file now covers Wi-Fi, OTA, OSC, and audio-react.)
Flash once over USB as normal. On boot, the serial console should print:
```
[OTA] Ready — hostname "patternflow.local", IP 192.168.x.x
```

Subsequent uploads

In Arduino IDE, open Tools → Port — you should see patternflow at 192.168.x.x (ESP32) alongside the USB ports.
Select that network port and hit Upload. The IDE compiles, pushes over Wi-Fi, and the device reboots into the new firmware.
Progress prints to serial as [OTA] 47% etc.

If the network port doesn't appear, make sure your computer and the device are on the same Wi-Fi subnet, and that no firewall is blocking mDNS (UDP port 5353) or the OTA port (3232).

The upload-password prompt (and how to avoid it)

The firmware ships with no OTA password (PF_OTA_PASSWORD ""), so the device never challenges for one. But Arduino IDE 2.x always pops a password dialog for network ports and refuses an empty field — that's an IDE limitation, not the firmware. Two ways around it:

Stay in the IDE: type any dummy character in the prompt. With no-auth firmware the device ignores it, so the upload still succeeds.
Skip the prompt entirely (recommended for no-password): upload from the command line, which never asks. With the ESP32 core's bundled espota.py:
```
python espota.py -i patternflow.local -p 3232 -f /path/to/patternflow.ino.bin
```
(no -a flag = no password). Or arduino-cli upload -p patternflow.local -b esp32:esp32:<board> patternflow/.

To require a password instead, set #define PF_OTA_PASSWORD "your-secret" in patternflow_secrets.h.

Known issue: `invalid int value: '{upload.port.properties.port}'`

Arduino IDE 2.x's mDNS discovery for ESP32 core 3.3.8 doesn't populate the {upload.port.properties.port} placeholder, so espota receives the literal string and fails:

espota.exe: error: argument -p/--port: invalid int value: '{upload.port.properties.port}'

One-time fix — create a platform.local.txt next to the ESP32 core's platform.txt (the IDE auto-merges local overrides, so this survives ESP32 core updates):

Path on Windows: %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\3.3.8\platform.local.txt
Path on macOS: ~/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/platform.local.txt
Path on Linux: ~/.arduino15/packages/esp32/hardware/esp32/3.3.8/platform.local.txt

Contents (one line):

tools.esp_ota.upload.pattern={cmd} -i {upload.port.address} -p 3232 "--auth={upload.field.password}" -f "{build.path}/{build.project_name}.bin"

This hardcodes the OTA port to 3232 (which is what ArduinoOTA always listens on anyway). Restart the Arduino IDE after creating the file.

Disabling / customizing

Set #define PF_OTA_ENABLED 0 in patternflow_secrets.h to compile OTA out entirely (no Wi-Fi stack pulled in unless OSC or audio-react is also enabled).
Set #define PF_OTA_HOSTNAME "yourname" to advertise as yourname.local instead of patternflow.local — useful if multiple devices are on the same network.
OTA ships with no password by default (PF_OTA_PASSWORD ""). Set #define PF_OTA_PASSWORD "your-secret" in patternflow_secrets.h to require one on a shared network. See "The upload-password prompt" above for the Arduino IDE 2.x quirk.

License

MIT - see root LICENSE-MIT file.

FilesExpand file tree

README.md

Latest commit

History