From f5db090499dca9ecb37f018225f5947767d076f4 Mon Sep 17 00:00:00 2001 From: DevomB Date: Thu, 15 Oct 2026 09:42:22 -0700 Subject: [PATCH 1/3] Laptop sound past HD Audio and MIPI cameras: what each needs, and why 1.0 ships without them laptop.md and ADR-021 named Intel's SOF, AMD's coprocessor and the smart amplifiers as gaps, and camera.md named MIPI cameras, without saying what closing them takes or what each kind of laptop does meanwhile; the guide's known limitations did not mention laptop sound at all. docs/design/laptop-audio.md says what each needs from boot.fragment, firmware.list and the mixer, against Linux 6.18's Kconfig and linux-firmware 20260916: the amplifiers and AMD's microphones need the ASoC core and modules, with firmware already in linux-firmware or none; SOF needs sof-bin as a second firmware source, a softdep, and UCM in the mixer, and once built it takes Intel laptops with digital microphones from snd-hda-intel. Everything is a module but the SPI core, which SoundWire and the amplifiers on SPI need. It recommends a named gap in 1.0, and an order after it. camera.md gains the same for MIPI cameras, and the guide's known limitations name what laptops lose. --- docs/design/camera.md | 37 ++++++- docs/design/laptop-audio.md | 198 ++++++++++++++++++++++++++++++++++++ docs/design/laptop.md | 12 +-- docs/user-guide.md | 6 ++ 4 files changed, 245 insertions(+), 8 deletions(-) create mode 100644 docs/design/laptop-audio.md diff --git a/docs/design/camera.md b/docs/design/camera.md index 70bc93c2..5d421ffd 100644 --- a/docs/design/camera.md +++ b/docs/design/camera.md @@ -104,6 +104,41 @@ itself waits for the tests on physical hardware. ## Not covered -- MIPI cameras behind the IPU6, and infrared cameras. +- MIPI cameras behind the IPU6 (below), and infrared cameras. - Which USB devices are accepted at all. - Screen sharing, which is the compositor's question, not the camera's. + +## MIPI cameras + +Many Intel laptops since 2022 have a MIPI camera behind the image +processing unit (the IPU6, and the IPU7 from Lunar Lake on) instead of a +USB one. The kernel's half is upstream: `VIDEO_INTEL_IPU6` captures the +sensor's raw frames, `IPU_BRIDGE` describes the sensors from ACPI tables +made for Windows, `INTEL_SKL_INT3472` powers them, each sensor model has a +driver of its own, and many models of 2022 and 2023 put Intel's Visual +Sensing Controller between sensor and IPU (`INTEL_VSC`, with its MEI +driver). The IPU7's driver is still in staging in 6.18. Their firmware is +in linux-firmware 20260916: the IPU6's five files are 2.3 MB, the IPU7's +two 0.5 MB, and the Visual Sensing Controller's 29 files 11.5 MB, before +compression. + +What the kernel gives is raw Bayer frames through a media-controller +graph: `/dev/media0`, several `/dev/video*` nodes and the sensors' +`/dev/v4l-subdev*`. Making a picture of them is userspace's work: +libcamera's software image processing, or Intel's closed one. There it +leaves ADR-026, which grants a zone one node that hands Firefox a format it +reads: + +- **libcamera in the zone,** given the graph: the zone's kernel reaches the + IPU6 driver's, the media controller's and the sensors' ioctls, and a + grant has a graph to revoke rather than a node. +- **libcamera in zone 0,** handing the zone pictures: a camera stack and + its image processing become trusted code, and the zone still needs a + V4L2 node the pictures come out of, which only an out-of-tree module + (v4l2loopback) makes. + +Neither fits 1.0, which leaves these cameras unsupported, as ADR-026 has +it; a USB camera works on those laptops. The firmware is the small part of +the cost. Where the image processing runs and what a grant hands over are +the questions, and they wait for a machine with such a camera to answer +them on. diff --git a/docs/design/laptop-audio.md b/docs/design/laptop-audio.md new file mode 100644 index 00000000..81193e8d --- /dev/null +++ b/docs/design/laptop-audio.md @@ -0,0 +1,198 @@ +# Laptop sound past HD Audio + +Kryptik 1.0 plays and records through the HD Audio driver and USB audio +([the laptop](laptop.md#sound), ADR-021). Many laptops put part of their +sound where that driver cannot reach it: microphones and codecs behind +Intel's audio DSP, microphones on AMD's audio coprocessor, and amplifiers in +front of the speakers that need a driver and a tuning file of their own. +This note says what each needs from `boot.fragment`, `firmware.list` and +`kryptik-sound`, what that costs against ADR-013 and ADR-012, and why 1.0 +ships without them. Linux 6.18 and linux-firmware 20260916 are the pinned +versions; sof-bin, which nothing pins yet, is at 2026.09.2, its newest +release. + +## What 1.0 does on each kind of laptop + +Which driver takes Intel's audio controller is decided as it probes, by +`snd-intel-dspcfg`. Its table has rows for a platform only when that +platform's SOF driver is built, so with none built it finds no row and +`snd-hda-intel` takes every controller, as it takes AMD's. + +- **An HD Audio codec alone,** as on desktops and many laptops: the + speakers, the headphone jack, a headset's microphone, HDMI and + DisplayPort all work. +- **An HD Audio codec with digital microphones on Intel's DSP,** common on + Intel laptops: the same, except the built-in microphones, which are wired + to the DSP. +- **Codecs on SoundWire,** on many recent Intel laptops: the HD Audio link + carries only the graphics' HDMI codec, so the speakers, the jack and the + microphones are all silent. HDMI and DisplayPort should still play. +- **AMD laptops:** the codec is on HD Audio, so the speakers, the jack and + HDMI work. The built-in microphones are on the audio coprocessor (ACP) + and do not. +- **Smart amplifiers** in front of the speakers (Cirrus's CS35L41, CS35L54, + CS35L56 and CS35L57, TI's TAS2781), on Intel and AMD laptops alike: + linux-firmware has tuning files for about 200 models, most of them ASUS, + HP, Lenovo and Dell. The jack works, and the speakers stay silent. + +USB audio works on all of them. A machine shows which kinds it is without +any of these drivers: the audio controller's PCI class (0x040100 or +0x040380 means a DSP, 0x040300 none), an `NHLT` table under +`/sys/firmware/acpi/tables`, and whether its ACPI tables describe SoundWire +links or amplifiers (`CSC3551` for Cirrus's CS35L41, `TIAS2781` for TI's). +Testing on physical hardware is planned for October 2026, and its Intel and +AMD laptops will show which kinds they are. + +## What each needs + +### Amplifiers + +- **Kernel:** the ASoC core (`SND_SOC`), which these HD Audio drivers + depend on; `SND_HDA_SCODEC_CS35L41_I2C`, `SND_HDA_SCODEC_CS35L56_I2C` and + `SND_HDA_SCODEC_TAS2781_I2C` with their `_SPI` twins; and + `SERIAL_MULTI_INSTANTIATE`, which splits the one ACPI device a pair of + amplifiers is described as into one per amplifier. Those on SPI need the + SPI core and the platform's SPI controller (`SPI_PXA2XX` for Intel's + LPSS, `SPI_AMD`). The drivers are modules. eudev loads them from the ACPI + devices, and they join the Realtek codec's driver, already built, as its + components, so `sound.conf` keeps its list. +- **Firmware:** the tuning files, each named for a laptop's PCI subsystem + ID: about 750 files and 6 MB at the top of linux-firmware's `cirrus/`, + and about 90 files and 4 MB in `ti/audio/tas2781/` with their links, + before compression. The source and its licence rows are those ADR-012 + already has, and the list takes them whole, as it takes `amdgpu/*`. +- **Mixer:** nothing new, as the amplifiers play what the codec plays. The + CS35L56 also sits on SoundWire, behind SOF, on some laptops; those come + with SOF. + +### AMD's microphones + +- **Kernel:** the ASoC core and, per generation of the coprocessor, as + modules: `SND_SOC_AMD_RENOIR` and `SND_SOC_AMD_RENOIR_MACH` (Renoir, + Ryzen 4000); `SND_SOC_AMD_ACP6x` and `SND_SOC_AMD_YC_MACH` (Yellow Carp, + Ryzen 6000); `SND_SOC_AMD_ACP63_TOPLEVEL`, `SND_SOC_AMD_PS` and + `SND_SOC_AMD_PS_MACH` (ACP 6.3 and 7.x, Ryzen 7040 and later). + `SND_AMD_ACP_CONFIG`, which they select, decides which driver takes a + machine. None needs firmware. +- **Which laptops:** Yellow Carp's machine driver looks for the ACPI + property `AcpDmicConnected`, then the method `_WOV`, then a table of 88 + laptops by DMI; the driver of the 7040 and later looks for + `acp-audio-device-type` or `_WOV`. A laptop that says neither and is not + in the table has no microphone until it is added upstream. +- **Mixer:** the microphones are a card of their own, for capture alone. + `kryptik-sound` records from the first card it finds with a capture + device 0, a USB one first, and nothing makes that the microphones' card + rather than the codec's, whose input is the headset jack. It would have + to prefer the built-in microphones, and the headset's while one is + plugged in, which the codec's jack control says. + +### Intel's DSP: Sound Open Firmware + +- **Kernel:** the ASoC core, `SND_SOC_SOF_TOPLEVEL`, `SND_SOC_SOF_PCI` and + `SND_SOC_SOF_INTEL_TOPLEVEL`. Every Intel platform's option defaults to + on with `SND_SOC_SOF_PCI`, so the fragment keeps the generations it + supports (Tiger Lake, Alder Lake with Raptor Lake, Meteor Lake with Arrow + Lake, Lunar Lake, Panther Lake with Wildcat Lake) and turns the older + ones off. `SND_SOC_SOF_HDA_LINK` and `SND_SOC_SOF_HDA_AUDIO_CODEC` let + SOF drive the HD Audio codec and HDMI, and + `SND_SOC_INTEL_SKL_HDA_DSP_GENERIC_MACH` is the machine driver for a + codec with digital microphones. SoundWire adds `SOUNDWIRE`, + `SOUNDWIRE_INTEL`, `SND_SOC_SOF_INTEL_SOUNDWIRE` and + `SND_SOC_INTEL_SOUNDWIRE_SOF_MACH`, the generic SoundWire machine driver, + which selects about twenty codec drivers (Realtek's, Cirrus's CS42L43 and + CS35L56, Maxim's) and needs `SND_SOC_INTEL_USER_FRIENDLY_LONG_NAMES` and + the SPI core. +- **Loading:** SOF binds an HD Audio codec as `snd-hda-intel` does, by + `request_module`, which `STATIC_USERMODEHELPER` turns away, so + `sound.conf` gains the same softdep ahead of SOF's Intel PCI drivers, and + stage 05 holds that list too. The machine drivers and SoundWire's codec + drivers bind to devices eudev sees, and load by alias. +- **Firmware, from a second source:** linux-firmware 20260916 carries no + Intel SOF file (its `WHENCE` has no `intel/sof` row). The SOF project + publishes them in sof-bin, which amends ADR-012: + - sof-bin publishes no signature: 2026.09.2 is a lightweight tag on an + unsigned commit. Its tarball would be pinned by SHA-256 in + `sources.lock` and join the sources the signature gate accepts by + note. + - Only the Intel-signed builds ship (the community-signed ones are for + boards that take the SOF project's key), at the paths the kernel asks + for, in sof-bin's layout under `intel/`, compressed like the rest. + - From Tiger Lake on, that is 20 firmware files (14.0 MB) and 347 + topology files (18.7 MB): about 6.9 MB with each file compressed (xz + -9, standing in for the image's zstd -19), against about 81 MB of + firmware today. The firmware is most of it: the 41 generic HD Audio + topologies are 0.1 MB compressed. + - The kernel picks a machine's topology from its ACPI tables and its + microphones, so the image carries every topology of the platforms it + keeps. +- **Mixer:** an SOF card puts the codec, HDMI and the microphones on + different devices of one card, and a SoundWire card splits the speakers + from the jack, at numbers the topology sets. `kryptik-sound` opens device + 0 and sets only the controls it knows by name, while SoundWire codecs + need controls set that ALSA's use-case manager (UCM) sets for each codec + combination. So SOF brings UCM with it: alsa-lib built with it and + `alsa-ucm-conf` pinned, read by the mixer alone, which picks its devices + by UCM's names for them (speaker, headphones, microphones). +- **The handover:** once a platform's SOF driver is built, + `snd-intel-dspcfg` gives SOF every controller of that platform whose ACPI + tables describe digital microphones or SoundWire links, laptops whose + speakers work under `snd-hda-intel` today among them. There, a firmware + the board refuses or a topology the image lacks silences the machine, not + only its microphones. The way back is `options snd_intel_dspcfg + dsp_driver=1`, which root would have to write in `/etc/modprobe.d`. + +## What else it touches + +- **The zones: nothing.** No zone has `/dev/snd` (ADR-021). Every driver + here is reached by zone 0's mixer alone, a user of its own in the `audio` + group, and a zone's sound is still a fixed-format stream on its socket. +- **The kernel image: the SPI core.** The drivers are modules, as ADR-013 + asks, and two built-in parts are on the way. The parser of the ACPI + `NHLT` table (`ACPI_NHLT`), which SOF reads, is in the image already: + `snd-hda-intel` selects `SND_INTEL_DSP_CONFIG`, which selects it. The SPI + core (`SPI`), which SoundWire's machine driver and the amplifiers on SPI + need, is not, and it cannot be a module, which is ADR-013's exception; + the controllers under it can. Stage 05's size step and its fragment + check, which fails an option dropped or forced on, settle the cost on the + branch that adds them; `size-budget` allows 23,087,770 bytes. +- **What runs beside the CPU.** The SOF firmware runs on the DSP, which + reaches memory by DMA behind the IOMMU (on and strict, ADR-012), like the + Wi-Fi and graphics firmware. The kernel parses the topology files, from + the verified root. The tuning files go to the amplifiers' own DSPs. + +## Recommendation: a named gap in 1.0 + +1.0 ships without them, and the user guide's known limitations say so: no +built-in microphones on Intel laptops with a DSP or on AMD laptops, no +built-in sound on SoundWire laptops, and no speakers behind smart +amplifiers. + +- **None of it can be tested first.** QEMU emulates no DSP, coprocessor, + SoundWire link or amplifier, and every check runs in QEMU. +- **SOF changes what works today.** Built, it takes the Intel laptops with + digital microphones from `snd-hda-intel`, whose speakers work now, and a + fault there costs their speakers too. +- **It is three pieces of work, not a fragment line:** the kernel's, a + firmware source with its provenance, and a mixer that picks its devices + by role, with UCM for SOF. + +After 1.0, ordered by what each gives for its cost, and each only once a +machine that has it has run it: + +- the amplifiers, then AMD's microphones: the ASoC core and a few modules + (and the SPI core for amplifiers on SPI), firmware from linux-firmware or + none, and the mixer's choice of microphone; +- SOF for HD Audio laptops with digital microphones: sof-bin's firmware and + generic topologies, the softdep, and the mixer's devices by role; +- SoundWire, with UCM in the mixer. + +## Check, when each comes + +- Stage 05: the fragment check, the size step, and `sound.conf` held to the + codec drivers built and to SOF's PCI drivers. +- The acceptance's firmware check: what a module names and linux-firmware + has ships, or `firmware-accepted.txt` says why not. +- On a machine of the kind: the speakers, the jack and each microphone + through `kryptik-sound`, a zone's grant of the microphone among them, and + again after a suspend. +- The sound suite keeps running in QEMU, where none of it loads. diff --git a/docs/design/laptop.md b/docs/design/laptop.md index 183b5ae3..458205ff 100644 --- a/docs/design/laptop.md +++ b/docs/design/laptop.md @@ -45,12 +45,10 @@ codec driver ahead of `snd-hda-intel` (`build/config/modprobe.d/sound.conf`, which stage 05 holds to the drivers built). Not yet: Intel's Sound Open Firmware drivers for laptops with digital -microphones, AMD's audio coprocessor drivers, and the smart-amplifier -codecs (Cirrus, TI) many laptops put on their speakers. Most of their -firmware is in linux-firmware, but Intel's SOF firmware and topology files -are not: the SOF project publishes them separately (sof-bin). That is a -second firmware source, which ADR-012 has to name, with its own provenance -row. +microphones or SoundWire codecs, AMD's audio coprocessor drivers, and the +smart-amplifier codecs (Cirrus, TI) many laptops put on their speakers. +[Laptop sound past HD Audio](laptop-audio.md) says what each needs, what a +laptop does without it, and the order they come in after 1.0. ### Who holds `/dev/snd` @@ -65,7 +63,7 @@ row. - **A small mixer in zone 0.** `kryptik-sound` runs as a user of its own in the `audio` group, never as root. It reads fixed-format sound from one socket per zone, mixes it, and plays it through alsa-lib, whose UCM - configurations the SOF and AMD laptops need. The microphone's gate is + configurations the SOF laptops would need. The microphone's gate is trusted code, and only zone 0 reaches the sound drivers' ioctls. It is C, built in stage 04 with the hardening flags like the screen lock, not Rust like kryptikd: alsa-lib is a glibc library that loads its plugins, and diff --git a/docs/user-guide.md b/docs/user-guide.md index 2f2d5ecc..51d12122 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -614,6 +614,12 @@ shows each timer, its timeout and whether it is running. planned for October 2026. QEMU has no lid, no microphone or brightness keys, no backlight, no built-in panel and no camera of the USB Video Class, so those parts wait for it. +- Laptop sound goes through the HD Audio driver and USB audio. On many + recent laptops part of it does not work: built-in microphones on Intel's + or AMD's audio processor, all built-in sound on Intel laptops whose audio + chips sit on SoundWire, and speakers behind a smart amplifier. A USB + headset or sound card works on all of them + ([laptop sound](design/laptop-audio.md)). - A disk with 4096-byte logical sectors (4Kn) is refused, as the medium's boot partition is a FAT of 512-byte sectors that Linux does not mount there. Disks that present 512-byte sectors (512e), as most do, are fine. From a77a2ae2ea0238f723a74ff19b79f1410434c6db Mon Sep 17 00:00:00 2001 From: DevomB Date: Thu, 15 Oct 2026 09:44:34 -0700 Subject: [PATCH 2/3] The hardware sentence stands alone in the laptop sound note --- docs/design/laptop-audio.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/laptop-audio.md b/docs/design/laptop-audio.md index 81193e8d..568cfd21 100644 --- a/docs/design/laptop-audio.md +++ b/docs/design/laptop-audio.md @@ -40,8 +40,8 @@ any of these drivers: the audio controller's PCI class (0x040100 or 0x040380 means a DSP, 0x040300 none), an `NHLT` table under `/sys/firmware/acpi/tables`, and whether its ACPI tables describe SoundWire links or amplifiers (`CSC3551` for Cirrus's CS35L41, `TIAS2781` for TI's). -Testing on physical hardware is planned for October 2026, and its Intel and -AMD laptops will show which kinds they are. +Testing on physical hardware is planned for October 2026. The Intel and AMD +laptops tested then will show which kinds they are. ## What each needs From dc74302130f915a49e87a5fb38a1201cc257a6e9 Mon Sep 17 00:00:00 2001 From: DevomB Date: Fri, 16 Oct 2026 01:42:17 -0700 Subject: [PATCH 3/3] The laptop sound note and the guide say no monitor plays sound over HDMI or DisplayPort The note said HDMI and DisplayPort work wherever the HD Audio driver takes the codec, and should still play on SoundWire laptops. The driver takes those codecs, but kryptik-sound plays only to a card's device 0, the codec's own (pick_card, plughw:N,0), so a monitor's speakers stay silent, and a SoundWire laptop's card, which has only HDMI devices, gets nothing at all. The note's three bullets say so, and the guide's limitation line adds the monitor's speakers. --- docs/design/laptop-audio.md | 12 +++++++----- docs/user-guide.md | 3 ++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/design/laptop-audio.md b/docs/design/laptop-audio.md index 568cfd21..3a14642d 100644 --- a/docs/design/laptop-audio.md +++ b/docs/design/laptop-audio.md @@ -19,16 +19,18 @@ platform's SOF driver is built, so with none built it finds no row and `snd-hda-intel` takes every controller, as it takes AMD's. - **An HD Audio codec alone,** as on desktops and many laptops: the - speakers, the headphone jack, a headset's microphone, HDMI and - DisplayPort all work. + speakers, the headphone jack and a headset's microphone work. The driver + takes the HDMI and DisplayPort codecs too, but the mixer plays only to a + card's device 0, the codec's own, so a monitor's speakers stay silent. - **An HD Audio codec with digital microphones on Intel's DSP,** common on Intel laptops: the same, except the built-in microphones, which are wired to the DSP. - **Codecs on SoundWire,** on many recent Intel laptops: the HD Audio link carries only the graphics' HDMI codec, so the speakers, the jack and the - microphones are all silent. HDMI and DisplayPort should still play. -- **AMD laptops:** the codec is on HD Audio, so the speakers, the jack and - HDMI work. The built-in microphones are on the audio coprocessor (ACP) + microphones are all silent, and as that card has no device 0 the mixer + plays to none of it, a monitor's speakers included. +- **AMD laptops:** the codec is on HD Audio, so the speakers and the jack + work. The built-in microphones are on the audio coprocessor (ACP) and do not. - **Smart amplifiers** in front of the speakers (Cirrus's CS35L41, CS35L54, CS35L56 and CS35L57, TI's TAS2781), on Intel and AMD laptops alike: diff --git a/docs/user-guide.md b/docs/user-guide.md index 51d12122..a269fc91 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -619,7 +619,8 @@ shows each timer, its timeout and whether it is running. or AMD's audio processor, all built-in sound on Intel laptops whose audio chips sit on SoundWire, and speakers behind a smart amplifier. A USB headset or sound card works on all of them - ([laptop sound](design/laptop-audio.md)). + ([laptop sound](design/laptop-audio.md)). Nothing plays through a + monitor's speakers over HDMI or DisplayPort. - A disk with 4096-byte logical sectors (4Kn) is refused, as the medium's boot partition is a FAT of 512-byte sectors that Linux does not mount there. Disks that present 512-byte sectors (512e), as most do, are fine.