Laptop sound past HD Audio and MIPI cameras: what each needs, and why 1.0 ships without them

docs/design/camera.md

@@ -104,6 +104,41 @@ itself waits for the tests on physical hardware.
 
 ## Not covered
 
-- MIPI cameras behind the IPU6, and infrared cameras.
+- MIPI cameras behind the IPU6 (below), and infrared cameras.
 - Which USB devices are accepted at all.
 - Screen sharing, which is the compositor's question, not the camera's.
+
+## MIPI cameras
+
+Many Intel laptops since 2022 have a MIPI camera behind the image
+processing unit (the IPU6, and the IPU7 from Lunar Lake on) instead of a
+USB one. The kernel's half is upstream: `VIDEO_INTEL_IPU6` captures the
+sensor's raw frames, `IPU_BRIDGE` describes the sensors from ACPI tables
+made for Windows, `INTEL_SKL_INT3472` powers them, each sensor model has a
+driver of its own, and many models of 2022 and 2023 put Intel's Visual
+Sensing Controller between sensor and IPU (`INTEL_VSC`, with its MEI
+driver). The IPU7's driver is still in staging in 6.18. Their firmware is
+in linux-firmware 20260916: the IPU6's five files are 2.3 MB, the IPU7's
+two 0.5 MB, and the Visual Sensing Controller's 29 files 11.5 MB, before
+compression.
+
+What the kernel gives is raw Bayer frames through a media-controller
+graph: `/dev/media0`, several `/dev/video*` nodes and the sensors'
+`/dev/v4l-subdev*`. Making a picture of them is userspace's work:
+libcamera's software image processing, or Intel's closed one. There it
+leaves ADR-026, which grants a zone one node that hands Firefox a format it
+reads:
+
+- **libcamera in the zone,** given the graph: the zone's kernel reaches the
+  IPU6 driver's, the media controller's and the sensors' ioctls, and a
+  grant has a graph to revoke rather than a node.
+- **libcamera in zone 0,** handing the zone pictures: a camera stack and
+  its image processing become trusted code, and the zone still needs a
+  V4L2 node the pictures come out of, which only an out-of-tree module
+  (v4l2loopback) makes.
+
+Neither fits 1.0, which leaves these cameras unsupported, as ADR-026 has
+it; a USB camera works on those laptops. The firmware is the small part of
+the cost. Where the image processing runs and what a grant hands over are
+the questions, and they wait for a machine with such a camera to answer
+them on.

docs/design/laptop-audio.md

@@ -0,0 +1,200 @@
+# Laptop sound past HD Audio
+
+Kryptik 1.0 plays and records through the HD Audio driver and USB audio
+([the laptop](laptop.md#sound), ADR-021). Many laptops put part of their
+sound where that driver cannot reach it: microphones and codecs behind
+Intel's audio DSP, microphones on AMD's audio coprocessor, and amplifiers in
+front of the speakers that need a driver and a tuning file of their own.
+This note says what each needs from `boot.fragment`, `firmware.list` and
+`kryptik-sound`, what that costs against ADR-013 and ADR-012, and why 1.0
+ships without them. Linux 6.18 and linux-firmware 20260916 are the pinned
+versions; sof-bin, which nothing pins yet, is at 2026.09.2, its newest
+release.
+
+## What 1.0 does on each kind of laptop
+
+Which driver takes Intel's audio controller is decided as it probes, by
+`snd-intel-dspcfg`. Its table has rows for a platform only when that
+platform's SOF driver is built, so with none built it finds no row and
+`snd-hda-intel` takes every controller, as it takes AMD's.
+
+- **An HD Audio codec alone,** as on desktops and many laptops: the
+  speakers, the headphone jack and a headset's microphone work. The driver
+  takes the HDMI and DisplayPort codecs too, but the mixer plays only to a
+  card's device 0, the codec's own, so a monitor's speakers stay silent.
+- **An HD Audio codec with digital microphones on Intel's DSP,** common on
+  Intel laptops: the same, except the built-in microphones, which are wired
+  to the DSP.
+- **Codecs on SoundWire,** on many recent Intel laptops: the HD Audio link
+  carries only the graphics' HDMI codec, so the speakers, the jack and the
+  microphones are all silent, and as that card has no device 0 the mixer
+  plays to none of it, a monitor's speakers included.
+- **AMD laptops:** the codec is on HD Audio, so the speakers and the jack
+  work. The built-in microphones are on the audio coprocessor (ACP)
+  and do not.
+- **Smart amplifiers** in front of the speakers (Cirrus's CS35L41, CS35L54,
+  CS35L56 and CS35L57, TI's TAS2781), on Intel and AMD laptops alike:
+  linux-firmware has tuning files for about 200 models, most of them ASUS,
+  HP, Lenovo and Dell. The jack works, and the speakers stay silent.
+
+USB audio works on all of them. A machine shows which kinds it is without
+any of these drivers: the audio controller's PCI class (0x040100 or
+0x040380 means a DSP, 0x040300 none), an `NHLT` table under
+`/sys/firmware/acpi/tables`, and whether its ACPI tables describe SoundWire
+links or amplifiers (`CSC3551` for Cirrus's CS35L41, `TIAS2781` for TI's).
+Testing on physical hardware is planned for October 2026. The Intel and AMD
+laptops tested then will show which kinds they are.
+
+## What each needs
+
+### Amplifiers
+
+- **Kernel:** the ASoC core (`SND_SOC`), which these HD Audio drivers
+  depend on; `SND_HDA_SCODEC_CS35L41_I2C`, `SND_HDA_SCODEC_CS35L56_I2C` and
+  `SND_HDA_SCODEC_TAS2781_I2C` with their `_SPI` twins; and
+  `SERIAL_MULTI_INSTANTIATE`, which splits the one ACPI device a pair of
+  amplifiers is described as into one per amplifier. Those on SPI need the
+  SPI core and the platform's SPI controller (`SPI_PXA2XX` for Intel's
+  LPSS, `SPI_AMD`). The drivers are modules. eudev loads them from the ACPI
+  devices, and they join the Realtek codec's driver, already built, as its
+  components, so `sound.conf` keeps its list.
+- **Firmware:** the tuning files, each named for a laptop's PCI subsystem
+  ID: about 750 files and 6 MB at the top of linux-firmware's `cirrus/`,
+  and about 90 files and 4 MB in `ti/audio/tas2781/` with their links,
+  before compression. The source and its licence rows are those ADR-012
+  already has, and the list takes them whole, as it takes `amdgpu/*`.
+- **Mixer:** nothing new, as the amplifiers play what the codec plays. The
+  CS35L56 also sits on SoundWire, behind SOF, on some laptops; those come
+  with SOF.
+
+### AMD's microphones
+
+- **Kernel:** the ASoC core and, per generation of the coprocessor, as
+  modules: `SND_SOC_AMD_RENOIR` and `SND_SOC_AMD_RENOIR_MACH` (Renoir,
+  Ryzen 4000); `SND_SOC_AMD_ACP6x` and `SND_SOC_AMD_YC_MACH` (Yellow Carp,
+  Ryzen 6000); `SND_SOC_AMD_ACP63_TOPLEVEL`, `SND_SOC_AMD_PS` and
+  `SND_SOC_AMD_PS_MACH` (ACP 6.3 and 7.x, Ryzen 7040 and later).
+  `SND_AMD_ACP_CONFIG`, which they select, decides which driver takes a
+  machine. None needs firmware.
+- **Which laptops:** Yellow Carp's machine driver looks for the ACPI
+  property `AcpDmicConnected`, then the method `_WOV`, then a table of 88
+  laptops by DMI; the driver of the 7040 and later looks for
+  `acp-audio-device-type` or `_WOV`. A laptop that says neither and is not
+  in the table has no microphone until it is added upstream.
+- **Mixer:** the microphones are a card of their own, for capture alone.
+  `kryptik-sound` records from the first card it finds with a capture
+  device 0, a USB one first, and nothing makes that the microphones' card
+  rather than the codec's, whose input is the headset jack. It would have
+  to prefer the built-in microphones, and the headset's while one is
+  plugged in, which the codec's jack control says.
+
+### Intel's DSP: Sound Open Firmware
+
+- **Kernel:** the ASoC core, `SND_SOC_SOF_TOPLEVEL`, `SND_SOC_SOF_PCI` and
+  `SND_SOC_SOF_INTEL_TOPLEVEL`. Every Intel platform's option defaults to
+  on with `SND_SOC_SOF_PCI`, so the fragment keeps the generations it
+  supports (Tiger Lake, Alder Lake with Raptor Lake, Meteor Lake with Arrow
+  Lake, Lunar Lake, Panther Lake with Wildcat Lake) and turns the older
+  ones off. `SND_SOC_SOF_HDA_LINK` and `SND_SOC_SOF_HDA_AUDIO_CODEC` let
+  SOF drive the HD Audio codec and HDMI, and
+  `SND_SOC_INTEL_SKL_HDA_DSP_GENERIC_MACH` is the machine driver for a
+  codec with digital microphones. SoundWire adds `SOUNDWIRE`,
+  `SOUNDWIRE_INTEL`, `SND_SOC_SOF_INTEL_SOUNDWIRE` and
+  `SND_SOC_INTEL_SOUNDWIRE_SOF_MACH`, the generic SoundWire machine driver,
+  which selects about twenty codec drivers (Realtek's, Cirrus's CS42L43 and
+  CS35L56, Maxim's) and needs `SND_SOC_INTEL_USER_FRIENDLY_LONG_NAMES` and
+  the SPI core.
+- **Loading:** SOF binds an HD Audio codec as `snd-hda-intel` does, by
+  `request_module`, which `STATIC_USERMODEHELPER` turns away, so
+  `sound.conf` gains the same softdep ahead of SOF's Intel PCI drivers, and
+  stage 05 holds that list too. The machine drivers and SoundWire's codec
+  drivers bind to devices eudev sees, and load by alias.
+- **Firmware, from a second source:** linux-firmware 20260916 carries no
+  Intel SOF file (its `WHENCE` has no `intel/sof` row). The SOF project
+  publishes them in sof-bin, which amends ADR-012:
+  - sof-bin publishes no signature: 2026.09.2 is a lightweight tag on an
+    unsigned commit. Its tarball would be pinned by SHA-256 in
+    `sources.lock` and join the sources the signature gate accepts by
+    note.
+  - Only the Intel-signed builds ship (the community-signed ones are for
+    boards that take the SOF project's key), at the paths the kernel asks
+    for, in sof-bin's layout under `intel/`, compressed like the rest.
+  - From Tiger Lake on, that is 20 firmware files (14.0 MB) and 347
+    topology files (18.7 MB): about 6.9 MB with each file compressed (xz
+    -9, standing in for the image's zstd -19), against about 81 MB of
+    firmware today. The firmware is most of it: the 41 generic HD Audio
+    topologies are 0.1 MB compressed.
+  - The kernel picks a machine's topology from its ACPI tables and its
+    microphones, so the image carries every topology of the platforms it
+    keeps.
+- **Mixer:** an SOF card puts the codec, HDMI and the microphones on
+  different devices of one card, and a SoundWire card splits the speakers
+  from the jack, at numbers the topology sets. `kryptik-sound` opens device
+  0 and sets only the controls it knows by name, while SoundWire codecs
+  need controls set that ALSA's use-case manager (UCM) sets for each codec
+  combination. So SOF brings UCM with it: alsa-lib built with it and
+  `alsa-ucm-conf` pinned, read by the mixer alone, which picks its devices
+  by UCM's names for them (speaker, headphones, microphones).
+- **The handover:** once a platform's SOF driver is built,
+  `snd-intel-dspcfg` gives SOF every controller of that platform whose ACPI
+  tables describe digital microphones or SoundWire links, laptops whose
+  speakers work under `snd-hda-intel` today among them. There, a firmware
+  the board refuses or a topology the image lacks silences the machine, not
+  only its microphones. The way back is `options snd_intel_dspcfg
+  dsp_driver=1`, which root would have to write in `/etc/modprobe.d`.
+
+## What else it touches
+
+- **The zones: nothing.** No zone has `/dev/snd` (ADR-021). Every driver
+  here is reached by zone 0's mixer alone, a user of its own in the `audio`
+  group, and a zone's sound is still a fixed-format stream on its socket.
+- **The kernel image: the SPI core.** The drivers are modules, as ADR-013
+  asks, and two built-in parts are on the way. The parser of the ACPI
+  `NHLT` table (`ACPI_NHLT`), which SOF reads, is in the image already:
+  `snd-hda-intel` selects `SND_INTEL_DSP_CONFIG`, which selects it. The SPI
+  core (`SPI`), which SoundWire's machine driver and the amplifiers on SPI
+  need, is not, and it cannot be a module, which is ADR-013's exception;
+  the controllers under it can. Stage 05's size step and its fragment
+  check, which fails an option dropped or forced on, settle the cost on the
+  branch that adds them; `size-budget` allows 23,087,770 bytes.
+- **What runs beside the CPU.** The SOF firmware runs on the DSP, which
+  reaches memory by DMA behind the IOMMU (on and strict, ADR-012), like the
+  Wi-Fi and graphics firmware. The kernel parses the topology files, from
+  the verified root. The tuning files go to the amplifiers' own DSPs.
+
+## Recommendation: a named gap in 1.0
+
+1.0 ships without them, and the user guide's known limitations say so: no
+built-in microphones on Intel laptops with a DSP or on AMD laptops, no
+built-in sound on SoundWire laptops, and no speakers behind smart
+amplifiers.
+
+- **None of it can be tested first.** QEMU emulates no DSP, coprocessor,
+  SoundWire link or amplifier, and every check runs in QEMU.
+- **SOF changes what works today.** Built, it takes the Intel laptops with
+  digital microphones from `snd-hda-intel`, whose speakers work now, and a
+  fault there costs their speakers too.
+- **It is three pieces of work, not a fragment line:** the kernel's, a
+  firmware source with its provenance, and a mixer that picks its devices
+  by role, with UCM for SOF.
+
+After 1.0, ordered by what each gives for its cost, and each only once a
+machine that has it has run it:
+
+- the amplifiers, then AMD's microphones: the ASoC core and a few modules
+  (and the SPI core for amplifiers on SPI), firmware from linux-firmware or
+  none, and the mixer's choice of microphone;
+- SOF for HD Audio laptops with digital microphones: sof-bin's firmware and
+  generic topologies, the softdep, and the mixer's devices by role;
+- SoundWire, with UCM in the mixer.
+
+## Check, when each comes
+
+- Stage 05: the fragment check, the size step, and `sound.conf` held to the
+  codec drivers built and to SOF's PCI drivers.
+- The acceptance's firmware check: what a module names and linux-firmware
+  has ships, or `firmware-accepted.txt` says why not.
+- On a machine of the kind: the speakers, the jack and each microphone
+  through `kryptik-sound`, a zone's grant of the microphone among them, and
+  again after a suspend.
+- The sound suite keeps running in QEMU, where none of it loads.

docs/design/laptop.md

@@ -45,12 +45,10 @@ codec driver ahead of `snd-hda-intel` (`build/config/modprobe.d/sound.conf`,
 which stage 05 holds to the drivers built).
 
 Not yet: Intel's Sound Open Firmware drivers for laptops with digital
-microphones, AMD's audio coprocessor drivers, and the smart-amplifier
-codecs (Cirrus, TI) many laptops put on their speakers. Most of their
-firmware is in linux-firmware, but Intel's SOF firmware and topology files
-are not: the SOF project publishes them separately (sof-bin). That is a
-second firmware source, which ADR-012 has to name, with its own provenance
-row.
+microphones or SoundWire codecs, AMD's audio coprocessor drivers, and the
+smart-amplifier codecs (Cirrus, TI) many laptops put on their speakers.
+[Laptop sound past HD Audio](laptop-audio.md) says what each needs, what a
+laptop does without it, and the order they come in after 1.0.
 
 ### Who holds `/dev/snd`
 
@@ -65,7 +63,7 @@ row.
 - **A small mixer in zone 0.** `kryptik-sound` runs as a user of its own
   in the `audio` group, never as root. It reads fixed-format sound from one
   socket per zone, mixes it, and plays it through alsa-lib, whose UCM
-  configurations the SOF and AMD laptops need. The microphone's gate is
+  configurations the SOF laptops would need. The microphone's gate is
   trusted code, and only zone 0 reaches the sound drivers' ioctls. It is C,
   built in stage 04 with the hardening flags like the screen lock, not Rust
   like kryptikd: alsa-lib is a glibc library that loads its plugins, and

docs/user-guide.md

@@ -614,6 +614,13 @@ shows each timer, its timeout and whether it is running.
   planned for October 2026. QEMU has no lid, no microphone or brightness
   keys, no backlight, no built-in panel and no camera of the USB Video
   Class, so those parts wait for it.
+- Laptop sound goes through the HD Audio driver and USB audio. On many
+  recent laptops part of it does not work: built-in microphones on Intel's
+  or AMD's audio processor, all built-in sound on Intel laptops whose audio
+  chips sit on SoundWire, and speakers behind a smart amplifier. A USB
+  headset or sound card works on all of them
+  ([laptop sound](design/laptop-audio.md)). Nothing plays through a
+  monitor's speakers over HDMI or DisplayPort.
 - A disk with 4096-byte logical sectors (4Kn) is refused, as the medium's
   boot partition is a FAT of 512-byte sectors that Linux does not mount
   there. Disks that present 512-byte sectors (512e), as most do, are fine.