Skip to content

femu/zns: write caches per open zone, and two fixes on the cache path - #205

Merged
huaicheng merged 3 commits into
MoatLab:masterfrom
wnsah814:zns-write-cache-fixes
Sep 17, 2026
Merged

huaicheng merged 3 commits into
MoatLab:masterfrom
wnsah814:zns-write-cache-fixes

Conversation

@wnsah814

@wnsah814 wnsah814 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Description

The ZNS write cache stages one zone's partial stripe, so a device needs one cache per zone the host may be writing at the same time. Three things on that path made the emulated device collapse under concurrent sequential writers, which a real ZNS SSD does not do (a WD ZN540 keeps its throughput flat up to its open limit of 14 zones):

  1. The count was hard-wired to 3. zns_num_wc now sets it; left at 0 it follows zns_max_open, and stays 3 when that is unlimited, so existing configurations are unchanged. Real devices bound these per-zone buffers with the open-zone limit (Bjørling et al., ATC '21, §3.1).
  2. The free-cache scan skipped cache 0. It started at index 1 with cache 0 as the initial candidate, so an empty cache 0 was never taken while any other cache was in use: the device served one writer fewer than it had caches, and the third concurrent zone already evicted on every write.
  3. A flush of a partial cache burned a whole stripe. An evicted cache holding a few LPNs still walked every plane and page of the stripe, so one 4 KiB write consumed 2 planes × 3 pages, the zone's blocks ran out of pages long before the zone was full, later LPNs were mapped past their block, and reads of them failed valid_ppa() and completed with no NAND time at all. The flush now stops once the cache's LPNs are placed (the last page is still padded, as a real program is).

8 ch × 4 chips × 2 planes, TLC, tR 65 µs / tPROG 450 µs; N writers, one zone each, 4 KiB sequential, psync QD1, 96 MiB per writer; then 4 KiB random reads of the first writer's zone for 10 s. Read latency below tR means the read never reached the NAND model.

writers master: write IOPS / read µs this branch, defaults (3 caches) this branch, zns_max_open=14
2 103,044 / 75.6 112,476 / 75.5 105,251 / 76.2
3 36,919 / 60.2 162,040 / 76.9 160,278 / 75.6
4 13,333 / 24.5 19,355 / 62.2 206,088 / 75.5
8 19,718 / 20.5 24,524 / 50.2 274,592 / 74.9

With the defaults the third writer no longer thrashes (fix 2) and a thrashing device no longer reads faster than tR (fix 3); with the cache count following the open limit there is no thrash to 8 writers.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Performance improvement
  • Code refactoring

Testing

  • I have tested my changes locally
  • I have added tests that prove my fix is effective or that my feature works
  • All existing tests pass
  • I have tested across multiple FEMU modes (if applicable)

FEMU Modes Tested

  • BlackBox SSD (BBSSD)
  • WhiteBox SSD (OCSSD)
  • Zoned Namespace SSD (ZNSSD)
  • NoSSD
  • Not applicable

Platform Testing

  • Ubuntu 20.04/22.04
  • Other distributions (specify): ___________
  • Build verification completed

Checklist

  • My code follows QEMU coding standards
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new compiler warnings
  • I have updated documentation if necessary
  • No trailing whitespace or C++ style comments in C files

A write cache stages the partial stripe of one zone, so the device
needs one per zone the host may be writing at the same time. The count
was hard-wired to 3: with more concurrent writers than that every write
misses, evicts another zone's cache and pays that zone's flush
synchronously, and the sequential-write throughput of the emulated
device collapses (8 channels x 4 chips x 2 planes, TLC, 4 KiB writes:
89K IOPS with 2 writers, 6K with 4).

Real ZNS SSDs bound these per-zone resources with the open-zone limit
(Bjorling et al., ATC'21, sec. 3.1), and a WD ZN540 (max open 14) keeps
its throughput flat up to 14 concurrent zones. Take the number of caches
from a zns_num_wc property; when it is left at 0 follow zns_max_open,
and keep the previous 3 when that is unlimited too.
The scan for a free write cache started at index 1 with cache 0 as the
initial candidate, so an empty cache 0 was only picked when no other
cache held more than its zero entries, i.e. never while any other cache
was in use. The device therefore served one write stream fewer than it
had caches: with the default 3, a third concurrent zone already evicted
on every write. Scan every cache, and prefer an empty one.
A flush walked every plane and every page of a stripe even when the
cache being flushed held a handful of LPNs, as it does when a write
miss evicts another zone's cache. Each such eviction burned a full
stripe of physical pages (2 planes x 3 pages for one 4 KiB write in a
TLC configuration), the zone's blocks ran out of pages well before the
zone was full, and later LPNs were mapped past the end of their block.
Reads of those LPNs then failed valid_ppa() and completed without any
NAND time, so a thrashing device looked faster on reads than the NAND
read latency allows.

Stop the walk once the cache's LPNs are placed. The last page is still
padded to a whole page, as a real program is; only planes and pages
that received nothing are left untouched.
@huaicheng
huaicheng merged commit 320bdc6 into MoatLab:master Sep 17, 2026
6 checks passed
@huaicheng

Copy link
Copy Markdown
Contributor

Thanks, @wnsah814 — merged. The cache-0 scan and the partial-flush fix are real bugs, and tying the cache count to the open-zone limit is the right default.

One follow-up I'll add separately: zns_num_wc isn't bounded, so a very large value aborts QEMU at realize; I'll cap it at the zone count, since a cache beyond that can never be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants