femu/zns: write caches per open zone, and two fixes on the cache path - #205
Merged
Merged
Conversation
A write cache stages the partial stripe of one zone, so the device needs one per zone the host may be writing at the same time. The count was hard-wired to 3: with more concurrent writers than that every write misses, evicts another zone's cache and pays that zone's flush synchronously, and the sequential-write throughput of the emulated device collapses (8 channels x 4 chips x 2 planes, TLC, 4 KiB writes: 89K IOPS with 2 writers, 6K with 4). Real ZNS SSDs bound these per-zone resources with the open-zone limit (Bjorling et al., ATC'21, sec. 3.1), and a WD ZN540 (max open 14) keeps its throughput flat up to 14 concurrent zones. Take the number of caches from a zns_num_wc property; when it is left at 0 follow zns_max_open, and keep the previous 3 when that is unlimited too.
The scan for a free write cache started at index 1 with cache 0 as the initial candidate, so an empty cache 0 was only picked when no other cache held more than its zero entries, i.e. never while any other cache was in use. The device therefore served one write stream fewer than it had caches: with the default 3, a third concurrent zone already evicted on every write. Scan every cache, and prefer an empty one.
A flush walked every plane and every page of a stripe even when the cache being flushed held a handful of LPNs, as it does when a write miss evicts another zone's cache. Each such eviction burned a full stripe of physical pages (2 planes x 3 pages for one 4 KiB write in a TLC configuration), the zone's blocks ran out of pages well before the zone was full, and later LPNs were mapped past the end of their block. Reads of those LPNs then failed valid_ppa() and completed without any NAND time, so a thrashing device looked faster on reads than the NAND read latency allows. Stop the walk once the cache's LPNs are placed. The last page is still padded to a whole page, as a real program is; only planes and pages that received nothing are left untouched.
Contributor
|
Thanks, @wnsah814 — merged. The cache-0 scan and the partial-flush fix are real bugs, and tying the cache count to the open-zone limit is the right default. One follow-up I'll add separately: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The ZNS write cache stages one zone's partial stripe, so a device needs one cache per zone the host may be writing at the same time. Three things on that path made the emulated device collapse under concurrent sequential writers, which a real ZNS SSD does not do (a WD ZN540 keeps its throughput flat up to its open limit of 14 zones):
zns_num_wcnow sets it; left at 0 it followszns_max_open, and stays 3 when that is unlimited, so existing configurations are unchanged. Real devices bound these per-zone buffers with the open-zone limit (Bjørling et al., ATC '21, §3.1).valid_ppa()and completed with no NAND time at all. The flush now stops once the cache's LPNs are placed (the last page is still padded, as a real program is).8 ch × 4 chips × 2 planes, TLC, tR 65 µs / tPROG 450 µs; N writers, one zone each, 4 KiB sequential, psync QD1, 96 MiB per writer; then 4 KiB random reads of the first writer's zone for 10 s. Read latency below tR means the read never reached the NAND model.
zns_max_open=14With the defaults the third writer no longer thrashes (fix 2) and a thrashing device no longer reads faster than tR (fix 3); with the cache count following the open limit there is no thrash to 8 writers.
Type of Change
Testing
FEMU Modes Tested
Platform Testing
Checklist