From 19b550047ba0fab3d43ac7ca1a5425450327da6a Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 28 Aug 2026 06:07:25 +0000 Subject: [PATCH 01/17] femu/nand: use measured QLC page read latency The QLC read latencies were extrapolated from TLC (Micron FMS'19) as fixed multipliers, giving 59.33 / 85.25 / 127.20 / 169.60 us. Replace them with measured values at 16 KB per page: 47.9 / 76.2 / 134.6 / 228.1 us. The extrapolation is both faster in the mean (110.34 vs 121.70 us) and narrower in spread (1 : 1.44 : 2.14 : 2.86 vs 1 : 1.59 : 2.81 : 4.76), so it understates what page placement is worth - a bit-plane layout worth 1.346x under the measured vector is worth only 1.221x under the extrapolated one. Write latencies are left as the TLC extrapolation; they were not measured, and the workload this is for writes once and then only reads. Co-Authored-By: Claude Opus 5 --- hw/femu/nand/nand.h | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/hw/femu/nand/nand.h b/hw/femu/nand/nand.h index 700f86adf95..bb757f83fed 100644 --- a/hw/femu/nand/nand.h +++ b/hw/femu/nand/nand.h @@ -43,18 +43,26 @@ /* * QLC NAND latency numbers in nanoseconds * - * Read Latency is extrapolated from TLC drives based on Micron FMS'19 - * presentation: "Component-Level Characterization of 3D TLC, QLC, and - * Low-Latency NAND" + * Read Latency is measured, at 16 KB per page. The four values are the four + * page types of a QLC wordline, fast to slow; relative to the fastest they are + * 1 : 1.59 : 2.81 : 4.76, mean 121.70 us. * - * Write Latency is increased similar to read latencies, but may be higher in - * practice. + * These replace an extrapolation from TLC (Micron FMS'19, "Component-Level + * Characterization of 3D TLC, QLC, and Low-Latency NAND"), which gave + * 59.33 / 85.25 / 127.20 / 169.60 us. That vector is both faster in the mean + * (110.34 us) and narrower in spread (1 : 1.44 : 2.14 : 2.86), so it understates + * what page placement is worth: a bit-plane layout that gains 1.346x under the + * measured vector gains only 1.221x under the extrapolated one. + * + * Write Latency is still the TLC extrapolation - it was not measured. Our + * workload writes once and then only reads, so program time does not enter the + * result; revisit before running anything write-sensitive. */ -#define QLC_LOWER_PAGE_READ_LATENCY_NS (TLC_LOWER_PAGE_READ_LATENCY_NS * 1.05) -#define QLC_CENTER_LOWER_PAGE_READ_LATENCY_NS (TLC_CENTER_PAGE_READ_LATENCY_NS * 1.1) -#define QLC_CENTER_UPPER_PAGE_READ_LATENCY_NS (TLC_UPPER_PAGE_READ_LATENCY_NS * 1.2) -#define QLC_UPPER_PAGE_READ_LATENCY_NS (TLC_UPPER_PAGE_READ_LATENCY_NS * 1.6) +#define QLC_LOWER_PAGE_READ_LATENCY_NS (47900) +#define QLC_CENTER_LOWER_PAGE_READ_LATENCY_NS (76200) +#define QLC_CENTER_UPPER_PAGE_READ_LATENCY_NS (134600) +#define QLC_UPPER_PAGE_READ_LATENCY_NS (228100) #define QLC_LOWER_PAGE_WRITE_LATENCY_NS (TLC_LOWER_PAGE_WRITE_LATENCY_NS * 1.05) #define QLC_CENTER_LOWER_PAGE_WRITE_LATENCY_NS (TLC_CENTER_PAGE_WRITE_LATENCY_NS * 1.1) From 741c837ced379223fda323fd593757fababd3834 Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Sun, 30 Aug 2026 05:27:32 +0000 Subject: [PATCH 02/17] femu: survive an unraisable memlock limit, and test the channel stage Two things found while getting FEMU to run and measure on a shared host. **Unpinned backing store.** init_dram_backend mlock()s the whole device and abort()s if that fails. RLIMIT_MEMLOCK is commonly 64 MB with the hard limit equal to the soft one, so an unprivileged user cannot raise it and cannot emulate a device larger than 64 MB at all. FEMU_ALLOW_UNPINNED=1 downgrades the failure to a warning; the default is unchanged, and the error message now names the limit that would have to be raised. Pinning exists so a page fault cannot land inside an emulated NAND access, so the variable is only sound on a host that is not swapping - the comment says to check vmstat si/so, and notes that swap merely occupied by stale pages is fine while active swap is not. **The channel stage is dead code, and enabling it is not free.** bbssd/ftl-media.c copies pg_xfer_lat into cfg.timing.page_xfer_ns and then sets channel_mode = NAND_CH_OFF unconditionally; nand-media.c reads page_xfer_ns only under NAND_CH_STAGED. Both call sites (bbssd, zns) select OFF, so NAND_CH_STAGED has no users and passing pg_xfer_lat on the command line changes nothing today. hw/femu/nand/test/test_channel.c characterises what turning it on would do. It builds and runs without QEMU, a guest, or KVM, because nand_media_op() is pure timing arithmetic over a caller-supplied timeline. It establishes: - STAGED differs from OFF even with every bus phase at zero, because the channel timeline is advanced to each op's data-out and the next command is clamped to it. So a channel_model option has to default to off, and "set the transfer to zero to reproduce the old numbers" does not work. - The staged model serialises the channel across LUNs. Reservations are taken in submission order, so an op's command phase waits for the previous op's data-out even on a different LUN. Two LUNs on one channel, both reads issued at t=0, slowest page: 228.1 us and 456.2 us - exactly 2x - and still 2x with the bus transfer set to zero, which is the clearest statement of the problem. Adding LUNs to a channel buys nothing; 800 reads take 139.31 ms at 1, 2, 4 and 8 LUNs per channel, unchanged to the nanosecond. Channels do scale. - Consequently the page-mapping gain it reports is (mean+xfer)/(aware+xfer) flat at every LUN count, 1.219x for our traffic mix, rather than falling toward 1.0 as a channel saturates. That number is a property of the model, not of a device. The header comment in nand-media.c states this reproduces bbssd's ssd_advance_status faithfully, so this is upstream behaviour rather than a defect introduced here: FEMU models a controller that does not pipeline within a channel. Co-Authored-By: Claude Opus 5 --- .gitignore | 1 + hw/femu/backend/dram.c | 30 +++++- hw/femu/nand/test/test_channel.c | 175 +++++++++++++++++++++++++++++++ 3 files changed, 203 insertions(+), 3 deletions(-) create mode 100644 hw/femu/nand/test/test_channel.c diff --git a/.gitignore b/.gitignore index b24a3d5a360..6c1e1f97161 100644 --- a/.gitignore +++ b/.gitignore @@ -133,3 +133,4 @@ trace-ust-all.c build-femu/ build/ *.md +subprojects/.wraplock diff --git a/hw/femu/backend/dram.c b/hw/femu/backend/dram.c index f5f1db9462a..c38e99b4621 100644 --- a/hw/femu/backend/dram.c +++ b/hw/femu/backend/dram.c @@ -63,9 +63,33 @@ int init_dram_backend(SsdDramBackend **mbe, int64_t nbytes) mbe_numa_bind(b->logical_space, nbytes); if (mlock(b->logical_space, nbytes) == -1) { - femu_err("Failed to pin the memory backend to the host DRAM\n"); - g_free(b->logical_space); - abort(); + /* + * Pinning keeps the backing store out of swap so a page fault cannot show + * up inside an emulated NAND access. It needs RLIMIT_MEMLOCK >= the device + * size, which an unprivileged user often cannot raise (the hard limit is + * commonly 64 MB and only root can lift it). + * + * FEMU_ALLOW_UNPINNED=1 downgrades the failure to a warning. Only set it + * on a host with no memory pressure -- check that `vmstat` reports si/so + * at 0 and that free RAM comfortably exceeds the device size. Swap that is + * merely *occupied* by stale pages is fine; swap that is *active* is not, + * because a fault during an emulated access lands directly in the measured + * latency. Default behaviour is unchanged. + */ + if (getenv("FEMU_ALLOW_UNPINNED")) { + femu_err("WARNING: memory backend is NOT pinned (mlock: %s).\n", + strerror(errno)); + femu_err("WARNING: FEMU_ALLOW_UNPINNED=1 is set, continuing anyway. " + "Latency measurements are only trustworthy while the host " + "is not swapping.\n"); + } else { + femu_err("Failed to pin the memory backend to the host DRAM\n"); + femu_err("Raise RLIMIT_MEMLOCK (ulimit -l) to at least %" PRId64 + " MB, or set FEMU_ALLOW_UNPINNED=1 to continue unpinned.\n", + nbytes >> 20); + g_free(b->logical_space); + abort(); + } } return 0; diff --git a/hw/femu/nand/test/test_channel.c b/hw/femu/nand/test/test_channel.c new file mode 100644 index 00000000000..5fcd3afcb67 --- /dev/null +++ b/hw/femu/nand/test/test_channel.c @@ -0,0 +1,175 @@ +/* Host unit tests for the NAND media timing model's channel stage. + * + * bbssd hardcodes channel_mode = NAND_CH_OFF (ftl-media.c:129), so pg_xfer_lat is + * copied into the config and never read. Before enabling NAND_CH_STAGED we need + * to know (a) that OFF is unchanged, and (b) that STAGED does what the analytical + * channel model assumed. Neither needs QEMU, a guest, or KVM: nand_media_op() is + * pure timing arithmetic over a caller-supplied timeline. + */ +#include +#include +#include +#include +#include +#include "nand-media.h" + +#define QLC 4 +static const int64_t QLC_RD[4] = {47900, 76200, 134600, 228100}; /* measured */ +static const int64_t XFER = 52433; /* QLC_CHNL_PAGE_TRANSFER_LATENCY_NS */ +#define MAXCH 8 +#define MAXLUN 16 + +typedef struct { uint64_t ch[MAXCH]; uint64_t lun[MAXCH][MAXLUN]; uint64_t pl[MAXCH][MAXLUN]; } State; +static State ST; +static uint64_t *t_ch(void *o, uint32_t c) { (void)o; return &ST.ch[c]; } +static uint64_t *t_lun(void *o, const NandLoc *l) { (void)o; return &ST.lun[l->ch][l->lun]; } +static uint64_t *t_pl(void *o, const NandLoc *l) { (void)o; return &ST.pl[l->ch][l->lun]; } +static const NandTimelineOps OPS = { .ch_avail=t_ch, .lun_avail=t_lun, .plane_avail=t_pl }; + +static void setup(NandMedia *m, uint32_t nch, uint32_t luns, NandChannelMode mode, bool bus) +{ + NandMediaConfig c; + memset(&c, 0, sizeof c); + memset(&ST, 0, sizeof ST); + c.nchs = nch; c.luns_per_ch = luns; c.planes_per_lun = 1; + for (int p = 0; p < 4; p++) c.timing.rd_table_ns[QLC][p] = QLC_RD[p]; + c.timing.wr_table_ns[QLC][0] = 1000000; + c.timing.er_table_ns[QLC] = 3000000; + if (bus) c.timing.page_xfer_ns = XFER; /* cmd_addr / status left 0 */ + c.policy.use_flat_timing = false; + c.policy.array_gate = NAND_GATE_LUN_ONLY; + c.policy.channel_mode = mode; + c.timeline = &OPS; c.timeline_opaque = NULL; + nand_media_init(m, &c); +} + +static uint64_t rd(NandMedia *m, uint32_t ch, uint32_t lun, int ptype, uint64_t at) +{ + NandLoc l; memset(&l, 0, sizeof l); + l.ch = ch; l.lun = lun; l.flash_type = QLC; l.page_type = ptype; + return nand_media_op(m, &l, NAND_MEDIA_READ, at).done_ns; +} + +static int fails; +static void ck(int ok, const char *what, const char *detail) +{ + printf(" [%s] %s%s%s\n", ok ? "PASS" : "FAIL", what, + detail && *detail ? " — " : "", detail ? detail : ""); + if (!ok) fails++; +} + +/* ---- 1. OFF must be untouched, and STAGED with a zero bus must equal it ---- */ +static void t_compat(void) +{ + printf("[1] 하위호환: OFF, 그리고 bus=0 인 STAGED\n"); + uint64_t off[64], staged0[64]; + NandMedia m; + setup(&m, 1, 4, NAND_CH_OFF, false); + for (int i = 0; i < 64; i++) off[i] = rd(&m, 0, i % 4, i % 4, 0); + setup(&m, 1, 4, NAND_CH_STAGED, false); + for (int i = 0; i < 64; i++) staged0[i] = rd(&m, 0, i % 4, i % 4, 0); + /* Documents the opposite of what one would hope: enabling the channel stage + * changes timing even with every bus phase at zero, because the channel + * timeline is advanced to each op's data-out and the next command is clamped + * to it. This is why a channel_model option has to default to off. */ + ck(memcmp(off, staged0, sizeof off) != 0, + "STAGED 는 bus=0 이어도 OFF 와 다름", + "channel_model 옵션은 반드시 off 를 기본값으로 해야 함"); + + /* OFF must ignore the bus entirely, even when pg_xfer_lat is set */ + uint64_t offbus[64]; + setup(&m, 1, 4, NAND_CH_OFF, true); + for (int i = 0; i < 64; i++) offbus[i] = rd(&m, 0, i % 4, i % 4, 0); + ck(!memcmp(off, offbus, sizeof off), "OFF 는 pg_xfer_lat 를 무시", + "오늘의 동작 — 값을 줘도 타이밍에 반영 안 됨"); +} + +/* ---- 2. one LUN: array read then data-out, serialised ---- */ +static void t_single(void) +{ + printf("[2] LUN 1개: array read 후 data-out 직렬화\n"); + NandMedia m; char b[160]; + setup(&m, 1, 1, NAND_CH_STAGED, true); + uint64_t d = rd(&m, 0, 0, 0, 0); + snprintf(b, sizeof b, "관측 %.1f us = array %.1f + xfer %.1f", + d/1000.0, QLC_RD[0]/1000.0, XFER/1000.0); + ck(d == (uint64_t)(QLC_RD[0] + XFER), "1회 읽기 = array + xfer", b); + + setup(&m, 1, 1, NAND_CH_STAGED, true); + uint64_t a = rd(&m, 0, 0, 0, 0), c = rd(&m, 0, 0, 0, 0); + snprintf(b, sizeof b, "1번째 %.1f us, 2번째 %.1f us", a/1000.0, c/1000.0); + ck(c >= a + QLC_RD[0], "같은 LUN 연속 읽기는 array 시간만큼 직렬화", b); +} + +/* ---- 3. two LUNs on one channel: sensing overlaps, data-out does not ---- */ +static void t_overlap(void) +{ + printf("[3] 한 채널의 LUN 2개: sensing 겹침, data-out 직렬화\n"); + NandMedia m; char b[160]; + setup(&m, 1, 2, NAND_CH_STAGED, true); + uint64_t a = rd(&m, 0, 0, 3, 0); /* slow page on LUN0 */ + uint64_t c = rd(&m, 0, 1, 3, 0); /* slow page on LUN1, same instant */ + snprintf(b, sizeof b, "LUN0 %.1f us, LUN1 %.1f us (직렬이면 %.1f)", + a/1000.0, c/1000.0, (QLC_RD[3]*2 + XFER*2)/1000.0); + /* Physical NAND would overlap here. This model does not: reservations are made + * in submission order, so LUN1's command waits for LUN0's data-out. */ + ck(c == (uint64_t)(QLC_RD[3] * 2 + XFER * 2), + "두 LUN 이 완전히 직렬화됨 (실제 NAND 와 다름)", b); + snprintf(b, sizeof b, "두 완료 간격 %.1f us, xfer %.1f us", (c-a)/1000.0, XFER/1000.0); + ck(c - a >= (uint64_t)XFER, "data-out 은 채널에서 직렬화", b); +} + +/* ---- 4. saturation: does the channel erase the page-type advantage? ---- */ +static void t_saturation(void) +{ + printf("[4] 포화: LUN/채널 수에 따라 page mapping 이득이 남는가\n"); + /* traffic shares by plane index, 2-tier unified-lru @1.8GB */ + const double w[4] = {0.381, 0.381, 0.119, 0.119}; + const int N = 4000; + /* The analytical channel model predicted 1.346 / 1.161 / 1.000 / 1.000 as LUNs + * per channel grow, assuming extra LUNs overlap sensing with another LUN's + * data burst. This model never overlaps them, so the gain is flat at + * (mean_array + xfer) / (aware_array + xfer). */ + const double pred[] = {1.219, 1.219, 1.219, 1.219}; + printf(" %-8s %10s %10s %8s %s\n", "LUN/ch", "oblivious", "aware", "gain", "모델 예상"); + int pi = 0; + for (uint32_t luns = 1; luns <= 8; luns *= 2, pi++) { + uint64_t mk[2]; + for (int arm = 0; arm < 2; arm++) { + NandMedia m; setup(&m, 1, luns, NAND_CH_STAGED, true); + uint64_t last = 0; int k = 0; + for (int i = 0; i < N; i++) { + /* pick a plane index by traffic share */ + double u = (double)(i % 1000) / 1000.0, acc = 0; int plane = 3; + for (int j = 0; j < 4; j++) { acc += w[j]; if (u < acc) { plane = j; break; } } + /* oblivious: traffic spread evenly over the four page classes. + aware: plane index maps one-to-one onto page class. */ + int ptype = arm == 0 ? (k++ % 4) : plane; + uint64_t d = rd(&m, 0, i % luns, ptype, 0); + if (d > last) last = d; + } + mk[arm] = last; + } + double gain = (double)mk[0] / (double)mk[1]; + char note[64]; + snprintf(note, sizeof note, "%.3fx", pred[pi]); + printf(" %-8u %9.2fms %9.2fms %7.3fx %s\n", + luns, mk[0]/1e6, mk[1]/1e6, gain, note); + if (fabs(gain - pred[pi]) > 0.05) { + printf(" ^ 예측에서 벗어남\n"); fails++; + } + } +} + +int main(void) +{ + printf("NAND media 채널 스테이지 단위 테스트 (QEMU/게스트/KVM 불필요)\n"); + printf("QLC read %.1f/%.1f/%.1f/%.1f us, channel page xfer %.2f us\n\n", + QLC_RD[0]/1000.0, QLC_RD[1]/1000.0, QLC_RD[2]/1000.0, QLC_RD[3]/1000.0, XFER/1000.0); + t_compat(); printf("\n"); + t_single(); printf("\n"); + t_overlap(); printf("\n"); + t_saturation(); printf("\n"); + printf(fails ? "실패 %d 건\n" : "전부 통과\n", fails); + return fails ? 1 : 0; +} From 706a89cdb8d369bf319b0dab41ba07b69e5bfb25 Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 11 Sep 2026 06:49:25 +0000 Subject: [PATCH 03/17] femu/nand: reach the last page of the QLC pairing cycle The cycle starts at page 8, so it needs rows - 1 iterations to cover the block; rows - 3 stopped at index 495 and left pages 496..511 holding their zero-initialised value, which reads as QLC_LOWER_PAGE. At 256 pages per block nothing reached those entries, so the bug was invisible in every run taken so far and only appeared once the geometry grew to 512. A host-side test extracts init_qlc_page_pairing from this file at build time and asserts the class histogram, so the fix cannot silently regress. Co-Authored-By: Claude Opus 5 --- hw/femu/nand/nand.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/hw/femu/nand/nand.c b/hw/femu/nand/nand.c index b4780ed0207..1272a6e00f1 100644 --- a/hw/femu/nand/nand.c +++ b/hw/femu/nand/nand.c @@ -111,7 +111,13 @@ static void init_qlc_page_pairing(FemuCtrl *n) for (i = 0; i < sizeof(centerup)/sizeof(centerup[0]); i++) qlc_tbl[centerup[i]] = QLC_UPPER_CENTER_PAGE; - for (i = 0; i < rows - 3; i++) { + /* + * The cycle starts at page 8, so it needs (rows - 1) iterations to reach + * the last page; rows - 3 stops at index 495 and leaves pages 496..511 at + * their zero-initialised value, which reads as QLC_LOWER_PAGE. That is + * invisible while pgs_per_blk <= 496, and wrong above it. + */ + for (i = 0; i < rows - 1; i++) { for (j = 0; j < page_per_row; j += 2) { int idx = 8 + (i * page_per_row) + j; qlc_tbl[idx] = qlc_tbl[idx+1] = lpflag; From 9f8fdadb7d451181444df6482e1fcc471918a848 Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 11 Sep 2026 06:49:39 +0000 Subject: [PATCH 04/17] femu/bbssd: count physical QLC page reads by class The placement work needs to know which of the four QLC page classes a read landed on, and how many pages of each class the workload actually touched. Counting at the NAND boundary rather than at the host queue includes mapping-table and GC reads, which is the right boundary for NAND-core energy: the array does that work whether or not the host asked for it. active_ns records the raw array latency chosen for the class and excludes queueing on purpose. Queueing belongs to the controller and would make the per-class figure depend on the queue depth rather than on the medium. Co-Authored-By: Claude Opus 5 --- hw/femu/backend/dram.c | 4 +++- hw/femu/bbssd/ftl-media.c | 25 +++++++++++++++++++++++++ hw/femu/bbssd/ftl.h | 28 ++++++++++++++++++++++++++++ hw/femu/nand/nand.h | 33 +++++++++++++-------------------- 4 files changed, 69 insertions(+), 21 deletions(-) diff --git a/hw/femu/backend/dram.c b/hw/femu/backend/dram.c index c38e99b4621..031b5ef9325 100644 --- a/hw/femu/backend/dram.c +++ b/hw/femu/backend/dram.c @@ -76,7 +76,9 @@ int init_dram_backend(SsdDramBackend **mbe, int64_t nbytes) * because a fault during an emulated access lands directly in the measured * latency. Default behaviour is unchanged. */ - if (getenv("FEMU_ALLOW_UNPINNED")) { + const char *allow_unpinned = getenv("FEMU_ALLOW_UNPINNED"); + if (allow_unpinned && allow_unpinned[0] && + strcmp(allow_unpinned, "0") != 0) { femu_err("WARNING: memory backend is NOT pinned (mlock: %s).\n", strerror(errno)); femu_err("WARNING: FEMU_ALLOW_UNPINNED=1 is set, continuing anyway. " diff --git a/hw/femu/bbssd/ftl-media.c b/hw/femu/bbssd/ftl-media.c index a113a4bd30a..5a64c65a191 100644 --- a/hw/femu/bbssd/ftl-media.c +++ b/hw/femu/bbssd/ftl-media.c @@ -171,5 +171,30 @@ uint64_t ssd_advance_status(struct ssd *ssd, struct ppa *ppa, return 0; } loc = bb_decode_loc(ssd, ppa); + + /* + * Count physical QLC page reads at the NAND boundary. This includes host, + * mapping-table and GC reads, which is the correct boundary for NAND-core + * energy. Queueing time is deliberately excluded from active_ns; it records + * the raw PACA array latency selected for this page class. + */ + if (op == NAND_MEDIA_READ && loc.flash_type == QLC && loc.page_type < 4) { + uint64_t page_bytes = (uint64_t)ssd->sp.secsz * ssd->sp.secs_per_pg; + uint64_t active_ns = + ssd->media.cfg.timing.rd_table_ns[loc.flash_type][loc.page_type]; + + if (!__atomic_load_n(&ssd->qlc_first_read_ns, __ATOMIC_RELAXED)) { + __atomic_store_n(&ssd->qlc_first_read_ns, + qemu_clock_get_ns(QEMU_CLOCK_REALTIME), + __ATOMIC_RELAXED); + } + __atomic_fetch_add(&ssd->qlc_read_pages[loc.page_type], 1, + __ATOMIC_RELAXED); + __atomic_fetch_add(&ssd->qlc_read_bytes[loc.page_type], page_bytes, + __ATOMIC_RELAXED); + __atomic_fetch_add(&ssd->qlc_read_active_ns[loc.page_type], active_ns, + __ATOMIC_RELAXED); + } + return nand_media_op(&ssd->media, &loc, op, stime).latency_ns; } diff --git a/hw/femu/bbssd/ftl.h b/hw/femu/bbssd/ftl.h index 0a6ac41658f..a02d3a5832a 100644 --- a/hw/femu/bbssd/ftl.h +++ b/hw/femu/bbssd/ftl.h @@ -54,6 +54,20 @@ enum { FEMU_RESET_ACCT = 5, FEMU_ENABLE_LOG = 6, FEMU_DISABLE_LOG = 7, + + /* + * The QLC read counters start at zero when the device is created, so a + * dump at process exit covers everything the device ever did: the guest + * probing it at boot, the catalog fill, FTL and GC traffic, the workload, + * and shutdown. Nothing in that total distinguishes the part under study. + * + * FEMU_RESET_QLC zeroes them without touching stored data, the LBA-to-PPA + * map or the page layout -- it resets the meter, not the drive. Issue it + * after the fill and before the workload; FEMU_SNAP_QLC writes the counters + * out at a chosen instant rather than waiting for teardown. + */ + FEMU_RESET_QLC = 8, + FEMU_SNAP_QLC = 9, }; @@ -504,6 +518,20 @@ struct ssd { uint64_t nand_write_pages; /* user pages programmed into NAND */ uint64_t gc_write_pages; /* pages the device relocated itself */ + /* + * QLC read-energy accounting. FEMU records physical media activity only; + * cited energy coefficients are applied offline so model assumptions stay + * explicit and replaceable. Index is QLC page class 0..3. + */ + uint64_t qlc_read_pages[4]; + uint64_t qlc_read_bytes[4]; + uint64_t qlc_read_active_ns[4]; + /* First counted NAND read, QEMU_CLOCK_REALTIME ns; 0 until the first one. + * Sum(t_active) adds per-LUN service time, so it exceeds elapsed time by the + * LUN parallelism. The controller is one resource, so its energy has to be + * charged against elapsed time instead - this is what makes that available. */ + uint64_t qlc_first_read_ns; + bool debug_ftl; /* check FTL invariants on the GC path (off by default) */ /* diff --git a/hw/femu/nand/nand.h b/hw/femu/nand/nand.h index bb757f83fed..11015bd0fb4 100644 --- a/hw/femu/nand/nand.h +++ b/hw/femu/nand/nand.h @@ -43,20 +43,13 @@ /* * QLC NAND latency numbers in nanoseconds * - * Read Latency is measured, at 16 KB per page. The four values are the four - * page types of a QLC wordline, fast to slow; relative to the fastest they are - * 1 : 1.59 : 2.81 : 4.76, mean 121.70 us. + * Measured on Intel 96-layer 3D QLC gen3 (1024Q3D3A): + * Q. Chen et al., "PACA: A Page Type Aware Read Cache Scheme in QLC + * Flash-based SSDs", ICCD 2022, Fig. 2 and Table I. * - * These replace an extrapolation from TLC (Micron FMS'19, "Component-Level - * Characterization of 3D TLC, QLC, and Low-Latency NAND"), which gave - * 59.33 / 85.25 / 127.20 / 169.60 us. That vector is both faster in the mean - * (110.34 us) and narrower in spread (1 : 1.44 : 2.14 : 2.86), so it understates - * what page placement is worth: a bit-plane layout that gains 1.346x under the - * measured vector gains only 1.221x under the extrapolated one. - * - * Write Latency is still the TLC extrapolation - it was not measured. Our - * workload writes once and then only reads, so program time does not enter the - * result; revisit before running anything write-sensitive. + * PACA reports one average program latency rather than a value for each page + * class. Keep the four program entries equal so only measured page-class + * variation is represented. */ #define QLC_LOWER_PAGE_READ_LATENCY_NS (47900) @@ -64,13 +57,14 @@ #define QLC_CENTER_UPPER_PAGE_READ_LATENCY_NS (134600) #define QLC_UPPER_PAGE_READ_LATENCY_NS (228100) -#define QLC_LOWER_PAGE_WRITE_LATENCY_NS (TLC_LOWER_PAGE_WRITE_LATENCY_NS * 1.05) -#define QLC_CENTER_LOWER_PAGE_WRITE_LATENCY_NS (TLC_CENTER_PAGE_WRITE_LATENCY_NS * 1.1) -#define QLC_CENTER_UPPER_PAGE_WRITE_LATENCY_NS (TLC_UPPER_PAGE_WRITE_LATENCY_NS * 1.2) -#define QLC_UPPER_PAGE_WRITE_LATENCY_NS (TLC_UPPER_PAGE_WRITE_LATENCY_NS * 1.6) +#define QLC_LOWER_PAGE_WRITE_LATENCY_NS (1860000) +#define QLC_CENTER_LOWER_PAGE_WRITE_LATENCY_NS (1860000) +#define QLC_CENTER_UPPER_PAGE_WRITE_LATENCY_NS (1860000) +#define QLC_UPPER_PAGE_WRITE_LATENCY_NS (1860000) -#define QLC_CHNL_PAGE_TRANSFER_LATENCY_NS (52433) -#define QLC_BLOCK_ERASE_LATENCY_NS (3000000) +/* 16 KiB at 800 MT/s. Sweep 40960/20480/13653 ns for 400/800/1200 MT/s. */ +#define QLC_CHNL_PAGE_TRANSFER_LATENCY_NS (20480) +#define QLC_BLOCK_ERASE_LATENCY_NS (6340000) enum { SLC_PAGE = 0, @@ -146,4 +140,3 @@ int64_t get_blk_erase_latency(int flash_type); int init_nand_flash(void *opaque); #endif - From 28cc6ce4c920fb796d381cca7ad756bd69c6a872 Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 11 Sep 2026 06:49:49 +0000 Subject: [PATCH 05/17] femu: report NAND read energy split into peripheral and array A per-class read coefficient alone cannot say where the energy goes. The array term scales with the number of sensing steps a class needs (1, 2, 4, 8) while the peripheral term scales with the time the page is held open, so the two move differently as the placement changes and reporting only their sum hides the mechanism the placement is acting on. The array coefficient is clamped to the read total, so a mis-set pair can only make the peripheral remainder zero, never negative. Coefficients are device properties with the measured defaults rather than constants, and the stats file records the pair it used so a CSV can be read years later without the binary that wrote it. Co-Authored-By: Claude Opus 5 --- hw/femu/bbssd/bb.c | 180 +++++++++++++++++++++++++++++++++++++++++++-- hw/femu/femu.c | 106 ++++++++++++++++++++++++++ hw/femu/nvme.h | 25 +++++++ 3 files changed, 304 insertions(+), 7 deletions(-) diff --git a/hw/femu/bbssd/bb.c b/hw/femu/bbssd/bb.c index a7f045f4a7f..84b6bcee97e 100644 --- a/hw/femu/bbssd/bb.c +++ b/hw/femu/bbssd/bb.c @@ -44,6 +44,8 @@ static void bb_init(FemuCtrl *n, NvmeNamespace *ns, Error **errp) * only whichever namespace brought its mode up first, and using it would leave * every other FTL-backed namespace on the previous setting. */ +static void bb_flush_stats(FemuCtrl *n); + static void bb_flip_apply(FemuCtrl *n, int64_t cdw10) { int i; @@ -119,6 +121,37 @@ static void bb_flip(FemuCtrl *n, NvmeCmd *cmd) femu_log("%s,Reset tt_late_ios/tt_ios,%ld/%ld\n", n->devname, late, tt); break; } + case FEMU_RESET_QLC: { + /* + * Zero the physical-read meters so what follows is attributable to the + * workload alone. Stored data, the mapping table and the page layout are + * untouched. qlc_first_read_ns goes too, so elapsed time restarts at the + * next counted read rather than at one from the fill. + */ + uint64_t before = 0; + for (int i = 0; i < n->num_namespaces; i++) { + struct ssd *ssd = n->namespaces[i].ssd; + if (!ssd) { + continue; + } + for (int c = 0; c < 4; c++) { + before += __atomic_load_n(&ssd->qlc_read_pages[c], __ATOMIC_RELAXED); + __atomic_store_n(&ssd->qlc_read_pages[c], 0, __ATOMIC_RELAXED); + __atomic_store_n(&ssd->qlc_read_bytes[c], 0, __ATOMIC_RELAXED); + __atomic_store_n(&ssd->qlc_read_active_ns[c], 0, __ATOMIC_RELAXED); + } + __atomic_store_n(&ssd->qlc_first_read_ns, 0, __ATOMIC_RELAXED); + } + /* Logged, not discarded: the pre-workload total is itself a measurement + * of what boot and fill cost, and it is the only record of it. */ + femu_log("%s,QLC counters reset, discarded %" PRIu64 " pages\n", + n->devname, before); + break; + } + case FEMU_SNAP_QLC: + bb_flush_stats(n); + femu_log("%s,QLC counters snapshotted\n", n->devname); + break; case FEMU_ENABLE_LOG: n->print_log = true; femu_log("%s,Log print [Enabled]!\n", n->devname); @@ -132,16 +165,149 @@ static void bb_flip(FemuCtrl *n, NvmeCmd *cmd) } } -/* - * Release what the namespace's FTL still holds. Reached for the mode the - * controller itself runs; a namespace running bbssd underneath a controller of - * another mode is not dispatched an exit at all, which is a gap in the generic - * teardown rather than one here. - */ +/* Snapshot physical QLC activity without freeing state; process-exit notifiers + * use this path because PCI device teardown is not guaranteed at VM shutdown. */ +static void bb_flush_stats(FemuCtrl *n) +{ + uint64_t pages[4] = {0}; + uint64_t bytes[4] = {0}; + uint64_t active_ns[4] = {0}; + uint64_t first_ns = 0; + uint64_t wall_ns = 0; + int n_luns = 0; + const char *stats_path = getenv("FEMU_QLC_STATS_PATH"); + FILE *stats = NULL; + uint64_t t; + int i; + + for (i = 0; i < n->num_namespaces; i++) { + struct ssd *ssd = n->namespaces[i].ssd; + + if (ssd) { + int page_class; + + for (page_class = 0; page_class < 4; page_class++) { + pages[page_class] += __atomic_load_n( + &ssd->qlc_read_pages[page_class], __ATOMIC_RELAXED); + bytes[page_class] += __atomic_load_n( + &ssd->qlc_read_bytes[page_class], __ATOMIC_RELAXED); + active_ns[page_class] += __atomic_load_n( + &ssd->qlc_read_active_ns[page_class], __ATOMIC_RELAXED); + } + + if (!n_luns) { + n_luns = ssd->sp.nchs * ssd->sp.luns_per_ch * ssd->sp.pls_per_lun; + } + + t = __atomic_load_n(&ssd->qlc_first_read_ns, __ATOMIC_RELAXED); + if (t && (!first_ns || t < first_ns)) { + first_ns = t; + } + } + } + + /* + * Elapsed time since the first counted read. Sum(t_active) is per-LUN service + * time added up, so it runs ahead of this by roughly the LUN parallelism; the + * controller is a single resource and has to be charged against elapsed time. + */ + if (first_ns) { + uint64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + + wall_ns = now > first_ns ? now - first_ns : 0; + } + + if (stats_path && stats_path[0]) { + stats = fopen(stats_path, "w"); + if (!stats) { + femu_log("QLC stats: cannot open %s: %s\n", + stats_path, strerror(errno)); + } + } + + /* + * Energy columns are Sum(count x cited coefficient), the same arithmetic the + * offline accounting does; the coefficients used are written into the file so + * the columns stay auditable. Coefficients are milli-pJ/bit, byte counts are + * exact, so uJ = bytes * 8 * mpj / 1e9. Sweeps (channel, controller, idle, + * scenario codes) stay offline in experiments/energy_account.py. + */ + if (stats) { + double e_nand_uj[4], e_xfer_uj[4], e_array_uj[4], e_periph_uj[4]; + double nand_total = 0, xfer_total = 0; + double array_total = 0, periph_total = 0; + + for (i = 0; i < 4; i++) { + double bits = (double)bytes[i] * 8.0; + /* + * Peripheral is the remainder rather than its own coefficient, so + * the two halves always add back to the total the offline + * accounting uses. An array share above the total would make it + * negative, which is a misconfiguration, not a measurement. + */ + uint32_t array_mpj = n->e_array_mpj[i] <= n->e_read_mpj[i] + ? n->e_array_mpj[i] : n->e_read_mpj[i]; + + e_nand_uj[i] = bits * n->e_read_mpj[i] / 1e9; + e_array_uj[i] = bits * array_mpj / 1e9; + e_periph_uj[i] = e_nand_uj[i] - e_array_uj[i]; + e_xfer_uj[i] = bits * n->e_xfer_mpj / 1e9; + nand_total += e_nand_uj[i]; + array_total += e_array_uj[i]; + periph_total += e_periph_uj[i]; + xfer_total += e_xfer_uj[i]; + + if (n->e_array_mpj[i] > n->e_read_mpj[i]) { + femu_log("QLC energy: class %d array coefficient %u exceeds the " + "read total %u; clamped\n", + i, n->e_array_mpj[i], n->e_read_mpj[i]); + } + } + + fprintf(stats, "# coeff_mpj_per_bit: c0=%u c1=%u c2=%u c3=%u xfer=%u\n", + n->e_read_mpj[0], n->e_read_mpj[1], n->e_read_mpj[2], + n->e_read_mpj[3], n->e_xfer_mpj); + fprintf(stats, "# array_mpj_per_bit: c0=%u c1=%u c2=%u c3=%u " + "(peripheral is the remainder of each read coefficient)\n", + n->e_array_mpj[0], n->e_array_mpj[1], n->e_array_mpj[2], + n->e_array_mpj[3]); + fprintf(stats, "# nand_cell_type=%u e_nand_uj_total=%.3f " + "e_periph_uj_total=%.3f e_array_uj_total=%.3f " + "e_xfer_uj_total=%.3f\n", + n->nand_cell_type, nand_total, periph_total, array_total, + xfer_total); + fprintf(stats, "# t_wall_us=%.3f t_active_sum_us=%.3f luns=%d\n", + wall_ns / 1000.0, + (active_ns[0] + active_ns[1] + active_ns[2] + active_ns[3]) + / 1000.0, n_luns); + fprintf(stats, "# t_wall is elapsed since the first counted read (the " + "observation window). t_active_sum is per-LUN service time " + "added up, so device busy time is about t_active_sum/luns; the " + "controller belongs on that, not on either raw number.\n"); + fprintf(stats, "page_class,n_read,bytes_read,t_active_us," + "e_nand_uj,e_periph_uj,e_array_uj,e_xfer_uj\n"); + for (i = 0; i < 4; i++) { + fprintf(stats, "%d,%" PRIu64 ",%" PRIu64 ",%.3f,%.3f,%.3f,%.3f,%.3f\n", + i, pages[i], bytes[i], active_ns[i] / 1000.0, + e_nand_uj[i], e_periph_uj[i], e_array_uj[i], e_xfer_uj[i]); + } + fclose(stats); + } else { + for (i = 0; i < 4; i++) { + femu_log("QLC_READ_STATS,class=%d,reads=%" PRIu64 + ",bytes=%" PRIu64 ",active_ns=%" PRIu64 + ",e_nand_uj=%.3f\n", + i, pages[i], bytes[i], active_ns[i], + (double)bytes[i] * 8.0 * n->e_read_mpj[i] / 1e9); + } + } +} + static void bb_exit(FemuCtrl *n) { int i; + bb_flush_stats(n); for (i = 0; i < n->num_namespaces; i++) { struct ssd *ssd = n->namespaces[i].ssd; @@ -186,6 +352,7 @@ int nvme_register_bbssd(FemuCtrl *n) .state = NULL, .init = bb_init, .exit = bb_exit, + .stats_flush = bb_flush_stats, .rw_check_req = NULL, .admin_cmd = bb_admin_cmd, .io_cmd = bb_io_cmd, @@ -194,4 +361,3 @@ int nvme_register_bbssd(FemuCtrl *n) return 0; } - diff --git a/hw/femu/femu.c b/hw/femu/femu.c index 5a5a8470f3a..b6bb37375e0 100644 --- a/hw/femu/femu.c +++ b/hw/femu/femu.c @@ -1,6 +1,8 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" +#include "qemu/timer.h" #include "hw/qdev-properties.h" +#include "system/system.h" #include "./nvme.h" @@ -1175,6 +1177,29 @@ static void nvme_register_extensions_ns(FemuCtrl *n, NvmeNamespace *ns) n->ext_ops = saved_ops; } +static void femu_flush_extension_stats(FemuCtrl *n); + +static void femu_process_exit_notify(Notifier *notifier, void *data) +{ + FemuCtrl *n = container_of(notifier, FemuCtrl, process_exit_notifier); + + femu_flush_extension_stats(n); +} + +/* + * Periodic snapshot so a long run can be watched while it is still going. + * Re-arms itself; the snapshot path does not free or reset any state, so the + * counters keep accumulating and the file is simply rewritten each tick. + */ +static void femu_stats_timer_cb(void *opaque) +{ + FemuCtrl *n = opaque; + + femu_flush_extension_stats(n); + timer_mod(n->stats_timer, + qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + n->stats_flush_ms); +} + static void femu_realize(PCIDevice *pci_dev, Error **errp) { FemuCtrl *n = FEMU(pci_dev); @@ -1285,6 +1310,20 @@ static void femu_realize(PCIDevice *pci_dev, Error **errp) n, QEMU_THREAD_JOINABLE); n->ftl_thread_running = true; } + + /* PCI exit is not guaranteed on whole-process shutdown. Keep experiment + * counters observable on normal guest poweroff and QMP quit as well. */ + n->process_exit_notifier.notify = femu_process_exit_notify; + qemu_add_exit_notifier(&n->process_exit_notifier); + n->process_exit_notifier_registered = true; + + if (n->stats_flush_ms) { + n->stats_timer = timer_new_ms(QEMU_CLOCK_REALTIME, + femu_stats_timer_cb, n); + timer_mod(n->stats_timer, + qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + n->stats_flush_ms); + femu_log("QLC stats snapshot every %u ms\n", n->stats_flush_ms); + } } /* @@ -1360,6 +1399,38 @@ static void femu_exit_extensions(FemuCtrl *n) } } +/* Run each distinct mode's lightweight stats snapshot without freeing state. */ +static void femu_flush_extension_stats(FemuCtrl *n) +{ + void (*seen[FEMU_NR_MODES])(struct FemuCtrl *); + int nseen = 0, i, j; + + if (n->ext_ops.stats_flush) { + seen[nseen++] = n->ext_ops.stats_flush; + } + + for (i = 0; n->namespaces && i < n->num_namespaces; i++) { + void (*flush)(struct FemuCtrl *) = + n->namespaces[i].ext_ops.stats_flush; + + if (!flush) { + continue; + } + for (j = 0; j < nseen; j++) { + if (seen[j] == flush) { + break; + } + } + if (j == nseen && nseen < (int)ARRAY_SIZE(seen)) { + seen[nseen++] = flush; + } + } + + for (j = 0; j < nseen; j++) { + seen[j](n); + } +} + static void femu_exit(PCIDevice *pci_dev) { FemuCtrl *n = FEMU(pci_dev); @@ -1367,6 +1438,14 @@ static void femu_exit(PCIDevice *pci_dev) femu_debug("femu_exit starting!\n"); femu_stop_ftl_thread(n); + if (n->stats_timer) { + timer_free(n->stats_timer); + n->stats_timer = NULL; + } + if (n->process_exit_notifier_registered) { + qemu_remove_exit_notifier(&n->process_exit_notifier); + n->process_exit_notifier_registered = false; + } femu_exit_extensions(n); nvme_clear_ctrl(n, true); @@ -1511,6 +1590,33 @@ static const Property femu_props[] = { DEFINE_PROP_STRING("mapping", FemuCtrl, bb_params.mapping_scheme), DEFINE_PROP_UINT32("mapping_cache_mb", FemuCtrl, mapping_cache_mb, 0), DEFINE_PROP_UINT8("nand_cell_type", FemuCtrl, nand_cell_type, 0), + /* QLC page-class read energy, milli-pJ/bit. docs/25 SS5.1 primary profile. */ + /* + * Read energy per class, milli-pJ/bit, as peripheral + array: + * peripheral = P_fix + rho_p * t_R(c), P_fix = 1.273, rho_p = 0.668 + * array = E_fix + (n(c) - 1) * E_sense + * with t_R(c) the measured class read latencies this device already models. + * The peripheral term therefore tracks read latency and, at 97-98% of the + * total, dominates it; see e_array_c*_mpj for the other half. + */ + DEFINE_PROP_UINT32("e_read_c0_mpj", FemuCtrl, e_read_mpj[0], 33821), + DEFINE_PROP_UINT32("e_read_c1_mpj", FemuCtrl, e_read_mpj[1], 53220), + DEFINE_PROP_UINT32("e_read_c2_mpj", FemuCtrl, e_read_mpj[2], 93219), + DEFINE_PROP_UINT32("e_read_c3_mpj", FemuCtrl, e_read_mpj[3], 157653), + /* + * Array share of each read coefficient, milli-pJ/bit: + * E_fix + (n(c) - 1) * E_sense, E_fix = 0.551, E_sense = 0.494 pJ/bit, + * n(c) = 1 / 2 / 4 / 8 senses. + * n(c) is an estimate of per-class sensing complexity, not a cited figure; + * the read latencies it sits beside are measured. Peripheral energy is the + * remainder, e_read_mpj - e_array_mpj. + */ + DEFINE_PROP_UINT32("e_array_c0_mpj", FemuCtrl, e_array_mpj[0], 551), + DEFINE_PROP_UINT32("e_array_c1_mpj", FemuCtrl, e_array_mpj[1], 1045), + DEFINE_PROP_UINT32("e_array_c2_mpj", FemuCtrl, e_array_mpj[2], 2033), + DEFINE_PROP_UINT32("e_array_c3_mpj", FemuCtrl, e_array_mpj[3], 4009), + DEFINE_PROP_UINT32("e_xfer_mpj", FemuCtrl, e_xfer_mpj, 28100), + DEFINE_PROP_UINT32("stats_flush_ms", FemuCtrl, stats_flush_ms, 0), DEFINE_PROP_INT32("cell_pages", FemuCtrl, bb_params.cell_pages, 0), DEFINE_PROP_INT32("pgtype_lat", FemuCtrl, bb_params.pgtype_lat, 0), DEFINE_PROP_INT32("ecc_step_ns", FemuCtrl, bb_params.ecc_step_ns, 0), diff --git a/hw/femu/nvme.h b/hw/femu/nvme.h index d6612f44b9c..611ea7c760d 100644 --- a/hw/femu/nvme.h +++ b/hw/femu/nvme.h @@ -6,6 +6,7 @@ #include "qemu/units.h" #include "qemu/cutils.h" #include "qemu/memalign.h" +#include "qemu/notify.h" #include "hw/pci/msix.h" #include "hw/pci/msi.h" #include "hw/virtio/vhost.h" @@ -1452,6 +1453,7 @@ typedef struct FemuExtCtrlOps { void *state; void (*init)(struct FemuCtrl *, NvmeNamespace *, Error **); void (*exit)(struct FemuCtrl *); + void (*stats_flush)(struct FemuCtrl *); uint16_t (*rw_check_req)(struct FemuCtrl *, NvmeCmd *, NvmeRequest *); int (*start_ctrl)(struct FemuCtrl *); uint16_t (*admin_cmd)(struct FemuCtrl *, NvmeCmd *); @@ -1740,6 +1742,8 @@ typedef struct FemuCtrl { /* Coperd: OC2.0 FIXME */ NvmeParams params; FemuExtCtrlOps ext_ops; + Notifier process_exit_notifier; + bool process_exit_notifier_registered; time_t start_time; uint16_t temperature; @@ -1830,6 +1834,27 @@ typedef struct FemuCtrl { uint32_t read_cache_mb; /* bbssd DRAM read cache size (0 = off) */ uint32_t mapping_cache_mb; /* bbssd DFTL translation cache size (0 = off) */ uint8_t nand_cell_type; /* bbssd NAND cell type: 0=off(flat), 1 SLC..4 QLC */ + + /* + * Read-energy coefficients for the QLC page-class accounting, in + * milli-pJ/bit (34000 = 34.0 pJ/bit). Integers because QEMU device + * properties carry no floating point. Defaults are the cited profile: + * MCFlash energy density 0.709 pJ/bit/us x PACA measured tR. + * The emitted CSV records the coefficients actually used, so the + * energy columns stay auditable and reproducible offline. + */ + uint32_t e_read_mpj[4]; + /* + * The array half of the read coefficient: base sensing plus (n-1) extra + * senses for a class that needs n of them. The rest of e_read_mpj is + * peripheral, which scales with the class read latency. Splitting the two + * is what shows that the peripheral term dominates -- reporting only the + * total leaves that as an offline assertion instead of a measurement. + */ + uint32_t e_array_mpj[4]; + uint32_t e_xfer_mpj; /* channel transfer, milli-pJ/bit */ + uint32_t stats_flush_ms; /* periodic stats snapshot; 0 = on exit only */ + QEMUTimer *stats_timer; uint32_t nand_bad_blocks; /* bbssd factory bad blocks reported via SMART; 0 = none */ uint32_t op_pcent; /* bbssd over-provisioning percent (0 = use devsz_mb) */ bool debug_ftl; /* check bbssd FTL invariants on the GC path */ From 462bd969c45e979247e36366c549ac28f8a3260c Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 11 Sep 2026 06:50:31 +0000 Subject: [PATCH 06/17] femu/nand: test the QLC pairing table against the device source The table is what every physical-layout prediction is checked against, and a mismatch is silent: the mapper still emits a plan, the device still serves the reads, and only the latency is wrong. Reading it back out of a boot log costs a boot; this costs a compile. init_qlc_page_pairing() is static and its translation unit pulls in QEMU, so the function text is extracted from nand.c on every build rather than copied into the test, where the two could drift apart while still both passing. Checked both ways: the current source gives 132/128/126/126 and passes, and restoring the rows - 3 bound makes it fail at page 496. Co-Authored-By: Claude Opus 5 --- hw/femu/nand/test/test_pairing.c | 83 ++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 hw/femu/nand/test/test_pairing.c diff --git a/hw/femu/nand/test/test_pairing.c b/hw/femu/nand/test/test_pairing.c new file mode 100644 index 00000000000..ab499bd64df --- /dev/null +++ b/hw/femu/nand/test/test_pairing.c @@ -0,0 +1,83 @@ +/* Host unit test for the QLC page-pairing table. + * + * The expert plane-major layout assigns bit-plane Bn to QLC class n-1, so every + * predicted address depends on qlc_tbl matching the device exactly. A mismatch + * is silent: the mapper still emits a plan, the device still serves the reads, + * and only the latency is wrong. Checking the table before booting is far + * cheaper than reading it back out of a FEMU WRITE log. + * + * init_qlc_page_pairing() is static and its translation unit pulls in QEMU, so + * the function text is extracted from nand.c at build time (see the Makefile + * rule) rather than copied here, which would let the two drift apart. + */ +#include +#include +#include +#include + +typedef struct FemuCtrl FemuCtrl; +#include "../nand.h" + +int slc_tbl[MAX_SUPPORTED_PAGES_PER_BLOCK]; +int mlc_tbl[MAX_SUPPORTED_PAGES_PER_BLOCK]; +int tlc_tbl[MAX_SUPPORTED_PAGES_PER_BLOCK]; +int qlc_tbl[MAX_SUPPORTED_PAGES_PER_BLOCK]; +struct NandFlashTiming nand_flash_timing; + +/* Upstream's own style: size_t/int comparisons and the unused FemuCtrl argument. + * Our test code stays under -Wall -Wextra -Werror; the extracted text does not. */ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wsign-compare" +#pragma GCC diagnostic ignored "-Wunused-parameter" +#include "pairing_extract.inc" +#pragma GCC diagnostic pop + +static int fails; + +static void check(int cond, const char *what) +{ + if (!cond) { printf("FAIL %s\n", what); fails++; } +} + +int main(void) +{ + int pg, counts[4] = {0}; + + memset(qlc_tbl, -1, sizeof(qlc_tbl)); + init_qlc_page_pairing(NULL); + + /* Prologue: the shadow-programming sequence leaves pages 0..7 special. */ + for (pg = 0; pg < 6; pg++) + check(qlc_tbl[pg] == QLC_LOWER_PAGE, "prologue pg 0..5 is class 0"); + for (pg = 6; pg < 8; pg++) + check(qlc_tbl[pg] == QLC_LOWER_CENTER_PAGE, "prologue pg 6..7 is class 1"); + + /* From page 8 the cycle is 0 0 1 1 2 2 3 3, to the last page of the block. */ + for (pg = 8; pg < MAX_SUPPORTED_PAGES_PER_BLOCK; pg++) { + int want = ((pg - 8) % 8) / 2; + if (qlc_tbl[pg] != want) { + printf("FAIL pg %d: class %d, expected %d\n", pg, qlc_tbl[pg], want); + fails++; + break; + } + } + + /* No page may keep the -1 poison: rows-3 used to leave 496..511 untouched, + * where the zero-initialised global reads as a valid QLC_LOWER_PAGE. */ + for (pg = 0; pg < MAX_SUPPORTED_PAGES_PER_BLOCK; pg++) { + if (qlc_tbl[pg] < 0 || qlc_tbl[pg] > 3) { + printf("FAIL pg %d never assigned (%d)\n", pg, qlc_tbl[pg]); + fails++; + break; + } + counts[qlc_tbl[pg]]++; + } + + printf("class page counts: %d %d %d %d\n", + counts[0], counts[1], counts[2], counts[3]); + check(counts[0] == 132 && counts[1] == 128 && + counts[2] == 126 && counts[3] == 126, "class counts for 512 pages"); + + printf(fails ? "test_pairing: %d failure(s)\n" : "test_pairing: ok\n", fails); + return fails != 0; +} From 860901b8a9c0cf50e42a2b101444b4a55416150a Mon Sep 17 00:00:00 2001 From: Kiwoong Kim Date: Fri, 11 Sep 2026 06:52:47 +0000 Subject: [PATCH 07/17] femu/nand: write down how to run the host tests Both tests build outside QEMU, which is the point of them, but the recipes are not obvious: test_channel needs a stub osdep.h because nand-media.c includes one it will not get here, and test_pairing needs the pairing function re-extracted from nand.c on every build so a stale include cannot quietly test nothing. Neither is discoverable from the sources alone. Co-Authored-By: Claude Opus 5 --- hw/femu/nand/test/README.md | 82 +++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 hw/femu/nand/test/README.md diff --git a/hw/femu/nand/test/README.md b/hw/femu/nand/test/README.md new file mode 100644 index 00000000000..254f23f2655 --- /dev/null +++ b/hw/femu/nand/test/README.md @@ -0,0 +1,82 @@ +# NAND host tests + +Both tests build and run outside QEMU — no guest, no KVM, no root. Run them from +the FEMU checkout root. + +## test_channel — the media timing model + +`nand_media_op()` is pure timing arithmetic over a caller-supplied timeline. + +```bash +mkdir -p /tmp/nandtest/qemu +printf '#include \n#include \n#include \n#include \n' \ + > /tmp/nandtest/qemu/osdep.h +gcc -c -I/tmp/nandtest -Ihw/femu/nand -o /tmp/nandtest/nand-media.o hw/femu/nand/nand-media.c +gcc -I/tmp/nandtest -Ihw/femu/nand -o /tmp/nandtest/t \ + hw/femu/nand/test/test_channel.c /tmp/nandtest/nand-media.o -lm +/tmp/nandtest/t +``` + +## test_pairing — the QLC page-class table + +`init_qlc_page_pairing()` is static and its translation unit pulls in QEMU, so the +function text is extracted from `nand.c` instead of being copied into the test. +Re-extract on every build; a stale `.inc` would test nothing. + +```bash +mkdir -p /tmp/nandtest +sed -n '/^static void init_qlc_page_pairing/,/^}/p' hw/femu/nand/nand.c \ + > /tmp/nandtest/pairing_extract.inc +gcc -std=c11 -Wall -Wextra -Werror -I/tmp/nandtest -o /tmp/nandtest/tp \ + hw/femu/nand/test/test_pairing.c +/tmp/nandtest/tp +``` + +Expected output is `class page counts: 132 128 126 126` and `test_pairing: ok`. + +The table is what every physical-layout experiment predicts against, and a +mismatch is silent — the mapper still emits a plan and the device still serves the +reads, only the latency is wrong. Upstream's loop bound was `rows - 3`, which +stops at page index 495 and leaves 496..511 at the zero-initialised value, a +valid-looking `QLC_LOWER_PAGE`. That is invisible at `pgs_per_blk <= 496` — which +is why the earlier 256-page runs were unaffected — and wrong at the 512 pages the +QLC-aligned expert layout requires. Against the unpatched source this test reports +`FAIL pg 496` and counts `128 124 122 122`. + +## What the timing tests establish + +**bbssd never enables the channel stage.** `ftl-media.c` copies `pg_xfer_lat` into +`cfg.timing.page_xfer_ns` and then sets `cfg.policy.channel_mode = NAND_CH_OFF` +unconditionally. `nand-media.c` reads `page_xfer_ns` only under `NAND_CH_STAGED`, +so passing `pg_xfer_lat=...` on the command line today changes nothing. +`NAND_CH_STAGED` is dead code: both call sites (`bbssd/ftl-media.c`, +`zns/zftl.c`) set `NAND_CH_OFF`. + +**Turning it on is not behaviour-preserving, even with a zero bus.** With every +bus phase at 0, `NAND_CH_STAGED` still differs from `NAND_CH_OFF`, because the +channel timeline is advanced to each op's data-out time and the next op's +command phase is clamped to it. Any `channel_model` option must therefore default +to `off`. + +**The staged model serialises the channel across LUNs.** Reservations are made in +op-submission order, so an op's command phase waits for the *previous* op's +data-out even when the two are on different LUNs. Measured with two LUNs on one +channel, both reads issued at t=0, slow page (228.1 us array): + +| bus transfer | LUN0 done | LUN1 done | if sensing overlapped | +|---|---:|---:|---:| +| 0 us | 228.1 us | 456.2 us | 228.1 us | +| 52.4 us | 280.5 us | 561.1 us | 280.5 us | + +Real NAND issues LUN1's command while LUN0 senses; the bus is needed only for the +command and the data burst. This model holds the channel from command through +data-out, so **adding LUNs per channel buys nothing** and per-op cost is always +`array + transfer`. It describes a controller that does not pipeline. + +The consequence for page-mapping studies: the gain is `(mean_array + xfer) / +(aware_array + xfer)` at every LUN count — 1.219x for the 2-tier unified-LRU +traffic mix at 1.8 GB — rather than falling toward 1.0 as the channel saturates. +That flat 1.219x is a property of the model, not of the device. Anything claiming +a LUN-count dependence needs the reservation order fixed first (event-driven +issue, or a separate command-phase timeline), with these tests extended to cover +it. From 14566d4c31ee2d281cd16564631b24de23d64968 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Fri, 11 Sep 2026 16:54:22 +0900 Subject: [PATCH 08/17] femu: commit the container harness it has been run from compose.yaml, the Dockerfile and its entrypoint have driven every measurement on this fork but were never tracked, so the geometry, the QLC stats path and the energy coefficients a run was given lived only in an untracked file. A result is only reproducible if the configuration that produced it is in the history beside the code. Co-Authored-By: Claude Opus 5 --- .dockerignore | 13 ++++ compose.yaml | 75 ++++++++++++++++++ docker/Dockerfile | 72 +++++++++++++++++ docker/femu-run | 193 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 353 insertions(+) create mode 100644 .dockerignore create mode 100644 compose.yaml create mode 100644 docker/Dockerfile create mode 100755 docker/femu-run diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 00000000000..1f4bd3bdec1 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,13 @@ +.git +.github +build +build-femu +build-docker +roms +*.qcow2 +*.img +*.iso +*.log +docker-data +images +guest diff --git a/compose.yaml b/compose.yaml new file mode 100644 index 00000000000..db11b035b2f --- /dev/null +++ b/compose.yaml @@ -0,0 +1,75 @@ +services: + femu: + build: + context: . + dockerfile: docker/Dockerfile + args: + BUILD_JOBS: ${FEMU_BUILD_JOBS:-8} + image: ${FEMU_DOCKER_IMAGE:-femu-qlc:latest} + container_name: ${FEMU_CONTAINER_NAME:-femu} + devices: + - /dev/kvm:/dev/kvm + cap_add: + - IPC_LOCK + ulimits: + memlock: + soft: -1 + hard: -1 + volumes: + - ${FEMU_GUEST_DIR:-../images}:/guest + - ${FEMU_DATA_DIR:-./docker-data}:/data + ports: + - "${FEMU_SSH_PORT:-2222}:2222" + environment: + FEMU_MODE: ${FEMU_MODE:-bbssd} + FEMU_IMAGE: /guest/${FEMU_IMAGE_NAME:-u20s.qcow2} + FEMU_KERNEL: ${FEMU_KERNEL:-} + FEMU_INITRD: ${FEMU_INITRD:-} + FEMU_KERNEL_APPEND: ${FEMU_KERNEL_APPEND:-root=LABEL=rootfs console=ttyS0} + FEMU_MEMORY: ${FEMU_MEMORY:-6G} + FEMU_CPUS: ${FEMU_CPUS:-6} + FEMU_GUEST_SSH_PORT: 2222 + FEMU_NAND_CELL_TYPE: ${FEMU_NAND_CELL_TYPE:-4} + # Counter-validated pilot geometry. 7,168 MiB is large enough to hold the + # complete 5,760-object routed-expert catalog (about 6.23 GB). + FEMU_SSD_SIZE_MB: ${FEMU_SSD_SIZE_MB:-7168} + FEMU_SECTORS_PER_PAGE: ${FEMU_SECTORS_PER_PAGE:-32} + FEMU_PAGES_PER_BLOCK: ${FEMU_PAGES_PER_BLOCK:-256} + FEMU_BLOCKS_PER_PLANE: ${FEMU_BLOCKS_PER_PLANE:-48} + FEMU_PLANES_PER_LUN: ${FEMU_PLANES_PER_LUN:-1} + FEMU_LUNS_PER_CHANNEL: ${FEMU_LUNS_PER_CHANNEL:-8} + FEMU_CHANNELS: ${FEMU_CHANNELS:-8} + FEMU_QLC_STATS_PATH: ${FEMU_QLC_STATS_PATH:-/data/qlc_counts.csv} + # 에너지 계수 (milli-pJ/bit). peripheral + array 분해 모델: + # peripheral = P_fix + rho_p * t_R(c), P_fix=1.273 rho_p=0.668 + # array = E_fix + (n(c)-1) * E_sense, E_fix=0.551 E_sense=0.494 + # n(c) = 1/2/4/8 은 class 별 sensing 횟수 추정값 (인용값 아님) + FEMU_E_READ_C0: ${FEMU_E_READ_C0:-33821} + FEMU_E_READ_C1: ${FEMU_E_READ_C1:-53220} + FEMU_E_READ_C2: ${FEMU_E_READ_C2:-93219} + FEMU_E_READ_C3: ${FEMU_E_READ_C3:-157653} + FEMU_E_ARRAY_C0: ${FEMU_E_ARRAY_C0:-551} + FEMU_E_ARRAY_C1: ${FEMU_E_ARRAY_C1:-1045} + FEMU_E_ARRAY_C2: ${FEMU_E_ARRAY_C2:-2033} + FEMU_E_ARRAY_C3: ${FEMU_E_ARRAY_C3:-4009} + FEMU_E_XFER: ${FEMU_E_XFER:-28100} + FEMU_STATS_FLUSH_MS: ${FEMU_STATS_FLUSH_MS:-0} + FEMU_IMAGE_FORMAT: ${FEMU_IMAGE_FORMAT:-qcow2} + FEMU_EXTRA_DEVICE_OPTS: ${FEMU_EXTRA_DEVICE_OPTS:-} + # Extra guest disks, ';'-separated QEMU -drive specs. A replay payload + # is far too large for a cloud-init seed and the guest has no network, + # so it comes in as a read-only disk and is copied in from inside. + FEMU_EXTRA_DRIVES: ${FEMU_EXTRA_DRIVES:-} + # FEMU_EXP_LOG switches the [EXP] log on; FEMU_SECRET is the marker + # string it looks for. Together they make FEMU report lpn -> PPA for every + # page whose content carries the marker, which is the only way to read the + # real address mapping instead of inferring it from read latency. + FEMU_EXP_LOG: ${FEMU_EXP_LOG:-} + FEMU_SECRET: ${FEMU_SECRET:-} + FEMU_DUMP_LPN: ${FEMU_DUMP_LPN:-} + FEMU_QMP_SOCKET: /data/qmp.sock + FEMU_ALLOW_UNPINNED: ${FEMU_ALLOW_UNPINNED:-} + command: ["${FEMU_MODE:-bbssd}"] + stdin_open: true + tty: true + stop_grace_period: 30s diff --git a/docker/Dockerfile b/docker/Dockerfile new file mode 100644 index 00000000000..1e72080406c --- /dev/null +++ b/docker/Dockerfile @@ -0,0 +1,72 @@ +FROM ubuntu:24.04 AS builder + +ARG DEBIAN_FRONTEND=noninteractive +ARG BUILD_JOBS=8 + +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + bash \ + build-essential \ + ca-certificates \ + git \ + libaio-dev \ + libdw-dev \ + libfdt-dev \ + libglib2.0-dev \ + libnuma-dev \ + libpixman-1-dev \ + libslirp-dev \ + ninja-build \ + pkg-config \ + python3 \ + python3-venv \ + zlib1g-dev \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /src/femu +COPY . . + +# FEMU carries a compatible Meson wheel in python/wheels. QEMU may still fetch +# pinned source subprojects that are not populated in the source checkout. +RUN mkdir -p build-docker \ + && cd build-docker \ + && ../configure \ + --enable-kvm \ + --enable-slirp \ + --target-list=x86_64-softmmu \ + --disable-docs \ + --disable-gtk \ + --disable-sdl \ + --disable-werror \ + --enable-strip \ + --prefix=/opt/femu \ + && ninja -j "${BUILD_JOBS}" \ + && ninja install \ + && /opt/femu/bin/qemu-system-x86_64 --version + +FROM ubuntu:24.04 AS runtime + +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + bash \ + libaio1t64 \ + libdw1t64 \ + libfdt1 \ + libglib2.0-0t64 \ + libnuma1 \ + libpixman-1-0 \ + libslirp0 \ + zlib1g \ + && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /opt/femu /opt/femu + +COPY docker/femu-run /usr/local/bin/femu-run +RUN chmod 0755 /usr/local/bin/femu-run \ + && mkdir -p /images /data + +ENV PATH="/opt/femu/bin:${PATH}" +ENTRYPOINT ["/usr/local/bin/femu-run"] +CMD ["help"] diff --git a/docker/femu-run b/docker/femu-run new file mode 100755 index 00000000000..0085130e62c --- /dev/null +++ b/docker/femu-run @@ -0,0 +1,193 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat <<'EOF' +Usage: femu-run MODE [additional QEMU options] + +MODE: + bbssd Run a BlackBox SSD (default cell type: QLC) + zns Run a Zoned Namespace SSD (default flash type: QLC) + shell Open a shell in the container + version Print the FEMU/QEMU version + help Show this message + +Required runtime input: + FEMU_IMAGE=/guest/u20s.qcow2 + +Common environment variables: + FEMU_KERNEL=/guest/kernel Optional external guest kernel + FEMU_INITRD=/guest/initrd Optional initrd used with FEMU_KERNEL + FEMU_KERNEL_APPEND=... Kernel command line + FEMU_MEMORY=4G Guest RAM + FEMU_CPUS=4 Guest vCPUs + FEMU_NAND_CELL_TYPE=4 1=SLC, 2=MLC, 3=TLC, 4=QLC + FEMU_SSD_SIZE_MB=16384 Emulated SSD capacity + FEMU_QLC_STATS_PATH=... Write page-class read counters as CSV on shutdown + FEMU_E_READ_C0..C3=... Read energy per page class, milli-pJ/bit + (default 33821/53220/93219/157653) + FEMU_E_ARRAY_C0..C3=... Array share of that coefficient; the rest is + peripheral, which scales with read latency + FEMU_E_XFER=28100 Channel transfer energy, milli-pJ/bit + FEMU_STATS_FLUSH_MS=0 Rewrite the CSV every N ms while running (0 = on exit only) + FEMU_GUEST_SSH_PORT=2222 Port inside the container forwarded to guest :22 + FEMU_EXTRA_DEVICE_OPTS=... Extra comma-separated femu device properties + FEMU_EXTRA_DRIVES=... Extra guest disks, one QEMU -drive spec each, + separated by ';'. The payload image a replay + experiment loads onto the emulated SSD is far too + large for a cloud-init seed, so it is attached as a + read-only disk and copied in from inside the guest. + FEMU_ALLOW_TCG=1 Permit slow TCG fallback when /dev/kvm is absent +EOF +} + +mode="${1:-${FEMU_MODE:-bbssd}}" +if [[ $# -gt 0 ]]; then + shift +fi + +case "${mode}" in + help|-h|--help) + usage + exit 0 + ;; + version) + exec qemu-system-x86_64 --version + ;; + shell|bash) + exec /bin/bash "$@" + ;; + bbssd|zns) + ;; + *) + echo "femu-run: unsupported mode '${mode}'" >&2 + usage >&2 + exit 2 + ;; +esac + +femu_image="${FEMU_IMAGE:-/guest/u20s.qcow2}" +if [[ ! -f "${femu_image}" ]]; then + echo "femu-run: guest image not found: ${femu_image}" >&2 + echo "Mount the host guest directory at /guest or set FEMU_IMAGE." >&2 + exit 1 +fi + +boot_args=() +femu_kernel="${FEMU_KERNEL:-}" +femu_initrd="${FEMU_INITRD:-}" +if [[ -n "${femu_kernel}" ]]; then + if [[ ! -f "${femu_kernel}" ]]; then + echo "femu-run: guest kernel not found: ${femu_kernel}" >&2 + exit 1 + fi + boot_args+=(-kernel "${femu_kernel}") + if [[ -n "${femu_initrd}" ]]; then + if [[ ! -f "${femu_initrd}" ]]; then + echo "femu-run: guest initrd not found: ${femu_initrd}" >&2 + exit 1 + fi + boot_args+=(-initrd "${femu_initrd}") + fi + boot_args+=(-append "${FEMU_KERNEL_APPEND:-root=LABEL=rootfs console=ttyS0}") +fi + +accel_args=() +if [[ -c /dev/kvm && -r /dev/kvm && -w /dev/kvm ]]; then + accel_args=(-enable-kvm -cpu host) +elif [[ "${FEMU_ALLOW_TCG:-0}" == "1" ]]; then + echo "femu-run: warning: /dev/kvm unavailable; using slow TCG emulation" >&2 + accel_args=(-accel tcg -cpu max) +else + echo "femu-run: /dev/kvm is unavailable or not accessible" >&2 + echo "Start the container with: --device /dev/kvm" >&2 + echo "Set FEMU_ALLOW_TCG=1 only if slow software emulation is acceptable." >&2 + exit 1 +fi + +cell_type="${FEMU_NAND_CELL_TYPE:-4}" +if [[ ! "${cell_type}" =~ ^[1-4]$ ]]; then + echo "femu-run: FEMU_NAND_CELL_TYPE must be 1, 2, 3, or 4" >&2 + exit 2 +fi + +ssd_size_mb="${FEMU_SSD_SIZE_MB:-16384}" +extra_device_opts="${FEMU_EXTRA_DEVICE_OPTS:-}" + +if [[ "${mode}" == "bbssd" ]]; then + device="femu,devsz_mb=${ssd_size_mb},namespaces=1,femu_mode=1" + device+=",secsz=${FEMU_SECTOR_SIZE:-512}" + device+=",secs_per_pg=${FEMU_SECTORS_PER_PAGE:-8}" + device+=",pgs_per_blk=${FEMU_PAGES_PER_BLOCK:-256}" + device+=",blks_per_pl=${FEMU_BLOCKS_PER_PLANE:-256}" + device+=",pls_per_lun=${FEMU_PLANES_PER_LUN:-1}" + device+=",luns_per_ch=${FEMU_LUNS_PER_CHANNEL:-8}" + device+=",nchs=${FEMU_CHANNELS:-8}" + device+=",gc_thres_pcent=${FEMU_GC_THRESHOLD:-75}" + device+=",gc_thres_pcent_high=${FEMU_GC_THRESHOLD_HIGH:-95}" + device+=",nand_cell_type=${cell_type}" + device+=",e_read_c0_mpj=${FEMU_E_READ_C0:-34000}" + device+=",e_read_c1_mpj=${FEMU_E_READ_C1:-54000}" + device+=",e_read_c2_mpj=${FEMU_E_READ_C2:-95400}" + device+=",e_read_c3_mpj=${FEMU_E_READ_C3:-161700}" + device+=",e_array_c0_mpj=${FEMU_E_ARRAY_C0:-551}" + device+=",e_array_c1_mpj=${FEMU_E_ARRAY_C1:-1045}" + device+=",e_array_c2_mpj=${FEMU_E_ARRAY_C2:-2033}" + device+=",e_array_c3_mpj=${FEMU_E_ARRAY_C3:-4009}" + device+=",e_xfer_mpj=${FEMU_E_XFER:-28100}" + device+=",stats_flush_ms=${FEMU_STATS_FLUSH_MS:-0}" +else + # The supplied FEMU ZNS model supports SLC, TLC, and QLC (not MLC). + if [[ "${cell_type}" == "2" ]]; then + echo "femu-run: the ZNS model does not support MLC (cell type 2)" >&2 + exit 2 + fi + device="femu,devsz_mb=${ssd_size_mb},namespaces=1,femu_mode=3" + device+=",zns_num_ch=${FEMU_ZNS_CHANNELS:-8}" + device+=",zns_num_lun=${FEMU_ZNS_LUNS_PER_CHANNEL:-4}" + device+=",zns_num_plane=${FEMU_ZNS_PLANES_PER_LUN:-2}" + device+=",zns_num_blk=${FEMU_ZNS_BLOCKS_PER_PLANE:-32}" + device+=",zns_flash_type=${cell_type}" +fi + +if [[ -n "${extra_device_opts}" ]]; then + device+=",${extra_device_opts#,}" +fi + +memory="${FEMU_MEMORY:-4G}" +cpus="${FEMU_CPUS:-4}" +ssh_port="${FEMU_GUEST_SSH_PORT:-2222}" +qmp_socket="${FEMU_QMP_SOCKET:-/data/qmp.sock}" +image_format="${FEMU_IMAGE_FORMAT:-qcow2}" + +# Extra disks, if any. Split on ';' rather than whitespace: a -drive spec is a +# comma-separated list that may itself contain paths with spaces. +extra_drive_args=() +if [[ -n "${FEMU_EXTRA_DRIVES:-}" ]]; then + # `read` returns non-zero on a final line with no newline, which would end + # the loop before that last spec is used, so terminate the stream explicitly. + while IFS= read -r spec; do + [[ -z "${spec}" ]] && continue + extra_drive_args+=(-drive "${spec}") + done < <(printf '%s\n' "${FEMU_EXTRA_DRIVES}" | tr ';' '\n') +fi + +echo "FEMU mode=${mode}, NAND cell type=${cell_type}, image=${femu_image}" +echo "Guest SSH is forwarded to container port ${ssh_port}" + +exec qemu-system-x86_64 \ + -name "FEMU-${mode^^}-VM" \ + "${accel_args[@]}" \ + -smp "${cpus}" \ + -m "${memory}" \ + "${boot_args[@]}" \ + -device virtio-scsi-pci,id=scsi0 \ + -device scsi-hd,drive=hd0 \ + -drive "file=${femu_image},if=none,aio=native,cache=none,format=${image_format},id=hd0" \ + -device "${device}" \ + "${extra_drive_args[@]}" \ + -netdev "user,id=net0,hostfwd=tcp::${ssh_port}-:22" \ + -device virtio-net-pci,netdev=net0 \ + -nographic \ + -qmp "unix:${qmp_socket},server=on,wait=off" \ + "$@" From dbbee8d9930403a77fff4c3df776290c3077d741 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 15:51:54 +0900 Subject: [PATCH 09/17] femu/docker: default to the geometry the measurements were taken on compose.yaml carried the pilot geometry -- 7,168 MiB over 8 channels x 8 LUNs at 256 pages per block -- while every measurement in this repository was taken on 64 GiB over 2 channels x 4 LUNs at 512 pages per block with op_pcent=7. The harness passes those explicitly, so its runs were right, but a bare `docker compose up` emulated a different device and said nothing about it. The page count is the part that bites. Upstream's QLC pairing table only assigns classes for pages 0..495, so at 256 pages per block every class is correct and the bug this fork fixes (nand.c, rows - 1) cannot appear. Someone reproducing at the old default would run this source and see physics it does not model, with no error anywhere. Defaults only. The run harness exports these before calling compose, so nothing about the runs already recorded changes; what changes is what happens when a reader sets nothing, which now reproduces the measured device instead of quietly substituting another one. Verified by starting a container with no overrides: SSD_SIZE_MB=65536 CH=2 LUN=4 PPB=512 BPP=1024 CELL=4 OPTS=op_pcent=7 MEM=8G. Co-Authored-By: Claude Opus 5 --- compose.yaml | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/compose.yaml b/compose.yaml index db11b035b2f..4fb1218c1f7 100644 --- a/compose.yaml +++ b/compose.yaml @@ -26,19 +26,30 @@ services: FEMU_KERNEL: ${FEMU_KERNEL:-} FEMU_INITRD: ${FEMU_INITRD:-} FEMU_KERNEL_APPEND: ${FEMU_KERNEL_APPEND:-root=LABEL=rootfs console=ttyS0} - FEMU_MEMORY: ${FEMU_MEMORY:-6G} + FEMU_MEMORY: ${FEMU_MEMORY:-8G} FEMU_CPUS: ${FEMU_CPUS:-6} FEMU_GUEST_SSH_PORT: 2222 FEMU_NAND_CELL_TYPE: ${FEMU_NAND_CELL_TYPE:-4} - # Counter-validated pilot geometry. 7,168 MiB is large enough to hold the - # complete 5,760-object routed-expert catalog (about 6.23 GB). - FEMU_SSD_SIZE_MB: ${FEMU_SSD_SIZE_MB:-7168} + # The QLC-aligned expert placement geometry: 64 GiB over 2 channels x 4 + # LUNs, 512 pages per block. These are defaults, not a suggestion -- they + # are the device the measurements in this repository were taken on, and + # a run that leaves them alone reproduces it. + # + # 512 pages per block is the part that matters most. The QLC pairing table + # only covers pages 0..495 upstream, so at the old 256-page default every + # page class is correct and the bug this fork fixes (nand.c, rows - 1) + # cannot appear. Same source, different physics, silently. + # + # op_pcent=7 is over-provisioning: it sets the exposed namespace to 59.8 G + # of the 64 GiB raw, which is the capacity every layout here is planned + # against. + FEMU_SSD_SIZE_MB: ${FEMU_SSD_SIZE_MB:-65536} FEMU_SECTORS_PER_PAGE: ${FEMU_SECTORS_PER_PAGE:-32} - FEMU_PAGES_PER_BLOCK: ${FEMU_PAGES_PER_BLOCK:-256} - FEMU_BLOCKS_PER_PLANE: ${FEMU_BLOCKS_PER_PLANE:-48} + FEMU_PAGES_PER_BLOCK: ${FEMU_PAGES_PER_BLOCK:-512} + FEMU_BLOCKS_PER_PLANE: ${FEMU_BLOCKS_PER_PLANE:-1024} FEMU_PLANES_PER_LUN: ${FEMU_PLANES_PER_LUN:-1} - FEMU_LUNS_PER_CHANNEL: ${FEMU_LUNS_PER_CHANNEL:-8} - FEMU_CHANNELS: ${FEMU_CHANNELS:-8} + FEMU_LUNS_PER_CHANNEL: ${FEMU_LUNS_PER_CHANNEL:-4} + FEMU_CHANNELS: ${FEMU_CHANNELS:-2} FEMU_QLC_STATS_PATH: ${FEMU_QLC_STATS_PATH:-/data/qlc_counts.csv} # 에너지 계수 (milli-pJ/bit). peripheral + array 분해 모델: # peripheral = P_fix + rho_p * t_R(c), P_fix=1.273 rho_p=0.668 @@ -55,7 +66,7 @@ services: FEMU_E_XFER: ${FEMU_E_XFER:-28100} FEMU_STATS_FLUSH_MS: ${FEMU_STATS_FLUSH_MS:-0} FEMU_IMAGE_FORMAT: ${FEMU_IMAGE_FORMAT:-qcow2} - FEMU_EXTRA_DEVICE_OPTS: ${FEMU_EXTRA_DEVICE_OPTS:-} + FEMU_EXTRA_DEVICE_OPTS: ${FEMU_EXTRA_DEVICE_OPTS:-op_pcent=7} # Extra guest disks, ';'-separated QEMU -drive specs. A replay payload # is far too large for a cloud-init seed and the guest has no network, # so it comes in as a read-only disk and is copied in from inside. From eaa0aec5bd3d8f35373bcd6f5c3a8ae46d46aaf2 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 16:04:03 +0900 Subject: [PATCH 10/17] moe-harness: commit what actually runs the measurement The emulator was here; what drives it was not. Every result in this fork came out of a set of scripts that lived only on one host's filesystem, untracked, so a clone could build the device but had no way to fill it, prove the placement landed, or replay anything against it. Only the FEMU-host half is committed. The GPU host's analysis, plotting and quantiser tooling stays there: it needs the model weights and a GPU, and the two halves being one rsync'd directory is what let an edit to run_policy.sh get reverted twice mid-sweep. Splitting them by the machine they run on is the point, not a side effect. The layout is mirrored rather than flattened because the scripts resolve each other by relative path -- run_device.sh takes its root three levels up, and femu_compose.sh one level up from itself. run_device.sh gains "$ROOT/.." as a FEMU checkout candidate, which is this arrangement: the harness inside the emulator's tree, where before the emulator sat under the harness's _deps. The blanket *.md in .gitignore, which arrived with the QEMU 10.1.0 upgrade, would have dropped the README explaining the procedure. Excepted rather than force-added, so the next document does not vanish the same way. Not committed because it is data, not code: the payload (about 15 GB), the collected traces, the layouts built from them, and a guest image. The README says so and says what shape they take. Co-Authored-By: Claude Opus 5 --- .gitignore | 4 + moe-harness/README.md | 87 +++ moe-harness/exp/gating_nand/femu/make_seed.py | 96 +++ .../femu_handoff/packages/LAYER_TRACE_V1.md | 87 +++ .../packages/QLC_ALIGNED_LAYOUT_V1.md | 124 ++++ .../femu_handoff/packages/REPLAYER_V1.md | 134 +++++ .../moe_bcq/femu_handoff/packages/bundle.py | 208 +++++++ .../packages/layer_read_groups.py | 246 ++++++++ .../femu_handoff/packages/logical_reads.py | 100 ++++ .../packages/qlc_aligned_mapper.py | 549 ++++++++++++++++++ .../moe_bcq/femu_handoff/packages/replay_v1.c | 467 +++++++++++++++ .../femu_handoff/packages/trace_compiler.py | 230 ++++++++ .../exp/moe_bcq/femu_run/audit_handoff.py | 110 ++++ .../exp/moe_bcq/femu_run/class_confusion.c | 85 +++ .../exp/moe_bcq/femu_run/drive_multi.sh | 127 ++++ moe-harness/exp/moe_bcq/femu_run/drive_run.sh | 120 ++++ .../exp/moe_bcq/femu_run/guest_bringup.sh | 19 + .../exp/moe_bcq/femu_run/guest_replay.sh | 91 +++ moe-harness/exp/moe_bcq/femu_run/mark_write.c | 58 ++ moe-harness/exp/moe_bcq/femu_run/preflight.sh | 45 ++ moe-harness/exp/moe_bcq/femu_run/probe_map.c | 104 ++++ .../exp/moe_bcq/femu_run/run_device.sh | 78 +++ .../exp/moe_bcq/femu_run/run_policy.sh | 65 +++ moe-harness/exp/moe_bcq/femu_run/run_sweep.sh | 32 + .../exp/moe_bcq/femu_run/verify_fill_256.py | 162 ++++++ moe-harness/runs/femu/run01.env | 17 + moe-harness/scripts/build_replay.sh | 19 + moe-harness/scripts/femu_compose.sh | 32 + 28 files changed, 3496 insertions(+) create mode 100644 moe-harness/README.md create mode 100644 moe-harness/exp/gating_nand/femu/make_seed.py create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/REPLAYER_V1.md create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/layer_read_groups.py create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/replay_v1.c create mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py create mode 100644 moe-harness/exp/moe_bcq/femu_run/audit_handoff.py create mode 100644 moe-harness/exp/moe_bcq/femu_run/class_confusion.c create mode 100755 moe-harness/exp/moe_bcq/femu_run/drive_multi.sh create mode 100755 moe-harness/exp/moe_bcq/femu_run/drive_run.sh create mode 100755 moe-harness/exp/moe_bcq/femu_run/guest_bringup.sh create mode 100755 moe-harness/exp/moe_bcq/femu_run/guest_replay.sh create mode 100644 moe-harness/exp/moe_bcq/femu_run/mark_write.c create mode 100755 moe-harness/exp/moe_bcq/femu_run/preflight.sh create mode 100644 moe-harness/exp/moe_bcq/femu_run/probe_map.c create mode 100755 moe-harness/exp/moe_bcq/femu_run/run_device.sh create mode 100755 moe-harness/exp/moe_bcq/femu_run/run_policy.sh create mode 100755 moe-harness/exp/moe_bcq/femu_run/run_sweep.sh create mode 100644 moe-harness/exp/moe_bcq/femu_run/verify_fill_256.py create mode 100644 moe-harness/runs/femu/run01.env create mode 100755 moe-harness/scripts/build_replay.sh create mode 100755 moe-harness/scripts/femu_compose.sh diff --git a/.gitignore b/.gitignore index 6c1e1f97161..9215d642714 100644 --- a/.gitignore +++ b/.gitignore @@ -133,4 +133,8 @@ trace-ust-all.c build-femu/ build/ *.md +# The blanket *.md above came in with the QEMU 10.1.0 upgrade and silently +# drops documentation. The harness's own docs are the part a reader needs +# most, so they are excepted rather than force-added one at a time. +!moe-harness/**/*.md subprojects/.wraplock diff --git a/moe-harness/README.md b/moe-harness/README.md new file mode 100644 index 00000000000..28dfee719d3 --- /dev/null +++ b/moe-harness/README.md @@ -0,0 +1,87 @@ +# QLC placement measurement harness + +What drives the experiment: it boots a FEMU device from this checkout, fills it +with an image whose byte layout decides which QLC page class every bit-plane +lands on, proves the placement landed with the device's own counters, and +replays a recorded MoE inference trace against it. + +Only the FEMU-host half is here. The GPU host's analysis and plotting +(`compose_e2e.py`, `make_figures.py`, `timed_replay.py`, and the quantiser +tooling) stays there, because it needs the model weights and a GPU. + +## Layout + + exp/moe_bcq/femu_run/ run the experiment + run_device.sh one device: boot, fill, then N replays + drive_multi.sh confusion check once, then a replay per trace + run_policy.sh older path: one device, one replay + drive_run.sh its driver + run_sweep.sh, preflight.sh sweep wrapper and its guard + guest_replay.sh runs inside the guest: fill and read-back + class_confusion.c reads one class back and counts what the + device says it read -- the placement check + probe_map.c, mark_write.c LPN -> PPA probes, for diagnosing a fill + verify_fill_256.py checks the queue satisfies the fill contract + audit_handoff.py re-derives a binary's totals from the JSONL + + exp/moe_bcq/femu_handoff/packages/ + replay_v1.c the guest replayer, QD=32 O_DIRECT AIO + qlc_aligned_mapper.py plan / validate / materialize an image + trace_compiler.py mapped JSONL -> replay_qd32.bin + layer_read_groups.py trace -> per-layer read groups under a cache + bundle.py, logical_reads.py payload access + + exp/gating_nand/femu/make_seed.py cloud-init seed carrying the binaries + scripts/femu_compose.sh compose wrapper + scripts/build_replay.sh builds the guest replayer + runs/femu/run01.env per-run environment template + +## What it needs that is not here + +The payload (`planes.bin`, `scales.bin`, about 15 GB for the two models), the +collected traces, and the layouts built from them. They are data, not code, and +are distributed separately. A guest image is also needed: an Ubuntu 22.04 cloud +image, with a per-run qcow2 overlay -- 20.04 will not do, because replay_v1 +needs a kernel new enough for the AIO path. + +## Running one device + + bash exp/moe_bcq/femu_run/run_device.sh DEVICE_TAG IMAGE_BASENAME IMAGE_PAGES SPECFILE + +`IMAGE_BASENAME` is an image under `runs/femu/images/`, `IMAGE_PAGES` its page +count from the layout summary, and `SPECFILE` a list of ` ` +lines, one replay each. All of them share the device, which is the point: the +placement is a property of the image, so every trace that shares the placement +has to be replayed on the same fill. Re-filling per trace would re-run +out-of-place allocation and land the pages on different physical classes. + +The device geometry comes from the compose defaults in the parent checkout +(64 GiB, 2 channels x 4 LUNs, 512 pages per block, `op_pcent=7`) and needs no +argument. `runs/femu/run01.env` restates them and adds the per-run pieces: the +QLC counter path, the payload disk, the container name. + +## Two things that will waste a day if you skip them + +**Fill in 256 KiB writes.** This queue's `max_segments` is 127, so a larger +O_DIRECT write splits at 508 KiB, which falls in the middle of the 32nd 16 KiB +flash page. That page then belongs to both fragments and is programmed twice; +out-of-place update spends an extra physical page, and every later page shifts +one slot along. `guest_replay.sh` checks the queue before filling and dies if +the contract does not hold. This is configuration-dependent, which is why the +PPA check below stays in the procedure. + +**A matching read-back hash does not mean the placement is right.** Three early +runs passed SHA-256 read-back and were still wrong: correct data says nothing +about which cell holds it. `drive_multi.sh` therefore reads every class back +across the whole image and asserts the counter diagonal before any replay. A +partial range would miss exactly the pages a drifted fill puts out of place. + +## Checking a result + +Each replay writes `groups.jsonl.gz` (per-group timing, the only record of the +prefill/decode split) and `replay.csv` (the QLC counter snapshot for that +replay) into `records//`. `replay_v1` resets the counters when it +starts and snapshots them when it ends, so consecutive replays on one device +report independently -- verified by comparing every run's per-class page counts +against the counts its own compiled binary implies, which agreed exactly across +24 runs. diff --git a/moe-harness/exp/gating_nand/femu/make_seed.py b/moe-harness/exp/gating_nand/femu/make_seed.py new file mode 100644 index 00000000000..39dd8073add --- /dev/null +++ b/moe-harness/exp/gating_nand/femu/make_seed.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""Build a cloud-init seed ISO that runs a program in the guest and exits. + +FEMU's own scripts expect a VM you SSH into over slirp. This host has no libslirp, +so FEMU is built without it and the guest has no network at all: nothing can be +installed and nothing can be copied in after boot. Everything the run needs -- the +binary, its shared library, the trace -- is embedded here, gzip+base64, and the +results come back over the serial console. + + python3 make_seed.py -o ~/images/seed.iso \ + --file /usr/local/bin/replay=../../../build/guest/replay:0755 \ + --file /usr/local/lib/liburing.so.2=$CONDA/lib/liburing.so.2.14 \ + --file /root/objects.csv= \ + --file /root/stream.csv= \ + --run "LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/replay /dev/nvme0n1 0 /root/objects.csv /root/stream.csv" +""" + +import argparse +import base64 +import gzip +import io +import sys +from pathlib import Path + +import pycdlib + +HEAD = """#cloud-config +password: femu +chpasswd: {{ expire: False }} +ssh_pwauth: true +users: + - name: femu + plain_text_passwd: femu + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash +{keys}write_files: +{files}runcmd: + - [ sh, -c, "echo '==={tag}-START===' > /dev/ttyS0" ] +{cmds} - [ sh, -c, "echo '==={tag}-DONE===' > /dev/ttyS0" ] +""" + + +def parse_args(): + p = argparse.ArgumentParser() + p.add_argument("-o", "--output", type=Path, required=True) + p.add_argument("--file", action="append", default=[], metavar="GUEST=HOST[:MODE]", + help="embed HOST at GUEST, default mode 0644") + p.add_argument("--run", action="append", default=[], + help="shell command; stdout and stderr go to the serial console") + p.add_argument("--ssh-key", type=Path, default=None, + help="public key authorised for the femu user. The serial " + "console is one-shot: it runs what the seed says and " + "nothing more. A key turns each further question into " + "an ssh command instead of another boot and refill.") + p.add_argument("--tag", default="RUN", help="marker wrapping the output") + p.add_argument("--instance-id", default="femu-01") + return p.parse_args() + + +def main() -> int: + args = parse_args() + blocks = [] + for spec in args.file: + guest, _, rest = spec.partition("=") + host, _, mode = rest.partition(":") + data = gzip.compress(Path(host).read_bytes()) + blocks.append( + f" - path: {guest}\n" + f" permissions: '{mode or '0644'}'\n" + f" encoding: gz+b64\n" + f" content: {base64.b64encode(data).decode()}\n") + cmds = "".join(f' - [ sh, -c, "{c} > /dev/ttyS0 2>&1" ]\n' for c in args.run) + keys = "" + if args.ssh_key: + keys = " ssh_authorized_keys:\n - {}\n".format( + args.ssh_key.read_text().strip()) + user = HEAD.format(files="".join(blocks), cmds=cmds, tag=args.tag, + keys=keys).encode() + meta = f"instance-id: {args.instance_id}\nlocal-hostname: femu\n".encode() + + iso = pycdlib.PyCdlib() + iso.new(interchange_level=3, joliet=3, vol_ident="cidata", rock_ridge="1.09") + for data, path, rr, jol in ((user, "/USERDATA.;1", "user-data", "/user-data"), + (meta, "/METADATA.;1", "meta-data", "/meta-data")): + iso.add_fp(io.BytesIO(data), len(data), path, rr_name=rr, joliet_path=jol) + args.output.parent.mkdir(parents=True, exist_ok=True) + iso.write(str(args.output)) + iso.close() + print(f"{args.output} ({args.output.stat().st_size:,} bytes, " + f"{len(args.file)} files, {len(args.run)} commands)") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md b/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md new file mode 100644 index 00000000000..992e565dccc --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md @@ -0,0 +1,87 @@ +# 레이어별 읽기 그룹 v1 — 2026-09-09 + +이번 기본 입력은 `traces/prefill_w4_decode_mixed_v1/`이다. 기존 `traces/smoke/`는 DRAM 적용 전의 별도 수집이며 그대로 보존했다. 모델 payload는 기존 C(일반 4-bit BCQ + 공유 FP16 scale, MRE 없음)를 사용한다. + +**2026-09-09 QLC mapper용 파생 trace를 추가했다.** 기존 `layer_groups_lru_2147483648B/`는 scale 별도 상주 조건이고, 새 `layer_groups_lru_2147483648B_scales_on_demand/`는 plane과 요청된 scale column이 같은 2 GiB LRU를 공유한다. QLC-aligned 실험은 새 파생 trace를 사용한다. 배치 규격과 결과는 [QLC_ALIGNED_LAYOUT_V1.md](QLC_ALIGNED_LAYOUT_V1.md)에 있다. + +## 실행 조건 + +- 모델별 WikiText-2의 128-token 입력 2개, batch=1, 입력마다 32-token greedy 생성. 첫 생성 token은 prefill에서 나오므로 요청당 decode forward는 31회다. 모델별 tokenizer 및 생성 결과가 달라 동일한 token 경로를 비교하는 실험은 아니다. +- Prefill: 실제로 선택된 expert만 W4로 실행. Decode: gate score 기반 W2/W3/W4, 임계값 0.16/0.075. 변경된 정책으로 실제 모델을 다시 실행해 수집했다. +- Host DRAM plane-cache: **2 GiB = 2,147,483,648 B**, projection별 plane 단위 LRU. 각 요청은 빈 캐시에서 시작하고 prefill 이후 상태를 decode까지 유지한다. 두 요청 사이에는 초기화한다. +- 동일 레이어의 token들이 요구하는 expert-plane을 중복 제거한다. 현재 레이어가 요구한 항목은 해당 그룹 처리 중 퇴출하지 않는다. 같은 그룹 내 LRU 순서는 item ID 사전순으로 정하며 실제 GPU 실행 순서를 뜻하지 않는다. +- Scale은 별도 DRAM 상주: Qwen 778,567,680 B, DeepSeek 899,678,208 B. **2 GiB에 포함되지 않는다.** 비양자화 가중치, KV cache, 관리 인덱스, allocator, 전송 staging 및 GPU 작업 공간도 이 plane 예산 밖이다. +- QLC mapper용 파생 trace에서는 위 scale 상주 가정을 해제했다. Wp가 요구하는 공유 `alpha_4` column 1…p를 SSD 대상에 포함하고 plane과 scale을 합쳐 2 GiB LRU를 적용했다. +- xPU의 지속적인 routed weight cache와 prefetch는 가정하지 않는다. 읽기 완료 후 cache에 적재하고, 현재 그룹의 읽기와 계산이 완료된 다음 그룹을 진행하는 순서만 표현한다. 시간·지연 측정은 없다. + +## 결과와 파일 + +| 모델 | Prefill 그룹 | Decode 그룹 | Decode byte hit 비율 | Decode 읽기 없는 그룹 | +|---|---:|---:|---:|---:| +| Qwen | 48 | 1,488 | 55.07% | 228 | +| DeepSeek | 52 | 1,612 | 58.99% | 185 | + +이 수치는 짧은 입력에서의 캐시 시뮬레이션 결과이며 성능 벤치마크나 SSD 측정값이 아니다. Byte hit 비율은 `hit_bytes / demand_bytes`다. 요청마다 초기화하므로 prefill의 routed plane hit는 0이다. + +- [Qwen 요약](qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B/summary.json) +- [DeepSeek 요약](deepseek_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B/summary.json) +- [Qwen scale 포함 요약](qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/summary.json) +- [DeepSeek scale 포함 요약](deepseek_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/summary.json) + +각 모델의 위 요약과 같은 디렉터리에 다음 두 파일이 있다. + +| 파일 | 의미 | +|---|---| +| `layer_demands.jsonl` | 레이어 실행에 필요한 전체 plane 집합. DRAM 정책 적용 전 입력 | +| `layer_reads.jsonl` | DRAM miss로 남은 읽기 구간과 hit·eviction·cache byte 기록 | +| `summary.json` | 정책, 단계별 통계, 원본과 출력의 SHA256. 이 파일이 있어야 변환 완료 | +| `validation.json` | 출력 파일을 다시 읽어 캐시 상태·그룹 순서·통계를 독립적으로 확인한 결과. `verify_layer_groups.py`가 만든다 | + +이 표는 `validation.json`을 오래 전부터 적어 두었지만 실제로 만드는 코드는 없었다. 그동안 +캐시 시뮬레이션은 파이프라인에서 유일하게 검사받지 않는 단계였다. FEMU 카운터 일치는 이 +단계를 덮지 못한다. `layer_reads.jsonl`이 mapper의 **입력**이라 캐시가 틀린 항목을 miss로 +판정해도 매핑·재생·집계가 그대로 일관되게 따라가기 때문이다. 자기일관성은 정확성이 아니다. + +`verify_layer_groups.py`는 쓰여진 파일만 다시 읽어 네 가지를 확인한다. + +| 확인 | 내용 | +|---|---| +| 순서 | group id, layer 주기, forward 번호, 요청 경계와 decode 위치를 `layer_reads.jsonl`만으로 재유도 | +| 수요 | `layer_demands.jsonl`과 `layer_reads.jsonl`이 그룹마다 일치 | +| 캐시 | 모든 hit·miss·eviction·점유량을 `online_cache.py`의 `ReferenceLRU`가 재현. 이 구현은 숫자를 쓴 `LayerLRU`와 코드를 공유하지 않는다 | +| 통계 | `summary.json`의 단계별·요청별 합계와 최대 점유량을 그룹 줄에서 다시 누적 | + +`ReferenceLRU`가 검사로서 힘이 있는지는 무작위 차등 시험으로 확인했다. 300회 × 120단계 +동안 두 구현은 miss 집합·바이트·eviction·recency 순서까지 일치했고, 캐시에 넣은 세 가지 +결함(현재 레이어 항목을 evict, 삽입 순서를 item_id 대신 수요 순서로, hit의 recency 미갱신)은 +모두 잡혔다. + +원본 `logical_trace.jsonl`, `trace_meta.json`, `inputs.json`, `generations.json`, `code/`는 한 단계 위 디렉터리에 있다. token별 선택과 실행 당시 코드까지 확인할 수 있다. + +JSONL 한 줄은 `(request, forward, layer)` 하나다. `group_id`와 `release_after_group_id`가 순서를 나타낸다. 새 요청 첫 그룹은 `cache_reset=true`, 이전 그룹 ID는 null이다. 전부 hit인 그룹도 `reads=[]`로 남겨 계산 순서를 보존한다. 각 read의 `file, offset, nbytes`는 원본 payload 파일 구간이며 `item_id`는 projection/plane 식별자다. + +## FEMU에 넘기기 전 남은 주소 매핑 + +원본 `layer_reads.jsonl`은 LBA를 부여하기 전의 DRAM miss trace다. `lba_start`와 `sector_count`는 null, `address_status`는 `unmapped`다. QLC-aligned mapper가 이를 변환한 `layouts/.../mapped_reads.jsonl`에는 LBA와 NVMe command가 들어 있지만, 실제 FEMU replay는 아직 수행하지 않았다. + +다음 단계에서 각 SSD 레이아웃의 실제 image extent map을 만든 뒤 원본 파일 구간을 LBA로 변환한다. 이때 sector 크기, 정렬·padding, 요청 분할·병합, 최대 요청 크기를 명시하고 image/layout 체크섬을 연결해야 한다. 원본 offset을 곧바로 LBA로 간주하지 않는다. 하나의 logical read가 여러 NVMe 요청으로 나뉘거나 인접 read와 합쳐질 수 있다. Scale 초기 적재 I/O는 현재 trace에 없으므로 필요하면 별도 초기화 단계로 측정한다. + +같은 DRAM 정책과 논리 요청에서 레이아웃만 비교하려면 이 miss trace를 공통 입력으로 쓴다. 페이지 단위 caching이나 layout별 read-ahead로 cache admission이 바뀌는 실험이라면 `layer_demands.jsonl`부터 해당 정책으로 다시 시뮬레이션해야 한다. + +## 재현 + +`packages/`에 `layer_read_groups.py`, `test_layer_groups.py`와 의존 도구 `bundle.py`, `logical_reads.py`를 함께 넣었다. 변환 자체는 CPU와 NumPy만 필요하고 GPU나 체크포인트는 필요 없다. 다른 용량의 예: + +```bash +python layer_read_groups.py qwen_C --trace prefill_w4_decode_mixed_v1 --cache-bytes 1073741824 +``` + +제공된 2 GiB 출력은 이미 존재한다. 같은 경로 재실행은 덮어쓰지 않고 실패한다. 현재 레이어의 전체 작업 집합이 용량보다 크면 도구가 실패한다. 그런 용량에서는 먼저 그룹 내 streaming 순서를 정의해야 한다. + +캐시 fixture 검증: + +```bash +python -m unittest discover -s . -p 'test_layer_groups.py' +``` + +새 GPU 수집은 원래 저장소의 `collect_trace.py`에 `--mode generate --prefill-policy w4 --window 128 --samples 2 --max-new-tokens 32`를 지정한다. Qwen은 `models/Qwen1.5-MoE-A2.7B`와 `exp/moe_bcq/results/model/qwen_plain4_state.pt`, DeepSeek은 해당 모델 디렉터리와 `exp/moe_bcq/results/model/deepseek_plain4_state.pt`를 사용했다. 수집 정책과 코드·입력 식별자는 `trace_meta.json`, 원본 state 식별자는 manifest에도 기록되어 있다. diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md b/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md new file mode 100644 index 00000000000..988b71d974e --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md @@ -0,0 +1,124 @@ +# QLC-aligned expert plane-major mapper v1 — 2026-09-09 + +이 산출물은 C arm(일반 4-bit BCQ, 공유 `alpha_4` prefix, MRE 없음)의 routed expert plane과 scale을 QLC page class에 맞춘 LBA 주소표다. 아직 FEMU 장치에 적재하거나 latency를 측정하지 않았다. + +## 고정한 FEMU geometry + +```text +sector_bytes = 512 +sectors_per_page = 32 # 16 KiB +pages_per_block = 512 # patched pairing table 필수 +blocks_per_plane = 1024 +planes_per_lun = 1 +luns_per_channel = 4 +channels = 2 # 총 8 LUN +op_percent = 7 +nand_cell_type = 4 +gc_threshold = 75 +``` + +Raw capacity는 64 GiB이고 명목상 7% OP 적용 후 공간은 약 59.52 GiB다. 실제 FEMU가 노출하는 namespace 크기는 부팅 후 반드시 확인한다. + +이 mapper는 수정된 512-row QLC pairing을 전제로 한다. pg 0–5는 class 0, pg 6–7은 class 1인 특수 prologue라 사용하지 않는다. pg 8–511에서는 다음 주기가 반복되어야 한다. + +```text +page index mod 8: 0 1 2 3 4 5 6 7 +QLC class: 0 0 1 1 2 2 3 3 +``` + +원본 FEMU의 `rows-1` 문제를 고치지 않으면 pg 496–511이 class 0으로 남는다. 이 상태에서는 mapper 예측과 실제 장치가 다르므로 실험하면 안 된다. + +## Plane과 scale 배치 + +배치 순서는 `layer → expert → tier → gate/up/down`이고, 대응은 다음과 같다. + +| 정밀도 구성요소 | QLC class | +|---|---:| +| B1와 alpha1 | 0 | +| B2와 alpha2 | 1 | +| B3와 alpha3 | 2 | +| B4와 alpha4 | 3 | + +여기서 alpha1은 별도 최적화된 scale 세트가 아니라 공유 `alpha_4`의 첫 번째 column이다. Wp는 B1…Bp와 `alpha_4` column 1…p를 요구한다. + +총 8 LUN에서 class 하나의 slot은 `2 pages × 8 LUN = 16 LPN`이다. 현재 expert 한 개의 tier별 크기는 다음과 같다. + +```text +gate/up/down plane = 22 + 22 + 22 = 66 pages +gate/up/down scale = ceil(2.75) × 3 = 9 allocated pages +합계 = 75 pages +필요한 class cycle = ceil(75 / 16) = 5 +``` + +따라서 한 expert는 5개의 8-page pairing cycle, 즉 `5 × 8 page-index × 8 LUN = 320 LPN = 5 MiB`의 주소 공간을 사용한다. 각 tier의 자료는 같은 다섯 cycle에서 대응되는 class slot에 놓인다. 66-page plane은 같은 class의 여러 slot으로 나뉘므로 한 logical item이 `extent_map.json`에서 여러 fragment를 가질 수 있다. + +Scale extent 하나는 45,056 B로 sector에는 정확히 맞지만 NAND page에는 맞지 않는다. 각 projection scale column을 page 경계에서 시작하도록 3 pages를 할당하고 마지막 4 KiB는 filler로 둔다. 실제 read는 유효한 45,056 B만 요청한다. + +## 생성 결과 + +| 모델 | 실제 plane+scale | 순차 image 크기 | filler | payload 비율 | +|---|---:|---:|---:|---:| +| Qwen | 7,007,109,120 B | 7,717,519,360 B (7.19 GiB) | 710,410,240 B | 90.79% | +| DeepSeek | 8,097,103,872 B | 8,925,478,912 B (8.31 GiB) | 828,375,040 B | 90.72% | + +- [Qwen layout summary](qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/layout_summary.json) +- [DeepSeek layout summary](deepseek_C/layouts/qlc_aligned_epm_aif_2ch4lun/layout_summary.json) + +각 layout 디렉터리에는 다음 파일이 있다. + +| 파일 | 역할 | +|---|---| +| `extent_map.json` | 전체 plane·scale item의 원본 구간, target LPN/LBA, fragment와 목표 class | +| `mapped_reads.jsonl` | 2 GiB LRU miss를 LBA로 바꾼 레이어 그룹과 병합된 NVMe command | +| `layout_summary.json` | geometry, 용량, 입력·출력 SHA256, QD=32 권장값 | +| `layout_validation.json` | catalog·fragment·class·주소 중복·trace byte 보존 검증 결과 | +| `replay_qd32.bin` | strict group barrier와 QD=32 기본값을 담은 compact replay 입력 | +| `replay_qd32.bin.json` | binary hash, 원본 hash, record 총계와 request index 표 | + +현재 mapped trace는 plane과 scale column이 **같은 2 GiB LRU cache를 공유**한 결과를 사용한다. + +```text +traces/prefill_w4_decode_mixed_v1/ + layer_groups_lru_2147483648B_scales_on_demand/ +``` + +Mapper는 같은 class에서 LBA가 바로 이어진 fragment만 최대 4 MiB까지 하나의 command로 병합한다. 서로 다른 class를 가로질러 병합하지 않는다. Replayer는 이 command에 QD=32를 적용한다. 게스트의 실제 최대 request 크기가 4 MiB보다 작으면 block layer가 다시 나눌 수 있으므로 `max_sectors_kb`와 실제 NVMe command 수를 기록해야 한다. + +Binary 생성과 실제 실행 방법, rolling QD와 timestamp 의미는 [REPLAYER_V1.md](REPLAYER_V1.md)를 따른다. + +## 도구 사용 + +주소표와 mapped trace 생성: + +```bash +python qlc_aligned_mapper.py plan qwen_C \ + --layer-reads qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/layer_reads.jsonl \ + --output qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun +``` + +검증: + +```bash +python qlc_aligned_mapper.py validate qwen_C \ + --layer-reads qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/layer_reads.jsonl \ + qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun +``` + +실제 image가 필요할 때만 materialize한다. 제공된 package에는 image가 없다. + +```bash +python qlc_aligned_mapper.py materialize qwen_C \ + qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun --image qwen_C.img +``` + +생성된 image는 filler를 포함한다. FEMU 재시작 직후 host write가 0인지 확인한 다음, **LPN 0부터 끝까지 빠짐없이 순차 write**해야 한다. 최종 device command의 분할 경계도 16 KiB NAND page에 정렬되어야 한다. page 중간에서 나뉘면 같은 LPN이 두 번 program되어 후속 PPA가 밀릴 수 있다. 현재 guest에서는 `dd bs=256K`로 적재한다. 기존 `bs=4M`은 이 조건을 보장하지 못했다. queue 제한과 적재 후 FEMU WRITE 로그의 LPN→PPA class 규칙을 검증하고, 데이터를 되읽어 원본 SHA와 별도로 비교한 뒤 replay한다. + +## 아직 측정되지 않은 것 + +- FEMU에서 실제 PPA/page class 일치 여부 +- materialized image의 byte-for-byte read-back +- QD=32 replay의 제출·완료 timestamp와 latency +- OS/NVMe 계층의 실제 요청 split·merge +- GPU 전송 및 BCQ 연산을 포함한 end-to-end latency + +따라서 현재 결과는 **검증된 주소 계획과 replay 입력**이며 FEMU 성능 결과가 아니다. diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/REPLAYER_V1.md b/moe-harness/exp/moe_bcq/femu_handoff/packages/REPLAYER_V1.md new file mode 100644 index 00000000000..d30d5420541 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/REPLAYER_V1.md @@ -0,0 +1,134 @@ +# Binary trace compiler와 FEMU replayer v1 — 2026-09-09 + +`mapped_reads.jsonl`은 사람이 주소와 cache 결과를 감사하는 기준 파일이고, `replay_qd32.bin`은 게스트가 재생하는 입력이다. Qwen JSONL 101.6 MB는 2.14 MB, DeepSeek JSONL 124.4 MB는 2.59 MB의 고정 길이 binary record로 줄었다. 두 형식은 binary header의 `mapped_reads_sha256`으로 묶인다. + +현재 binary는 실제 C 모델에서 수집한 **WikiText-2 pilot generation trace**를 2 GiB LRU와 QLC-aligned mapper에 통과시킨 결과다. 정식 WikiText-2 전체, GSM8K, MMLU trace는 아니다. + +## 포함된 파일 + +| 파일 | 역할 | +|---|---| +| `trace_compiler.py` | 검증된 layout의 mapped JSONL을 binary로 컴파일하거나 binary를 정적으로 검사 | +| `replay_v1.c` | Linux native AIO로 binary trace를 재생하는 게스트 프로그램 | +| `test_trace_replayer.py` | compiler, O_DIRECT, rolling QD, 빈 그룹 barrier를 검사하는 작은 통합 테스트 | +| `{model}/layouts/.../replay_qd32.bin` | QD=32 기본값을 담은 실제 pilot replay 입력 | +| `replay_qd32.bin.json` | 요청 index 표, 원본 hash, record 크기와 총계 | + +replayer는 `linux/aio_abi.h`의 syscall을 직접 사용하므로 `libaio`나 `liburing`에 링크하지 않는다. + +## Binary 형식 + +모든 정수는 little-endian이고 record에는 포인터나 가변 길이 문자열이 없다. + +| record | 크기 | 핵심 내용 | +|---|---:|---| +| header | 136 B | magic/version, sector·alignment, 기본 QD, group/command/byte 총계, extent map과 mapped JSONL SHA256 | +| group | 40 B | group ID, 이전 group dependency, request index, forward/layer/phase, cache reset, command 수 | +| command | 16 B | LBA, sector 수, QLC page class | + +group record 뒤에 그 group의 command record가 바로 온다. command가 0개인 group도 record를 남긴다. 따라서 DRAM hit로 SSD read가 없어진 레이어도 순서에서 사라지지 않는다. 문자열 `request_id`는 sidecar JSON의 `requests[]`가 `request_index`로 복원한다. + +Compiler는 다음을 실패 조건으로 둔다. + +- `extent_map.json`, `mapped_reads.jsonl`의 hash가 layout summary와 다름 +- `layout_validation.json`이 통과 상태가 아님 +- group ID·dependency·command ID가 연속 규칙과 다름 +- 4 KiB O_DIRECT 정렬, 512 B sector, 4 MiB 최대 command 규칙 위반 +- JSONL과 layout summary의 group·command·byte 총계 불일치 + +## 컴파일과 정적 검사 + +전달 폴더 `packages/`에서 실행한다. 제공된 output은 이미 있으므로 재생성할 때는 새 이름을 쓴다. + +```bash +python trace_compiler.py compile \ + qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun \ + --output qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/replay_check.bin \ + --queue-depth 32 + +python trace_compiler.py inspect \ + qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/replay_qd32.bin +``` + +게스트에서 replayer를 빌드하고 trace만 먼저 검사한다. + +```bash +cc -O2 -std=c11 -Wall -Wextra -Werror -o replay_v1 replay_v1.c + +./replay_v1 \ + --trace qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/replay_qd32.bin \ + --dry-run +``` + +Qwen은 1,536 groups, 130,090 commands, 22,608,650,240 requested bytes이고 DeepSeek은 1,664 groups, 157,515 commands, 27,372,060,672 bytes여야 한다. + +## Image 적재와 실제 replay + +먼저 payload와 filler를 합친 image를 생성한다. 이 파일은 패키지에 미리 넣지 않았다. + +```bash +python qlc_aligned_mapper.py materialize qwen_C \ + qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun \ + --image qwen_C.img +``` + +요청한 FEMU geometry와 patched 512-row pairing으로 새 장치를 부팅한다. namespace를 마운트하지 않고 LBA 0부터 image 끝까지 한 번의 순차 stream으로 적재한다. + +```bash +sudo dd if=qwen_C.img of=/dev/nvme0n1 bs=256K \ + iflag=fullblock oflag=direct conv=fsync status=progress +``` + +2026-09-09 실장 검증에서 기존 `bs=4M` fill이 NAND page 중간에서 분할되어 같은 LPN을 두 번 프로그램하고 PPA 순서를 밀어내는 현상이 확인됐다. 따라서 fill 크기를 256 KiB로 수정했다. 관측된 guest는 4 KiB 메모리 page, `max_segments=127`이며 256 KiB는 page-aligned buffer에서 64개 메모리 page다. 이 설정에서 여유를 둔 크기이지 모든 장치에서 무분할을 보장하는 상수는 아니다. `getconf PAGESIZE`와 queue의 `max_segments`, `max_segment_size`, `max_sectors_kb`, `max_hw_sectors_kb`를 기록하고 실제 program 순서를 확인한다. `max_sectors_kb=4096`만으로 무분할을 판정하지 않는다. + +적재 전후에 다른 host write가 없어야 한다. 최종 device command도 NAND page 경계에서 나뉘고 각 LPN이 정확히 한 번씩 program되어야 mapper의 순차 배치 가정이 성립한다. WRITE log의 LPN→PPA class와 image read-back을 각각 확인한 다음 replay한다. byte hash 일치는 page class 일치를 보장하지 않는다. output 파일은 덮어쓰지 않으므로 run마다 새 디렉터리나 이름을 사용한다. + +```bash +sudo ./replay_v1 \ + --trace qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/replay_qd32.bin \ + --device /dev/nvme0n1 --controller /dev/nvme0 \ + --qd 32 \ + --group-log run01.groups.jsonl \ + --summary run01.summary.json +``` + +기본 동작은 O_DIRECT다. `--buffered`는 작은 개발용 파일에만 쓰는 fallback이다. +`--skip-qlc-counters`는 stock NVMe나 unit test용이며 논문 측정에서는 쓰지 않는다. 기본 +실행은 vendor admin command `0xef/cdw10=8`로 적재 후 QLC counter를 초기화한다. + +각 group은 barrier이므로 phase가 바뀌는 시점에는 직전 phase의 I/O가 모두 완료되어 있다. +replayer는 이 경계와 replay 끝에서 `cdw10=10/11/12`를 보내 직전 physical-counter 차분을 +각각 prefill/decode/teacher-forced bank에 누적한 후, 마지막에 `cdw10=9`로 snapshot한다. +요청마다 `prefill→decode`가 반복돼도 각 bank에 합산된다. + +최종 CSV의 기존 total 열 뒤에는 phase별 `n_read`, `bytes_read`, `e_nand_uj`가 추가된다. +모든 class에서 `total = prefill + decode + teacher_forced`가 성립해야 한다. 명령 중 하나라도 +실패하면 run을 실패 처리한다. plain FEMU가 알 수 없는 selector를 성공으로 돌려줄 수 있으므로 +실행 스크립트에서도 phase 열과 위 closure를 검사해야 한다. + +## 제출과 timestamp 의미 + +그룹 사이에는 strict barrier가 있다. 이전 그룹의 모든 command가 완료된 뒤 다음 그룹으로 이동한다. 그룹 안에서는 outstanding read를 최대 QD까지 채우고, 하나 이상 완료될 때마다 다시 채우는 rolling QD를 쓴다. QD=32는 동시에 완료되는 수가 아니라 host가 아직 completion을 받지 않은 command의 상한이다. + +`group-log`의 시간은 `CLOCK_MONOTONIC_RAW`이며 replay 시작을 0으로 둔다. + +| 필드 | 의미 | +|---|---| +| `group_ready_ns` | command record를 읽어 준비했고 이전 barrier도 끝난 시점 | +| `first_submit_ns` | 첫 `io_submit` 호출 직전 | +| `last_submit_ns` | 마지막 `io_submit`가 반환된 직후 | +| `last_complete_ns` | 마지막 completion을 `io_getevents`에서 관찰한 시점 | +| `group_io_ns` | 첫 submit 직전부터 마지막 completion 관찰까지 | +| `peak_outstanding` | 실제 host-side 최대 outstanding command 수 | + +빈 그룹은 submit 시점이 `null`이고 `group_io_ns=0`이다. `summary`의 `sum_group_io_ns`를 SSD service 구간 합으로 사용한다. `wall_ns_including_log_overhead`에는 binary 해석과 group JSON 기록 비용도 들어간다. + +`--io-log`는 command별 submit과 completion 관찰 시점을 남기는 진단 옵션이다. 한 번의 `io_getevents`로 여러 completion을 받으면 같은 관찰 timestamp가 기록되며 실제 device completion 순간과는 다를 수 있다. 이 로그 자체도 timing을 교란하므로 주 측정에는 group log만 사용한다. + +group log는 제공된 pilot 전체가 메모리 buffer에 머물도록 4 MiB buffering한 뒤 replay 후 flush한다. 그래도 결과 파일은 측정 대상 namespace가 아닌 root disk나 `/dev/shm`에 둔다. target namespace에 로그를 쓰면 순차 적재 계약과 QLC counter가 모두 오염된다. + +## 검증 범위 + +로컬 fixture에서 C11 `-Werror` 빌드, binary dry-run, 실제 O_DIRECT native-AIO read, QD=2 rolling 제출, 빈 그룹 barrier, 총 command/byte 보존을 검사했다. 실제 Qwen과 DeepSeek binary도 Python inspector와 C preflight를 모두 통과했다. + +FEMU에서는 먼저 class가 알려진 작은 LBA 집합을 fio와 replayer에서 QD=1/4/32로 각각 읽어 bytes, command 수, group makespan, QLC counter가 맞는지 교차 확인해야 한다. 현재 패키지에는 **FEMU에서 얻은 latency 결과가 아직 없다.** diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py new file mode 100644 index 00000000000..2efc5c5a7b8 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py @@ -0,0 +1,208 @@ +#!/usr/bin/env python3 +"""Portable, unpadded routed-BCQ payloads. No CUDA dependency.""" +from __future__ import annotations + +import argparse +from contextlib import ExitStack +import hashlib +import json +import re +from pathlib import Path + +import numpy as np + +SCHEMA = "moe-bcq-handoff-v1" +ROUTED = re.compile(r"^model\.layers\.(\d+)\.mlp\.experts\.(\d+)\.(gate|up|down)_proj\.qweight$") + + +def sha256(path): + h = hashlib.sha256() + with open(path, "rb") as f: + for b in iter(lambda: f.read(8 << 20), b""): + h.update(b) + return h.hexdigest() + + +def write_json(path, value): + Path(path).write_text(json.dumps(value, indent=2, ensure_ascii=False) + "\n") + + +def emit(f, tensor, dtype): + arr = np.asarray(tensor.numpy(), dtype=dtype, order="C") + data = arr.tobytes() + rec = dict(file=Path(f.name).name, offset=f.tell(), nbytes=len(data), + dtype=np.dtype(dtype).str, shape=list(arr.shape), + sha256=hashlib.sha256(data).hexdigest()) + f.write(data) + return rec + + +def export(state_path, model_dir, arm, out): + import torch + torch.set_num_threads(2) + out, model_dir, state_path = Path(out), Path(model_dir), Path(state_path) + out.mkdir(parents=True, exist_ok=False) + config = json.loads((model_dir / "config.json").read_text()) + state = torch.load(state_path, map_location="cpu", weights_only=True, mmap=True) + keys = [(ROUTED.fullmatch(k), k) for k in state if ROUTED.fullmatch(k)] + keys.sort(key=lambda t: (int(t[0][1]), int(t[0][2]), t[0][3])) + if not keys: + raise ValueError("No routed expert qweight tensors") + manifest = dict(schema=SCHEMA, model=model_dir.name, arm=arm, + scale_mode="per_precision" if arm == "A" else "shared_alpha4_prefix", + supported_bits=[2, 3, 4], source_state=dict(name=state_path.name, + sha256=sha256(state_path)), model_config=config, + source_config_sha256=sha256(model_dir / "config.json"), + mre_steps="not inferred from tensor data; see source quantization report", + exporter_sha256=sha256(__file__), byte_order="little", padding_bytes=0, + beta="implicit_zero_verified", bias="absent_verified", + native_qweight_order=["input_word32", "plane", "output"], + plane_order=["input_word32", "output"], + scale_column_order=["input_group", "output"], + bit_encoding="bit t of word k is input 32*k+t; 0=-1, 1=+1", + placement="unassigned: file offsets are NOT LBA or NAND pages", + projections=[]) + with (out / "planes.bin").open("xb") as pf, (out / "scales.bin").open("xb") as sf: + for m, key in keys: + prefix = key.removesuffix("qweight") + q = state[key] + a4 = state[prefix + "alpha_4"] + assert q.dtype == torch.int32 and q.ndim == 3 and q.shape[1] == 4 + i, o = q.shape[0] * 32, q.shape[2] + assert a4.dtype == torch.float16 and a4.shape[1:] == (4, o) + assert i % a4.shape[0] == 0 + g = i // a4.shape[0] + assert g % 32 == 0 + assert prefix + "bias" not in state, "Nonzero/explicit bias needs schema extension" + use = [2, 3, 4] if arm == "A" else [4] + for p in use: + a, b = state[prefix + f"alpha_{p}"], state[prefix + f"beta_{p}"] + assert a.shape == (i // g, p, o) and a.dtype == torch.float16 + assert torch.isfinite(a).all(), f"Invalid alpha: {prefix}" + assert b.shape == (i // g, o) and torch.count_nonzero(b) == 0, prefix + rec = dict(id=prefix.rstrip("."), layer=int(m[1]), expert=int(m[2]), + projection=m[3] + "_proj", weight_shape_out_in=[o, i], + group_size=g, planes=[], scales={}) + for j in range(4): + rec["planes"].append(emit(pf, q[:, j, :], " self.capacity: + raise ValueError("Layer working set exceeds cache capacity; define a streaming schedule first") + for key, e in wanted.items(): + if key in self.items and self.items[key] != e['nbytes']: + raise ValueError("Cache item size changed") + hits = [k for k in wanted if k in self.items] + misses = [dict(item_id=k, **e) for k, e in wanted.items() if k not in self.items] + miss_bytes = sum(e['nbytes'] for e in misses) + before = self.used + evicted = [] + # Decide every hit before insertion; never evict a current-layer demand. + for key in list(self.items): + if self.used + miss_bytes <= self.capacity: + break + if key not in wanted: + size = self.items.pop(key) + self.used -= size + evicted.append(dict(item_id=key, nbytes=size)) + assert self.used + miss_bytes <= self.capacity + # Stable tie break within a layer; not an assertion of GPU expert order. + for key in sorted(wanted): + size = wanted[key]['nbytes'] + if key in self.items: + del self.items[key] + else: + self.used += size + self.items[key] = size + hit_bytes = sum(wanted[k]['nbytes'] for k in hits) + assert hit_bytes + miss_bytes == sum(e['nbytes'] for e in extents) + assert self.used == before - sum(e['nbytes'] for e in evicted) + miss_bytes + assert self.used <= self.capacity + return dict(hit_item_ids=hits, hit_bytes=hit_bytes, misses=misses, + miss_bytes=miss_bytes, evictions=evicted, + cache_bytes_before=before, cache_bytes_after=self.used) + + +def build(bundle, trace_name, capacity_bytes, scales='resident'): + bundle = Path(bundle) + trace = bundle / 'traces' / trace_name + mpath = bundle / 'manifest.json' + manifest = json.loads(mpath.read_text()) + meta = json.loads((trace / 'trace_meta.json').read_text()) + assert manifest['schema'] == meta['schema'] == SCHEMA + assert meta['manifest_sha256'] == sha256(mpath) + assert meta['state_sha256'] == manifest['source_state']['sha256'] + assert meta['mode'] == 'generate', 'Keep teacher-forced workloads separate' + assert meta['phase_policies'] == dict(prefill='w4', decode='gated_mixed', teacher_forced='gated_mixed') + for filename, info in meta['files'].items(): + assert sha256(trace / filename) == info['sha256'] + assert scales in ('resident', 'on_demand') + suffix = '' if scales == 'resident' else '_scales_on_demand' + out = trace / f'layer_groups_lru_{capacity_bytes}B{suffix}' + # If the collector ran a cache beside the model, this derivation has to + # reproduce it group for group. That is the only check on the cache itself: + # the FEMU counters agree with the mapper, but the mapper's input is this + # file, so a cache that misses wrongly is replayed wrongly and consistently. + record = trace / 'online_cache.jsonl' + online, online_rows = None, {} + if record.exists(): + header, online_rows = online_cache.load(record) + assert header['manifest_sha256'] == meta['manifest_sha256'] + if header['scales'] == scales and capacity_bytes in header['capacities']: + online = str(capacity_bytes) + else: + online_rows = {} + out.mkdir(exist_ok=False) + cache = LayerLRU(capacity_bytes) + previous_request, previous_forward, previous_layer = None, -1, None + previous_group = None + seen_requests = set() + frame_info = None + next_position = None + phases = {'prefill': Counter(), 'decode': Counter()} + request_stats = {} + peak = 0 + groups = 0 + verified = 0 + source = trace / 'logical_trace.jsonl' + with source.open() as src, (out / 'layer_demands.jsonl').open('x') as df, (out / 'layer_reads.jsonl').open('x') as rf: + for line in src: + event = json.loads(line) + gid = event['event_id'] + rid = event['request_id'] + phase = event['phase'] + fid, layer = event['forward_id'], event['layer'] + assert gid == groups and phase in phases + new_forward = fid != previous_forward + reset = rid != previous_request + if new_forward: + assert fid == previous_forward + 1 + if previous_layer is not None: + assert previous_layer == manifest['totals']['layers'][-1] + assert layer == manifest['totals']['layers'][0] + frame_info = (rid, phase, event['input_ids'], event['token_positions']) + if reset: + assert rid not in seen_requests and phase == 'prefill' + assert event['token_positions'] == list(range(len(event['input_ids']))) + seen_requests.add(rid) + cache.clear() + previous_group = None + next_position = len(event['input_ids']) + request_stats[rid] = {'prefill': Counter(), 'decode': Counter()} + else: + assert phase == 'decode' and event['token_positions'] == [next_position] + assert len(event['input_ids']) == 1 + next_position += 1 + else: + assert not reset and frame_info == (rid, phase, event['input_ids'], event['token_positions']) + order = manifest['totals']['layers'] + assert layer == order[order.index(previous_layer) + 1] + if phase == 'prefill': + assert all(b == 4 for row in event['precision_bits'] for b in row) + extents = demand(manifest, event, scales_resident=(scales == 'resident')) + result = cache.serve(extents) + common = dict(group_id=gid, request_id=rid, forward_id=fid, layer=layer, + phase=phase, batch_size=event['batch_size'], + token_positions=event['token_positions'], cache_reset=reset, + release_after_group_id=previous_group, + barrier='all reads and compute of this group precede next group', + demand_items=len(extents), demand_bytes=sum(e['nbytes'] for e in extents)) + df.write(json.dumps(dict(**common, demands=[dict(item_id=item_key(e), **e) for e in extents]), separators=(',', ':'))+'\n') + misses = result.pop('misses') + reads = [dict(read_id=f'{gid}:{j}', **e, lba_start=None, sector_count=None, + address_status='unmapped', operation='read') for j, e in enumerate(misses)] + if online is not None: + row = online_rows[gid] + assert row['p'] == phase + observed = dict(zip(online_cache.ROW_FIELDS, row['c'][online])) + derived = dict(miss_items=len(reads), miss_bytes=result['miss_bytes'], + hit_bytes=result['hit_bytes'], + evicted_items=len(result['evictions']), + evicted_bytes=sum(e['nbytes'] for e in result['evictions']), + used_after=result['cache_bytes_after'], + miss_digest=online_cache.digest(sorted(m['item_id'] for m in misses))) + if [len(extents), common['demand_bytes']] != row['d'] or observed != derived: + raise AssertionError(f'Online cache disagrees at group {gid}: ' + f'observed {row["d"]} {observed} vs derived ' + f'{[len(extents), common["demand_bytes"]]} {derived}') + verified += 1 + rf.write(json.dumps(dict(**common, **result, reads=reads), separators=(',', ':'))+'\n') + stat = dict(groups=1, demand_items=len(extents), hit_items=len(result['hit_item_ids']), + miss_items=len(reads), demand_bytes=common['demand_bytes'], + hit_bytes=result['hit_bytes'], miss_bytes=result['miss_bytes'], + evicted_items=len(result['evictions']), evicted_bytes=sum(e['nbytes'] for e in result['evictions']), + all_hit_groups=int(not reads)) + phases[phase].update(stat) + request_stats[rid][phase].update(stat) + peak = max(peak, cache.used) + previous_request, previous_forward, previous_layer = rid, fid, layer + previous_group = gid + groups += 1 + assert groups == meta['events'] and previous_layer == manifest['totals']['layers'][-1] + if online is not None: + assert verified == groups == len(online_rows), 'Online record does not cover every group' + check = dict(status='verified', origin=header['origin'], groups=verified, + record_sha256=sha256(record), + implementation=('online_cache.py ReferenceLRU, run in-process during GPU execution' + if header['origin'] == 'in_process' else + 'online_cache.py ReferenceLRU, replayed from the trace file; ' + 'checks the cache logic, not the trace'), + covers='per-group miss count, miss/hit bytes, eviction count and bytes, ' + 'occupancy, and a digest of which items missed') + elif record.exists(): + check = dict(status='not_applicable', groups=0, + reason=f'record holds scales={header["scales"]} capacities={header["capacities"]}') + else: + check = dict(status='absent', groups=0, + reason='trace was collected before the in-process cache existed; ' + 'this derivation is unchecked') + summary = dict(schema='moe-layer-read-groups-v1', groups=groups, requests=len(seen_requests), + model=manifest['model'], arm=manifest['arm'], phase_policies=meta['phase_policies'], + cache=dict(policy='LRU', capacity_bytes=capacity_bytes, granularity='projection-plane', + recency='whole layer; ties use lexicographic item_id', reset='each request; prefill retained for decode', + protected='all demanded items until layer completion', peak_payload_bytes=peak, + prefetch=False, admission='all misses on completion before next layer'), + scales=scales, online_check=check, + resident_scales_bytes=(manifest['totals']['scales_bytes'] if scales == 'resident' else 0), + budget_note=('Cache capacity is plane payload only; scales are additional.' if scales == 'resident' + else 'Planes and requested scale columns share this cache capacity.') + + ' Python index, allocator, GPU working buffers and transport staging not modeled.', + xpu_policy='no persistent routed-weight cache; current call working data only', + timing='barrier order only; no timestamps or compute/transfer latency', + mapping=dict(status='pending', layout_sha256=None, sector_bytes=None, + note='Source offsets are NOT LBA. Map to actual image extents before FEMU replay.'), + phase_stats=phases, request_stats=request_stats, source_manifest_sha256=sha256(mpath), + source_trace_sha256=sha256(source), source_meta_sha256=sha256(trace / 'trace_meta.json'), + builder_sha256=sha256(__file__), files={name:dict(sha256=sha256(out/name), nbytes=(out/name).stat().st_size) + for name in ['layer_demands.jsonl','layer_reads.jsonl']}) + write_json(out / 'summary.json', summary) + print(json.dumps(dict(output=str(out), groups=groups, phase_stats=phases)), flush=True) + return summary + + +def main(): + p = argparse.ArgumentParser(description=__doc__) + p.add_argument('bundle', type=Path) + p.add_argument('--trace', default='prefill_w4_decode_mixed_v1') + p.add_argument('--cache-bytes', type=int, default=2*1024**3) + p.add_argument('--scales', choices=['resident', 'on_demand'], default='resident') + a = p.parse_args() + build(a.bundle, a.trace, a.cache_bytes, a.scales) + + +if __name__ == '__main__': + main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py new file mode 100644 index 00000000000..7be03c0178b --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py @@ -0,0 +1,100 @@ +#!/usr/bin/env python3 +"""Validate a trace and resolve it to file extents, NOT SSD page addresses. + +One cold demand set per forward/layer; deduplicate prefix planes within the set. +For A, keep EVERY used precision's scale set, even if a higher tier is present. +""" +import argparse +import json +import math +from collections import defaultdict +from pathlib import Path + +from bundle import SCHEMA, sha256, write_json + + +def demand(manifest, event, scales_resident=True): + catalog = defaultdict(list) + for p in manifest["projections"]: + if p["layer"] == event["layer"]: + catalog[p["expert"]].append(p) + wanted = defaultdict(set) + positions = event["token_positions"] + assert len(positions) == len(set(positions)) + assert len(positions) == len(event["input_ids"]) == len(event["selected_experts"]) == len(event["precision_bits"]) + assert len(positions) == len(event["gate_scores"]) + assert event["batch_size"] == 1 + assert positions == sorted(positions) and all(type(x) is int and x >= 0 for x in positions) + for es, bs, gs in zip(event["selected_experts"], event["precision_bits"], event["gate_scores"]): + assert len(es) == len(bs) == len(gs) == manifest["model_config"]["num_experts_per_tok"] + assert len(es) == len(set(es)) + assert all(math.isfinite(g) and g >= 0 for g in gs) + for expert, bits in zip(es, bs): + assert expert in catalog and bits in manifest["supported_bits"] + wanted[expert].add(bits) + extents = [] + for expert, precisions in sorted(wanted.items()): + for p in catalog[expert]: + for j, e in enumerate(p["planes"][:max(precisions)], 1): + extents.append(dict(projection_id=p["id"], kind="plane", plane=j, **e)) + if not scales_resident: + for b in sorted(precisions) if manifest["scale_mode"] == "per_precision" else [4]: + take = b if manifest["scale_mode"] == "per_precision" else max(precisions) + for j, e in enumerate(p["scales"][str(b)][:take], 1): + extents.append(dict(projection_id=p["id"], kind="scale", scale_set=b, column=j, **e)) + return extents + + +def main(): + p = argparse.ArgumentParser(description=__doc__) + p.add_argument("bundle", type=Path) + p.add_argument("--trace", default="smoke") + p.add_argument("--scales", choices=["resident", "on_demand"], default="resident") + a = p.parse_args() + mpath = a.bundle / "manifest.json" + m = json.loads(mpath.read_text()) + tdir = a.bundle / "traces" / a.trace + meta = json.loads((tdir / "trace_meta.json").read_text()) + assert m["schema"] == meta["schema"] == SCHEMA + assert meta["manifest_sha256"] == sha256(mpath) + for name, info in meta["files"].items(): + assert sha256(tdir / name) == info["sha256"] + out = tdir / f"reads_{a.scales}.jsonl" + expected, total, last = 0, 0, None + layers_seen = [] + frame_info = None + with (tdir / "logical_trace.jsonl").open() as f, out.open("x") as dst: + for line in f: + e = json.loads(line) + assert e["event_id"] == expected + assert e["phase"] in ("prefill", "decode", "teacher_forced") + if e["phase"] == "decode": + assert len(e["input_ids"]) == 1 + order = (e["forward_id"], e["layer"]) + assert last is None or order > last + if last is None or last[0] != e["forward_id"]: + assert e["forward_id"] == (0 if last is None else last[0] + 1) + if last is not None: + assert layers_seen == m["totals"]["layers"], "Incomplete forward" + layers_seen = [] + frame_info = (e["request_id"], e["phase"], e["token_positions"], e["input_ids"]) + assert frame_info == (e["request_id"], e["phase"], e["token_positions"], e["input_ids"]) + layers_seen.append(e["layer"]) + last = order + extents = demand(m, e, a.scales == "resident") + nbytes = sum(x["nbytes"] for x in extents) + dst.write(json.dumps(dict(event_id=expected, forward_id=e["forward_id"], + layer=e["layer"], request_id=e["request_id"], phase=e["phase"], + demand_bytes=nbytes, extents=extents), separators=(",", ":")) + "\n") + expected += 1 + total += nbytes + assert expected == meta["events"] + assert layers_seen == m["totals"]["layers"], "Incomplete final forward" + write_json(tdir / f"reads_{a.scales}_summary.json", dict(events=expected, + cold_demand_bytes=total, scales=a.scales, sha256=sha256(out), + semantics="Deduplicated within forward/layer; no reuse across events. No LBA/page rounding/timing.")) + print(f"{expected} events, {total} logical bytes, scales={a.scales}") + + +if __name__ == "__main__": + main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py new file mode 100644 index 00000000000..248da6402ee --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py @@ -0,0 +1,549 @@ +#!/usr/bin/env python3 +"""Map routed BCQ planes and shared scale columns to aligned QLC LBAs. + +The mapping assumes a freshly reset FEMU device is filled sequentially from +LPN 0. Under that contract, allocation ordinal equals LPN and the closed-form +FEMU channel->LUN->page write pointer determines the physical page class. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import math +from collections import defaultdict +from dataclasses import asdict, dataclass +from itertools import zip_longest +from pathlib import Path + +from bundle import SCHEMA, sha256, write_json + + +LAYOUT_SCHEMA = "moe-bcq-qlc-aligned-layout-v1" +MAPPED_SCHEMA = "moe-bcq-mapped-layer-reads-v1" +PROJECTION_ORDER = ("gate_proj", "up_proj", "down_proj") + + +@dataclass(frozen=True) +class Geometry: + sector_bytes: int = 512 + sectors_per_page: int = 32 + pages_per_block: int = 512 + blocks_per_plane: int = 1024 + planes_per_lun: int = 1 + luns_per_channel: int = 4 + channels: int = 2 + op_percent: int = 7 + pairing_profile: str = "patched-512" + + @property + def page_bytes(self): + return self.sector_bytes * self.sectors_per_page + + @property + def parallel_luns(self): + return self.channels * self.luns_per_channel * self.planes_per_lun + + @property + def line_pages(self): + return self.pages_per_block * self.parallel_luns + + @property + def raw_bytes(self): + return self.line_pages * self.blocks_per_plane * self.page_bytes + + @property + def exposed_bytes_nominal(self): + return self.raw_bytes * (100 - self.op_percent) // 100 + + def validate(self): + if self.sector_bytes <= 0 or self.sectors_per_page <= 0: + raise ValueError("Sector/page geometry must be positive") + if self.pages_per_block != 512 or self.pairing_profile != "patched-512": + raise ValueError("This mapper requires the patched 512-row FEMU QLC pairing table") + if self.pages_per_block % 8 or self.parallel_luns <= 0: + raise ValueError("Unsupported FEMU geometry") + + +def qlc_class(page_in_block): + """Expected init_qlc_page_pairing class after the rows-1 fix.""" + if not 0 <= page_in_block < 512: + raise ValueError(page_in_block) + if page_in_block <= 5: + return 0 + if page_in_block <= 7: + return 1 + return (page_in_block % 8) // 2 + + +# Which QLC class slot each bit-plane tier is placed in. A policy is a +# permutation of the four slots, so every policy allocates exactly the same +# pages, the same fragments and the same NVMe commands -- only the physical page +# class under each plane changes. That is what makes the comparison controlled: +# the placement is the single variable, with byte layout and command structure +# held fixed. A baseline built by packing sequentially instead would also change +# the request pattern and confound the two. +def slot_for(tier, expert_ordinal, policy): + if policy == "aligned": + # B1 on the fastest class, B4 on the slowest. B1/B2 are read on every + # expert selection; B4 only for the top tier. + return tier + if policy == "inverted": + # The worst case: the always-read planes on the slowest pages. + return 3 - tier + if policy == "rotated": + # Class-oblivious control. Each expert shifts the permutation by one, so + # across experts every tier meets every class equally often and the + # read-frequency skew buys nothing. + return (tier + expert_ordinal) % 4 + raise ValueError(f"Unknown placement policy: {policy}") + + +PLACEMENT_POLICIES = ("aligned", "inverted", "rotated") + + +def item_key(kind, projection_id, index, scale_set=4): + if kind == "plane": + return f"{projection_id}/B{index}" + return f"{projection_id}/alpha{scale_set}/C{index}" + + +def canonical_hash(value): + data = json.dumps(value, sort_keys=True, separators=(",", ":")).encode() + return hashlib.sha256(data).hexdigest() + + +def group_catalog(manifest): + grouped = defaultdict(dict) + for rec in manifest["projections"]: + key = (rec["layer"], rec["expert"]) + if rec["projection"] in grouped[key]: + raise ValueError(f"Duplicate projection: {key} {rec['projection']}") + grouped[key][rec["projection"]] = rec + for key, projections in grouped.items(): + if set(projections) != set(PROJECTION_ORDER): + raise ValueError(f"Expected gate/up/down projections for {key}") + return [(key, grouped[key]) for key in sorted(grouped)] + + +def usable_cycle_base_lpn(cycle_index, geom): + """Each usable 8-page cycle contains class 0/1/2/3 slots.""" + cycles_per_block = (geom.pages_per_block - 8) // 8 + block, within = divmod(cycle_index, cycles_per_block) + page = 8 + 8 * within + return block * geom.line_pages + page * geom.parallel_luns + + +def target_fragments(source, item, tier, slot, expert_cycle, cursor_pages, geom): + """Place one source extent in same-class slots, returning page-padded fragments.""" + slot_pages = 2 * geom.parallel_luns + remaining = source["nbytes"] + source_delta = 0 + fragments = [] + while remaining: + cycle_delta, within = divmod(cursor_pages, slot_pages) + room_pages = slot_pages - within + take = min(remaining, room_pages * geom.page_bytes) + allocated_pages = math.ceil(take / geom.page_bytes) + base = usable_cycle_base_lpn(expert_cycle + cycle_delta, geom) + target_lpn = base + slot * slot_pages + within + for page_delta in range(allocated_pages): + physical_page = ((target_lpn + page_delta) // geom.parallel_luns) % geom.pages_per_block + if qlc_class(physical_page) != slot: + raise AssertionError((item, tier, slot, target_lpn, physical_page)) + fragments.append(dict(source_offset=source["offset"] + source_delta, + source_nbytes=take, target_lpn=target_lpn, + target_offset=target_lpn * geom.page_bytes, + lba_start=target_lpn * geom.sectors_per_page, + sector_count=math.ceil(take / geom.sector_bytes), + allocated_pages=allocated_pages, + allocated_bytes=allocated_pages * geom.page_bytes, + page_class=slot)) + remaining -= take + source_delta += take + cursor_pages += allocated_pages + return fragments, cursor_pages + + +def make_extent_map(bundle, manifest, geom, policy="aligned"): + if manifest["schema"] != SCHEMA: + raise ValueError("Unsupported bundle schema") + if manifest["scale_mode"] != "shared_alpha4_prefix": + raise ValueError("v1 QLC mapper expects shared alpha4-prefix scales") + if policy not in PLACEMENT_POLICIES: + raise ValueError(f"Unknown placement policy: {policy}") + geom.validate() + groups = group_catalog(manifest) + slot_pages = 2 * geom.parallel_luns + entries = [] + seen = set() + cycles_per_expert = None + for expert_ordinal, ((layer, expert), projections) in enumerate(groups): + # All four tiers must consume identical page positions to preserve pairing. + layouts = [] + for tier in range(4): + sources = [] + for projection in PROJECTION_ORDER: + rec = projections[projection] + sources.append(("plane", rec, rec["planes"][tier])) + for projection in PROJECTION_ORDER: + rec = projections[projection] + sources.append(("scale", rec, rec["scales"]["4"][tier])) + layouts.append(sources) + allocated_shape = [sum(math.ceil(src["nbytes"] / geom.page_bytes) + for _, _, src in sources) for sources in layouts] + if len(set(allocated_shape)) != 1: + raise ValueError(f"Tier allocation mismatch for layer={layer}, expert={expert}") + need_cycles = math.ceil(allocated_shape[0] / slot_pages) + if cycles_per_expert is None: + cycles_per_expert = need_cycles + elif cycles_per_expert != need_cycles: + raise ValueError("v1 requires a uniform expert footprint") + expert_cycle = expert_ordinal * cycles_per_expert + for tier, sources in enumerate(layouts): + cursor = 0 + for kind, rec, src in sources: + index = tier + 1 + key = item_key(kind, rec["id"], index) + if key in seen: + raise ValueError(f"Duplicate item: {key}") + seen.add(key) + slot = slot_for(tier, expert_ordinal, policy) + fragments, cursor = target_fragments(src, key, tier, slot, + expert_cycle, cursor, geom) + entries.append(dict(item_id=key, kind=kind, layer=layer, + expert=expert, projection=rec["projection"], + projection_id=rec["id"], plane=(index if kind == "plane" else None), + scale_set=(4 if kind == "scale" else None), + column=(index if kind == "scale" else None), + source_file=src["file"], source_offset=src["offset"], + nbytes=src["nbytes"], source_sha256=src["sha256"], + dtype=src["dtype"], shape=src["shape"], tier=tier, + target_class=slot, + allocated_bytes=sum(f["allocated_bytes"] for f in fragments), + fragments=fragments)) + if cursor != allocated_shape[tier]: + raise AssertionError("Allocation cursor mismatch") + used_cycles = len(groups) * cycles_per_expert + cycles_per_block = (geom.pages_per_block - 8) // 8 + blocks_used = math.ceil(used_cycles / cycles_per_block) + image_pages = blocks_used * geom.line_pages + image_bytes = image_pages * geom.page_bytes + if image_bytes > geom.exposed_bytes_nominal: + raise ValueError(f"Layout needs {image_bytes} bytes, nominal exposed capacity is " + f"{geom.exposed_bytes_nominal} bytes") + data_bytes = sum(e["nbytes"] for e in entries) + allocated_bytes = sum(e["allocated_bytes"] for e in entries) + return dict(schema=LAYOUT_SCHEMA, policy="qlc_aligned_expert_plane_major", + placement_policy=policy, + placement_slots={f"B{t+1}": slot_for(t, 0, policy) for t in range(4)}, + mapping_contract="fresh FEMU; sequential full fill from LPN 0; no intervening writes", + scale_policy="alpha4 column j shares QLC class j-1 with Bj", + projection_order=list(PROJECTION_ORDER), geometry=asdict(geom), + cycles_per_block=cycles_per_block, cycles_per_expert=cycles_per_expert, + experts=len(groups), entries=entries, + totals=dict(image_pages=image_pages, image_bytes=image_bytes, + blocks_used=blocks_used, data_bytes=data_bytes, + page_allocated_data_bytes=allocated_bytes, + filler_bytes=image_bytes-data_bytes, + payload_fraction=data_bytes/image_bytes)) + + +def source_matches(read, entry): + expected = dict(file=entry["source_file"], offset=entry["source_offset"], + nbytes=entry["nbytes"], dtype=entry["dtype"], shape=entry["shape"], + sha256=entry["source_sha256"]) + return all(read.get(k) == v for k, v in expected.items()) + + +def coalesce_commands(mapped, max_io_bytes, group_id): + commands = [] + for frag in sorted(mapped, key=lambda x: x["lba_start"]): + if frag["nbytes"] % 512: + raise ValueError("Mapped fragment is not sector aligned") + if (commands and commands[-1]["lba_start"] + commands[-1]["sector_count"] == frag["lba_start"] + and commands[-1]["page_class"] == frag["page_class"] + and commands[-1]["nbytes"] + frag["nbytes"] <= max_io_bytes): + cmd = commands[-1] + cmd["sector_count"] += frag["sector_count"] + cmd["nbytes"] += frag["nbytes"] + if frag["item_id"] not in cmd["item_ids"]: + cmd["item_ids"].append(frag["item_id"]) + else: + commands.append(dict(command_id=f"{group_id}:{len(commands)}", + lba_start=frag["lba_start"], sector_count=frag["sector_count"], + nbytes=frag["nbytes"], page_class=frag["page_class"], + item_ids=[frag["item_id"]], operation="read")) + return commands + + +def map_layer_reads(layer_reads, output_path, extent_map, max_io_bytes): + catalog = {e["item_id"]: e for e in extent_map["entries"]} + groups = commands = command_bytes = items = fragments = 0 + with Path(layer_reads).open() as src, Path(output_path).open("x") as dst: + for line in src: + group = json.loads(line) + mapped = [] + for read in group["reads"]: + entry = catalog.get(read["item_id"]) + if entry is None or not source_matches(read, entry): + raise ValueError(f"Read does not match manifest: {read['item_id']}") + for fragment_index, frag in enumerate(entry["fragments"]): + mapped.append(dict(item_id=entry["item_id"], kind=entry["kind"], + projection_id=entry["projection_id"], plane=entry["plane"], + scale_set=entry["scale_set"], column=entry["column"], + fragment_index=fragment_index, source_file=entry["source_file"], + source_offset=frag["source_offset"], image_offset=frag["target_offset"], + nbytes=frag["source_nbytes"], lba_start=frag["lba_start"], + sector_count=frag["sector_count"], page_class=frag["page_class"], + operation="read")) + cmds = coalesce_commands(mapped, max_io_bytes, group["group_id"]) + out = {k: v for k, v in group.items() if k != "reads"} + out.update(schema=MAPPED_SCHEMA, mapped_fragments=mapped, commands=cmds, + mapped_fragment_count=len(mapped), command_count=len(cmds), + command_bytes=sum(c["nbytes"] for c in cmds)) + dst.write(json.dumps(out, separators=(",", ":")) + "\n") + groups += 1 + items += len(group["reads"]) + fragments += len(mapped) + commands += len(cmds) + command_bytes += sum(c["nbytes"] for c in cmds) + return dict(groups=groups, items=items, fragments=fragments, commands=commands, + command_bytes=command_bytes, max_io_bytes=max_io_bytes, + recommended_queue_depth=32) + + +def plan(bundle, layer_reads, output, geom, max_io_bytes, policy="aligned"): + bundle, layer_reads, output = Path(bundle), Path(layer_reads), Path(output) + output.mkdir(parents=True, exist_ok=False) + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text()) + extent_map = make_extent_map(bundle, manifest, geom, policy) + extent_map.update(source_manifest_sha256=sha256(manifest_path), + layout_spec_sha256=canonical_hash({k: v for k, v in extent_map.items() + if k != "entries"})) + extent_path = output / "extent_map.json" + write_json(extent_path, extent_map) + mapped_path = output / "mapped_reads.jsonl" + replay = map_layer_reads(layer_reads, mapped_path, extent_map, max_io_bytes) + try: + portable_reads = layer_reads.relative_to(bundle).as_posix() + except ValueError: + portable_reads = str(layer_reads) + summary = dict(schema=LAYOUT_SCHEMA, status="planned_not_materialized", + source_bundle=bundle.name, source_manifest_sha256=sha256(manifest_path), + source_layer_reads=portable_reads, source_layer_reads_sha256=sha256(layer_reads), + mapper_sha256=sha256(__file__), + extent_map_sha256=sha256(extent_path), mapped_reads_sha256=sha256(mapped_path), + policy=extent_map["policy"], placement_policy=policy, + placement_slots=extent_map["placement_slots"], + geometry=extent_map["geometry"], + totals=extent_map["totals"], replay_input=replay, + validation_required=["fresh device and zero host writes", "sequential fill from LPN 0", + "FEMU WRITE-log PPA/page-class match", "read-back byte equality"], + warning="pgs_per_blk=512 requires the FEMU QLC pairing rows-1 fix") + write_json(output / "layout_summary.json", summary) + return summary + + +def validate_layout(bundle, layer_reads, layout): + bundle, layer_reads, layout = Path(bundle), Path(layer_reads), Path(layout) + manifest_path = bundle / "manifest.json" + manifest = json.loads(manifest_path.read_text()) + extent_path, mapped_path = layout / "extent_map.json", layout / "mapped_reads.jsonl" + extent_map = json.loads(extent_path.read_text()) + summary = json.loads((layout / "layout_summary.json").read_text()) + if extent_map["schema"] != LAYOUT_SCHEMA or summary["schema"] != LAYOUT_SCHEMA: + raise ValueError("Layout schema mismatch") + if extent_map["source_manifest_sha256"] != sha256(manifest_path): + raise ValueError("Manifest checksum mismatch") + if summary["extent_map_sha256"] != sha256(extent_path): + raise ValueError("Extent-map checksum mismatch") + if summary["mapped_reads_sha256"] != sha256(mapped_path): + raise ValueError("Mapped-trace checksum mismatch") + if summary["source_layer_reads_sha256"] != sha256(layer_reads): + raise ValueError("Layer-read checksum mismatch") + geom = Geometry(**extent_map["geometry"]) + geom.validate() + + expected = {} + for rec in manifest["projections"]: + for index, source in enumerate(rec["planes"], 1): + expected[item_key("plane", rec["id"], index)] = source + for index, source in enumerate(rec["scales"]["4"], 1): + expected[item_key("scale", rec["id"], index)] = source + entries = {entry["item_id"]: entry for entry in extent_map["entries"]} + if len(entries) != len(extent_map["entries"]) or set(entries) != set(expected): + raise ValueError("Extent-map catalog is incomplete or duplicated") + + allocated = [] + class_allocated_pages = [0, 0, 0, 0] + data_bytes = allocated_bytes = 0 + for key, entry in entries.items(): + source = expected[key] + if not (entry["source_offset"] == source["offset"] + and entry["nbytes"] == source["nbytes"] + and entry["source_sha256"] == source["sha256"] + and entry["dtype"] == source["dtype"] + and entry["shape"] == source["shape"]): + raise ValueError(f"Source metadata mismatch: {key}") + cursor = source["offset"] + item_bytes = 0 + for frag in entry["fragments"]: + if frag["source_offset"] != cursor or frag["target_offset"] % geom.page_bytes: + raise ValueError(f"Fragment alignment/continuity mismatch: {key}") + if frag["lba_start"] * geom.sector_bytes != frag["target_offset"]: + raise ValueError(f"LBA mismatch: {key}") + if frag["sector_count"] * geom.sector_bytes != frag["source_nbytes"]: + raise ValueError(f"Sector count mismatch: {key}") + if frag["page_class"] != entry["target_class"]: + raise ValueError(f"Class metadata mismatch: {key}") + for page_delta in range(frag["allocated_pages"]): + lpn = frag["target_lpn"] + page_delta + page = (lpn // geom.parallel_luns) % geom.pages_per_block + if page < 8 or qlc_class(page) != entry["target_class"]: + raise ValueError(f"Physical QLC class mismatch: {key}, LPN {lpn}") + start = frag["target_offset"] + end = start + frag["allocated_bytes"] + allocated.append((start, end, key)) + class_allocated_pages[entry["target_class"]] += frag["allocated_pages"] + cursor += frag["source_nbytes"] + item_bytes += frag["source_nbytes"] + allocated_bytes += frag["allocated_bytes"] + if item_bytes != source["nbytes"]: + raise ValueError(f"Fragment byte coverage mismatch: {key}") + data_bytes += item_bytes + allocated.sort() + for previous, current in zip(allocated, allocated[1:]): + if previous[1] > current[0]: + raise ValueError(f"Overlapping targets: {previous[2]}, {current[2]}") + totals = extent_map["totals"] + if allocated and allocated[-1][1] > totals["image_bytes"]: + raise ValueError("Target exceeds image") + if (data_bytes != totals["data_bytes"] or allocated_bytes != totals["page_allocated_data_bytes"] + or totals["filler_bytes"] != totals["image_bytes"] - data_bytes): + raise ValueError("Layout byte totals mismatch") + if len(set(class_allocated_pages)) != 1: + raise ValueError("B/alpha tiers do not have symmetric allocated footprints") + + groups = original_items = mapped_fragments = commands = command_bytes = 0 + with layer_reads.open() as source, mapped_path.open() as mapped: + for original_line, mapped_line in zip_longest(source, mapped): + if original_line is None or mapped_line is None: + raise ValueError("Mapped trace group count mismatch") + original, result = json.loads(original_line), json.loads(mapped_line) + if result["schema"] != MAPPED_SCHEMA or result["group_id"] != original["group_id"]: + raise ValueError("Mapped group identity mismatch") + expected_fragments = [] + for read in original["reads"]: + entry = entries[read["item_id"]] + if not source_matches(read, entry): + raise ValueError(f"Trace source mismatch: {read['item_id']}") + for fragment_index, frag in enumerate(entry["fragments"]): + expected_fragments.append((entry["item_id"], fragment_index, + frag["lba_start"], frag["sector_count"], + frag["source_nbytes"], frag["page_class"])) + actual_fragments = [(x["item_id"], x["fragment_index"], x["lba_start"], + x["sector_count"], x["nbytes"], x["page_class"]) + for x in result["mapped_fragments"]] + if actual_fragments != expected_fragments: + raise ValueError(f"Mapped fragments mismatch in group {original['group_id']}") + if sum(c["nbytes"] for c in result["commands"]) != sum(r["nbytes"] for r in original["reads"]): + raise ValueError(f"Command byte mismatch in group {original['group_id']}") + for command in result["commands"]: + if command["nbytes"] != command["sector_count"] * geom.sector_bytes: + raise ValueError("Non-sector command") + groups += 1 + original_items += len(original["reads"]) + mapped_fragments += len(result["mapped_fragments"]) + commands += len(result["commands"]) + command_bytes += sum(c["nbytes"] for c in result["commands"]) + replay = summary["replay_input"] + observed = dict(groups=groups, items=original_items, fragments=mapped_fragments, + commands=commands, command_bytes=command_bytes) + if any(replay[k] != v for k, v in observed.items()): + raise ValueError("Mapped-trace totals mismatch") + report = dict(passed=True, schema=LAYOUT_SCHEMA, catalog_items=len(entries), + allocated_ranges=len(allocated), class_allocated_pages=class_allocated_pages, + mapped_trace=observed, + checks=["complete plane and scale catalog", "source metadata equality", + "fragment source coverage", "target non-overlap and image bound", + "patched-512 QLC class for every allocated page", "symmetric tier footprint", + "mapped group and fragment equality", "command byte conservation", + "source and output checksums"]) + write_json(layout / "layout_validation.json", report) + return report + + +def materialize(bundle, layout, image): + bundle, layout, image = Path(bundle), Path(layout), Path(image) + extent_map = json.loads((layout / "extent_map.json").read_text()) + manifest_path = bundle / "manifest.json" + if extent_map["source_manifest_sha256"] != sha256(manifest_path): + raise ValueError("Bundle manifest changed") + segments = [] + for entry in extent_map["entries"]: + for frag in entry["fragments"]: + segments.append((frag["target_offset"], frag["source_offset"], + frag["source_nbytes"], entry)) + segments.sort() + cursor = 0 + fill = b"\xA5" * (8 << 20) + image_hash = hashlib.sha256() + with image.open("xb") as dst: + for target, source, nbytes, entry in segments: + if target < cursor: + raise ValueError("Overlapping target extents") + gap = target - cursor + while gap: + block = fill[:min(gap, len(fill))] + dst.write(block); image_hash.update(block); gap -= len(block) + with (bundle / entry["source_file"]).open("rb") as src: + src.seek(source) + data = src.read(nbytes) + if len(data) != nbytes: + raise ValueError("Short source read") + dst.write(data); image_hash.update(data) + cursor = target + nbytes + total = extent_map["totals"]["image_bytes"] + while cursor < total: + block = fill[:min(total-cursor, len(fill))] + dst.write(block); image_hash.update(block); cursor += len(block) + report = dict(image=str(image), nbytes=cursor, sha256=image_hash.hexdigest(), + extent_map_sha256=sha256(layout / "extent_map.json")) + write_json(Path(str(image) + ".json"), report) + return report + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + pp = sub.add_parser("plan") + pp.add_argument("bundle", type=Path) + pp.add_argument("--layer-reads", type=Path, required=True) + pp.add_argument("--output", type=Path, required=True) + pp.add_argument("--max-io-bytes", type=int, default=4 << 20) + pp.add_argument("--policy", choices=PLACEMENT_POLICIES, default="aligned", + help="which QLC class each bit-plane tier is placed on; " + "the default reproduces the original layout byte for byte") + mp = sub.add_parser("materialize") + mp.add_argument("bundle", type=Path) + mp.add_argument("layout", type=Path) + mp.add_argument("--image", type=Path, required=True) + vp = sub.add_parser("validate") + vp.add_argument("bundle", type=Path) + vp.add_argument("--layer-reads", type=Path, required=True) + vp.add_argument("layout", type=Path) + args = parser.parse_args() + if args.command == "plan": + result = plan(args.bundle, args.layer_reads, args.output, Geometry(), + args.max_io_bytes, args.policy) + elif args.command == "materialize": + result = materialize(args.bundle, args.layout, args.image) + else: + result = validate_layout(args.bundle, args.layer_reads, args.layout) + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/replay_v1.c b/moe-harness/exp/moe_bcq/femu_handoff/packages/replay_v1.c new file mode 100644 index 00000000000..d974efdb24d --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/replay_v1.c @@ -0,0 +1,467 @@ +/* Replay a compiled routed-MoE trace against a preconditioned FEMU namespace. + * + * Groups are strict barriers. Within a group, variable-size O_DIRECT reads are + * kept at a rolling queue depth. The program does not fill the device: use the + * QLC layout materializer and a fresh sequential fill before running it. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define VERSION 1u +#define FEMU_ADM_FLIP 0xef +#define FEMU_RESET_QLC 8 +#define FEMU_SNAP_QLC 9 +#define FEMU_ACCUM_QLC_PREFILL 10 +#define FEMU_ACCUM_QLC_DECODE 11 +#define FEMU_ACCUM_QLC_TEACHER_FORCED 12 + +static const unsigned char MAGIC[8] = {'M','B','Q','R','P','L','1','\0'}; + +struct __attribute__((packed)) trace_header { + unsigned char magic[8]; + uint32_t version, header_bytes, sector_bytes, direct_alignment; + uint32_t default_qd, max_command_bytes, group_record_bytes, command_record_bytes; + uint64_t group_count, command_count, total_bytes, max_end_byte; + unsigned char extent_map_sha256[32], mapped_reads_sha256[32]; +}; + +struct __attribute__((packed)) group_record { + uint64_t group_id; + int64_t release_after_group_id; + uint64_t forward_id; + int32_t layer; + uint8_t phase, cache_reset; + uint16_t reserved; + uint32_t command_count, request_index; +}; + +struct __attribute__((packed)) command_record { + uint64_t lba_start; + uint32_t sector_count; + uint8_t page_class; + uint8_t reserved[3]; +}; + +_Static_assert(sizeof(struct trace_header) == 136, "trace header size"); +_Static_assert(sizeof(struct group_record) == 40, "group record size"); +_Static_assert(sizeof(struct command_record) == 16, "command record size"); + +struct slot { + struct iocb cb; + void *buffer; + struct command_record command; + uint32_t local_index; + uint64_t submit_ns; + bool active; +}; + +static void usage(const char *program) +{ + fprintf(stderr, + "usage:\n" + " %s --trace TRACE --dry-run\n" + " %s --trace TRACE --device /dev/nvme0n1 --controller /dev/nvme0\n" + " --group-log GROUPS.jsonl --summary RUN.json [--io-log IOS.jsonl]\n" + " [--qd 32] [--skip-qlc-counters] [--buffered]\n", + program, program); +} + +static void fail(const char *message) +{ + if (errno) fprintf(stderr, "fatal: %s: %s\n", message, strerror(errno)); + else fprintf(stderr, "fatal: %s\n", message); + exit(1); +} + +static void read_exact(FILE *stream, void *buffer, size_t size, const char *what) +{ + if (fread(buffer, 1, size, stream) != size) { + errno = 0; + fprintf(stderr, "fatal: truncated %s\n", what); + exit(1); + } +} + +static uint64_t now_ns(void) +{ + struct timespec value; + if (clock_gettime(CLOCK_MONOTONIC_RAW, &value)) fail("clock_gettime"); + return (uint64_t)value.tv_sec * 1000000000ull + (uint64_t)value.tv_nsec; +} + +static const char *phase_name(uint8_t phase) +{ + static const char *names[] = {"prefill", "decode", "teacher_forced"}; + return phase < 3 ? names[phase] : "invalid"; +} + +static int qlc_accum_command(uint8_t phase) +{ + static const int commands[] = { + FEMU_ACCUM_QLC_PREFILL, + FEMU_ACCUM_QLC_DECODE, + FEMU_ACCUM_QLC_TEACHER_FORCED, + }; + if (phase >= sizeof commands / sizeof commands[0]) { + errno = 0; + fail("invalid phase for QLC accounting"); + } + return commands[phase]; +} + +static FILE *open_exclusive(const char *path) +{ + int fd = open(path, O_WRONLY | O_CREAT | O_EXCL, 0644); + if (fd < 0) fail(path); + FILE *stream = fdopen(fd, "w"); + if (!stream) fail("fdopen"); + return stream; +} + +static int aio_setup_sys(unsigned entries, aio_context_t *context) +{ + return (int)syscall(__NR_io_setup, entries, context); +} +static int aio_submit_sys(aio_context_t context, long nr, struct iocb **iocbs) +{ + return (int)syscall(__NR_io_submit, context, nr, iocbs); +} +static int aio_getevents_sys(aio_context_t context, long min_nr, long nr, + struct io_event *events) +{ + return (int)syscall(__NR_io_getevents, context, min_nr, nr, events, NULL); +} +static int aio_destroy_sys(aio_context_t context) +{ + return (int)syscall(__NR_io_destroy, context); +} + +static void hash_hex(const unsigned char hash[32], char output[65]) +{ + static const char digits[] = "0123456789abcdef"; + for (int i = 0; i < 32; i++) { + output[2*i] = digits[hash[i] >> 4]; + output[2*i+1] = digits[hash[i] & 15]; + } + output[64] = 0; +} + +static int femu_flip(const char *controller, uint32_t selector) +{ + int fd = open(controller, O_RDONLY); + if (fd < 0) return -1; + struct nvme_admin_cmd command; + memset(&command, 0, sizeof command); + command.opcode = FEMU_ADM_FLIP; + command.cdw10 = selector; + int result = ioctl(fd, NVME_IOCTL_ADMIN_CMD, &command); + int saved = errno; + close(fd); + errno = saved; + return result; +} + +static struct trace_header read_header(FILE *trace) +{ + struct trace_header header; + read_exact(trace, &header, sizeof header, "trace header"); + uint16_t endian = 1; + if (*(unsigned char *)&endian != 1) { errno = 0; fail("little-endian host required"); } + if (memcmp(header.magic, MAGIC, 8) || header.version != VERSION + || header.header_bytes != sizeof header + || header.group_record_bytes != sizeof(struct group_record) + || header.command_record_bytes != sizeof(struct command_record) + || header.sector_bytes != 512 || header.direct_alignment < 512 + || !header.default_qd || !header.max_command_bytes) { + errno = 0; fail("binary trace header mismatch"); + } + return header; +} + +static uint32_t preflight(FILE *trace, const struct trace_header *header) +{ + uint64_t commands = 0, bytes = 0, max_end = 0; + int64_t previous = -1; + uint32_t max_group = 0; + for (uint64_t expected = 0; expected < header->group_count; expected++) { + struct group_record group; + read_exact(trace, &group, sizeof group, "group record"); + if (group.group_id != expected || group.phase > 2 || group.reserved + || (group.cache_reset && group.release_after_group_id != -1) + || (!group.cache_reset && group.release_after_group_id != previous)) { + errno = 0; fail("group ordering/dependency mismatch"); + } + if (group.command_count > max_group) max_group = group.command_count; + for (uint32_t index = 0; index < group.command_count; index++) { + struct command_record command; + read_exact(trace, &command, sizeof command, "command record"); + if (command.lba_start > UINT64_MAX / header->sector_bytes) { + errno = 0; fail("command offset overflow"); + } + uint64_t size = (uint64_t)command.sector_count * header->sector_bytes; + uint64_t offset = command.lba_start * header->sector_bytes; + if (!command.sector_count || command.page_class > 3 + || size > header->max_command_bytes + || offset % header->direct_alignment || size % header->direct_alignment + || command.reserved[0] || command.reserved[1] || command.reserved[2]) { + errno = 0; fail("invalid command record"); + } + if (offset > UINT64_MAX - size || bytes > UINT64_MAX - size) { + errno = 0; fail("command total overflow"); + } + commands++; + bytes += size; + if (offset + size > max_end) max_end = offset + size; + } + previous = (int64_t)group.group_id; + } + if (fgetc(trace) != EOF || commands != header->command_count + || bytes != header->total_bytes || max_end != header->max_end_byte) { + errno = 0; fail("trace length/totals mismatch"); + } + return max_group; +} + +static int free_slot(struct slot *slots, uint32_t qd) +{ + for (uint32_t i = 0; i < qd; i++) if (!slots[i].active) return (int)i; + return -1; +} + +int main(int argc, char **argv) +{ + const char *trace_path = NULL, *device = NULL, *controller = NULL; + const char *group_path = NULL, *io_path = NULL, *summary_path = NULL; + uint32_t qd_override = 0; + bool dry_run = false, skip_counters = false, buffered = false; + for (int i = 1; i < argc; i++) { + if (!strcmp(argv[i], "--trace") && ++i < argc) trace_path = argv[i]; + else if (!strcmp(argv[i], "--device") && ++i < argc) device = argv[i]; + else if (!strcmp(argv[i], "--controller") && ++i < argc) controller = argv[i]; + else if (!strcmp(argv[i], "--group-log") && ++i < argc) group_path = argv[i]; + else if (!strcmp(argv[i], "--io-log") && ++i < argc) io_path = argv[i]; + else if (!strcmp(argv[i], "--summary") && ++i < argc) summary_path = argv[i]; + else if (!strcmp(argv[i], "--qd") && ++i < argc) qd_override = (uint32_t)strtoul(argv[i], NULL, 10); + else if (!strcmp(argv[i], "--dry-run")) dry_run = true; + else if (!strcmp(argv[i], "--skip-qlc-counters")) skip_counters = true; + else if (!strcmp(argv[i], "--buffered")) buffered = true; + else { usage(argv[0]); return 2; } + } + if (!trace_path || (!dry_run && (!device || !group_path || !summary_path + || (!skip_counters && !controller)))) { + usage(argv[0]); return 2; + } + FILE *trace = fopen(trace_path, "rb"); + if (!trace) fail(trace_path); + struct trace_header header = read_header(trace); + uint32_t max_group = preflight(trace, &header); + uint32_t qd = qd_override ? qd_override : header.default_qd; + if (!qd || qd > 4096) { errno = 0; fail("queue depth must be in [1, 4096]"); } + char layout_hash[65], mapped_hash[65]; + hash_hex(header.extent_map_sha256, layout_hash); + hash_hex(header.mapped_reads_sha256, mapped_hash); + if (dry_run) { + printf("trace valid: groups=%" PRIu64 " commands=%" PRIu64 + " bytes=%" PRIu64 " max_group=%u qd=%u max_end=%" PRIu64 "\n", + header.group_count, header.command_count, header.total_bytes, + max_group, qd, header.max_end_byte); + fclose(trace); return 0; + } + + int flags = O_RDONLY | (buffered ? 0 : O_DIRECT); + int device_fd = open(device, flags); + if (device_fd < 0) fail(device); + uint64_t device_bytes = 0; + if (ioctl(device_fd, BLKGETSIZE64, &device_bytes)) { + struct stat st; + if (fstat(device_fd, &st) || !S_ISREG(st.st_mode)) fail("BLKGETSIZE64/fstat"); + device_bytes = (uint64_t)st.st_size; + } + if (header.max_end_byte > device_bytes) { errno = 0; fail("trace exceeds device size"); } + + struct slot *slots = calloc(qd, sizeof *slots); + struct io_event *events = calloc(qd, sizeof *events); + if (!slots || !events) fail("allocate queue state"); + for (uint32_t i = 0; i < qd; i++) { + int rc = posix_memalign(&slots[i].buffer, header.direct_alignment, + header.max_command_bytes); + if (rc) { errno = rc; fail("posix_memalign"); } + } + aio_context_t context = 0; + if (aio_setup_sys(qd, &context)) fail("io_setup"); + FILE *group_log = open_exclusive(group_path); + FILE *io_log = io_path ? open_exclusive(io_path) : NULL; + FILE *run_summary = open_exclusive(summary_path); + /* The complete group log is below 1 MiB for the supplied traces. Keep it + * out of the timed groups and flush after replay. Per-I/O logging remains + * a diagnostic mode whose overhead is explicitly visible. */ + if (setvbuf(group_log, NULL, _IOFBF, 4u << 20)) fail("setvbuf group log"); + if (setvbuf(run_summary, NULL, _IOFBF, 4096)) fail("setvbuf summary"); + + if (!skip_counters) { + if (!controller || femu_flip(controller, FEMU_RESET_QLC)) + fail("FEMU QLC counter reset"); + } + rewind(trace); + (void)read_header(trace); + uint64_t origin = now_ns(), sum_group_io = 0; + uint64_t observed_commands = 0, observed_bytes = 0; + uint8_t counter_phase = UINT8_MAX; + for (uint64_t expected = 0; expected < header.group_count; expected++) { + struct group_record group; + read_exact(trace, &group, sizeof group, "group record during replay"); + /* Every preceding group is complete here. Attribute its physical NAND + * counter delta before the next phase starts; requests may alternate + * prefill/decode multiple times in one compiled trace. */ + if (!skip_counters && counter_phase != UINT8_MAX + && group.phase != counter_phase) { + if (femu_flip(controller, qlc_accum_command(counter_phase))) + fail("FEMU QLC phase accumulation"); + } + counter_phase = group.phase; + struct command_record *commands = NULL; + if (group.command_count) { + commands = malloc((size_t)group.command_count * sizeof *commands); + if (!commands) fail("allocate group commands"); + read_exact(trace, commands, (size_t)group.command_count * sizeof *commands, + "group commands during replay"); + } + uint64_t ready = now_ns(), first_submit = 0, last_submit = 0, last_complete = ready; + uint64_t group_requested_bytes = 0; + for (uint32_t index = 0; index < group.command_count; index++) + group_requested_bytes += + (uint64_t)commands[index].sector_count * header.sector_bytes; + uint32_t next = 0, completed = 0, in_flight = 0, peak = 0; + while (completed < group.command_count) { + while (next < group.command_count && in_flight < qd) { + int index = free_slot(slots, qd); + if (index < 0) { errno = 0; fail("queue bookkeeping"); } + struct slot *slot = &slots[index]; + slot->command = commands[next]; + slot->local_index = next; + memset(&slot->cb, 0, sizeof slot->cb); + slot->cb.aio_data = (uint64_t)index + 1; + slot->cb.aio_lio_opcode = IOCB_CMD_PREAD; + slot->cb.aio_fildes = (uint32_t)device_fd; + slot->cb.aio_buf = (uint64_t)(uintptr_t)slot->buffer; + slot->cb.aio_nbytes = (uint64_t)slot->command.sector_count * header.sector_bytes; + slot->cb.aio_offset = (int64_t)(slot->command.lba_start * header.sector_bytes); + struct iocb *pointer = &slot->cb; + slot->submit_ns = now_ns(); + int submitted; + do submitted = aio_submit_sys(context, 1, &pointer); while (submitted < 0 && errno == EINTR); + if (submitted != 1) fail("io_submit"); + slot->active = true; + if (!first_submit) first_submit = slot->submit_ns; + last_submit = now_ns(); + next++; in_flight++; + if (in_flight > peak) peak = in_flight; + } + int count; + do count = aio_getevents_sys(context, 1, qd, events); while (count < 0 && errno == EINTR); + if (count <= 0) fail("io_getevents"); + uint64_t observed = now_ns(); + for (int i = 0; i < count; i++) { + if (!events[i].data || events[i].data > qd) { errno = 0; fail("invalid AIO user data"); } + struct slot *slot = &slots[events[i].data - 1]; + uint64_t expected_bytes = (uint64_t)slot->command.sector_count * header.sector_bytes; + if (!slot->active || events[i].res != (int64_t)expected_bytes || events[i].res2) { + errno = events[i].res < 0 ? (int)-events[i].res : 0; + fail("asynchronous read completion"); + } + if (io_log) fprintf(io_log, + "{\"group_id\":%" PRIu64 ",\"command_index\":%u," + "\"lba_start\":%" PRIu64 ",\"sector_count\":%u,\"page_class\":%u," + "\"submit_ns\":%" PRIu64 ",\"completion_observed_ns\":%" PRIu64 "," + "\"observed_latency_ns\":%" PRIu64 "}\n", + group.group_id, slot->local_index, slot->command.lba_start, + slot->command.sector_count, slot->command.page_class, + slot->submit_ns-origin, observed-origin, observed-slot->submit_ns); + slot->active = false; + in_flight--; completed++; + observed_commands++; + observed_bytes += expected_bytes; + } + last_complete = observed; + } + uint64_t group_io = first_submit ? last_complete - first_submit : 0; + sum_group_io += group_io; + if (group.command_count) { + fprintf(group_log, + "{\"group_id\":%" PRIu64 ",\"request_index\":%u,\"forward_id\":%" PRIu64 + ",\"layer\":%d,\"phase\":\"%s\",\"cache_reset\":%s," + "\"release_after_group_id\":%" PRId64 ",\"command_count\":%u," + "\"requested_bytes\":%" PRIu64 ",\"peak_outstanding\":%u," + "\"group_ready_ns\":%" PRIu64 ",\"first_submit_ns\":%" PRIu64 "," + "\"last_submit_ns\":%" PRIu64 ",\"last_complete_ns\":%" PRIu64 "," + "\"group_io_ns\":%" PRIu64 "}\n", + group.group_id, group.request_index, group.forward_id, group.layer, + phase_name(group.phase), group.cache_reset ? "true" : "false", + group.release_after_group_id, group.command_count, + group_requested_bytes, + peak, ready-origin, first_submit-origin, last_submit-origin, + last_complete-origin, group_io); + } else { + fprintf(group_log, + "{\"group_id\":%" PRIu64 ",\"request_index\":%u,\"forward_id\":%" PRIu64 + ",\"layer\":%d,\"phase\":\"%s\",\"cache_reset\":%s," + "\"release_after_group_id\":%" PRId64 ",\"command_count\":0," + "\"requested_bytes\":0,\"peak_outstanding\":0," + "\"group_ready_ns\":%" PRIu64 ",\"first_submit_ns\":null," + "\"last_submit_ns\":null,\"last_complete_ns\":%" PRIu64 "," + "\"group_io_ns\":0}\n", + group.group_id, group.request_index, group.forward_id, group.layer, + phase_name(group.phase), group.cache_reset ? "true" : "false", + group.release_after_group_id, ready-origin, ready-origin); + } + free(commands); + } + uint64_t wall = now_ns() - origin; + if (observed_commands != header.command_count || observed_bytes != header.total_bytes) { + errno = 0; fail("replay totals mismatch"); + } + fflush(group_log); if (io_log) fflush(io_log); + if (!skip_counters) { + if (counter_phase != UINT8_MAX + && femu_flip(controller, qlc_accum_command(counter_phase))) + fail("FEMU final QLC phase accumulation"); + if (femu_flip(controller, FEMU_SNAP_QLC)) + fail("FEMU QLC counter snapshot"); + } + fprintf(run_summary, + "{\n \"schema\": \"moe-bcq-replay-result-v1\",\n" + " \"queue_depth\": %u,\n \"direct_io\": %s,\n" + " \"qlc_counters\": \"%s\",\n" + " \"groups\": %" PRIu64 ",\n \"commands\": %" PRIu64 ",\n" + " \"requested_bytes\": %" PRIu64 ",\n" + " \"wall_ns_including_log_overhead\": %" PRIu64 ",\n" + " \"sum_group_io_ns\": %" PRIu64 ",\n" + " \"extent_map_sha256\": \"%s\",\n" + " \"mapped_reads_sha256\": \"%s\"\n}\n", + qd, buffered ? "false" : "true", + skip_counters ? "skipped" : "reset_phase_accumulated_and_snapshotted", + header.group_count, + observed_commands, observed_bytes, wall, sum_group_io, layout_hash, mapped_hash); + fflush(run_summary); + + fclose(run_summary); if (io_log) fclose(io_log); fclose(group_log); + if (aio_destroy_sys(context)) fail("io_destroy"); + for (uint32_t i = 0; i < qd; i++) free(slots[i].buffer); + free(events); free(slots); close(device_fd); fclose(trace); + return 0; +} diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py new file mode 100644 index 00000000000..19d8ad51a5f --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Compile mapped layer-read JSONL into a validated little-endian replay stream.""" +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import struct +import tempfile +from collections import Counter +from pathlib import Path + +from bundle import sha256, write_json + + +MAGIC = b"MBQRPL1\0" +VERSION = 1 +SECTOR_BYTES = 512 +DIRECT_ALIGNMENT = 4096 +PHASES = {"prefill": 0, "decode": 1, "teacher_forced": 2} +HEADER = struct.Struct("<8s8I4Q32s32s") +GROUP = struct.Struct(" (1 << 64) - 1 or sectors <= 0 + or sectors > (1 << 32) - 1): + raise ValueError(f"Invalid read command: {command}") + if nbytes != sectors * SECTOR_BYTES or nbytes > max_command_bytes: + raise ValueError(f"Invalid read size: {command['command_id']}") + if ((lba * SECTOR_BYTES) % DIRECT_ALIGNMENT + or nbytes % DIRECT_ALIGNMENT): + raise ValueError(f"O_DIRECT alignment violation: {command['command_id']}") + if page_class not in range(4): + raise ValueError(f"Invalid QLC class: {page_class}") + stream.write(COMMAND.pack(lba, sectors, page_class)) + group_bytes += nbytes + command_count += 1 + total_bytes += nbytes + max_end_byte = max(max_end_byte, (lba + sectors) * SECTOR_BYTES) + if group_bytes != record["command_bytes"]: + raise ValueError(f"Byte mismatch in group {gid}") + request_groups[request_id] += 1 + phase_groups[phase] += 1 + previous_group = gid + group_count += 1 + replay = summary["replay_input"] + if (group_count != replay["groups"] or command_count != replay["commands"] + or total_bytes != replay["command_bytes"]): + raise ValueError("Compiled totals differ from layout summary") + stream.seek(0) + stream.write(HEADER.pack(MAGIC, VERSION, HEADER.size, SECTOR_BYTES, + DIRECT_ALIGNMENT, queue_depth, max_command_bytes, GROUP.size, COMMAND.size, + group_count, command_count, total_bytes, max_end_byte, + raw_digest(summary["extent_map_sha256"]), raw_digest(summary["mapped_reads_sha256"]))) + stream.flush() + os.fsync(stream.fileno()) + os.chmod(temporary, 0o644) + os.link(temporary, output) + temporary.unlink() + temporary = None + finally: + if temporary is not None and temporary.exists(): + temporary.unlink() + + metadata = dict(schema="moe-bcq-replay-binary-v1", magic=MAGIC.rstrip(b"\0").decode(), + version=VERSION, byte_order="little", binary_file=output.name, + binary_sha256=sha256(output), binary_bytes=output.stat().st_size, + layout_directory=layout.name, layout_summary_sha256=sha256(summary_path), + layout_validation_sha256=sha256(validation_path), + extent_map_sha256=summary["extent_map_sha256"], + mapped_reads_sha256=summary["mapped_reads_sha256"], + sector_bytes=SECTOR_BYTES, direct_alignment=DIRECT_ALIGNMENT, + default_queue_depth=queue_depth, max_command_bytes=max_command_bytes, + record_bytes=dict(header=HEADER.size, group=GROUP.size, command=COMMAND.size), + totals=dict(groups=group_count, commands=command_count, + command_bytes=total_bytes, max_end_byte=max_end_byte), + phase_groups=dict(phase_groups), + requests=[dict(index=index, request_id=request_id, + groups=request_groups[request_id]) + for request_id, index in request_indices.items()], + semantics="Groups are barriers. Replayer uses rolling QD within a group; empty groups remain.") + write_json(Path(str(output) + ".json"), metadata) + return metadata + + +def inspect_trace(path): + path = Path(path) + with path.open("rb") as stream: + raw = stream.read(HEADER.size) + if len(raw) != HEADER.size: + raise ValueError("Truncated header") + values = HEADER.unpack(raw) + (magic, version, header_bytes, sector_bytes, alignment, default_qd, + max_command_bytes, group_bytes, command_bytes, groups, commands, + total_bytes, max_end_byte, layout_hash, mapped_hash) = values + if (magic != MAGIC or version != VERSION or header_bytes != HEADER.size + or group_bytes != GROUP.size or command_bytes != COMMAND.size + or sector_bytes != SECTOR_BYTES or alignment < sector_bytes + or not default_qd or not max_command_bytes): + raise ValueError("Binary header mismatch") + seen_commands = seen_bytes = seen_max_end = 0 + previous_group = -1 + for expected_group in range(groups): + raw = stream.read(GROUP.size) + if len(raw) != GROUP.size: + raise ValueError("Truncated group") + gid, release, forward, layer, phase, reset, reserved, count, request = GROUP.unpack(raw) + if (gid != expected_group or phase not in PHASES.values() or reserved + or (reset and release != -1) + or (not reset and release != previous_group)): + raise ValueError("Invalid group record") + for _ in range(count): + raw = stream.read(COMMAND.size) + if len(raw) != COMMAND.size: + raise ValueError("Truncated command") + lba, sectors, page_class = COMMAND.unpack(raw) + size, offset = sectors * sector_bytes, lba * sector_bytes + if (not sectors or page_class > 3 or size > max_command_bytes + or offset % alignment or size % alignment): + raise ValueError("Invalid command record") + seen_commands += 1 + seen_bytes += size + seen_max_end = max(seen_max_end, offset + size) + previous_group = gid + if (stream.read(1) or seen_commands != commands or seen_bytes != total_bytes + or seen_max_end != max_end_byte): + raise ValueError("Binary length/totals mismatch") + return dict(groups=groups, commands=commands, command_bytes=total_bytes, + max_end_byte=max_end_byte, default_queue_depth=default_qd, + max_command_bytes=max_command_bytes, sector_bytes=sector_bytes, + direct_alignment=alignment, extent_map_sha256=layout_hash.hex(), + mapped_reads_sha256=mapped_hash.hex(), binary_sha256=sha256(path)) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + cp = sub.add_parser("compile") + cp.add_argument("layout", type=Path) + cp.add_argument("--output", type=Path, required=True) + cp.add_argument("--queue-depth", type=int, default=32) + cp.add_argument("--max-command-bytes", type=int, default=4 << 20) + ip = sub.add_parser("inspect") + ip.add_argument("trace", type=Path) + args = parser.parse_args() + result = (compile_trace(args.layout, args.output, args.queue_depth, args.max_command_bytes) + if args.command == "compile" else inspect_trace(args.trace)) + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/moe-harness/exp/moe_bcq/femu_run/audit_handoff.py b/moe-harness/exp/moe_bcq/femu_run/audit_handoff.py new file mode 100644 index 00000000000..77b3cc4e6e3 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/audit_handoff.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +"""Read-only audit of the supplied Qwen binary and saved FEMU evidence. + +Run from the repository root. This does not boot FEMU or touch a device. +""" +import collections +import csv +import hashlib +import io +import json +from pathlib import Path +import re +import struct + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def audit(root): + layout = root / 'exp/moe_bcq/femu_handoff/packages/qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun' + runs = root / 'runs/femu' + binary = layout / 'replay_qd32.bin' + header = struct.Struct('<8s8I4Q32s32s') + group = struct.Struct(' ch=(\d+) lun=(\d+) pl=(\d+) blk=(\d+) pg=(\d+)') + rows = [tuple(map(int, m.groups())) for m in pattern.finditer(marker.read_text())] + # Absolute PPA equality is meaningful for the first fresh-device prefix. + prefix = [r for r in rows if r[0] < 4096] + prefix_mismatches = sum((ch, lun, pl, blk, pg) != + (lpn % 2, lpn // 2 % 4, 0, lpn // 4096, lpn // 8 % 512) + for lpn, ch, lun, pl, blk, pg in prefix) + assert len(prefix) == 4096 and prefix_mismatches == 0 + # Split into continuous LPN runs; these are observations, not inferred + # command boundaries. Later probes deliberately start at different LBAs. + sequences = [] + for row in rows: + if not sequences or row[0] not in (sequences[-1][-1][0], sequences[-1][-1][0] + 1): + sequences.append([]) + sequences[-1].append(row) + phys = lambda r: r[4] * 4096 + r[5] * 8 + r[2] * 2 + r[1] + sequence_reports = [] + for seq in sequences: + counts = collections.Counter(r[0] for r in seq) + extra = len(seq) - len(counts) + advance_extra = phys(seq[-1]) - phys(seq[0]) - (seq[-1][0] - seq[0][0]) + assert advance_extra == extra + sequence_reports.append(dict(first_lpn=seq[0][0], last_lpn=seq[-1][0], + write_records=len(seq), duplicate_programs=extra, + extra_physical_slots=advance_extra, + duplicate_lpn_mod32=dict(collections.Counter(k % 32 for k, v in counts.items() if v > 1)))) + + saved_counters = {} + for path in sorted(runs.glob('*qlc_counts.csv*')): + values = [0] * 4 + for row in csv.DictReader(io.StringIO(''.join( + line for line in path.read_text().splitlines(True) if not line.startswith('#')))): + values[int(row['page_class'])] += int(row['n_read']) + saved_counters[path.name] = dict(sha256=digest(path), pages=values, + total=sum(values), matches_replay_prediction=values == pages) + + return dict(binary=dict(sha256=digest(binary), groups=h[9], commands=commands, + host_read_bytes=byte_count, expected_nand_page_reads=pages, + total_nand_page_reads=sum(pages), incorrect_floor_counts=floor_pages, + command_size_histogram=dict(sorted(sizes.items())), max_command_bytes=max(sizes), + mapper_class_mismatches=class_mismatches, binary_matches_mapped_jsonl=True), + marker_log=dict(sha256=digest(marker), fresh_prefix_records=len(prefix), + fresh_prefix_ppa_mismatches=prefix_mismatches, sequences=sequence_reports), + saved_counters=saved_counters, + limitations=['No new FEMU run; audit uses saved files only.', + 'WRITE markers show program allocation, not NVMe command boundaries or Linux segment counts.', + '256 KiB fill still requires queue-limit and actual PPA verification on each configuration.']) + + +if __name__ == '__main__': + print(json.dumps(audit(Path.cwd()), indent=2)) diff --git a/moe-harness/exp/moe_bcq/femu_run/class_confusion.c b/moe-harness/exp/moe_bcq/femu_run/class_confusion.c new file mode 100644 index 00000000000..a5d14dbb2df --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/class_confusion.c @@ -0,0 +1,85 @@ +/* Read a chosen set of LPNs so FEMU's own per-class counters report which + * physical page classes they actually landed on. + * + * The latency probe infers a class from a measured time and can be fooled by + * host jitter. The device counts every physical page read by class itself, so + * reading only the LPNs a layout model calls class k turns the counter vector + * into that model's confusion row exactly, with no timing inference at all. + * + * Counters are reset here and snapshotted at the end, so the vector covers this + * program's reads and nothing else. The snapshot is written by FEMU to the path + * it was started with; this program cannot see it. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#define PGSZ (16 * 1024) +#define PGS_PER_BLK 512 +#define LPN_PER_PAGE_INDEX 8 /* nchs * luns_per_ch, the modelled stride */ +#define FEMU_FLIP_OPCODE 0xef +#define FEMU_RESET_QLC 8 +#define FEMU_SNAP_QLC 9 + +static int femu_flip(const char *ctrl, int selector) +{ + int fd = open(ctrl, O_RDONLY); + if (fd < 0) { perror(ctrl); return -1; } + struct nvme_admin_cmd cmd; + memset(&cmd, 0, sizeof cmd); + cmd.opcode = FEMU_FLIP_OPCODE; + cmd.cdw10 = selector; + int rc = ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd); + close(fd); + return rc; +} + +/* mirrors init_qlc_page_pairing() with the rows-1 fix */ +static int page_class(long pg) +{ + if (pg < 6) return 0; + if (pg < 8) return 1; + return (int)(((pg - 8) % 8) / 2); +} + +int main(int argc, char **argv) +{ + if (argc < 5) { + fprintf(stderr, "usage: %s DEV CTRL WANT_CLASS LPN_LIMIT\n" + " reads every LPN below LPN_LIMIT the model calls WANT_CLASS\n", + argv[0]); + return 2; + } + const char *dev = argv[1], *ctrl = argv[2]; + int want = atoi(argv[3]); + long limit = atol(argv[4]); + + int fd = open(dev, O_RDONLY | O_DIRECT); + if (fd < 0) { perror(dev); return 1; } + void *buf; + if (posix_memalign(&buf, 4096, PGSZ)) { perror("memalign"); return 1; } + + if (femu_flip(ctrl, FEMU_RESET_QLC)) { + fprintf(stderr, "fatal: QLC counter reset failed\n"); + return 1; + } + + long n = 0; + for (long lpn = 0; lpn < limit; lpn++) { + if (page_class((lpn / LPN_PER_PAGE_INDEX) % PGS_PER_BLK) != want) continue; + if (pread(fd, buf, PGSZ, (off_t)lpn * PGSZ) != PGSZ) { perror("pread"); return 1; } + n++; + } + + if (femu_flip(ctrl, FEMU_SNAP_QLC)) { + fprintf(stderr, "fatal: QLC counter snapshot failed\n"); + return 1; + } + printf("CONFUSION want_class=%d lpn_limit=%ld reads=%ld\n", want, limit, n); + return 0; +} diff --git a/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh b/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh new file mode 100755 index 00000000000..af40b409ee7 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh @@ -0,0 +1,127 @@ +#!/bin/bash +# Replay several traces against one already-filled device. +# +# A placement is a property of the image, not of the trace: re-filling the +# device for every trace would re-run out-of-place allocation and land the pages +# on different physical classes, so the fill has to happen once and every trace +# that shares that placement has to be replayed on it. replay_v1 resets the QLC +# counters when it starts and snapshots them when it ends, so consecutive +# replays on one device still report independent per-class totals. +# +# Two things from drive_run.sh carry over because they are what made earlier +# runs unrecoverable: FEMU rewrites one stats file per snapshot and the guest +# cannot read it, so the host copies it between steps; and replay_v1 refuses to +# overwrite its outputs, so every replay needs unique paths or it exits without +# running and leaves the previous counters in place, which reads as success. +set -uo pipefail + +TAG=${1:?usage: drive_multi.sh DEVICE_TAG SPECFILE} +SPEC=${2:?usage: drive_multi.sh DEVICE_TAG SPECFILE} +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +CSV=$ROOT/runs/femu/${TAG}_qlc.csv +RECORDS=$ROOT/exp/moe_bcq/femu_run/records +# -n on every ssh that is not being fed a file: without it ssh inherits the +# loop's stdin and swallows the rest of the spec, so the first replay runs and +# the loop then reads EOF and exits reporting success. SSHIN is the one that +# does take stdin, for streaming a binary in. +SSHOPT="-p ${SSH_PORT:-2222} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=10" +SSH="ssh -n $SSHOPT femu@127.0.0.1" +SSHIN="ssh $SSHOPT femu@127.0.0.1" + +echo "== device $TAG ==" +echo " queue: $($SSH 'cat /sys/block/nvme0n1/queue/max_segments /sys/block/nvme0n1/queue/max_sectors_kb' 2>/dev/null | tr '\n' ' ')" + +# ---- placement check, once per device ------------------------------------- +# Asserted, not printed. A drifted fill still produces a plausible counter row, +# and the whole reason the earlier runs went unnoticed is that a human had to +# spot the difference. The whole image, not a prefix: the pages past a partial +# range are exactly the ones that check would miss. +IMAGE_PAGES=${IMAGE_PAGES:?IMAGE_PAGES must be set (Qwen 471040, DeepSeek 544768)} +CONF=$ROOT/runs/femu/$TAG +mkdir -p "$CONF" +echo " class c0 c1 c2 c3" +for c in 0 1 2 3; do + before=$(stat -c %Y.%N "$CSV" 2>/dev/null || echo none) + $SSH "sudo /usr/local/bin/class_confusion /dev/nvme0n1 /dev/nvme0 $c $IMAGE_PAGES" \ + > "$CONF/confusion_c$c.stdout" 2>"$CONF/confusion_c$c.stderr" || { + echo " confusion_c$c FAILED"; tail -3 "$CONF/confusion_c$c.stderr" | sed 's/^/ /'; exit 1; } + for _ in $(seq 40); do + [ -f "$CSV" ] && [ "$(stat -c %Y.%N "$CSV" 2>/dev/null)" != "$before" ] && break + sleep 0.5 + done + cp "$CSV" "$CONF/confusion_c$c.csv" || exit 1 + read -r -a got <<<"$(grep -v '^#' "$CONF/confusion_c$c.csv" | grep -v page_class | cut -d, -f2 | tr '\n' ' ')" + printf ' confusion_c%-3s' "$c"; printf '%9d ' "${got[@]:0:4}"; printf '\n' + for k in 0 1 2 3; do + if { [ "$k" = "$c" ] && [ "${got[$k]}" -eq 0 ]; } || + { [ "$k" != "$c" ] && [ "${got[$k]}" -ne 0 ]; }; then + echo " FAIL class $c leaked into class $k -- the fill did not land as planned" + echo " counters: ${got[*]}"; exit 1 + fi + done +done +echo " confusion diagonal over $IMAGE_PAGES pages: clean" + +# ---- one replay per spec line --------------------------------------------- +rc=0 +mapfile -t SPEC_LINES < "$SPEC" +for LINE in "${SPEC_LINES[@]}"; do + read -r NAME BIN <<<"$LINE" + [ -n "${NAME:-}" ] || continue + case $NAME in \#*) continue;; esac + OUT=$RECORDS/$NAME + if [ -f "$OUT/replay.csv" ] && [ -f "$OUT/groups.jsonl.gz" ]; then + echo " -- $NAME already collected, skipping" + continue + fi + [ -f "$ROOT/$BIN" ] || { echo " -- $NAME MISSING binary $BIN"; rc=1; continue; } + mkdir -p "$OUT" + echo " -- $NAME ($(du -h "$ROOT/$BIN" | cut -f1))" + + # Fresh guest-side paths each time: replay_v1 opens its outputs O_EXCL, so a + # reused path makes it exit without replaying while the old counters stay + # put -- indistinguishable from a successful repeat unless rc is checked. + $SSH "rm -f /home/femu/current.bin /dev/shm/$NAME.*" >/dev/null 2>&1 + if ! $SSHIN "cat > /home/femu/current.bin" < "$ROOT/$BIN"; then + echo " binary transfer FAILED"; rc=1; continue + fi + want=$(stat -c %s "$ROOT/$BIN"); got=$($SSH "stat -c %s /home/femu/current.bin" 2>/dev/null) + [ "$want" = "$got" ] || { echo " binary truncated in transit ($got of $want)"; rc=1; continue; } + + before=$(stat -c %Y.%N "$CSV" 2>/dev/null || echo none) + if ! $SSH "sudo /usr/local/bin/replay_v1 --trace /home/femu/current.bin --device /dev/nvme0n1 \ + --controller /dev/nvme0 --qd 32 --group-log /dev/shm/$NAME.groups.jsonl \ + --summary /dev/shm/$NAME.summary.json && cat /dev/shm/$NAME.summary.json" \ + > "$OUT/replay.stdout" 2>"$OUT/replay.stderr"; then + echo " replay FAILED"; tail -3 "$OUT/replay.stderr" | sed 's/^/ /'; rc=1 + $SSH "rm -f /home/femu/current.bin /dev/shm/$NAME.*" >/dev/null 2>&1 + continue + fi + for _ in $(seq 60); do + [ -f "$CSV" ] && [ "$(stat -c %Y.%N "$CSV" 2>/dev/null)" != "$before" ] && break + sleep 0.5 + done + cp "$CSV" "$OUT/replay.csv" || { echo " counters not captured"; rc=1; } + if $SSH "sudo gzip -c /dev/shm/$NAME.groups.jsonl" > "$OUT/groups.jsonl.gz" 2>/dev/null && + gzip -t "$OUT/groups.jsonl.gz" 2>/dev/null; then + echo " group log $(zcat "$OUT/groups.jsonl.gz" | wc -l) lines" + else + echo " WARNING group log not retrieved -- per-phase timing unavailable"; rc=1 + fi + python3 - "$OUT" <<'PY' +import json, sys +from pathlib import Path +out = Path(sys.argv[1]) +s = json.loads((out/'replay.stdout').read_text()) +cls = [int(l.split(',')[1]) for l in (out/'replay.csv').read_text().splitlines() + if l[:1].isdigit()] +print(f" groups {s['groups']:,} commands {s['commands']:,} " + f"{s['requested_bytes']/2**30:.1f} GiB io {s['sum_group_io_ns']/1e9:.1f}s " + f"pages {sum(cls):,}") +PY + # tmpfs is small and the gsm8k logs are tens of MB; the record is on the host now. + $SSH "rm -f /home/femu/current.bin /dev/shm/$NAME.*" >/dev/null 2>&1 +done + +echo "== device $TAG done (rc=$rc) ==" +exit $rc diff --git a/moe-harness/exp/moe_bcq/femu_run/drive_run.sh b/moe-harness/exp/moe_bcq/femu_run/drive_run.sh new file mode 100755 index 00000000000..8a8643cf48d --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/drive_run.sh @@ -0,0 +1,120 @@ +#!/bin/bash +# Drive one measurement run from the host, preserving a counter snapshot per step. +# +# FEMU rewrites the same stats file on every snapshot and the guest cannot see +# that file, so a step's counters survive only if the host copies them before the +# next step runs. The previous run lost its replay-only snapshot exactly this way, +# leaving a reported result that could not be re-checked. +# +# Every guest command's exit status is checked. replay_v1 refuses to overwrite an +# existing output file, so a repeated run with the same paths exits non-zero and +# leaves the counters untouched -- which reads as a successful repeat unless the +# status is examined. +set -uo pipefail + +RUN=${1:?usage: drive_run.sh RUN_TAG} +# Derive the checkout root from this script rather than naming it, so the same +# script drives a run on whichever machine it was copied to. +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +CSV=$ROOT/runs/femu/${RUN}_qlc.csv +OUT=$ROOT/runs/femu/$RUN +SSH="ssh -p ${SSH_PORT:-2222} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=10 femu@127.0.0.1" + +mkdir -p "$OUT" + +# Wait for FEMU to rewrite the stats file, then keep it under a step-specific name. +keep() { + local label=$1 before=$2 + for _ in $(seq 40); do + [ -f "$CSV" ] && [ "$(stat -c %Y.%N "$CSV" 2>/dev/null)" != "$before" ] && break + sleep 0.5 + done + cp "$CSV" "$OUT/$label.csv" || return 1 + printf ' %-14s ' "$label" + grep -v '^#' "$OUT/$label.csv" | tail -4 | awk -F, '{printf "%9d ", $2} END {print ""}' +} + +step() { # step LABEL COMMAND... + local label=$1; shift + local before; before=$(stat -c %Y.%N "$CSV" 2>/dev/null || echo none) + if ! $SSH "$@" > "$OUT/$label.stdout" 2>"$OUT/$label.stderr"; then + echo " $label FAILED (rc=$?); see $OUT/$label.stderr" + tail -3 "$OUT/$label.stderr" | sed 's/^/ /' + return 1 + fi + keep "$label" "$before" +} + +echo "== $RUN ==" +echo " queue: $($SSH 'cat /sys/block/nvme0n1/queue/max_segments /sys/block/nvme0n1/queue/max_sectors_kb' 2>/dev/null | tr '\n' ' ')" +echo " class c0 c1 c2 c3" +# Assert the diagonal rather than print it. A drifted fill still produces a +# plausible-looking counter row, and the whole reason the earlier runs went +# unnoticed for so long is that a human had to spot the difference. Reading the +# whole image, not a prefix: pages past the checked range are exactly the ones a +# partial check would miss. +IMAGE_PAGES=${IMAGE_PAGES:-471040} +for c in 0 1 2 3; do + step "confusion_c$c" "sudo /usr/local/bin/class_confusion /dev/nvme0n1 /dev/nvme0 $c $IMAGE_PAGES" || exit 1 + read -r -a got <<<"$(grep -v '^#' "$OUT/confusion_c$c.csv" | grep -v page_class | cut -d, -f2 | tr '\n' ' ')" + for k in 0 1 2 3; do + if { [ "$k" = "$c" ] && [ "${got[$k]}" -eq 0 ]; } || + { [ "$k" != "$c" ] && [ "${got[$k]}" -ne 0 ]; }; then + echo " FAIL class $c leaked into class $k — the fill did not land as planned" + echo " counters: ${got[*]}" + exit 1 + fi + done +done +echo " confusion diagonal over $IMAGE_PAGES pages: clean" +step replay "sudo /usr/local/bin/replay_v1 --trace /root/replay.bin --device /dev/nvme0n1 \ + --controller /dev/nvme0 --qd 32 --group-log /dev/shm/$RUN.groups.jsonl \ + --summary /dev/shm/$RUN.summary.json && cat /dev/shm/$RUN.summary.json" || exit 1 +# Phase-aware counters are optional. The replayer announces each phase change +# with an admin FLIP (selectors 10-12), but FEMU's bb_flip only implements 8 and +# 9, so those announcements currently fall through to default and no phase +# columns are produced. That is not a reason to throw the run away: NAND energy +# is exactly linear in per-class page count, and compose_e2e.py splits it on the +# host from the page_class each compiled command carries, checking that its +# per-class totals equal the ones below. If a rebuilt FEMU does emit the +# columns, they are checked here and become a second, independent split. +python3 - "$OUT/replay.csv" <<'PY' || exit 1 +import csv +import sys + +with open(sys.argv[1], encoding="utf-8") as source: + rows = list(csv.DictReader(line for line in source if not line.startswith("#"))) +if len(rows) != 4: + raise SystemExit(f"expected 4 page-class rows, found {len(rows)}") +required = { + "prefill_n_read", "prefill_bytes_read", + "decode_n_read", "decode_bytes_read", + "teacher_forced_n_read", "teacher_forced_bytes_read", +} +if not required.issubset(rows[0]): + print(" phase counters: absent (FEMU bb_flip lacks selectors 10-12); " + "the prefill/decode split comes from the compiled commands on the host") + raise SystemExit(0) +for row in rows: + for suffix in ("n_read", "bytes_read"): + total = int(row[suffix]) + phases = sum(int(row[f"{phase}_{suffix}"]) + for phase in ("prefill", "decode", "teacher_forced")) + if total != phases: + raise SystemExit( + f"class {row['page_class']} {suffix}: total {total} != phase sum {phases}") +if not any(int(row["prefill_n_read"]) for row in rows): + raise SystemExit("prefill physical-read counter is empty") +if not any(int(row["decode_n_read"]) for row in rows): + raise SystemExit("decode physical-read counter is empty") +print(" phase counter closure: total = prefill + decode + teacher-forced") +PY +# The group log carries per-phase timing and is the only record of it; the guest +# writes it to tmpfs, so it dies with the container unless it is pulled here. +if $SSH "sudo gzip -c /dev/shm/$RUN.groups.jsonl" > "$OUT/groups.jsonl.gz" 2>/dev/null && + gunzip -f -k "$OUT/groups.jsonl.gz" 2>/dev/null; then + echo " group log $(wc -l < "$OUT/groups.jsonl") lines" +else + echo " WARNING group log not retrieved; per-phase timing will be unavailable" +fi +echo " saved under $OUT/" diff --git a/moe-harness/exp/moe_bcq/femu_run/guest_bringup.sh b/moe-harness/exp/moe_bcq/femu_run/guest_bringup.sh new file mode 100755 index 00000000000..f710b54ed11 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/guest_bringup.sh @@ -0,0 +1,19 @@ +#!/bin/bash +# Bring the guest to a usable state and stop. Everything else runs over SSH. +# +# The serial console is one-shot: cloud-init runs what the seed says and then the +# machine just sits there. Putting an experiment in the seed therefore costs a +# boot and a device fill per question. This script only prepares the device and +# gets out of the way. +set -uo pipefail +exec 2>&1 +Q=/sys/block/nvme0n1/queue +echo "[guest] kernel $(uname -r)" +echo "[guest] nvme $(lsblk -dno NAME,SIZE /dev/nvme0n1 2>/dev/null)" +cat "$Q/max_hw_sectors_kb" > "$Q/max_sectors_kb" 2>/dev/null || true +echo none > "$Q/scheduler" 2>/dev/null || true +echo "[guest] max_sectors_kb $(cat $Q/max_sectors_kb), scheduler $(cat $Q/scheduler)" +for d in /dev/vd?; do + [ -b "$d" ] && echo "[guest] virtio $d $(blockdev --getsize64 "$d") bytes" +done +echo "[guest] READY, device NOT filled; drive the rest over ssh" diff --git a/moe-harness/exp/moe_bcq/femu_run/guest_replay.sh b/moe-harness/exp/moe_bcq/femu_run/guest_replay.sh new file mode 100755 index 00000000000..5221733f54d --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/guest_replay.sh @@ -0,0 +1,91 @@ +#!/bin/bash +# Guest side of one QLC-aligned replay: load the payload, replay the trace, +# and hand everything back over the serial console. +# +# The payload arrives as a read-only virtio disk. The root disk is virtio-scsi (/dev/sda), +# so the only virtio-blk disks are the cloud-init seed and the payload, and the +# payload is by far the larger -- picked by size rather than by enumeration +# order, which is not something to bet a 7 GiB dd on. +set -euo pipefail +exec 2>&1 + +SRC="" +best=0 +for d in /dev/vd?; do + [ -b "$d" ] || continue + sz=$(blockdev --getsize64 "$d" 2>/dev/null || echo 0) + echo "[guest] virtio disk $d $sz bytes" + if [ "$sz" -gt "$best" ]; then best=$sz; SRC=$d; fi +done +DEV=/dev/nvme0n1 +CTRL=/dev/nvme0 +TRACE=/root/replay.bin +OUT=/dev/shm + +say() { echo "[guest] $*"; } + +say "kernel $(uname -r)" +say "nvme: $(lsblk -dno NAME,SIZE /dev/nvme0n1 2>/dev/null || echo MISSING)" +[ -b "$DEV" ] || { say "FATAL no $DEV"; exit 1; } +[ -n "$SRC" ] && [ -b "$SRC" ] || { say "FATAL no payload disk found"; lsblk; exit 1; } +say "payload disk is $SRC" + +# 4 MiB is the compiler's cap; current Qwen replay commands are <=256 KiB. +# max_sectors_kb is only one split limit. Record segment limits as well; +# a userspace AIO command count need not equal the NVMe command count. +Q=/sys/block/nvme0n1/queue +say "max_hw_sectors_kb=$(cat $Q/max_hw_sectors_kb) max_sectors_kb=$(cat $Q/max_sectors_kb)" +cat "$Q/max_hw_sectors_kb" > "$Q/max_sectors_kb" 2>/dev/null || true +echo none > "$Q/scheduler" 2>/dev/null || true +say "max_sectors_kb now $(cat $Q/max_sectors_kb), scheduler $(cat $Q/scheduler)" +say "memory_page_bytes=$(getconf PAGESIZE) max_segments=$(cat $Q/max_segments) max_segment_size=$(cat $Q/max_segment_size)" +if [ "$(cat $Q/max_sectors_kb)" -lt 256 ]; then + say "WARNING below the 256 KiB largest replay command; the device will see more commands than the trace has" +fi + +IMG_BYTES=$(blockdev --getsize64 "$SRC") +say "payload disk $IMG_BYTES bytes" +[ "$IMG_BYTES" -le "$(blockdev --getsize64 "$DEV")" ] || { say "FATAL payload exceeds namespace"; exit 1; } +[ "$((IMG_BYTES % 16384))" -eq 0 ] || { say "FATAL image is not NAND-page aligned"; exit 1; } + +# Leave one memory-page segment for a potentially unaligned userspace buffer. +# Fail rather than silently returning to a fill size that can split mid-page. +MEM_PAGE=$(getconf PAGESIZE) +SEGMENTS=$(cat "$Q/max_segments") +SEG_SIZE=$(cat "$Q/max_segment_size") +MAX_KB=$(cat "$Q/max_sectors_kb") +HW_KB=$(cat "$Q/max_hw_sectors_kb") +[ "$SEG_SIZE" -ge "$MEM_PAGE" ] && [ "$SEGMENTS" -ge "$((262144 / MEM_PAGE + 1))" ] && + [ "$MAX_KB" -ge 256 ] && [ "$HW_KB" -ge 256 ] || { + say "FATAL queue limits do not satisfy the 256 KiB fill contract"; exit 1; +} + +say "=== fill: sequential write from LBA 0 ===" +t0=$(date +%s.%N) +# bs is capped well under the queue's max_segments (127) on purpose. A larger +# O_DIRECT write is split by the block layer at a 127-segment = 508 KiB boundary, +# which falls in the middle of the 32nd 16 KiB flash page. That page then belongs +# to both fragments, so it is written twice, and out-of-place update spends an +# extra physical page -- shifting every later page one slot along and scrambling +# the QLC class the layout counted on. With the observed 4 KiB pages and +# max_segments=127, a page-aligned 256 KiB buffer needs at most 64 page segments. +# This is configuration-dependent: PPA verification is still required. +dd if="$SRC" of="$DEV" bs=256k iflag=fullblock oflag=direct conv=fsync status=none || { say "FATAL fill"; exit 1; } +t1=$(date +%s.%N) +say "fill done in $(echo "$t1 - $t0" | bc) s" + +say "=== read-back verification ===" +SRC_SHA=$(dd if="$SRC" bs=4M iflag=direct,fullblock status=none | sha256sum | cut -d' ' -f1) +DST_SHA=$(dd if="$DEV" bs=4M count="$IMG_BYTES" iflag=direct,fullblock,count_bytes status=none | sha256sum | cut -d' ' -f1) +say "payload sha256 $SRC_SHA" +say "device sha256 $DST_SHA" +[ "$SRC_SHA" = "$DST_SHA" ] && say "read-back OK" || { say "FATAL read-back mismatch"; exit 1; } + +# The replay's class prediction is only as good as the assumed LPN->PPA order. +# Read it back off the clock before trusting it: two windows, one at the start of +# the image and one deep inside it, so a pattern that drifts is visible. +# Stop here. Everything past the fill is driven over ssh instead, one step at a +# time, so the QLC counter file can be read on the host between steps -- the +# guest cannot see that file, and each snapshot overwrites the last. +say "device is filled and verified; driving the rest over ssh" +say "ALL DONE" diff --git a/moe-harness/exp/moe_bcq/femu_run/mark_write.c b/moe-harness/exp/moe_bcq/femu_run/mark_write.c new file mode 100644 index 00000000000..fb9700f8dde --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/mark_write.c @@ -0,0 +1,58 @@ +/* Write marker-bearing pages so FEMU logs the real LPN -> PPA for each one. + * + * Every attempt to predict the physical page class from the LPN has been a model + * of the write pointer, and the counters say the model is wrong. FEMU will name + * the physical address itself for any page whose content carries the marker + * string it was started with, so write that string into each page and read the + * mapping out of the emulator's log rather than inferring it. + * + * Write sequentially from LPN 0 on a fresh device: that is exactly the fill the + * layout assumes, so the logged addresses are the ones the layout would get. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include + +#define PGSZ (16 * 1024) + +int main(int argc, char **argv) +{ + if (argc < 4) { + fprintf(stderr, "usage: %s DEV MARKER COUNT [FIRST] [PAGES_PER_WRITE]\n" + " PAGES_PER_WRITE > 1 issues one multi-page command, the way a\n" + " bulk fill does, rather than a command per page.\n", argv[0]); + return 2; + } + const char *dev = argv[1], *marker = argv[2]; + long count = atol(argv[3]); + long first = argc > 4 ? atol(argv[4]) : 0; + long batch = argc > 5 ? atol(argv[5]) : 1; + if (batch < 1) batch = 1; + + int fd = open(dev, O_WRONLY | O_DIRECT); + if (fd < 0) { perror(dev); return 1; } + void *buf; + size_t span = (size_t)batch * PGSZ; + if (posix_memalign(&buf, 4096, span)) { perror("memalign"); return 1; } + + for (long i = 0; i < count; i += batch) { + long n = count - i < batch ? count - i : batch; + memset(buf, 0, span); + /* the marker plus the LPN, so a dump can be checked against the write */ + for (long j = 0; j < n; j++) + snprintf((char *)buf + (size_t)j * PGSZ, PGSZ, "%s lpn=%ld", + marker, first + i + j); + size_t want = (size_t)n * PGSZ; + if (pwrite(fd, buf, want, (off_t)(first + i) * PGSZ) != (ssize_t)want) { + perror("pwrite"); + return 1; + } + } + if (fsync(fd)) { perror("fsync"); return 1; } + printf("MARKWRITE first=%ld count=%ld pages_per_write=%ld marker=%s\n", + first, count, batch, marker); + return 0; +} diff --git a/moe-harness/exp/moe_bcq/femu_run/preflight.sh b/moe-harness/exp/moe_bcq/femu_run/preflight.sh new file mode 100755 index 00000000000..49839503b3c --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/preflight.sh @@ -0,0 +1,45 @@ +#!/bin/bash +# Verify run_policy.sh is the patched version before committing to a run. +# +# On 2026-09-10 this file reverted, mid-sweep and silently, to its pre-patch +# state: byte-identical to the original and carrying the original mtime. What +# reverted it was never identified. Unpatched, two defaults come back and both +# fail quietly rather than loudly: +# +# BUNDLE -> qwen_C, so a DeepSeek run replays the Qwen binary against a +# DeepSeek image. Fill and read-back both still pass; neither +# check knows whose bytes it is looking at. +# IMAGE_PAGES -> 471040, so the last 73,728 pages of a DeepSeek image are +# never class-checked and confusion still reports "clean". +# +# So the guard cannot live inside run_policy.sh -- a revert takes it too. The +# canonical copy and this check sit outside the tree that reverted. +set -uo pipefail + +CANON=${FEMU_CANONICAL_DIR:-$HOME/.femu_canonical} +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +TARGET=$ROOT/exp/moe_bcq/femu_run/run_policy.sh + +[ -f "$CANON/SHA256SUMS" ] || { echo "preflight: no canonical copy at $CANON"; exit 1; } +want=$(awk '$2=="run_policy.sh"{print $1}' "$CANON/SHA256SUMS") +[ -n "$want" ] || { echo "preflight: SHA256SUMS has no run_policy.sh entry"; exit 1; } + +got=$(sha256sum "$TARGET" 2>/dev/null | cut -d' ' -f1) +if [ "$got" = "$want" ]; then + echo "preflight: run_policy.sh matches canonical (${want:0:12})" + exit 0 +fi + +echo "preflight: run_policy.sh DOES NOT match the canonical copy" +echo " expected ${want:0:12} got ${got:0:12}" +# Restoring is right only when the canonical copy is itself intact; otherwise a +# corrupted canonical would be copied over a good working file. +canon_now=$(sha256sum "$CANON/run_policy.sh" 2>/dev/null | cut -d' ' -f1) +if [ "$canon_now" != "$want" ]; then + echo " canonical copy is itself altered -- refusing to restore. Fix $CANON by hand." + exit 1 +fi +cp -p "$CANON/run_policy.sh" "$TARGET" +chmod +x "$TARGET" +echo " restored from $CANON/run_policy.sh" +sha256sum "$TARGET" | sed 's/^/ now /' diff --git a/moe-harness/exp/moe_bcq/femu_run/probe_map.c b/moe-harness/exp/moe_bcq/femu_run/probe_map.c new file mode 100644 index 00000000000..a06a253ec9c --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/probe_map.c @@ -0,0 +1,104 @@ +/* Measure the LPN -> QLC page-class mapping the device actually uses. + * + * The layout planner predicts a class for every page it places, and the whole + * experiment rests on that prediction matching the device. Read latency is the + * one signal that reports the class directly: the four classes are 47.9, 76.2, + * 134.6 and 228.1 us apart, far enough to separate under any constant host + * overhead. So time single-page reads at known LPNs and read the class back off + * the clock instead of trusting a model of the write pointer. + * + * The device must already be filled: a read of an unmapped LPN never reaches the + * media, so it would time as class 0 and look like a mapping that starts fast. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include + +#define PGSZ (16 * 1024) +static const double EXPECT_US[4] = {47.9, 76.2, 134.6, 228.1}; + +static int cmp(const void *a, const void *b) +{ + double x = *(const double *)a, y = *(const double *)b; + return x < y ? -1 : x > y; +} + +static double now_us(void) +{ + struct timespec t; + clock_gettime(CLOCK_MONOTONIC_RAW, &t); + return t.tv_sec * 1e6 + t.tv_nsec / 1e3; +} + +int main(int argc, char **argv) +{ + const char *dev = argc > 1 ? argv[1] : "/dev/nvme0n1"; + long first = argc > 2 ? atol(argv[2]) : 0; + long count = argc > 3 ? atol(argv[3]) : 1024; + int reps = argc > 4 ? atoi(argv[4]) : 5; + + int fd = open(dev, O_RDONLY | O_DIRECT); + if (fd < 0) { perror(dev); return 1; } + void *buf; + if (posix_memalign(&buf, 4096, PGSZ)) { perror("memalign"); return 1; } + + double *med = malloc(count * sizeof *med); + double *s = malloc(reps * sizeof *s); + if (!med || !s) { fprintf(stderr, "oom\n"); return 1; } + + /* Sweep repetition-major rather than LPN-major: consecutive reads of the + * same page would sit behind one another on the same LUN and measure queue + * time as well as array time. */ + for (long i = 0; i < count; i++) med[i] = 0; + for (int r = 0; r < reps; r++) { + for (long i = 0; i < count; i++) { + off_t off = (off_t)(first + i) * PGSZ; + double t0 = now_us(); + if (pread(fd, buf, PGSZ, off) != PGSZ) { perror("pread"); return 1; } + double dt = now_us() - t0; + /* keep the running minimum: the cleanest estimate of array time */ + if (r == 0 || dt < med[i]) med[i] = dt; + } + } + + /* Calibrate the constant host overhead from the observed spread, then class + * each page by nearest expected latency. The offset is whatever makes the + * fastest pages land on class 0. */ + double *sorted = malloc(count * sizeof *sorted); + memcpy(sorted, med, count * sizeof *sorted); + qsort(sorted, count, sizeof *sorted, cmp); + double floor_us = sorted[count / 100]; /* 1st percentile */ + double offset = floor_us - EXPECT_US[0]; + + printf("PROBE dev=%s first=%ld count=%ld reps=%d\n", dev, first, count, reps); + printf("PROBE floor=%.1fus implied_host_offset=%.1fus\n", floor_us, offset); + printf("PROBE quartiles %.1f %.1f %.1f %.1f\n", + sorted[count / 8], sorted[count * 3 / 8], + sorted[count * 5 / 8], sorted[count * 7 / 8]); + + long hist[4] = {0}; + printf("CLASSMAP %ld ", first); + for (long i = 0; i < count; i++) { + int best = 0; + double bd = 1e18; + for (int c = 0; c < 4; c++) { + double d = med[i] - offset - EXPECT_US[c]; + if (d < 0) d = -d; + if (d < bd) { bd = d; best = c; } + } + hist[best]++; + putchar('0' + best); + if ((i + 1) % 128 == 0 && i + 1 < count) printf("\nCLASSMAP %ld ", first + i + 1); + } + printf("\nPROBE counts %ld %ld %ld %ld\n", hist[0], hist[1], hist[2], hist[3]); + + /* A few raw samples so the classification can be audited by hand. */ + printf("PROBE raw"); + for (long i = 0; i < 16 && i < count; i++) printf(" %ld:%.1f", first + i, med[i]); + printf("\n"); + return 0; +} diff --git a/moe-harness/exp/moe_bcq/femu_run/run_device.sh b/moe-harness/exp/moe_bcq/femu_run/run_device.sh new file mode 100755 index 00000000000..efc574be552 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/run_device.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# Boot one FEMU device, fill it from a given image, then replay a list of traces. +# +# This is deliberately NOT a patch to run_policy.sh. That file also exists on the +# GPU host and the trace delivery rsyncs the whole femu_run directory over, which +# silently reverted two edits to it mid-sweep on 2026-09-10. Files that exist +# only here survive that sync, so the multi-replay path lives in its own. +# +# One device per placement, several traces per device: the image decides which +# physical class each plane lands on, and every trace sharing that image shares +# that placement. Re-filling per trace would re-run out-of-place allocation. +set -uo pipefail + +TAG=${1:?usage: run_device.sh DEVICE_TAG IMAGE_BASENAME IMAGE_PAGES SPECFILE} +IMG_BASE=${2:?}; PAGES=${3:?}; SPEC=${4:?} +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +IMAGES=${FEMU_GUEST_DIR:-$HOME/images} +SSH_PUBKEY=${FEMU_SSH_PUBKEY:-$HOME/.ssh/id_rsa.pub} +QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$ROOT/_deps/FEMU-MoE/build/qemu-img")} +cd "$ROOT" + +# make_seed.py needs pycdlib, which is installed for the system interpreter and +# not for whatever python3 a shell resolves first. Fail here, not three lines +# into building the seed after the caller has committed to the run. +if [ -z "${PYTHON:-}" ]; then + for c in python3 /usr/bin/python3 python3.8; do + command -v "$c" >/dev/null 2>&1 || continue + "$c" -c 'import pycdlib' 2>/dev/null && { PYTHON=$c; break; } + done +fi +[ -n "${PYTHON:-}" ] || { echo " no python3 with pycdlib; pip install --user pycdlib"; exit 1; } + +IMG=/data/images/$IMG_BASE # container path; /data is runs/femu +HOST_IMG=$ROOT/runs/femu/images/$IMG_BASE +[ -f "$HOST_IMG" ] || { echo " no image at $HOST_IMG"; exit 1; } +[ -s "$SPEC" ] || { echo " empty or missing spec $SPEC"; exit 1; } +SEED=$IMAGES/seed-$TAG.iso +OVL=$IMAGES/femu-root-$TAG.qcow2 + +echo "=== device $TAG image $IMG_BASE pages $PAGES ($(grep -cve '^\s*$' "$SPEC") traces) ===" +rm -f "$SEED" "$OVL" "runs/femu/${TAG}_qlc.csv" +# No replay.bin in the seed: the binaries are streamed in per replay, and the +# gsm8k ones are ~100 MB each. +(cd exp/gating_nand/femu && "$PYTHON" make_seed.py -o "$SEED" --tag "${TAG^^}" \ + --instance-id "femu-$TAG" --ssh-key "$SSH_PUBKEY" \ + --file /usr/local/bin/replay_v1=$ROOT/build/guest/replay_v1:0755 \ + --file /usr/local/bin/class_confusion=$ROOT/build/guest/class_confusion:0755 \ + --file /usr/local/bin/guest_replay.sh=$ROOT/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ + --run "/usr/local/bin/guest_replay.sh > /dev/ttyS0 2>&1") >/dev/null || exit 1 +"$QEMU_IMG" create -f qcow2 -F qcow2 -b jammy-server-cloudimg-amd64.img "$OVL" 32G >/dev/null + +sed -e "s|^FEMU_QLC_STATS_PATH=.*|FEMU_QLC_STATS_PATH=/data/${TAG}_qlc.csv|" \ + -e "s|^FEMU_IMAGE_NAME=.*|FEMU_IMAGE_NAME=$(basename "$OVL")|" \ + -e "s|^FEMU_CONTAINER_NAME=.*|FEMU_CONTAINER_NAME=femu-$TAG|" \ + -e "s|FEMU_EXTRA_DRIVES=.*|FEMU_EXTRA_DRIVES='file=/guest/$(basename "$SEED"),if=virtio,format=raw,readonly=on;file=$IMG,if=virtio,format=raw,readonly=on'|" \ + runs/femu/run01.env > "runs/femu/${TAG}.env" + +set -a; . "runs/femu/${TAG}.env"; set +a +if [ -z "${FEMU_SOURCE_DIR:-}" ]; then + # "$ROOT/.." is this harness living inside the FEMU checkout itself, which + # is how the published repository is laid out; the _deps forms are the + # original layout, where the harness was the outer project. + for c in "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE" "$ROOT/.."; do + [ -f "$c/compose.yaml" ] && { FEMU_SOURCE_DIR=$c; break; } + done +fi +[ -n "${FEMU_SOURCE_DIR:-}" ] || { echo " no FEMU checkout with compose.yaml"; exit 1; } +export FEMU_SOURCE_DIR +nohup bash scripts/femu_compose.sh up femu > "runs/femu/${TAG}.console.log" 2>&1 & +until grep -qa "ALL DONE\|FATAL" "runs/femu/${TAG}.console.log" 2>/dev/null; do sleep 10; done +grep -qa FATAL "runs/femu/${TAG}.console.log" && { echo " FATAL during fill"; exit 1; } +grep -a "fill done\|read-back OK" "runs/femu/${TAG}.console.log" | tr -d '\r' | sed 's/^femu[^|]*| / /' + +IMAGE_PAGES=$PAGES bash exp/moe_bcq/femu_run/drive_multi.sh "$TAG" "$SPEC"; rc=$? +export FEMU_CONTAINER_NAME=femu-$TAG +bash scripts/femu_compose.sh down >/dev/null 2>&1 +echo " device torn down (rc=$rc)" +exit $rc diff --git a/moe-harness/exp/moe_bcq/femu_run/run_policy.sh b/moe-harness/exp/moe_bcq/femu_run/run_policy.sh new file mode 100755 index 00000000000..19d183c6917 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/run_policy.sh @@ -0,0 +1,65 @@ +#!/bin/bash +# One placement policy end to end: fresh device, 256 KiB fill, class check, replay. +# +# Each policy needs its own device. Refilling an existing one would land every +# page on a fresh PPA (out-of-place update), so the second fill would not be the +# layout the image describes. +set -uo pipefail +POL=${1:?usage: run_policy.sh POLICY} +# Same reason as drive_run.sh: the path is where the script is, not a constant. +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +# Guest images and the key authorised inside them are per-machine. +IMAGES=${FEMU_GUEST_DIR:-$HOME/images} +SSH_PUBKEY=${FEMU_SSH_PUBKEY:-$HOME/.ssh/id_rsa.pub} +# The patched build ships one; a system qemu-img works too and is preferred when +# present, since it does not depend on the checkout having been built yet. +QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$ROOT/_deps/FEMU-MoE/build/qemu-img")} +cd "$ROOT" + +# One knob, not two. The run tag defaulted to something different from the +# layout prefix once, and the results then carried a name that did not say which +# layout produced them. +TAG=${TAG_PREFIX:-${LAYOUT_PREFIX:-pol}}_$POL +IMG=/data/images/${IMG_PREFIX:-qwen_C}_$POL.img +SEED=$IMAGES/seed-$TAG.iso +OVL=$IMAGES/femu-root-$TAG.qcow2 +LAYOUT=${LAYOUT_PREFIX:-qlc}_epm_aif_2ch4lun_$POL +BIN=exp/moe_bcq/femu_handoff/packages/qwen_C/layouts/$LAYOUT/replay_qd32.bin + +echo "=== $POL ===" +rm -f "$SEED" "$OVL" "runs/femu/${TAG}_qlc.csv" +(cd exp/gating_nand/femu && python3 make_seed.py -o "$SEED" --tag "${TAG^^}" \ + --instance-id "femu-$TAG" --ssh-key "$SSH_PUBKEY" \ + --file /usr/local/bin/replay_v1=$ROOT/build/guest/replay_v1:0755 \ + --file /usr/local/bin/class_confusion=$ROOT/build/guest/class_confusion:0755 \ + --file /root/replay.bin=$ROOT/$BIN \ + --file /usr/local/bin/guest_replay.sh=$ROOT/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ + --run "/usr/local/bin/guest_replay.sh > /dev/ttyS0 2>&1") >/dev/null || exit 1 +"$QEMU_IMG" create -f qcow2 -F qcow2 -b jammy-server-cloudimg-amd64.img "$OVL" 32G >/dev/null + +sed -e "s|^FEMU_QLC_STATS_PATH=.*|FEMU_QLC_STATS_PATH=/data/${TAG}_qlc.csv|" \ + -e "s|^FEMU_IMAGE_NAME=.*|FEMU_IMAGE_NAME=$(basename "$OVL")|" \ + -e "s|^FEMU_CONTAINER_NAME=.*|FEMU_CONTAINER_NAME=femu-$TAG|" \ + -e "s|FEMU_EXTRA_DRIVES=.*|FEMU_EXTRA_DRIVES='file=/guest/$(basename "$SEED"),if=virtio,format=raw,readonly=on;file=$IMG,if=virtio,format=raw,readonly=on'|" \ + runs/femu/run01.env > "runs/femu/${TAG}.env" + +set -a; . "runs/femu/${TAG}.env"; set +a +# setup_femu.sh puts the checkout under _deps; a working tree kept beside it is +# the older layout. Prefer whichever actually has the compose file rather than +# naming one, or the run stalls waiting for a container that was never started. +if [ -z "${FEMU_SOURCE_DIR:-}" ]; then + for c in "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE"; do + [ -f "$c/compose.yaml" ] && { FEMU_SOURCE_DIR=$c; break; } + done +fi +[ -n "${FEMU_SOURCE_DIR:-}" ] || { echo " no FEMU checkout with compose.yaml; run scripts/setup_femu.sh"; exit 1; } +export FEMU_SOURCE_DIR +nohup bash scripts/femu_compose.sh up femu > "runs/femu/${TAG}.console.log" 2>&1 & +until grep -qa "ALL DONE\|FATAL" "runs/femu/${TAG}.console.log" 2>/dev/null; do sleep 10; done +grep -qa FATAL "runs/femu/${TAG}.console.log" && { echo " FATAL during fill"; exit 1; } +grep -a "fill done\|read-back OK" "runs/femu/${TAG}.console.log" | tr -d '\r' | sed 's/^femu[^|]*| / /' + +bash exp/moe_bcq/femu_run/drive_run.sh "$TAG" || exit 1 +export FEMU_CONTAINER_NAME=femu-$TAG +bash scripts/femu_compose.sh down >/dev/null 2>&1 +echo " device torn down" diff --git a/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh b/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh new file mode 100755 index 00000000000..bd665e44d83 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# Run one bundle's placement policies end to end, preflight first. +# +# Usage: run_sweep.sh BUNDLE LAYOUT_PREFIX IMG_PREFIX TAG_PREFIX [policy ...] +# run_sweep.sh deepseek_C gen256 ds256 ds256 +# run_sweep.sh qwen_C gen256 qw256 qw256 inverted +# +# The preflight call is the point of this wrapper: run_policy.sh reverted once +# between two policies of a live sweep, and the unpatched defaults fail quietly +# (see preflight.sh). Going through here means no sweep can start on a reverted +# script, and a revert mid-sweep is caught at the next policy. +set -uo pipefail +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +HERE=$ROOT/exp/moe_bcq/femu_run + +BUNDLE=${1:?usage: run_sweep.sh BUNDLE LAYOUT_PREFIX IMG_PREFIX TAG_PREFIX [policy ...]} +LAYOUT_PREFIX=${2:?}; IMG_PREFIX=${3:?}; TAG_PREFIX=${4:?} +shift 4 +POLICIES=("$@"); [ ${#POLICIES[@]} -gt 0 ] || POLICIES=(aligned rotated inverted) + +cd "$ROOT" +rc=0 +for pol in "${POLICIES[@]}"; do + # Re-checked per policy, not once at the top: the observed revert landed + # between two policies of a running sweep. + bash "$HERE/preflight.sh" || { echo "!!!! preflight failed; not running $pol"; rc=1; break; } + echo "############ $(date +%T) $TAG_PREFIX $pol ############" + BUNDLE=$BUNDLE LAYOUT_PREFIX=$LAYOUT_PREFIX IMG_PREFIX=$IMG_PREFIX TAG_PREFIX=$TAG_PREFIX \ + bash "$HERE/run_policy.sh" "$pol" || { echo "!!!! $pol FAILED rc=$?"; rc=1; } +done +echo "############ $(date +%T) sweep done (rc=$rc) ############" +exit $rc diff --git a/moe-harness/exp/moe_bcq/femu_run/verify_fill_256.py b/moe-harness/exp/moe_bcq/femu_run/verify_fill_256.py new file mode 100644 index 00000000000..fed96356779 --- /dev/null +++ b/moe-harness/exp/moe_bcq/femu_run/verify_fill_256.py @@ -0,0 +1,162 @@ +#!/usr/bin/env python3 +"""Run the isolated, pre-provisioned verify256 VM; preserve each snapshot. + +Requires seed-verify256-20260910.iso and its private root overlay. Never uses +the existing femu-map VM. Results/output paths are exclusive. +""" +import csv +import hashlib +import io +import json +from pathlib import Path +import subprocess +import time + +ROOT = Path(__file__).resolve().parents[3] +OUT = ROOT / 'runs/femu/verify256_20260910' +NAME = 'femu-verify256-20260910' +IMAGE = 'sha256:a91a26ae90eb2f193e1d98b8b180f587fcb463c418a76a7f56360e1cfb9453b5' +EXPECTED = [420294, 420294, 315822, 237450] +SSH = ['ssh', '-o', 'BatchMode=yes', '-o', 'StrictHostKeyChecking=accept-new', + '-o', 'ConnectTimeout=5', '-p', '2223', 'femu@localhost'] + + +def run(args, timeout=120): + return subprocess.run(args, check=True, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, timeout=timeout).stdout + + +def save(path, data): + with path.open('xb') as stream: + stream.write(data) + + +def remote(command, timeout=120): + return run(SSH + [command], timeout) + + +def counts(raw): + values = [0] * 4 + lines = ''.join(x for x in raw.decode().splitlines(True) if not x.startswith('#')) + for row in csv.DictReader(io.StringIO(lines)): + values[int(row['page_class'])] += int(row['n_read']) + return values + + +def snapshot(directory, stem): + # Must be called after successful guest completion and before another probe. + raw = (OUT / 'counter.live.csv').read_bytes() + save(directory / (stem + '.qlc.csv'), raw) + return counts(raw) + + +def verify_groups(records, summary): + assert len(records) == summary['groups'] == 1536 + assert sum(g['command_count'] for g in records) == summary['commands'] == 130090 + assert sum(g['requested_bytes'] for g in records) == summary['requested_bytes'] == 22608650240 + assert sum(g['group_io_ns'] for g in records) == summary['sum_group_io_ns'] + previous_complete = 0 + for index, g in enumerate(records): + assert g['group_id'] == index and g['group_ready_ns'] >= previous_complete + assert g['peak_outstanding'] <= 32 + if g['command_count']: + assert g['first_submit_ns'] >= g['group_ready_ns'] + assert g['last_complete_ns'] >= g['last_submit_ns'] + else: + assert g['first_submit_ns'] is None and g['group_io_ns'] == 0 + previous_complete = g['last_complete_ns'] + + +def main(): + env = dict(FEMU_IMAGE='/guest/femu-root-verify256-20260910.qcow2', + FEMU_MEMORY='8G', FEMU_CPUS='6', FEMU_NAND_CELL_TYPE='4', + FEMU_SSD_SIZE_MB='65536', FEMU_SECTORS_PER_PAGE='32', + FEMU_PAGES_PER_BLOCK='512', FEMU_BLOCKS_PER_PLANE='1024', + FEMU_PLANES_PER_LUN='1', FEMU_LUNS_PER_CHANNEL='4', FEMU_CHANNELS='2', + FEMU_EXTRA_DEVICE_OPTS='op_pcent=7', + FEMU_QLC_STATS_PATH='/data/verify256_20260910/counter.live.csv', + FEMU_QMP_SOCKET='/data/verify256_20260910/qmp.sock', + FEMU_EXTRA_DRIVES='file=/guest/seed-verify256-20260910.iso,if=virtio,format=raw,readonly=on;' + 'file=/data/images/qwen_C.img,if=virtio,format=raw,readonly=on') + save(OUT / 'configuration.json', json.dumps(dict(image=IMAGE, env=env, + fill_bytes=262144, expected_counts=EXPECTED), indent=2).encode()) + cmd = ['docker', 'run', '-d', '--name', NAME, '--device', '/dev/kvm', + '--cap-add', 'IPC_LOCK', '--ulimit', 'memlock=-1:-1', + '-p', '127.0.0.1:2223:2222', + '-v', '/data01/kwkim02/images:/guest', + '-v', str(ROOT / 'runs/femu') + ':/data'] + for key, value in env.items(): + cmd.extend(['-e', key + '=' + value]) + cmd.extend([IMAGE, 'bbssd']) + boot_reports = [] + launched = False + try: + for boot in (1, 2): + directory = OUT / f'boot{boot}' + directory.mkdir(exist_ok=False) + print(f'boot{boot}: starting fresh FEMU SSD', flush=True) + if boot == 1: + run(cmd) + launched = True + else: + run(['docker', 'start', NAME]) + for attempt in range(120): + try: + remote('sudo test -x /root/guest_fill.sh && sudo test -f /root/replay.bin', 10) + break + except (subprocess.CalledProcessError, subprocess.TimeoutExpired): + time.sleep(2) + else: + raise RuntimeError('guest SSH/cloud-init did not become ready') + save(directory / 'container.json', run(['docker', 'inspect', NAME])) + save(directory / 'guest_trace_sha256.txt', remote('sudo sha256sum /root/replay.bin')) + print(f'boot{boot}: sequential 256 KiB fill + complete read-back', flush=True) + fill = remote('sudo bash /root/guest_fill.sh', 1200) + save(directory / 'fill.log', fill) + expected_sha = json.loads((ROOT / 'runs/femu/images/qwen_C.img.json').read_text())['sha256'] + assert fill.count(expected_sha.encode()) == 2 and b'read-back OK' in fill + print(f'boot{boot}: read-back passed', flush=True) + reports = [] + for repeat in (1, 2, 3): + stem = f'replay{repeat}' + guest = f'/dev/shm/verify_boot{boot}_{stem}' + # Exclusive output paths and checked exit prevent stale success. + result = remote(f'sudo replay_v1 --trace /root/replay.bin --device /dev/nvme0n1 ' + f'--controller /dev/nvme0 --qd 32 --group-log {guest}.groups.jsonl ' + f'--summary {guest}.summary.json', 300) + save(directory / (stem + '.stdout'), result) + actual = snapshot(directory, stem) + summary_raw = remote(f'sudo cat {guest}.summary.json') + groups_raw = remote(f'sudo cat {guest}.groups.jsonl') + save(directory / (stem + '.summary.json'), summary_raw) + save(directory / (stem + '.groups.jsonl'), groups_raw) + summary = json.loads(summary_raw) + verify_groups([json.loads(line) for line in groups_raw.splitlines()], summary) + assert actual == EXPECTED, (boot, repeat, actual) + reports.append(dict(repeat=repeat, counts=actual, group_io_ns=summary['sum_group_io_ns'])) + print(f'boot{boot} {stem}: exact class counts, group_io={summary["sum_group_io_ns"]/1e9:.3f}s', flush=True) + matrix = [] + # Covers all 471040 LPNs in the image, including filler. + for klass in range(4): + result = remote(f'sudo class_confusion /dev/nvme0n1 /dev/nvme0 {klass} 471040', 180) + save(directory / f'class{klass}.stdout', result) + row = snapshot(directory, f'class{klass}') + n = int(result.decode().split('reads=')[1].strip()) + assert sum(row) == n and row[klass] == n, row + matrix.append(row) + print(f'boot{boot}: class {klass} full-image check passed ({n} pages)', flush=True) + boot_reports.append(dict(boot=boot, repeats=reports, full_image_confusion=matrix)) + run(['docker', 'stop', '-t', '15', NAME], 60) + save(directory / 'console.log', run(['docker', 'logs', NAME])) + report = dict(passed=True, boots=boot_reports, image_sha256=expected_sha) + save(OUT / 'verification.json', json.dumps(report, indent=2).encode()) + print('PASS: two fresh boots, six replays, two full-image class checks', flush=True) + finally: + if launched: + subprocess.run(['docker', 'stop', '-t', '10', NAME], stdout=subprocess.DEVNULL, timeout=30) + if not (OUT / 'console.final.log').exists(): + save(OUT / 'console.final.log', run(['docker', 'logs', NAME])) + + +if __name__ == '__main__': + main() diff --git a/moe-harness/runs/femu/run01.env b/moe-harness/runs/femu/run01.env new file mode 100644 index 00000000000..247e218f025 --- /dev/null +++ b/moe-harness/runs/femu/run01.env @@ -0,0 +1,17 @@ +FEMU_GUEST_DIR=/data/kwkim02/images +FEMU_DATA_DIR=/data/kwkim02/MoE_SSD/runs/femu +FEMU_IMAGE_NAME=placeholder.qcow2 +FEMU_MEMORY=8G +FEMU_CPUS=6 +FEMU_NAND_CELL_TYPE=4 +FEMU_SSD_SIZE_MB=65536 +FEMU_SECTORS_PER_PAGE=32 +FEMU_PAGES_PER_BLOCK=512 +FEMU_BLOCKS_PER_PLANE=1024 +FEMU_PLANES_PER_LUN=1 +FEMU_LUNS_PER_CHANNEL=4 +FEMU_CHANNELS=2 +FEMU_EXTRA_DEVICE_OPTS=op_pcent=7 +FEMU_QLC_STATS_PATH=/data/placeholder.csv +FEMU_EXTRA_DRIVES=placeholder +FEMU_CONTAINER_NAME=placeholder diff --git a/moe-harness/scripts/build_replay.sh b/moe-harness/scripts/build_replay.sh new file mode 100755 index 00000000000..7267b11a484 --- /dev/null +++ b/moe-harness/scripts/build_replay.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +src="${repo_root}/exp/gating_nand/femu/replay.c" +out_dir="${repo_root}/build/guest" + +mkdir -p "${out_dir}" + +if ! pkg-config --exists liburing; then + echo "build_replay: liburing development files are missing" >&2 + echo "install liburing-dev, or activate the conda environment containing liburing" >&2 + exit 1 +fi + +read -r -a uring_flags <<< "$(pkg-config --cflags --libs liburing)" +cc -O2 -Wall -Wextra "${src}" -o "${out_dir}/replay" "${uring_flags[@]}" + +echo "build_replay: wrote ${out_dir}/replay" diff --git a/moe-harness/scripts/femu_compose.sh b/moe-harness/scripts/femu_compose.sh new file mode 100755 index 00000000000..805a807faef --- /dev/null +++ b/moe-harness/scripts/femu_compose.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +femu_root="${FEMU_SOURCE_DIR:-${repo_root}/_deps/FEMU-MoE}" + +if [[ ! -f "${femu_root}/compose.yaml" ]]; then + echo "femu_compose: ${femu_root}/compose.yaml is missing" >&2 + echo "run ${repo_root}/scripts/setup_femu.sh first" >&2 + exit 1 +fi + +export FEMU_GUEST_DIR="${FEMU_GUEST_DIR:-${repo_root}/images}" +export FEMU_DATA_DIR="${FEMU_DATA_DIR:-${repo_root}/runs/femu}" +mkdir -p "${FEMU_GUEST_DIR}" "${FEMU_DATA_DIR}" + +# Run from the FEMU source tree instead of relying on --project-directory. The +# latter is unavailable in old Compose installations and can be misparsed as a +# top-level Docker flag when the Compose CLI plugin is absent. +cd "${femu_root}" + +if docker compose version >/dev/null 2>&1; then + exec docker compose -f compose.yaml "$@" +fi + +if command -v docker-compose >/dev/null 2>&1; then + exec docker-compose -f compose.yaml "$@" +fi + +echo "femu_compose: Docker Compose is not installed" >&2 +echo "install the Docker Compose plugin, or the legacy docker-compose command" >&2 +exit 1 From 62701fb954e6ad0c7cf18c6b0fa3bb15b57ce38e Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 17:30:05 +0900 Subject: [PATCH 11/17] moe-harness: let the code and the data it works on live apart The harness resolved one root from its own location and used it for both the scripts it calls and the images, guest binaries and payload packages it reads. That was fine while the harness was the project, but it is now inside the FEMU checkout, and the data -- tens of gigabytes, distributed separately -- is not. Resolving both from the same place meant moving the code silently took the data lookups with it. HARNESS is where these scripts are. ROOT is the project holding the data, and FEMU_PROJECT_ROOT sets it. It defaults to HARNESS, so unpacking the data under the harness, which is what a clone gets, still works with no argument. Verified from the published layout with the data elsewhere: HARNESS resolves to the harness, ROOT to the data project, and every reference on both sides -- guest_replay.sh, make_seed.py, femu_compose.sh, the FEMU checkout, the images, the built guest binaries, run01.env, the packages and the records -- resolves. Co-Authored-By: Claude Opus 5 --- .../exp/moe_bcq/femu_run/drive_multi.sh | 4 +++- .../exp/moe_bcq/femu_run/run_device.sh | 23 ++++++++++++------- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh b/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh index af40b409ee7..76376073502 100755 --- a/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh +++ b/moe-harness/exp/moe_bcq/femu_run/drive_multi.sh @@ -17,7 +17,9 @@ set -uo pipefail TAG=${1:?usage: drive_multi.sh DEVICE_TAG SPECFILE} SPEC=${2:?usage: drive_multi.sh DEVICE_TAG SPECFILE} -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +# Everything this script touches is data -- counters, records, replay binaries +# -- so it follows the project root, not where the script itself lives. +ROOT=${FEMU_PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)} CSV=$ROOT/runs/femu/${TAG}_qlc.csv RECORDS=$ROOT/exp/moe_bcq/femu_run/records # -n on every ssh that is not being fed a file: without it ssh inherits the diff --git a/moe-harness/exp/moe_bcq/femu_run/run_device.sh b/moe-harness/exp/moe_bcq/femu_run/run_device.sh index efc574be552..e877f7f691e 100755 --- a/moe-harness/exp/moe_bcq/femu_run/run_device.sh +++ b/moe-harness/exp/moe_bcq/femu_run/run_device.sh @@ -13,10 +13,17 @@ set -uo pipefail TAG=${1:?usage: run_device.sh DEVICE_TAG IMAGE_BASENAME IMAGE_PAGES SPECFILE} IMG_BASE=${2:?}; PAGES=${3:?}; SPEC=${4:?} -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +# Two roots, because the code and the data it works on need not live together. +# HARNESS is where these scripts are -- inside the FEMU checkout, tracked in +# git. ROOT is the project holding the images, the built guest binaries and the +# payload packages, which are tens of gigabytes and are distributed separately. +# They coincide when someone unpacks the data under the harness, which is what +# a fresh clone does, so the default keeps that case working with no argument. +HARNESS=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +ROOT=${FEMU_PROJECT_ROOT:-$HARNESS} IMAGES=${FEMU_GUEST_DIR:-$HOME/images} SSH_PUBKEY=${FEMU_SSH_PUBKEY:-$HOME/.ssh/id_rsa.pub} -QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$ROOT/_deps/FEMU-MoE/build/qemu-img")} +QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$HARNESS/../build/qemu-img")} cd "$ROOT" # make_seed.py needs pycdlib, which is installed for the system interpreter and @@ -41,11 +48,11 @@ echo "=== device $TAG image $IMG_BASE pages $PAGES ($(grep -cve '^\s*$' "$SPE rm -f "$SEED" "$OVL" "runs/femu/${TAG}_qlc.csv" # No replay.bin in the seed: the binaries are streamed in per replay, and the # gsm8k ones are ~100 MB each. -(cd exp/gating_nand/femu && "$PYTHON" make_seed.py -o "$SEED" --tag "${TAG^^}" \ +(cd "$HARNESS/exp/gating_nand/femu" && "$PYTHON" make_seed.py -o "$SEED" --tag "${TAG^^}" \ --instance-id "femu-$TAG" --ssh-key "$SSH_PUBKEY" \ --file /usr/local/bin/replay_v1=$ROOT/build/guest/replay_v1:0755 \ --file /usr/local/bin/class_confusion=$ROOT/build/guest/class_confusion:0755 \ - --file /usr/local/bin/guest_replay.sh=$ROOT/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ + --file /usr/local/bin/guest_replay.sh=$HARNESS/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ --run "/usr/local/bin/guest_replay.sh > /dev/ttyS0 2>&1") >/dev/null || exit 1 "$QEMU_IMG" create -f qcow2 -F qcow2 -b jammy-server-cloudimg-amd64.img "$OVL" 32G >/dev/null @@ -60,19 +67,19 @@ if [ -z "${FEMU_SOURCE_DIR:-}" ]; then # "$ROOT/.." is this harness living inside the FEMU checkout itself, which # is how the published repository is laid out; the _deps forms are the # original layout, where the harness was the outer project. - for c in "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE" "$ROOT/.."; do + for c in "$HARNESS/.." "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE"; do [ -f "$c/compose.yaml" ] && { FEMU_SOURCE_DIR=$c; break; } done fi [ -n "${FEMU_SOURCE_DIR:-}" ] || { echo " no FEMU checkout with compose.yaml"; exit 1; } export FEMU_SOURCE_DIR -nohup bash scripts/femu_compose.sh up femu > "runs/femu/${TAG}.console.log" 2>&1 & +nohup bash "$HARNESS/scripts/femu_compose.sh" up femu > "runs/femu/${TAG}.console.log" 2>&1 & until grep -qa "ALL DONE\|FATAL" "runs/femu/${TAG}.console.log" 2>/dev/null; do sleep 10; done grep -qa FATAL "runs/femu/${TAG}.console.log" && { echo " FATAL during fill"; exit 1; } grep -a "fill done\|read-back OK" "runs/femu/${TAG}.console.log" | tr -d '\r' | sed 's/^femu[^|]*| / /' -IMAGE_PAGES=$PAGES bash exp/moe_bcq/femu_run/drive_multi.sh "$TAG" "$SPEC"; rc=$? +IMAGE_PAGES=$PAGES bash "$HARNESS/exp/moe_bcq/femu_run/drive_multi.sh" "$TAG" "$SPEC"; rc=$? export FEMU_CONTAINER_NAME=femu-$TAG -bash scripts/femu_compose.sh down >/dev/null 2>&1 +bash "$HARNESS/scripts/femu_compose.sh" down >/dev/null 2>&1 echo " device torn down (rc=$rc)" exit $rc From 5be27e2ed633b7821df2b2aa58f163d09ee17c05 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 22:46:56 +0900 Subject: [PATCH 12/17] moe-harness: leave the trace tooling to MoE_Trace The mapper, the trace compiler, the layer-group builder and bundle.py were copied in with the harness. They belong to MoE_Trace, and keeping a second copy here would rebuild the problem the split exists to end: two owners for one file, which is how a pair of mapper copies came to sit 64 lines apart. Nothing in the harness calls them, so this costs the run path nothing. The set was incomplete here anyway -- layer_read_groups.py imports online_cache, which was never copied across, so it could not have run. REPLAYER_V1.md stays. It specifies the binary format, and the program that consumes that format, replay_v1.c, is in this repository. The README now names all three owners and says the part that would otherwise be found the hard way: building an image needs MoE_Trace's mapper. The harness runs without it once an image exists, which is why nothing here breaks. Co-Authored-By: Claude Opus 5 --- moe-harness/README.md | 25 +- .../femu_handoff/packages/LAYER_TRACE_V1.md | 87 --- .../packages/QLC_ALIGNED_LAYOUT_V1.md | 124 ---- .../moe_bcq/femu_handoff/packages/bundle.py | 208 ------- .../packages/layer_read_groups.py | 246 -------- .../femu_handoff/packages/logical_reads.py | 100 ---- .../packages/qlc_aligned_mapper.py | 549 ------------------ .../femu_handoff/packages/trace_compiler.py | 230 -------- 8 files changed, 21 insertions(+), 1548 deletions(-) delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/layer_read_groups.py delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py delete mode 100644 moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py diff --git a/moe-harness/README.md b/moe-harness/README.md index 28dfee719d3..6fc61d578aa 100644 --- a/moe-harness/README.md +++ b/moe-harness/README.md @@ -26,16 +26,33 @@ tooling) stays there, because it needs the model weights and a GPU. exp/moe_bcq/femu_handoff/packages/ replay_v1.c the guest replayer, QD=32 O_DIRECT AIO - qlc_aligned_mapper.py plan / validate / materialize an image - trace_compiler.py mapped JSONL -> replay_qd32.bin - layer_read_groups.py trace -> per-layer read groups under a cache - bundle.py, logical_reads.py payload access + REPLAYER_V1.md the binary format it consumes exp/gating_nand/femu/make_seed.py cloud-init seed carrying the binaries scripts/femu_compose.sh compose wrapper scripts/build_replay.sh builds the guest replayer runs/femu/run01.env per-run environment template +## Where the rest lives + +Three things are kept apart by who owns them, because the copies that used to +exist on both hosts drifted -- one pair of mapper copies ended up 64 lines +apart, and an edit to run_policy.sh was reverted twice by a sync. + +| | | +|---|---| +| this repository | the harness, `replay_v1.c`, the emulator | +| `MoE_Trace` | the mapper, `bundle.py`, the rest of the trace tooling | +| neither, they are data | payload, layouts, `replay_qd32.bin`, images | + +The split has a consequence worth stating plainly: **building an image needs +`MoE_Trace`.** `qlc_aligned_mapper.py materialize` turns a layout and a payload +into the image this harness fills a device from, and it is not in this +repository. Clone `MoE_Trace` alongside, or have the image built where that +tooling already is and shipped as data. + +Nothing here calls it, so the harness runs without it once an image exists. + ## What it needs that is not here The payload (`planes.bin`, `scales.bin`, about 15 GB for the two models), the diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md b/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md deleted file mode 100644 index 992e565dccc..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/LAYER_TRACE_V1.md +++ /dev/null @@ -1,87 +0,0 @@ -# 레이어별 읽기 그룹 v1 — 2026-09-09 - -이번 기본 입력은 `traces/prefill_w4_decode_mixed_v1/`이다. 기존 `traces/smoke/`는 DRAM 적용 전의 별도 수집이며 그대로 보존했다. 모델 payload는 기존 C(일반 4-bit BCQ + 공유 FP16 scale, MRE 없음)를 사용한다. - -**2026-09-09 QLC mapper용 파생 trace를 추가했다.** 기존 `layer_groups_lru_2147483648B/`는 scale 별도 상주 조건이고, 새 `layer_groups_lru_2147483648B_scales_on_demand/`는 plane과 요청된 scale column이 같은 2 GiB LRU를 공유한다. QLC-aligned 실험은 새 파생 trace를 사용한다. 배치 규격과 결과는 [QLC_ALIGNED_LAYOUT_V1.md](QLC_ALIGNED_LAYOUT_V1.md)에 있다. - -## 실행 조건 - -- 모델별 WikiText-2의 128-token 입력 2개, batch=1, 입력마다 32-token greedy 생성. 첫 생성 token은 prefill에서 나오므로 요청당 decode forward는 31회다. 모델별 tokenizer 및 생성 결과가 달라 동일한 token 경로를 비교하는 실험은 아니다. -- Prefill: 실제로 선택된 expert만 W4로 실행. Decode: gate score 기반 W2/W3/W4, 임계값 0.16/0.075. 변경된 정책으로 실제 모델을 다시 실행해 수집했다. -- Host DRAM plane-cache: **2 GiB = 2,147,483,648 B**, projection별 plane 단위 LRU. 각 요청은 빈 캐시에서 시작하고 prefill 이후 상태를 decode까지 유지한다. 두 요청 사이에는 초기화한다. -- 동일 레이어의 token들이 요구하는 expert-plane을 중복 제거한다. 현재 레이어가 요구한 항목은 해당 그룹 처리 중 퇴출하지 않는다. 같은 그룹 내 LRU 순서는 item ID 사전순으로 정하며 실제 GPU 실행 순서를 뜻하지 않는다. -- Scale은 별도 DRAM 상주: Qwen 778,567,680 B, DeepSeek 899,678,208 B. **2 GiB에 포함되지 않는다.** 비양자화 가중치, KV cache, 관리 인덱스, allocator, 전송 staging 및 GPU 작업 공간도 이 plane 예산 밖이다. -- QLC mapper용 파생 trace에서는 위 scale 상주 가정을 해제했다. Wp가 요구하는 공유 `alpha_4` column 1…p를 SSD 대상에 포함하고 plane과 scale을 합쳐 2 GiB LRU를 적용했다. -- xPU의 지속적인 routed weight cache와 prefetch는 가정하지 않는다. 읽기 완료 후 cache에 적재하고, 현재 그룹의 읽기와 계산이 완료된 다음 그룹을 진행하는 순서만 표현한다. 시간·지연 측정은 없다. - -## 결과와 파일 - -| 모델 | Prefill 그룹 | Decode 그룹 | Decode byte hit 비율 | Decode 읽기 없는 그룹 | -|---|---:|---:|---:|---:| -| Qwen | 48 | 1,488 | 55.07% | 228 | -| DeepSeek | 52 | 1,612 | 58.99% | 185 | - -이 수치는 짧은 입력에서의 캐시 시뮬레이션 결과이며 성능 벤치마크나 SSD 측정값이 아니다. Byte hit 비율은 `hit_bytes / demand_bytes`다. 요청마다 초기화하므로 prefill의 routed plane hit는 0이다. - -- [Qwen 요약](qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B/summary.json) -- [DeepSeek 요약](deepseek_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B/summary.json) -- [Qwen scale 포함 요약](qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/summary.json) -- [DeepSeek scale 포함 요약](deepseek_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/summary.json) - -각 모델의 위 요약과 같은 디렉터리에 다음 두 파일이 있다. - -| 파일 | 의미 | -|---|---| -| `layer_demands.jsonl` | 레이어 실행에 필요한 전체 plane 집합. DRAM 정책 적용 전 입력 | -| `layer_reads.jsonl` | DRAM miss로 남은 읽기 구간과 hit·eviction·cache byte 기록 | -| `summary.json` | 정책, 단계별 통계, 원본과 출력의 SHA256. 이 파일이 있어야 변환 완료 | -| `validation.json` | 출력 파일을 다시 읽어 캐시 상태·그룹 순서·통계를 독립적으로 확인한 결과. `verify_layer_groups.py`가 만든다 | - -이 표는 `validation.json`을 오래 전부터 적어 두었지만 실제로 만드는 코드는 없었다. 그동안 -캐시 시뮬레이션은 파이프라인에서 유일하게 검사받지 않는 단계였다. FEMU 카운터 일치는 이 -단계를 덮지 못한다. `layer_reads.jsonl`이 mapper의 **입력**이라 캐시가 틀린 항목을 miss로 -판정해도 매핑·재생·집계가 그대로 일관되게 따라가기 때문이다. 자기일관성은 정확성이 아니다. - -`verify_layer_groups.py`는 쓰여진 파일만 다시 읽어 네 가지를 확인한다. - -| 확인 | 내용 | -|---|---| -| 순서 | group id, layer 주기, forward 번호, 요청 경계와 decode 위치를 `layer_reads.jsonl`만으로 재유도 | -| 수요 | `layer_demands.jsonl`과 `layer_reads.jsonl`이 그룹마다 일치 | -| 캐시 | 모든 hit·miss·eviction·점유량을 `online_cache.py`의 `ReferenceLRU`가 재현. 이 구현은 숫자를 쓴 `LayerLRU`와 코드를 공유하지 않는다 | -| 통계 | `summary.json`의 단계별·요청별 합계와 최대 점유량을 그룹 줄에서 다시 누적 | - -`ReferenceLRU`가 검사로서 힘이 있는지는 무작위 차등 시험으로 확인했다. 300회 × 120단계 -동안 두 구현은 miss 집합·바이트·eviction·recency 순서까지 일치했고, 캐시에 넣은 세 가지 -결함(현재 레이어 항목을 evict, 삽입 순서를 item_id 대신 수요 순서로, hit의 recency 미갱신)은 -모두 잡혔다. - -원본 `logical_trace.jsonl`, `trace_meta.json`, `inputs.json`, `generations.json`, `code/`는 한 단계 위 디렉터리에 있다. token별 선택과 실행 당시 코드까지 확인할 수 있다. - -JSONL 한 줄은 `(request, forward, layer)` 하나다. `group_id`와 `release_after_group_id`가 순서를 나타낸다. 새 요청 첫 그룹은 `cache_reset=true`, 이전 그룹 ID는 null이다. 전부 hit인 그룹도 `reads=[]`로 남겨 계산 순서를 보존한다. 각 read의 `file, offset, nbytes`는 원본 payload 파일 구간이며 `item_id`는 projection/plane 식별자다. - -## FEMU에 넘기기 전 남은 주소 매핑 - -원본 `layer_reads.jsonl`은 LBA를 부여하기 전의 DRAM miss trace다. `lba_start`와 `sector_count`는 null, `address_status`는 `unmapped`다. QLC-aligned mapper가 이를 변환한 `layouts/.../mapped_reads.jsonl`에는 LBA와 NVMe command가 들어 있지만, 실제 FEMU replay는 아직 수행하지 않았다. - -다음 단계에서 각 SSD 레이아웃의 실제 image extent map을 만든 뒤 원본 파일 구간을 LBA로 변환한다. 이때 sector 크기, 정렬·padding, 요청 분할·병합, 최대 요청 크기를 명시하고 image/layout 체크섬을 연결해야 한다. 원본 offset을 곧바로 LBA로 간주하지 않는다. 하나의 logical read가 여러 NVMe 요청으로 나뉘거나 인접 read와 합쳐질 수 있다. Scale 초기 적재 I/O는 현재 trace에 없으므로 필요하면 별도 초기화 단계로 측정한다. - -같은 DRAM 정책과 논리 요청에서 레이아웃만 비교하려면 이 miss trace를 공통 입력으로 쓴다. 페이지 단위 caching이나 layout별 read-ahead로 cache admission이 바뀌는 실험이라면 `layer_demands.jsonl`부터 해당 정책으로 다시 시뮬레이션해야 한다. - -## 재현 - -`packages/`에 `layer_read_groups.py`, `test_layer_groups.py`와 의존 도구 `bundle.py`, `logical_reads.py`를 함께 넣었다. 변환 자체는 CPU와 NumPy만 필요하고 GPU나 체크포인트는 필요 없다. 다른 용량의 예: - -```bash -python layer_read_groups.py qwen_C --trace prefill_w4_decode_mixed_v1 --cache-bytes 1073741824 -``` - -제공된 2 GiB 출력은 이미 존재한다. 같은 경로 재실행은 덮어쓰지 않고 실패한다. 현재 레이어의 전체 작업 집합이 용량보다 크면 도구가 실패한다. 그런 용량에서는 먼저 그룹 내 streaming 순서를 정의해야 한다. - -캐시 fixture 검증: - -```bash -python -m unittest discover -s . -p 'test_layer_groups.py' -``` - -새 GPU 수집은 원래 저장소의 `collect_trace.py`에 `--mode generate --prefill-policy w4 --window 128 --samples 2 --max-new-tokens 32`를 지정한다. Qwen은 `models/Qwen1.5-MoE-A2.7B`와 `exp/moe_bcq/results/model/qwen_plain4_state.pt`, DeepSeek은 해당 모델 디렉터리와 `exp/moe_bcq/results/model/deepseek_plain4_state.pt`를 사용했다. 수집 정책과 코드·입력 식별자는 `trace_meta.json`, 원본 state 식별자는 manifest에도 기록되어 있다. diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md b/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md deleted file mode 100644 index 988b71d974e..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/QLC_ALIGNED_LAYOUT_V1.md +++ /dev/null @@ -1,124 +0,0 @@ -# QLC-aligned expert plane-major mapper v1 — 2026-09-09 - -이 산출물은 C arm(일반 4-bit BCQ, 공유 `alpha_4` prefix, MRE 없음)의 routed expert plane과 scale을 QLC page class에 맞춘 LBA 주소표다. 아직 FEMU 장치에 적재하거나 latency를 측정하지 않았다. - -## 고정한 FEMU geometry - -```text -sector_bytes = 512 -sectors_per_page = 32 # 16 KiB -pages_per_block = 512 # patched pairing table 필수 -blocks_per_plane = 1024 -planes_per_lun = 1 -luns_per_channel = 4 -channels = 2 # 총 8 LUN -op_percent = 7 -nand_cell_type = 4 -gc_threshold = 75 -``` - -Raw capacity는 64 GiB이고 명목상 7% OP 적용 후 공간은 약 59.52 GiB다. 실제 FEMU가 노출하는 namespace 크기는 부팅 후 반드시 확인한다. - -이 mapper는 수정된 512-row QLC pairing을 전제로 한다. pg 0–5는 class 0, pg 6–7은 class 1인 특수 prologue라 사용하지 않는다. pg 8–511에서는 다음 주기가 반복되어야 한다. - -```text -page index mod 8: 0 1 2 3 4 5 6 7 -QLC class: 0 0 1 1 2 2 3 3 -``` - -원본 FEMU의 `rows-1` 문제를 고치지 않으면 pg 496–511이 class 0으로 남는다. 이 상태에서는 mapper 예측과 실제 장치가 다르므로 실험하면 안 된다. - -## Plane과 scale 배치 - -배치 순서는 `layer → expert → tier → gate/up/down`이고, 대응은 다음과 같다. - -| 정밀도 구성요소 | QLC class | -|---|---:| -| B1와 alpha1 | 0 | -| B2와 alpha2 | 1 | -| B3와 alpha3 | 2 | -| B4와 alpha4 | 3 | - -여기서 alpha1은 별도 최적화된 scale 세트가 아니라 공유 `alpha_4`의 첫 번째 column이다. Wp는 B1…Bp와 `alpha_4` column 1…p를 요구한다. - -총 8 LUN에서 class 하나의 slot은 `2 pages × 8 LUN = 16 LPN`이다. 현재 expert 한 개의 tier별 크기는 다음과 같다. - -```text -gate/up/down plane = 22 + 22 + 22 = 66 pages -gate/up/down scale = ceil(2.75) × 3 = 9 allocated pages -합계 = 75 pages -필요한 class cycle = ceil(75 / 16) = 5 -``` - -따라서 한 expert는 5개의 8-page pairing cycle, 즉 `5 × 8 page-index × 8 LUN = 320 LPN = 5 MiB`의 주소 공간을 사용한다. 각 tier의 자료는 같은 다섯 cycle에서 대응되는 class slot에 놓인다. 66-page plane은 같은 class의 여러 slot으로 나뉘므로 한 logical item이 `extent_map.json`에서 여러 fragment를 가질 수 있다. - -Scale extent 하나는 45,056 B로 sector에는 정확히 맞지만 NAND page에는 맞지 않는다. 각 projection scale column을 page 경계에서 시작하도록 3 pages를 할당하고 마지막 4 KiB는 filler로 둔다. 실제 read는 유효한 45,056 B만 요청한다. - -## 생성 결과 - -| 모델 | 실제 plane+scale | 순차 image 크기 | filler | payload 비율 | -|---|---:|---:|---:|---:| -| Qwen | 7,007,109,120 B | 7,717,519,360 B (7.19 GiB) | 710,410,240 B | 90.79% | -| DeepSeek | 8,097,103,872 B | 8,925,478,912 B (8.31 GiB) | 828,375,040 B | 90.72% | - -- [Qwen layout summary](qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun/layout_summary.json) -- [DeepSeek layout summary](deepseek_C/layouts/qlc_aligned_epm_aif_2ch4lun/layout_summary.json) - -각 layout 디렉터리에는 다음 파일이 있다. - -| 파일 | 역할 | -|---|---| -| `extent_map.json` | 전체 plane·scale item의 원본 구간, target LPN/LBA, fragment와 목표 class | -| `mapped_reads.jsonl` | 2 GiB LRU miss를 LBA로 바꾼 레이어 그룹과 병합된 NVMe command | -| `layout_summary.json` | geometry, 용량, 입력·출력 SHA256, QD=32 권장값 | -| `layout_validation.json` | catalog·fragment·class·주소 중복·trace byte 보존 검증 결과 | -| `replay_qd32.bin` | strict group barrier와 QD=32 기본값을 담은 compact replay 입력 | -| `replay_qd32.bin.json` | binary hash, 원본 hash, record 총계와 request index 표 | - -현재 mapped trace는 plane과 scale column이 **같은 2 GiB LRU cache를 공유**한 결과를 사용한다. - -```text -traces/prefill_w4_decode_mixed_v1/ - layer_groups_lru_2147483648B_scales_on_demand/ -``` - -Mapper는 같은 class에서 LBA가 바로 이어진 fragment만 최대 4 MiB까지 하나의 command로 병합한다. 서로 다른 class를 가로질러 병합하지 않는다. Replayer는 이 command에 QD=32를 적용한다. 게스트의 실제 최대 request 크기가 4 MiB보다 작으면 block layer가 다시 나눌 수 있으므로 `max_sectors_kb`와 실제 NVMe command 수를 기록해야 한다. - -Binary 생성과 실제 실행 방법, rolling QD와 timestamp 의미는 [REPLAYER_V1.md](REPLAYER_V1.md)를 따른다. - -## 도구 사용 - -주소표와 mapped trace 생성: - -```bash -python qlc_aligned_mapper.py plan qwen_C \ - --layer-reads qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/layer_reads.jsonl \ - --output qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun -``` - -검증: - -```bash -python qlc_aligned_mapper.py validate qwen_C \ - --layer-reads qwen_C/traces/prefill_w4_decode_mixed_v1/layer_groups_lru_2147483648B_scales_on_demand/layer_reads.jsonl \ - qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun -``` - -실제 image가 필요할 때만 materialize한다. 제공된 package에는 image가 없다. - -```bash -python qlc_aligned_mapper.py materialize qwen_C \ - qwen_C/layouts/qlc_aligned_epm_aif_2ch4lun --image qwen_C.img -``` - -생성된 image는 filler를 포함한다. FEMU 재시작 직후 host write가 0인지 확인한 다음, **LPN 0부터 끝까지 빠짐없이 순차 write**해야 한다. 최종 device command의 분할 경계도 16 KiB NAND page에 정렬되어야 한다. page 중간에서 나뉘면 같은 LPN이 두 번 program되어 후속 PPA가 밀릴 수 있다. 현재 guest에서는 `dd bs=256K`로 적재한다. 기존 `bs=4M`은 이 조건을 보장하지 못했다. queue 제한과 적재 후 FEMU WRITE 로그의 LPN→PPA class 규칙을 검증하고, 데이터를 되읽어 원본 SHA와 별도로 비교한 뒤 replay한다. - -## 아직 측정되지 않은 것 - -- FEMU에서 실제 PPA/page class 일치 여부 -- materialized image의 byte-for-byte read-back -- QD=32 replay의 제출·완료 timestamp와 latency -- OS/NVMe 계층의 실제 요청 split·merge -- GPU 전송 및 BCQ 연산을 포함한 end-to-end latency - -따라서 현재 결과는 **검증된 주소 계획과 replay 입력**이며 FEMU 성능 결과가 아니다. diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py deleted file mode 100644 index 2efc5c5a7b8..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/bundle.py +++ /dev/null @@ -1,208 +0,0 @@ -#!/usr/bin/env python3 -"""Portable, unpadded routed-BCQ payloads. No CUDA dependency.""" -from __future__ import annotations - -import argparse -from contextlib import ExitStack -import hashlib -import json -import re -from pathlib import Path - -import numpy as np - -SCHEMA = "moe-bcq-handoff-v1" -ROUTED = re.compile(r"^model\.layers\.(\d+)\.mlp\.experts\.(\d+)\.(gate|up|down)_proj\.qweight$") - - -def sha256(path): - h = hashlib.sha256() - with open(path, "rb") as f: - for b in iter(lambda: f.read(8 << 20), b""): - h.update(b) - return h.hexdigest() - - -def write_json(path, value): - Path(path).write_text(json.dumps(value, indent=2, ensure_ascii=False) + "\n") - - -def emit(f, tensor, dtype): - arr = np.asarray(tensor.numpy(), dtype=dtype, order="C") - data = arr.tobytes() - rec = dict(file=Path(f.name).name, offset=f.tell(), nbytes=len(data), - dtype=np.dtype(dtype).str, shape=list(arr.shape), - sha256=hashlib.sha256(data).hexdigest()) - f.write(data) - return rec - - -def export(state_path, model_dir, arm, out): - import torch - torch.set_num_threads(2) - out, model_dir, state_path = Path(out), Path(model_dir), Path(state_path) - out.mkdir(parents=True, exist_ok=False) - config = json.loads((model_dir / "config.json").read_text()) - state = torch.load(state_path, map_location="cpu", weights_only=True, mmap=True) - keys = [(ROUTED.fullmatch(k), k) for k in state if ROUTED.fullmatch(k)] - keys.sort(key=lambda t: (int(t[0][1]), int(t[0][2]), t[0][3])) - if not keys: - raise ValueError("No routed expert qweight tensors") - manifest = dict(schema=SCHEMA, model=model_dir.name, arm=arm, - scale_mode="per_precision" if arm == "A" else "shared_alpha4_prefix", - supported_bits=[2, 3, 4], source_state=dict(name=state_path.name, - sha256=sha256(state_path)), model_config=config, - source_config_sha256=sha256(model_dir / "config.json"), - mre_steps="not inferred from tensor data; see source quantization report", - exporter_sha256=sha256(__file__), byte_order="little", padding_bytes=0, - beta="implicit_zero_verified", bias="absent_verified", - native_qweight_order=["input_word32", "plane", "output"], - plane_order=["input_word32", "output"], - scale_column_order=["input_group", "output"], - bit_encoding="bit t of word k is input 32*k+t; 0=-1, 1=+1", - placement="unassigned: file offsets are NOT LBA or NAND pages", - projections=[]) - with (out / "planes.bin").open("xb") as pf, (out / "scales.bin").open("xb") as sf: - for m, key in keys: - prefix = key.removesuffix("qweight") - q = state[key] - a4 = state[prefix + "alpha_4"] - assert q.dtype == torch.int32 and q.ndim == 3 and q.shape[1] == 4 - i, o = q.shape[0] * 32, q.shape[2] - assert a4.dtype == torch.float16 and a4.shape[1:] == (4, o) - assert i % a4.shape[0] == 0 - g = i // a4.shape[0] - assert g % 32 == 0 - assert prefix + "bias" not in state, "Nonzero/explicit bias needs schema extension" - use = [2, 3, 4] if arm == "A" else [4] - for p in use: - a, b = state[prefix + f"alpha_{p}"], state[prefix + f"beta_{p}"] - assert a.shape == (i // g, p, o) and a.dtype == torch.float16 - assert torch.isfinite(a).all(), f"Invalid alpha: {prefix}" - assert b.shape == (i // g, o) and torch.count_nonzero(b) == 0, prefix - rec = dict(id=prefix.rstrip("."), layer=int(m[1]), expert=int(m[2]), - projection=m[3] + "_proj", weight_shape_out_in=[o, i], - group_size=g, planes=[], scales={}) - for j in range(4): - rec["planes"].append(emit(pf, q[:, j, :], " self.capacity: - raise ValueError("Layer working set exceeds cache capacity; define a streaming schedule first") - for key, e in wanted.items(): - if key in self.items and self.items[key] != e['nbytes']: - raise ValueError("Cache item size changed") - hits = [k for k in wanted if k in self.items] - misses = [dict(item_id=k, **e) for k, e in wanted.items() if k not in self.items] - miss_bytes = sum(e['nbytes'] for e in misses) - before = self.used - evicted = [] - # Decide every hit before insertion; never evict a current-layer demand. - for key in list(self.items): - if self.used + miss_bytes <= self.capacity: - break - if key not in wanted: - size = self.items.pop(key) - self.used -= size - evicted.append(dict(item_id=key, nbytes=size)) - assert self.used + miss_bytes <= self.capacity - # Stable tie break within a layer; not an assertion of GPU expert order. - for key in sorted(wanted): - size = wanted[key]['nbytes'] - if key in self.items: - del self.items[key] - else: - self.used += size - self.items[key] = size - hit_bytes = sum(wanted[k]['nbytes'] for k in hits) - assert hit_bytes + miss_bytes == sum(e['nbytes'] for e in extents) - assert self.used == before - sum(e['nbytes'] for e in evicted) + miss_bytes - assert self.used <= self.capacity - return dict(hit_item_ids=hits, hit_bytes=hit_bytes, misses=misses, - miss_bytes=miss_bytes, evictions=evicted, - cache_bytes_before=before, cache_bytes_after=self.used) - - -def build(bundle, trace_name, capacity_bytes, scales='resident'): - bundle = Path(bundle) - trace = bundle / 'traces' / trace_name - mpath = bundle / 'manifest.json' - manifest = json.loads(mpath.read_text()) - meta = json.loads((trace / 'trace_meta.json').read_text()) - assert manifest['schema'] == meta['schema'] == SCHEMA - assert meta['manifest_sha256'] == sha256(mpath) - assert meta['state_sha256'] == manifest['source_state']['sha256'] - assert meta['mode'] == 'generate', 'Keep teacher-forced workloads separate' - assert meta['phase_policies'] == dict(prefill='w4', decode='gated_mixed', teacher_forced='gated_mixed') - for filename, info in meta['files'].items(): - assert sha256(trace / filename) == info['sha256'] - assert scales in ('resident', 'on_demand') - suffix = '' if scales == 'resident' else '_scales_on_demand' - out = trace / f'layer_groups_lru_{capacity_bytes}B{suffix}' - # If the collector ran a cache beside the model, this derivation has to - # reproduce it group for group. That is the only check on the cache itself: - # the FEMU counters agree with the mapper, but the mapper's input is this - # file, so a cache that misses wrongly is replayed wrongly and consistently. - record = trace / 'online_cache.jsonl' - online, online_rows = None, {} - if record.exists(): - header, online_rows = online_cache.load(record) - assert header['manifest_sha256'] == meta['manifest_sha256'] - if header['scales'] == scales and capacity_bytes in header['capacities']: - online = str(capacity_bytes) - else: - online_rows = {} - out.mkdir(exist_ok=False) - cache = LayerLRU(capacity_bytes) - previous_request, previous_forward, previous_layer = None, -1, None - previous_group = None - seen_requests = set() - frame_info = None - next_position = None - phases = {'prefill': Counter(), 'decode': Counter()} - request_stats = {} - peak = 0 - groups = 0 - verified = 0 - source = trace / 'logical_trace.jsonl' - with source.open() as src, (out / 'layer_demands.jsonl').open('x') as df, (out / 'layer_reads.jsonl').open('x') as rf: - for line in src: - event = json.loads(line) - gid = event['event_id'] - rid = event['request_id'] - phase = event['phase'] - fid, layer = event['forward_id'], event['layer'] - assert gid == groups and phase in phases - new_forward = fid != previous_forward - reset = rid != previous_request - if new_forward: - assert fid == previous_forward + 1 - if previous_layer is not None: - assert previous_layer == manifest['totals']['layers'][-1] - assert layer == manifest['totals']['layers'][0] - frame_info = (rid, phase, event['input_ids'], event['token_positions']) - if reset: - assert rid not in seen_requests and phase == 'prefill' - assert event['token_positions'] == list(range(len(event['input_ids']))) - seen_requests.add(rid) - cache.clear() - previous_group = None - next_position = len(event['input_ids']) - request_stats[rid] = {'prefill': Counter(), 'decode': Counter()} - else: - assert phase == 'decode' and event['token_positions'] == [next_position] - assert len(event['input_ids']) == 1 - next_position += 1 - else: - assert not reset and frame_info == (rid, phase, event['input_ids'], event['token_positions']) - order = manifest['totals']['layers'] - assert layer == order[order.index(previous_layer) + 1] - if phase == 'prefill': - assert all(b == 4 for row in event['precision_bits'] for b in row) - extents = demand(manifest, event, scales_resident=(scales == 'resident')) - result = cache.serve(extents) - common = dict(group_id=gid, request_id=rid, forward_id=fid, layer=layer, - phase=phase, batch_size=event['batch_size'], - token_positions=event['token_positions'], cache_reset=reset, - release_after_group_id=previous_group, - barrier='all reads and compute of this group precede next group', - demand_items=len(extents), demand_bytes=sum(e['nbytes'] for e in extents)) - df.write(json.dumps(dict(**common, demands=[dict(item_id=item_key(e), **e) for e in extents]), separators=(',', ':'))+'\n') - misses = result.pop('misses') - reads = [dict(read_id=f'{gid}:{j}', **e, lba_start=None, sector_count=None, - address_status='unmapped', operation='read') for j, e in enumerate(misses)] - if online is not None: - row = online_rows[gid] - assert row['p'] == phase - observed = dict(zip(online_cache.ROW_FIELDS, row['c'][online])) - derived = dict(miss_items=len(reads), miss_bytes=result['miss_bytes'], - hit_bytes=result['hit_bytes'], - evicted_items=len(result['evictions']), - evicted_bytes=sum(e['nbytes'] for e in result['evictions']), - used_after=result['cache_bytes_after'], - miss_digest=online_cache.digest(sorted(m['item_id'] for m in misses))) - if [len(extents), common['demand_bytes']] != row['d'] or observed != derived: - raise AssertionError(f'Online cache disagrees at group {gid}: ' - f'observed {row["d"]} {observed} vs derived ' - f'{[len(extents), common["demand_bytes"]]} {derived}') - verified += 1 - rf.write(json.dumps(dict(**common, **result, reads=reads), separators=(',', ':'))+'\n') - stat = dict(groups=1, demand_items=len(extents), hit_items=len(result['hit_item_ids']), - miss_items=len(reads), demand_bytes=common['demand_bytes'], - hit_bytes=result['hit_bytes'], miss_bytes=result['miss_bytes'], - evicted_items=len(result['evictions']), evicted_bytes=sum(e['nbytes'] for e in result['evictions']), - all_hit_groups=int(not reads)) - phases[phase].update(stat) - request_stats[rid][phase].update(stat) - peak = max(peak, cache.used) - previous_request, previous_forward, previous_layer = rid, fid, layer - previous_group = gid - groups += 1 - assert groups == meta['events'] and previous_layer == manifest['totals']['layers'][-1] - if online is not None: - assert verified == groups == len(online_rows), 'Online record does not cover every group' - check = dict(status='verified', origin=header['origin'], groups=verified, - record_sha256=sha256(record), - implementation=('online_cache.py ReferenceLRU, run in-process during GPU execution' - if header['origin'] == 'in_process' else - 'online_cache.py ReferenceLRU, replayed from the trace file; ' - 'checks the cache logic, not the trace'), - covers='per-group miss count, miss/hit bytes, eviction count and bytes, ' - 'occupancy, and a digest of which items missed') - elif record.exists(): - check = dict(status='not_applicable', groups=0, - reason=f'record holds scales={header["scales"]} capacities={header["capacities"]}') - else: - check = dict(status='absent', groups=0, - reason='trace was collected before the in-process cache existed; ' - 'this derivation is unchecked') - summary = dict(schema='moe-layer-read-groups-v1', groups=groups, requests=len(seen_requests), - model=manifest['model'], arm=manifest['arm'], phase_policies=meta['phase_policies'], - cache=dict(policy='LRU', capacity_bytes=capacity_bytes, granularity='projection-plane', - recency='whole layer; ties use lexicographic item_id', reset='each request; prefill retained for decode', - protected='all demanded items until layer completion', peak_payload_bytes=peak, - prefetch=False, admission='all misses on completion before next layer'), - scales=scales, online_check=check, - resident_scales_bytes=(manifest['totals']['scales_bytes'] if scales == 'resident' else 0), - budget_note=('Cache capacity is plane payload only; scales are additional.' if scales == 'resident' - else 'Planes and requested scale columns share this cache capacity.') - + ' Python index, allocator, GPU working buffers and transport staging not modeled.', - xpu_policy='no persistent routed-weight cache; current call working data only', - timing='barrier order only; no timestamps or compute/transfer latency', - mapping=dict(status='pending', layout_sha256=None, sector_bytes=None, - note='Source offsets are NOT LBA. Map to actual image extents before FEMU replay.'), - phase_stats=phases, request_stats=request_stats, source_manifest_sha256=sha256(mpath), - source_trace_sha256=sha256(source), source_meta_sha256=sha256(trace / 'trace_meta.json'), - builder_sha256=sha256(__file__), files={name:dict(sha256=sha256(out/name), nbytes=(out/name).stat().st_size) - for name in ['layer_demands.jsonl','layer_reads.jsonl']}) - write_json(out / 'summary.json', summary) - print(json.dumps(dict(output=str(out), groups=groups, phase_stats=phases)), flush=True) - return summary - - -def main(): - p = argparse.ArgumentParser(description=__doc__) - p.add_argument('bundle', type=Path) - p.add_argument('--trace', default='prefill_w4_decode_mixed_v1') - p.add_argument('--cache-bytes', type=int, default=2*1024**3) - p.add_argument('--scales', choices=['resident', 'on_demand'], default='resident') - a = p.parse_args() - build(a.bundle, a.trace, a.cache_bytes, a.scales) - - -if __name__ == '__main__': - main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py deleted file mode 100644 index 7be03c0178b..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/logical_reads.py +++ /dev/null @@ -1,100 +0,0 @@ -#!/usr/bin/env python3 -"""Validate a trace and resolve it to file extents, NOT SSD page addresses. - -One cold demand set per forward/layer; deduplicate prefix planes within the set. -For A, keep EVERY used precision's scale set, even if a higher tier is present. -""" -import argparse -import json -import math -from collections import defaultdict -from pathlib import Path - -from bundle import SCHEMA, sha256, write_json - - -def demand(manifest, event, scales_resident=True): - catalog = defaultdict(list) - for p in manifest["projections"]: - if p["layer"] == event["layer"]: - catalog[p["expert"]].append(p) - wanted = defaultdict(set) - positions = event["token_positions"] - assert len(positions) == len(set(positions)) - assert len(positions) == len(event["input_ids"]) == len(event["selected_experts"]) == len(event["precision_bits"]) - assert len(positions) == len(event["gate_scores"]) - assert event["batch_size"] == 1 - assert positions == sorted(positions) and all(type(x) is int and x >= 0 for x in positions) - for es, bs, gs in zip(event["selected_experts"], event["precision_bits"], event["gate_scores"]): - assert len(es) == len(bs) == len(gs) == manifest["model_config"]["num_experts_per_tok"] - assert len(es) == len(set(es)) - assert all(math.isfinite(g) and g >= 0 for g in gs) - for expert, bits in zip(es, bs): - assert expert in catalog and bits in manifest["supported_bits"] - wanted[expert].add(bits) - extents = [] - for expert, precisions in sorted(wanted.items()): - for p in catalog[expert]: - for j, e in enumerate(p["planes"][:max(precisions)], 1): - extents.append(dict(projection_id=p["id"], kind="plane", plane=j, **e)) - if not scales_resident: - for b in sorted(precisions) if manifest["scale_mode"] == "per_precision" else [4]: - take = b if manifest["scale_mode"] == "per_precision" else max(precisions) - for j, e in enumerate(p["scales"][str(b)][:take], 1): - extents.append(dict(projection_id=p["id"], kind="scale", scale_set=b, column=j, **e)) - return extents - - -def main(): - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("bundle", type=Path) - p.add_argument("--trace", default="smoke") - p.add_argument("--scales", choices=["resident", "on_demand"], default="resident") - a = p.parse_args() - mpath = a.bundle / "manifest.json" - m = json.loads(mpath.read_text()) - tdir = a.bundle / "traces" / a.trace - meta = json.loads((tdir / "trace_meta.json").read_text()) - assert m["schema"] == meta["schema"] == SCHEMA - assert meta["manifest_sha256"] == sha256(mpath) - for name, info in meta["files"].items(): - assert sha256(tdir / name) == info["sha256"] - out = tdir / f"reads_{a.scales}.jsonl" - expected, total, last = 0, 0, None - layers_seen = [] - frame_info = None - with (tdir / "logical_trace.jsonl").open() as f, out.open("x") as dst: - for line in f: - e = json.loads(line) - assert e["event_id"] == expected - assert e["phase"] in ("prefill", "decode", "teacher_forced") - if e["phase"] == "decode": - assert len(e["input_ids"]) == 1 - order = (e["forward_id"], e["layer"]) - assert last is None or order > last - if last is None or last[0] != e["forward_id"]: - assert e["forward_id"] == (0 if last is None else last[0] + 1) - if last is not None: - assert layers_seen == m["totals"]["layers"], "Incomplete forward" - layers_seen = [] - frame_info = (e["request_id"], e["phase"], e["token_positions"], e["input_ids"]) - assert frame_info == (e["request_id"], e["phase"], e["token_positions"], e["input_ids"]) - layers_seen.append(e["layer"]) - last = order - extents = demand(m, e, a.scales == "resident") - nbytes = sum(x["nbytes"] for x in extents) - dst.write(json.dumps(dict(event_id=expected, forward_id=e["forward_id"], - layer=e["layer"], request_id=e["request_id"], phase=e["phase"], - demand_bytes=nbytes, extents=extents), separators=(",", ":")) + "\n") - expected += 1 - total += nbytes - assert expected == meta["events"] - assert layers_seen == m["totals"]["layers"], "Incomplete final forward" - write_json(tdir / f"reads_{a.scales}_summary.json", dict(events=expected, - cold_demand_bytes=total, scales=a.scales, sha256=sha256(out), - semantics="Deduplicated within forward/layer; no reuse across events. No LBA/page rounding/timing.")) - print(f"{expected} events, {total} logical bytes, scales={a.scales}") - - -if __name__ == "__main__": - main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py deleted file mode 100644 index 248da6402ee..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/qlc_aligned_mapper.py +++ /dev/null @@ -1,549 +0,0 @@ -#!/usr/bin/env python3 -"""Map routed BCQ planes and shared scale columns to aligned QLC LBAs. - -The mapping assumes a freshly reset FEMU device is filled sequentially from -LPN 0. Under that contract, allocation ordinal equals LPN and the closed-form -FEMU channel->LUN->page write pointer determines the physical page class. -""" -from __future__ import annotations - -import argparse -import hashlib -import json -import math -from collections import defaultdict -from dataclasses import asdict, dataclass -from itertools import zip_longest -from pathlib import Path - -from bundle import SCHEMA, sha256, write_json - - -LAYOUT_SCHEMA = "moe-bcq-qlc-aligned-layout-v1" -MAPPED_SCHEMA = "moe-bcq-mapped-layer-reads-v1" -PROJECTION_ORDER = ("gate_proj", "up_proj", "down_proj") - - -@dataclass(frozen=True) -class Geometry: - sector_bytes: int = 512 - sectors_per_page: int = 32 - pages_per_block: int = 512 - blocks_per_plane: int = 1024 - planes_per_lun: int = 1 - luns_per_channel: int = 4 - channels: int = 2 - op_percent: int = 7 - pairing_profile: str = "patched-512" - - @property - def page_bytes(self): - return self.sector_bytes * self.sectors_per_page - - @property - def parallel_luns(self): - return self.channels * self.luns_per_channel * self.planes_per_lun - - @property - def line_pages(self): - return self.pages_per_block * self.parallel_luns - - @property - def raw_bytes(self): - return self.line_pages * self.blocks_per_plane * self.page_bytes - - @property - def exposed_bytes_nominal(self): - return self.raw_bytes * (100 - self.op_percent) // 100 - - def validate(self): - if self.sector_bytes <= 0 or self.sectors_per_page <= 0: - raise ValueError("Sector/page geometry must be positive") - if self.pages_per_block != 512 or self.pairing_profile != "patched-512": - raise ValueError("This mapper requires the patched 512-row FEMU QLC pairing table") - if self.pages_per_block % 8 or self.parallel_luns <= 0: - raise ValueError("Unsupported FEMU geometry") - - -def qlc_class(page_in_block): - """Expected init_qlc_page_pairing class after the rows-1 fix.""" - if not 0 <= page_in_block < 512: - raise ValueError(page_in_block) - if page_in_block <= 5: - return 0 - if page_in_block <= 7: - return 1 - return (page_in_block % 8) // 2 - - -# Which QLC class slot each bit-plane tier is placed in. A policy is a -# permutation of the four slots, so every policy allocates exactly the same -# pages, the same fragments and the same NVMe commands -- only the physical page -# class under each plane changes. That is what makes the comparison controlled: -# the placement is the single variable, with byte layout and command structure -# held fixed. A baseline built by packing sequentially instead would also change -# the request pattern and confound the two. -def slot_for(tier, expert_ordinal, policy): - if policy == "aligned": - # B1 on the fastest class, B4 on the slowest. B1/B2 are read on every - # expert selection; B4 only for the top tier. - return tier - if policy == "inverted": - # The worst case: the always-read planes on the slowest pages. - return 3 - tier - if policy == "rotated": - # Class-oblivious control. Each expert shifts the permutation by one, so - # across experts every tier meets every class equally often and the - # read-frequency skew buys nothing. - return (tier + expert_ordinal) % 4 - raise ValueError(f"Unknown placement policy: {policy}") - - -PLACEMENT_POLICIES = ("aligned", "inverted", "rotated") - - -def item_key(kind, projection_id, index, scale_set=4): - if kind == "plane": - return f"{projection_id}/B{index}" - return f"{projection_id}/alpha{scale_set}/C{index}" - - -def canonical_hash(value): - data = json.dumps(value, sort_keys=True, separators=(",", ":")).encode() - return hashlib.sha256(data).hexdigest() - - -def group_catalog(manifest): - grouped = defaultdict(dict) - for rec in manifest["projections"]: - key = (rec["layer"], rec["expert"]) - if rec["projection"] in grouped[key]: - raise ValueError(f"Duplicate projection: {key} {rec['projection']}") - grouped[key][rec["projection"]] = rec - for key, projections in grouped.items(): - if set(projections) != set(PROJECTION_ORDER): - raise ValueError(f"Expected gate/up/down projections for {key}") - return [(key, grouped[key]) for key in sorted(grouped)] - - -def usable_cycle_base_lpn(cycle_index, geom): - """Each usable 8-page cycle contains class 0/1/2/3 slots.""" - cycles_per_block = (geom.pages_per_block - 8) // 8 - block, within = divmod(cycle_index, cycles_per_block) - page = 8 + 8 * within - return block * geom.line_pages + page * geom.parallel_luns - - -def target_fragments(source, item, tier, slot, expert_cycle, cursor_pages, geom): - """Place one source extent in same-class slots, returning page-padded fragments.""" - slot_pages = 2 * geom.parallel_luns - remaining = source["nbytes"] - source_delta = 0 - fragments = [] - while remaining: - cycle_delta, within = divmod(cursor_pages, slot_pages) - room_pages = slot_pages - within - take = min(remaining, room_pages * geom.page_bytes) - allocated_pages = math.ceil(take / geom.page_bytes) - base = usable_cycle_base_lpn(expert_cycle + cycle_delta, geom) - target_lpn = base + slot * slot_pages + within - for page_delta in range(allocated_pages): - physical_page = ((target_lpn + page_delta) // geom.parallel_luns) % geom.pages_per_block - if qlc_class(physical_page) != slot: - raise AssertionError((item, tier, slot, target_lpn, physical_page)) - fragments.append(dict(source_offset=source["offset"] + source_delta, - source_nbytes=take, target_lpn=target_lpn, - target_offset=target_lpn * geom.page_bytes, - lba_start=target_lpn * geom.sectors_per_page, - sector_count=math.ceil(take / geom.sector_bytes), - allocated_pages=allocated_pages, - allocated_bytes=allocated_pages * geom.page_bytes, - page_class=slot)) - remaining -= take - source_delta += take - cursor_pages += allocated_pages - return fragments, cursor_pages - - -def make_extent_map(bundle, manifest, geom, policy="aligned"): - if manifest["schema"] != SCHEMA: - raise ValueError("Unsupported bundle schema") - if manifest["scale_mode"] != "shared_alpha4_prefix": - raise ValueError("v1 QLC mapper expects shared alpha4-prefix scales") - if policy not in PLACEMENT_POLICIES: - raise ValueError(f"Unknown placement policy: {policy}") - geom.validate() - groups = group_catalog(manifest) - slot_pages = 2 * geom.parallel_luns - entries = [] - seen = set() - cycles_per_expert = None - for expert_ordinal, ((layer, expert), projections) in enumerate(groups): - # All four tiers must consume identical page positions to preserve pairing. - layouts = [] - for tier in range(4): - sources = [] - for projection in PROJECTION_ORDER: - rec = projections[projection] - sources.append(("plane", rec, rec["planes"][tier])) - for projection in PROJECTION_ORDER: - rec = projections[projection] - sources.append(("scale", rec, rec["scales"]["4"][tier])) - layouts.append(sources) - allocated_shape = [sum(math.ceil(src["nbytes"] / geom.page_bytes) - for _, _, src in sources) for sources in layouts] - if len(set(allocated_shape)) != 1: - raise ValueError(f"Tier allocation mismatch for layer={layer}, expert={expert}") - need_cycles = math.ceil(allocated_shape[0] / slot_pages) - if cycles_per_expert is None: - cycles_per_expert = need_cycles - elif cycles_per_expert != need_cycles: - raise ValueError("v1 requires a uniform expert footprint") - expert_cycle = expert_ordinal * cycles_per_expert - for tier, sources in enumerate(layouts): - cursor = 0 - for kind, rec, src in sources: - index = tier + 1 - key = item_key(kind, rec["id"], index) - if key in seen: - raise ValueError(f"Duplicate item: {key}") - seen.add(key) - slot = slot_for(tier, expert_ordinal, policy) - fragments, cursor = target_fragments(src, key, tier, slot, - expert_cycle, cursor, geom) - entries.append(dict(item_id=key, kind=kind, layer=layer, - expert=expert, projection=rec["projection"], - projection_id=rec["id"], plane=(index if kind == "plane" else None), - scale_set=(4 if kind == "scale" else None), - column=(index if kind == "scale" else None), - source_file=src["file"], source_offset=src["offset"], - nbytes=src["nbytes"], source_sha256=src["sha256"], - dtype=src["dtype"], shape=src["shape"], tier=tier, - target_class=slot, - allocated_bytes=sum(f["allocated_bytes"] for f in fragments), - fragments=fragments)) - if cursor != allocated_shape[tier]: - raise AssertionError("Allocation cursor mismatch") - used_cycles = len(groups) * cycles_per_expert - cycles_per_block = (geom.pages_per_block - 8) // 8 - blocks_used = math.ceil(used_cycles / cycles_per_block) - image_pages = blocks_used * geom.line_pages - image_bytes = image_pages * geom.page_bytes - if image_bytes > geom.exposed_bytes_nominal: - raise ValueError(f"Layout needs {image_bytes} bytes, nominal exposed capacity is " - f"{geom.exposed_bytes_nominal} bytes") - data_bytes = sum(e["nbytes"] for e in entries) - allocated_bytes = sum(e["allocated_bytes"] for e in entries) - return dict(schema=LAYOUT_SCHEMA, policy="qlc_aligned_expert_plane_major", - placement_policy=policy, - placement_slots={f"B{t+1}": slot_for(t, 0, policy) for t in range(4)}, - mapping_contract="fresh FEMU; sequential full fill from LPN 0; no intervening writes", - scale_policy="alpha4 column j shares QLC class j-1 with Bj", - projection_order=list(PROJECTION_ORDER), geometry=asdict(geom), - cycles_per_block=cycles_per_block, cycles_per_expert=cycles_per_expert, - experts=len(groups), entries=entries, - totals=dict(image_pages=image_pages, image_bytes=image_bytes, - blocks_used=blocks_used, data_bytes=data_bytes, - page_allocated_data_bytes=allocated_bytes, - filler_bytes=image_bytes-data_bytes, - payload_fraction=data_bytes/image_bytes)) - - -def source_matches(read, entry): - expected = dict(file=entry["source_file"], offset=entry["source_offset"], - nbytes=entry["nbytes"], dtype=entry["dtype"], shape=entry["shape"], - sha256=entry["source_sha256"]) - return all(read.get(k) == v for k, v in expected.items()) - - -def coalesce_commands(mapped, max_io_bytes, group_id): - commands = [] - for frag in sorted(mapped, key=lambda x: x["lba_start"]): - if frag["nbytes"] % 512: - raise ValueError("Mapped fragment is not sector aligned") - if (commands and commands[-1]["lba_start"] + commands[-1]["sector_count"] == frag["lba_start"] - and commands[-1]["page_class"] == frag["page_class"] - and commands[-1]["nbytes"] + frag["nbytes"] <= max_io_bytes): - cmd = commands[-1] - cmd["sector_count"] += frag["sector_count"] - cmd["nbytes"] += frag["nbytes"] - if frag["item_id"] not in cmd["item_ids"]: - cmd["item_ids"].append(frag["item_id"]) - else: - commands.append(dict(command_id=f"{group_id}:{len(commands)}", - lba_start=frag["lba_start"], sector_count=frag["sector_count"], - nbytes=frag["nbytes"], page_class=frag["page_class"], - item_ids=[frag["item_id"]], operation="read")) - return commands - - -def map_layer_reads(layer_reads, output_path, extent_map, max_io_bytes): - catalog = {e["item_id"]: e for e in extent_map["entries"]} - groups = commands = command_bytes = items = fragments = 0 - with Path(layer_reads).open() as src, Path(output_path).open("x") as dst: - for line in src: - group = json.loads(line) - mapped = [] - for read in group["reads"]: - entry = catalog.get(read["item_id"]) - if entry is None or not source_matches(read, entry): - raise ValueError(f"Read does not match manifest: {read['item_id']}") - for fragment_index, frag in enumerate(entry["fragments"]): - mapped.append(dict(item_id=entry["item_id"], kind=entry["kind"], - projection_id=entry["projection_id"], plane=entry["plane"], - scale_set=entry["scale_set"], column=entry["column"], - fragment_index=fragment_index, source_file=entry["source_file"], - source_offset=frag["source_offset"], image_offset=frag["target_offset"], - nbytes=frag["source_nbytes"], lba_start=frag["lba_start"], - sector_count=frag["sector_count"], page_class=frag["page_class"], - operation="read")) - cmds = coalesce_commands(mapped, max_io_bytes, group["group_id"]) - out = {k: v for k, v in group.items() if k != "reads"} - out.update(schema=MAPPED_SCHEMA, mapped_fragments=mapped, commands=cmds, - mapped_fragment_count=len(mapped), command_count=len(cmds), - command_bytes=sum(c["nbytes"] for c in cmds)) - dst.write(json.dumps(out, separators=(",", ":")) + "\n") - groups += 1 - items += len(group["reads"]) - fragments += len(mapped) - commands += len(cmds) - command_bytes += sum(c["nbytes"] for c in cmds) - return dict(groups=groups, items=items, fragments=fragments, commands=commands, - command_bytes=command_bytes, max_io_bytes=max_io_bytes, - recommended_queue_depth=32) - - -def plan(bundle, layer_reads, output, geom, max_io_bytes, policy="aligned"): - bundle, layer_reads, output = Path(bundle), Path(layer_reads), Path(output) - output.mkdir(parents=True, exist_ok=False) - manifest_path = bundle / "manifest.json" - manifest = json.loads(manifest_path.read_text()) - extent_map = make_extent_map(bundle, manifest, geom, policy) - extent_map.update(source_manifest_sha256=sha256(manifest_path), - layout_spec_sha256=canonical_hash({k: v for k, v in extent_map.items() - if k != "entries"})) - extent_path = output / "extent_map.json" - write_json(extent_path, extent_map) - mapped_path = output / "mapped_reads.jsonl" - replay = map_layer_reads(layer_reads, mapped_path, extent_map, max_io_bytes) - try: - portable_reads = layer_reads.relative_to(bundle).as_posix() - except ValueError: - portable_reads = str(layer_reads) - summary = dict(schema=LAYOUT_SCHEMA, status="planned_not_materialized", - source_bundle=bundle.name, source_manifest_sha256=sha256(manifest_path), - source_layer_reads=portable_reads, source_layer_reads_sha256=sha256(layer_reads), - mapper_sha256=sha256(__file__), - extent_map_sha256=sha256(extent_path), mapped_reads_sha256=sha256(mapped_path), - policy=extent_map["policy"], placement_policy=policy, - placement_slots=extent_map["placement_slots"], - geometry=extent_map["geometry"], - totals=extent_map["totals"], replay_input=replay, - validation_required=["fresh device and zero host writes", "sequential fill from LPN 0", - "FEMU WRITE-log PPA/page-class match", "read-back byte equality"], - warning="pgs_per_blk=512 requires the FEMU QLC pairing rows-1 fix") - write_json(output / "layout_summary.json", summary) - return summary - - -def validate_layout(bundle, layer_reads, layout): - bundle, layer_reads, layout = Path(bundle), Path(layer_reads), Path(layout) - manifest_path = bundle / "manifest.json" - manifest = json.loads(manifest_path.read_text()) - extent_path, mapped_path = layout / "extent_map.json", layout / "mapped_reads.jsonl" - extent_map = json.loads(extent_path.read_text()) - summary = json.loads((layout / "layout_summary.json").read_text()) - if extent_map["schema"] != LAYOUT_SCHEMA or summary["schema"] != LAYOUT_SCHEMA: - raise ValueError("Layout schema mismatch") - if extent_map["source_manifest_sha256"] != sha256(manifest_path): - raise ValueError("Manifest checksum mismatch") - if summary["extent_map_sha256"] != sha256(extent_path): - raise ValueError("Extent-map checksum mismatch") - if summary["mapped_reads_sha256"] != sha256(mapped_path): - raise ValueError("Mapped-trace checksum mismatch") - if summary["source_layer_reads_sha256"] != sha256(layer_reads): - raise ValueError("Layer-read checksum mismatch") - geom = Geometry(**extent_map["geometry"]) - geom.validate() - - expected = {} - for rec in manifest["projections"]: - for index, source in enumerate(rec["planes"], 1): - expected[item_key("plane", rec["id"], index)] = source - for index, source in enumerate(rec["scales"]["4"], 1): - expected[item_key("scale", rec["id"], index)] = source - entries = {entry["item_id"]: entry for entry in extent_map["entries"]} - if len(entries) != len(extent_map["entries"]) or set(entries) != set(expected): - raise ValueError("Extent-map catalog is incomplete or duplicated") - - allocated = [] - class_allocated_pages = [0, 0, 0, 0] - data_bytes = allocated_bytes = 0 - for key, entry in entries.items(): - source = expected[key] - if not (entry["source_offset"] == source["offset"] - and entry["nbytes"] == source["nbytes"] - and entry["source_sha256"] == source["sha256"] - and entry["dtype"] == source["dtype"] - and entry["shape"] == source["shape"]): - raise ValueError(f"Source metadata mismatch: {key}") - cursor = source["offset"] - item_bytes = 0 - for frag in entry["fragments"]: - if frag["source_offset"] != cursor or frag["target_offset"] % geom.page_bytes: - raise ValueError(f"Fragment alignment/continuity mismatch: {key}") - if frag["lba_start"] * geom.sector_bytes != frag["target_offset"]: - raise ValueError(f"LBA mismatch: {key}") - if frag["sector_count"] * geom.sector_bytes != frag["source_nbytes"]: - raise ValueError(f"Sector count mismatch: {key}") - if frag["page_class"] != entry["target_class"]: - raise ValueError(f"Class metadata mismatch: {key}") - for page_delta in range(frag["allocated_pages"]): - lpn = frag["target_lpn"] + page_delta - page = (lpn // geom.parallel_luns) % geom.pages_per_block - if page < 8 or qlc_class(page) != entry["target_class"]: - raise ValueError(f"Physical QLC class mismatch: {key}, LPN {lpn}") - start = frag["target_offset"] - end = start + frag["allocated_bytes"] - allocated.append((start, end, key)) - class_allocated_pages[entry["target_class"]] += frag["allocated_pages"] - cursor += frag["source_nbytes"] - item_bytes += frag["source_nbytes"] - allocated_bytes += frag["allocated_bytes"] - if item_bytes != source["nbytes"]: - raise ValueError(f"Fragment byte coverage mismatch: {key}") - data_bytes += item_bytes - allocated.sort() - for previous, current in zip(allocated, allocated[1:]): - if previous[1] > current[0]: - raise ValueError(f"Overlapping targets: {previous[2]}, {current[2]}") - totals = extent_map["totals"] - if allocated and allocated[-1][1] > totals["image_bytes"]: - raise ValueError("Target exceeds image") - if (data_bytes != totals["data_bytes"] or allocated_bytes != totals["page_allocated_data_bytes"] - or totals["filler_bytes"] != totals["image_bytes"] - data_bytes): - raise ValueError("Layout byte totals mismatch") - if len(set(class_allocated_pages)) != 1: - raise ValueError("B/alpha tiers do not have symmetric allocated footprints") - - groups = original_items = mapped_fragments = commands = command_bytes = 0 - with layer_reads.open() as source, mapped_path.open() as mapped: - for original_line, mapped_line in zip_longest(source, mapped): - if original_line is None or mapped_line is None: - raise ValueError("Mapped trace group count mismatch") - original, result = json.loads(original_line), json.loads(mapped_line) - if result["schema"] != MAPPED_SCHEMA or result["group_id"] != original["group_id"]: - raise ValueError("Mapped group identity mismatch") - expected_fragments = [] - for read in original["reads"]: - entry = entries[read["item_id"]] - if not source_matches(read, entry): - raise ValueError(f"Trace source mismatch: {read['item_id']}") - for fragment_index, frag in enumerate(entry["fragments"]): - expected_fragments.append((entry["item_id"], fragment_index, - frag["lba_start"], frag["sector_count"], - frag["source_nbytes"], frag["page_class"])) - actual_fragments = [(x["item_id"], x["fragment_index"], x["lba_start"], - x["sector_count"], x["nbytes"], x["page_class"]) - for x in result["mapped_fragments"]] - if actual_fragments != expected_fragments: - raise ValueError(f"Mapped fragments mismatch in group {original['group_id']}") - if sum(c["nbytes"] for c in result["commands"]) != sum(r["nbytes"] for r in original["reads"]): - raise ValueError(f"Command byte mismatch in group {original['group_id']}") - for command in result["commands"]: - if command["nbytes"] != command["sector_count"] * geom.sector_bytes: - raise ValueError("Non-sector command") - groups += 1 - original_items += len(original["reads"]) - mapped_fragments += len(result["mapped_fragments"]) - commands += len(result["commands"]) - command_bytes += sum(c["nbytes"] for c in result["commands"]) - replay = summary["replay_input"] - observed = dict(groups=groups, items=original_items, fragments=mapped_fragments, - commands=commands, command_bytes=command_bytes) - if any(replay[k] != v for k, v in observed.items()): - raise ValueError("Mapped-trace totals mismatch") - report = dict(passed=True, schema=LAYOUT_SCHEMA, catalog_items=len(entries), - allocated_ranges=len(allocated), class_allocated_pages=class_allocated_pages, - mapped_trace=observed, - checks=["complete plane and scale catalog", "source metadata equality", - "fragment source coverage", "target non-overlap and image bound", - "patched-512 QLC class for every allocated page", "symmetric tier footprint", - "mapped group and fragment equality", "command byte conservation", - "source and output checksums"]) - write_json(layout / "layout_validation.json", report) - return report - - -def materialize(bundle, layout, image): - bundle, layout, image = Path(bundle), Path(layout), Path(image) - extent_map = json.loads((layout / "extent_map.json").read_text()) - manifest_path = bundle / "manifest.json" - if extent_map["source_manifest_sha256"] != sha256(manifest_path): - raise ValueError("Bundle manifest changed") - segments = [] - for entry in extent_map["entries"]: - for frag in entry["fragments"]: - segments.append((frag["target_offset"], frag["source_offset"], - frag["source_nbytes"], entry)) - segments.sort() - cursor = 0 - fill = b"\xA5" * (8 << 20) - image_hash = hashlib.sha256() - with image.open("xb") as dst: - for target, source, nbytes, entry in segments: - if target < cursor: - raise ValueError("Overlapping target extents") - gap = target - cursor - while gap: - block = fill[:min(gap, len(fill))] - dst.write(block); image_hash.update(block); gap -= len(block) - with (bundle / entry["source_file"]).open("rb") as src: - src.seek(source) - data = src.read(nbytes) - if len(data) != nbytes: - raise ValueError("Short source read") - dst.write(data); image_hash.update(data) - cursor = target + nbytes - total = extent_map["totals"]["image_bytes"] - while cursor < total: - block = fill[:min(total-cursor, len(fill))] - dst.write(block); image_hash.update(block); cursor += len(block) - report = dict(image=str(image), nbytes=cursor, sha256=image_hash.hexdigest(), - extent_map_sha256=sha256(layout / "extent_map.json")) - write_json(Path(str(image) + ".json"), report) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - sub = parser.add_subparsers(dest="command", required=True) - pp = sub.add_parser("plan") - pp.add_argument("bundle", type=Path) - pp.add_argument("--layer-reads", type=Path, required=True) - pp.add_argument("--output", type=Path, required=True) - pp.add_argument("--max-io-bytes", type=int, default=4 << 20) - pp.add_argument("--policy", choices=PLACEMENT_POLICIES, default="aligned", - help="which QLC class each bit-plane tier is placed on; " - "the default reproduces the original layout byte for byte") - mp = sub.add_parser("materialize") - mp.add_argument("bundle", type=Path) - mp.add_argument("layout", type=Path) - mp.add_argument("--image", type=Path, required=True) - vp = sub.add_parser("validate") - vp.add_argument("bundle", type=Path) - vp.add_argument("--layer-reads", type=Path, required=True) - vp.add_argument("layout", type=Path) - args = parser.parse_args() - if args.command == "plan": - result = plan(args.bundle, args.layer_reads, args.output, Geometry(), - args.max_io_bytes, args.policy) - elif args.command == "materialize": - result = materialize(args.bundle, args.layout, args.image) - else: - result = validate_layout(args.bundle, args.layer_reads, args.layout) - print(json.dumps(result, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py b/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py deleted file mode 100644 index 19d8ad51a5f..00000000000 --- a/moe-harness/exp/moe_bcq/femu_handoff/packages/trace_compiler.py +++ /dev/null @@ -1,230 +0,0 @@ -#!/usr/bin/env python3 -"""Compile mapped layer-read JSONL into a validated little-endian replay stream.""" -from __future__ import annotations - -import argparse -import hashlib -import json -import os -import struct -import tempfile -from collections import Counter -from pathlib import Path - -from bundle import sha256, write_json - - -MAGIC = b"MBQRPL1\0" -VERSION = 1 -SECTOR_BYTES = 512 -DIRECT_ALIGNMENT = 4096 -PHASES = {"prefill": 0, "decode": 1, "teacher_forced": 2} -HEADER = struct.Struct("<8s8I4Q32s32s") -GROUP = struct.Struct(" (1 << 64) - 1 or sectors <= 0 - or sectors > (1 << 32) - 1): - raise ValueError(f"Invalid read command: {command}") - if nbytes != sectors * SECTOR_BYTES or nbytes > max_command_bytes: - raise ValueError(f"Invalid read size: {command['command_id']}") - if ((lba * SECTOR_BYTES) % DIRECT_ALIGNMENT - or nbytes % DIRECT_ALIGNMENT): - raise ValueError(f"O_DIRECT alignment violation: {command['command_id']}") - if page_class not in range(4): - raise ValueError(f"Invalid QLC class: {page_class}") - stream.write(COMMAND.pack(lba, sectors, page_class)) - group_bytes += nbytes - command_count += 1 - total_bytes += nbytes - max_end_byte = max(max_end_byte, (lba + sectors) * SECTOR_BYTES) - if group_bytes != record["command_bytes"]: - raise ValueError(f"Byte mismatch in group {gid}") - request_groups[request_id] += 1 - phase_groups[phase] += 1 - previous_group = gid - group_count += 1 - replay = summary["replay_input"] - if (group_count != replay["groups"] or command_count != replay["commands"] - or total_bytes != replay["command_bytes"]): - raise ValueError("Compiled totals differ from layout summary") - stream.seek(0) - stream.write(HEADER.pack(MAGIC, VERSION, HEADER.size, SECTOR_BYTES, - DIRECT_ALIGNMENT, queue_depth, max_command_bytes, GROUP.size, COMMAND.size, - group_count, command_count, total_bytes, max_end_byte, - raw_digest(summary["extent_map_sha256"]), raw_digest(summary["mapped_reads_sha256"]))) - stream.flush() - os.fsync(stream.fileno()) - os.chmod(temporary, 0o644) - os.link(temporary, output) - temporary.unlink() - temporary = None - finally: - if temporary is not None and temporary.exists(): - temporary.unlink() - - metadata = dict(schema="moe-bcq-replay-binary-v1", magic=MAGIC.rstrip(b"\0").decode(), - version=VERSION, byte_order="little", binary_file=output.name, - binary_sha256=sha256(output), binary_bytes=output.stat().st_size, - layout_directory=layout.name, layout_summary_sha256=sha256(summary_path), - layout_validation_sha256=sha256(validation_path), - extent_map_sha256=summary["extent_map_sha256"], - mapped_reads_sha256=summary["mapped_reads_sha256"], - sector_bytes=SECTOR_BYTES, direct_alignment=DIRECT_ALIGNMENT, - default_queue_depth=queue_depth, max_command_bytes=max_command_bytes, - record_bytes=dict(header=HEADER.size, group=GROUP.size, command=COMMAND.size), - totals=dict(groups=group_count, commands=command_count, - command_bytes=total_bytes, max_end_byte=max_end_byte), - phase_groups=dict(phase_groups), - requests=[dict(index=index, request_id=request_id, - groups=request_groups[request_id]) - for request_id, index in request_indices.items()], - semantics="Groups are barriers. Replayer uses rolling QD within a group; empty groups remain.") - write_json(Path(str(output) + ".json"), metadata) - return metadata - - -def inspect_trace(path): - path = Path(path) - with path.open("rb") as stream: - raw = stream.read(HEADER.size) - if len(raw) != HEADER.size: - raise ValueError("Truncated header") - values = HEADER.unpack(raw) - (magic, version, header_bytes, sector_bytes, alignment, default_qd, - max_command_bytes, group_bytes, command_bytes, groups, commands, - total_bytes, max_end_byte, layout_hash, mapped_hash) = values - if (magic != MAGIC or version != VERSION or header_bytes != HEADER.size - or group_bytes != GROUP.size or command_bytes != COMMAND.size - or sector_bytes != SECTOR_BYTES or alignment < sector_bytes - or not default_qd or not max_command_bytes): - raise ValueError("Binary header mismatch") - seen_commands = seen_bytes = seen_max_end = 0 - previous_group = -1 - for expected_group in range(groups): - raw = stream.read(GROUP.size) - if len(raw) != GROUP.size: - raise ValueError("Truncated group") - gid, release, forward, layer, phase, reset, reserved, count, request = GROUP.unpack(raw) - if (gid != expected_group or phase not in PHASES.values() or reserved - or (reset and release != -1) - or (not reset and release != previous_group)): - raise ValueError("Invalid group record") - for _ in range(count): - raw = stream.read(COMMAND.size) - if len(raw) != COMMAND.size: - raise ValueError("Truncated command") - lba, sectors, page_class = COMMAND.unpack(raw) - size, offset = sectors * sector_bytes, lba * sector_bytes - if (not sectors or page_class > 3 or size > max_command_bytes - or offset % alignment or size % alignment): - raise ValueError("Invalid command record") - seen_commands += 1 - seen_bytes += size - seen_max_end = max(seen_max_end, offset + size) - previous_group = gid - if (stream.read(1) or seen_commands != commands or seen_bytes != total_bytes - or seen_max_end != max_end_byte): - raise ValueError("Binary length/totals mismatch") - return dict(groups=groups, commands=commands, command_bytes=total_bytes, - max_end_byte=max_end_byte, default_queue_depth=default_qd, - max_command_bytes=max_command_bytes, sector_bytes=sector_bytes, - direct_alignment=alignment, extent_map_sha256=layout_hash.hex(), - mapped_reads_sha256=mapped_hash.hex(), binary_sha256=sha256(path)) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - sub = parser.add_subparsers(dest="command", required=True) - cp = sub.add_parser("compile") - cp.add_argument("layout", type=Path) - cp.add_argument("--output", type=Path, required=True) - cp.add_argument("--queue-depth", type=int, default=32) - cp.add_argument("--max-command-bytes", type=int, default=4 << 20) - ip = sub.add_parser("inspect") - ip.add_argument("trace", type=Path) - args = parser.parse_args() - result = (compile_trace(args.layout, args.output, args.queue_depth, args.max_command_bytes) - if args.command == "compile" else inspect_trace(args.trace)) - print(json.dumps(result, indent=2)) - - -if __name__ == "__main__": - main() From c22236eee6d0f9328e7d11855c8f04ed06f08ea9 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 23:10:01 +0900 Subject: [PATCH 13/17] moe-harness: split code and data roots in the rest of the scripts run_device.sh and drive_multi.sh already told HARNESS from ROOT; the sweep path still resolved one root for both. Moving the harness out of the data project would have taken its code lookups along and left the sweep calling scripts that are no longer where it looked. run_sweep.sh finds its siblings under HARNESS. run_policy.sh does the same for guest_replay.sh, make_seed.py, femu_compose.sh and the driver, while the image, the guest binaries and the replay binary stay under ROOT. drive_run.sh touches only counters and records, so it follows ROOT alone. Each also accepts the FEMU checkout as the harness's own parent, which is how the published layout sits. preflight.sh guards a code file, so its target follows HARNESS. Worth saying plainly: this guard exists because a sync reverted run_policy.sh twice, and git supersedes it the moment that sync stops carrying the harness -- a checkout that is overwritten shows up in git status. Until then it still earns its place. Co-Authored-By: Claude Opus 5 --- moe-harness/exp/moe_bcq/femu_run/drive_run.sh | 2 +- moe-harness/exp/moe_bcq/femu_run/preflight.sh | 4 ++-- moe-harness/exp/moe_bcq/femu_run/run_policy.sh | 13 +++++++------ moe-harness/exp/moe_bcq/femu_run/run_sweep.sh | 5 +++-- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/moe-harness/exp/moe_bcq/femu_run/drive_run.sh b/moe-harness/exp/moe_bcq/femu_run/drive_run.sh index 8a8643cf48d..a66eecc9a7b 100755 --- a/moe-harness/exp/moe_bcq/femu_run/drive_run.sh +++ b/moe-harness/exp/moe_bcq/femu_run/drive_run.sh @@ -15,7 +15,7 @@ set -uo pipefail RUN=${1:?usage: drive_run.sh RUN_TAG} # Derive the checkout root from this script rather than naming it, so the same # script drives a run on whichever machine it was copied to. -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +ROOT=${FEMU_PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)} CSV=$ROOT/runs/femu/${RUN}_qlc.csv OUT=$ROOT/runs/femu/$RUN SSH="ssh -p ${SSH_PORT:-2222} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=10 femu@127.0.0.1" diff --git a/moe-harness/exp/moe_bcq/femu_run/preflight.sh b/moe-harness/exp/moe_bcq/femu_run/preflight.sh index 49839503b3c..e27bdfe0e0d 100755 --- a/moe-harness/exp/moe_bcq/femu_run/preflight.sh +++ b/moe-harness/exp/moe_bcq/femu_run/preflight.sh @@ -17,8 +17,8 @@ set -uo pipefail CANON=${FEMU_CANONICAL_DIR:-$HOME/.femu_canonical} -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) -TARGET=$ROOT/exp/moe_bcq/femu_run/run_policy.sh +HARNESS=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +TARGET=$HARNESS/exp/moe_bcq/femu_run/run_policy.sh [ -f "$CANON/SHA256SUMS" ] || { echo "preflight: no canonical copy at $CANON"; exit 1; } want=$(awk '$2=="run_policy.sh"{print $1}' "$CANON/SHA256SUMS") diff --git a/moe-harness/exp/moe_bcq/femu_run/run_policy.sh b/moe-harness/exp/moe_bcq/femu_run/run_policy.sh index 19d183c6917..00c2c91dc67 100755 --- a/moe-harness/exp/moe_bcq/femu_run/run_policy.sh +++ b/moe-harness/exp/moe_bcq/femu_run/run_policy.sh @@ -7,13 +7,14 @@ set -uo pipefail POL=${1:?usage: run_policy.sh POLICY} # Same reason as drive_run.sh: the path is where the script is, not a constant. -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +HARNESS=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +ROOT=${FEMU_PROJECT_ROOT:-$HARNESS} # Guest images and the key authorised inside them are per-machine. IMAGES=${FEMU_GUEST_DIR:-$HOME/images} SSH_PUBKEY=${FEMU_SSH_PUBKEY:-$HOME/.ssh/id_rsa.pub} # The patched build ships one; a system qemu-img works too and is preferred when # present, since it does not depend on the checkout having been built yet. -QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$ROOT/_deps/FEMU-MoE/build/qemu-img")} +QEMU_IMG=${QEMU_IMG:-$(command -v qemu-img || echo "$HARNESS/../build/qemu-img")} cd "$ROOT" # One knob, not two. The run tag defaulted to something different from the @@ -33,7 +34,7 @@ rm -f "$SEED" "$OVL" "runs/femu/${TAG}_qlc.csv" --file /usr/local/bin/replay_v1=$ROOT/build/guest/replay_v1:0755 \ --file /usr/local/bin/class_confusion=$ROOT/build/guest/class_confusion:0755 \ --file /root/replay.bin=$ROOT/$BIN \ - --file /usr/local/bin/guest_replay.sh=$ROOT/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ + --file /usr/local/bin/guest_replay.sh=$HARNESS/exp/moe_bcq/femu_run/guest_replay.sh:0755 \ --run "/usr/local/bin/guest_replay.sh > /dev/ttyS0 2>&1") >/dev/null || exit 1 "$QEMU_IMG" create -f qcow2 -F qcow2 -b jammy-server-cloudimg-amd64.img "$OVL" 32G >/dev/null @@ -48,18 +49,18 @@ set -a; . "runs/femu/${TAG}.env"; set +a # the older layout. Prefer whichever actually has the compose file rather than # naming one, or the run stalls waiting for a container that was never started. if [ -z "${FEMU_SOURCE_DIR:-}" ]; then - for c in "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE"; do + for c in "$HARNESS/.." "$ROOT/_deps/FEMU-MoE" "$ROOT/FEMU-MoE"; do [ -f "$c/compose.yaml" ] && { FEMU_SOURCE_DIR=$c; break; } done fi [ -n "${FEMU_SOURCE_DIR:-}" ] || { echo " no FEMU checkout with compose.yaml; run scripts/setup_femu.sh"; exit 1; } export FEMU_SOURCE_DIR -nohup bash scripts/femu_compose.sh up femu > "runs/femu/${TAG}.console.log" 2>&1 & +nohup bash "$HARNESS/scripts/femu_compose.sh" up femu > "runs/femu/${TAG}.console.log" 2>&1 & until grep -qa "ALL DONE\|FATAL" "runs/femu/${TAG}.console.log" 2>/dev/null; do sleep 10; done grep -qa FATAL "runs/femu/${TAG}.console.log" && { echo " FATAL during fill"; exit 1; } grep -a "fill done\|read-back OK" "runs/femu/${TAG}.console.log" | tr -d '\r' | sed 's/^femu[^|]*| / /' bash exp/moe_bcq/femu_run/drive_run.sh "$TAG" || exit 1 export FEMU_CONTAINER_NAME=femu-$TAG -bash scripts/femu_compose.sh down >/dev/null 2>&1 +bash "$HARNESS/scripts/femu_compose.sh" down >/dev/null 2>&1 echo " device torn down" diff --git a/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh b/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh index bd665e44d83..e0bbe49ed17 100755 --- a/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh +++ b/moe-harness/exp/moe_bcq/femu_run/run_sweep.sh @@ -10,8 +10,9 @@ # (see preflight.sh). Going through here means no sweep can start on a reverted # script, and a revert mid-sweep is caught at the next policy. set -uo pipefail -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) -HERE=$ROOT/exp/moe_bcq/femu_run +HARNESS=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd) +ROOT=${FEMU_PROJECT_ROOT:-$HARNESS} +HERE=$HARNESS/exp/moe_bcq/femu_run BUNDLE=${1:?usage: run_sweep.sh BUNDLE LAYOUT_PREFIX IMG_PREFIX TAG_PREFIX [policy ...]} LAYOUT_PREFIX=${2:?}; IMG_PREFIX=${3:?}; TAG_PREFIX=${4:?} From 8f89162b3bea25649bc07080dd068b03100c9129 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 23:39:01 +0900 Subject: [PATCH 14/17] femu-scripts: give the container path the README's shape The README's build-and-run is not available on the host this fork is measured on, and until now the alternative was a compose invocation with six environment variables, which looks like a different project rather than the same one. femu-docker.sh is that path with the README's steps and names: build, verify, run, ssh, stop. Its header and the new README section give the mapping, so the question it kept raising -- why not just use run-blackbox.sh -- is answered where it gets asked rather than in a conversation. The reason is worth stating once here too. Two of the README's steps need root: pkgdep.sh installs packages, and run-blackbox.sh launches QEMU under sudo because FEMU pins its memory backend. Pinning needs RLIMIT_MEMLOCK past the device size; this host allows 64 MiB against 64 GiB, and has no passwordless sudo to raise it. The container gets IPC_LOCK and an unlimited memlock without the host granting root to anyone. Two things the README leaves to the reader are handled, because both are ways to lose an afternoon: the guest disk is a copy-on-write overlay per instance rather than the base image written in place, and the seed authorises an ssh key -- the cloud image ships no password, so without one there is no way in. It sits beside the scripts it mirrors, in hw/femu/scripts, which the femu-scripts symlink at the root points at. Checked against a running container: verify lists the femu device options, and status reports cell=4 size=65536MB 2ch x 4LUN 512pg/blk 1024blk/pl opts=op_pcent=7, which is the measured configuration. Co-Authored-By: Claude Opus 5 --- README.md | 43 ++++++++++ hw/femu/scripts/femu-docker.sh | 147 +++++++++++++++++++++++++++++++++ 2 files changed, 190 insertions(+) create mode 100755 hw/femu/scripts/femu-docker.sh diff --git a/README.md b/README.md index d3ea9c168e3..a8060e91349 100644 --- a/README.md +++ b/README.md @@ -163,6 +163,49 @@ OpenChannel needs a host that speaks it. LightNVM was removed from Linux in --- +## Running it in a container + +`femu-scripts/femu-docker.sh` is the path below done in a container, for hosts +where the path below cannot run. Two steps of it need root: `pkgdep.sh` +installs packages, and `run-blackbox.sh` launches QEMU under `sudo` because +FEMU pins its memory backend, which needs `RLIMIT_MEMLOCK` raised past the +device size. A host that allows the usual 64 MiB cannot start a 64 GiB device +at all. + +The container gets `IPC_LOCK` and an unlimited memlock without the host +granting root to anyone. The steps map one to one: + +| this README | container | +|---|---| +| `sudo ./pkgdep.sh` | `docker/Dockerfile`, builder stage | +| `./femu-compile.sh` | `docker/Dockerfile`, builder stage | +| `./qemu-system-x86_64 -device femu,help` | `femu-docker.sh verify` | +| `./run-blackbox.sh` | `femu-docker.sh run` | + +```bash +./femu-scripts/femu-docker.sh build # dependencies and compile, inside +./femu-scripts/femu-docker.sh verify # did the femu device register +./femu-scripts/femu-docker.sh image # guest overlay + cloud-init seed +./femu-scripts/femu-docker.sh run # start the device, boot the guest +./femu-scripts/femu-docker.sh ssh # a shell in the guest +./femu-scripts/femu-docker.sh stop +``` + +One difference worth knowing: this README has you build a VM image by hand and +reuse it. `image` cuts a copy-on-write overlay per instance instead and writes +a cloud-init seed that authorises your ssh key, so the base image is never +written and a broken guest is one file to delete. The cloud image ships no +password, so without that seed there is no way in. + +The device is the same on both paths -- geometry, cell type and the read-energy +coefficients come from `compose.yaml`, whose defaults are the configuration the +measurements in this repository were taken on. `femu-docker.sh status` prints +what the instance would get. + +Running an actual measurement is a different entry point, because it needs a +payload image, a compiled trace and the placement checks: see +[moe-harness/README.md](moe-harness/README.md). + ## Installation ### Build FEMU diff --git a/hw/femu/scripts/femu-docker.sh b/hw/femu/scripts/femu-docker.sh new file mode 100755 index 00000000000..0f2bdd4a46b --- /dev/null +++ b/hw/femu/scripts/femu-docker.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# The README's build-and-run path, done in a container. +# +# The README tells you to install dependencies with sudo, compile into +# build-femu/, and launch with run-blackbox.sh. Step one does not work on the +# host this fork is measured on -- there is no passwordless sudo -- and step +# three would not either: run-blackbox.sh launches QEMU under sudo, and FEMU +# pins its memory backend, which needs RLIMIT_MEMLOCK raised. This host allows +# 64 MiB against a 64 GiB device. +# +# So the same three steps happen in a container, which gets IPC_LOCK and an +# unlimited memlock without the host granting root to anyone: +# +# README here +# sudo ./pkgdep.sh docker/Dockerfile, builder stage +# ./femu-compile.sh docker/Dockerfile, builder stage +# ./run-blackbox.sh docker/femu-run bbssd, via compose +# +# The device is the same either way. Geometry, cell type and the energy +# coefficients come from compose.yaml, whose defaults are the configuration the +# measurements in this repository were taken on. +set -uo pipefail + +HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +REPO=$(cd "$HERE/.." && pwd) + +# Where guest disks live. The README has you build a VM image by hand and keep +# it beside the build; this is that directory. +IMAGES=${FEMU_GUEST_DIR:-$HOME/images} +# What the container sees as /data: the counter CSV and the console log land here. +DATA=${FEMU_DATA_DIR:-$REPO/docker-data} +BASE=${FEMU_BASE_IMAGE:-jammy-server-cloudimg-amd64.img} +TAG=${FEMU_INSTANCE:-femu} +SSH_PORT=${FEMU_SSH_PORT:-2222} +SSH_PUBKEY=${FEMU_SSH_PUBKEY:-$HOME/.ssh/id_rsa.pub} + +compose() { + FEMU_GUEST_DIR=$IMAGES FEMU_DATA_DIR=$DATA \ + FEMU_CONTAINER_NAME=femu-$TAG FEMU_SSH_PORT=$SSH_PORT \ + FEMU_IMAGE_NAME=femu-root-$TAG.qcow2 \ + FEMU_QLC_STATS_PATH=/data/${TAG}_qlc.csv \ + FEMU_EXTRA_DRIVES="${FEMU_EXTRA_DRIVES:-file=/guest/seed-$TAG.iso,if=virtio,format=raw,readonly=on}" \ + docker compose -f "$REPO/compose.yaml" "$@" +} + +usage() { + cat <<'EOF' +Usage: femu-docker.sh COMMAND + + build Build the FEMU image. Dependencies and compile both happen inside, + so nothing is installed on the host. + verify Ask the built binary whether the femu device registered. + image Create this instance's guest disk and cloud-init seed. The overlay + is copy-on-write over the base image, which is never written. + run Start the device and boot the guest. Ctrl-C detaches; the container + keeps running. + ssh Open a shell in the guest. + stop Stop the container and remove it. + status What is running, and the device this instance would get. + +Environment: + FEMU_INSTANCE name for this instance's disk, seed and container (femu) + FEMU_GUEST_DIR where guest disks live ($HOME/images) + FEMU_DATA_DIR what the container sees as /data (/docker-data) + FEMU_BASE_IMAGE base image the overlay is cut from + (jammy-server-cloudimg-amd64.img) + FEMU_SSH_PORT host port forwarded to the guest's sshd (2222) + FEMU_SSH_PUBKEY key authorised in the guest (~/.ssh/id_rsa.pub) + +Running an actual measurement is a different entry point: it needs a payload +image, a compiled trace and the placement checks. See moe-harness/README.md. +EOF +} + +case "${1:-help}" in +help|-h|--help) usage ;; + +build) + echo "== building (dependencies and compile are inside the image) ==" + compose build femu + ;; + +verify) + echo "== does the femu device register? ==" + compose run --rm --entrypoint qemu-system-x86_64 femu -device femu,help 2>&1 | + head -20 + ;; + +image) + mkdir -p "$IMAGES" "$DATA" + [ -f "$IMAGES/$BASE" ] || { + echo "no base image at $IMAGES/$BASE" + echo "download an Ubuntu 22.04 cloud image there, or set FEMU_BASE_IMAGE" + exit 1 + } + [ -f "$SSH_PUBKEY" ] || { echo "no ssh public key at $SSH_PUBKEY"; exit 1; } + ovl=$IMAGES/femu-root-$TAG.qcow2 + seed=$IMAGES/seed-$TAG.iso + # Never write the base image: a run gets its own overlay, so a broken guest + # is one file to delete rather than a re-download. + [ -e "$ovl" ] && { echo "already exists: $ovl (delete it to start over)"; exit 1; } + qemu-img create -f qcow2 -F qcow2 -b "$BASE" "$ovl" 32G >/dev/null + echo " overlay $ovl" + # The cloud image ships no password, so without a seed carrying a key there + # is no way in. No --run: this seed only authorises the key. + for c in python3 /usr/bin/python3 python3.8; do + command -v "$c" >/dev/null 2>&1 || continue + "$c" -c 'import pycdlib' 2>/dev/null && { PY=$c; break; } + done + [ -n "${PY:-}" ] || { echo "no python3 with pycdlib; pip install --user pycdlib"; exit 1; } + (cd "$REPO/moe-harness/exp/gating_nand/femu" && + "$PY" make_seed.py -o "$seed" --tag "${TAG^^}" \ + --instance-id "femu-$TAG" --ssh-key "$SSH_PUBKEY") >/dev/null || exit 1 + echo " seed $seed" + ;; + +run) + [ -f "$IMAGES/femu-root-$TAG.qcow2" ] || { + echo "no guest disk for instance '$TAG'; run: $0 image"; exit 1; } + mkdir -p "$DATA" + echo "== starting femu-$TAG (Ctrl-C detaches, container keeps running) ==" + compose up femu + ;; + +ssh) + exec ssh -p "$SSH_PORT" -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null femu@127.0.0.1 "${@:2}" + ;; + +stop) + compose down + ;; + +status) + echo "== containers ==" + docker ps --filter "name=femu-$TAG" --format ' {{.Names}} {{.Status}}' || true + echo "== the device this instance gets ==" + compose run --rm --entrypoint sh femu -c 'echo " \ +cell=$FEMU_NAND_CELL_TYPE size=${FEMU_SSD_SIZE_MB}MB \ +${FEMU_CHANNELS}ch x ${FEMU_LUNS_PER_CHANNEL}LUN \ +${FEMU_PAGES_PER_BLOCK}pg/blk ${FEMU_BLOCKS_PER_PLANE}blk/pl \ +opts=$FEMU_EXTRA_DEVICE_OPTS"' 2>/dev/null | tail -1 + ;; + +*) + echo "unknown command: $1"; echo; usage; exit 1 ;; +esac From 5646f85d5db6bdba13b0f6dd9626183e3565d77f Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 23:42:05 +0900 Subject: [PATCH 15/17] femu-scripts: say why run-blackbox.sh is not what runs in the container The previous message gave one reason for two different situations, and it was only right about the first. sudo and RLIMIT_MEMLOCK are why the README's path cannot run on this host. They say nothing about why it is not what runs inside the container, where the process is already root -- sudo is not even installed there, so run-blackbox.sh would fail for want of a binary, not a privilege. The actual reason is that run-blackbox.sh writes the SSD layout into itself: pgs_per_blk=256, luns_per_ch=8, nchs=8, ssd_size=12288 and a fixed u20s.qcow2, none of which is this fork's device, and it reads no environment. A run cannot be handed its geometry, its guest disk, its payload disk or its counter path. femu-run is the same script with those constants lifted out. Both reasons now sit in the README and the script header, separately, so neither is used to argue something it does not support. Co-Authored-By: Claude Opus 5 --- README.md | 30 +++++++++++++++++++++--------- hw/femu/scripts/femu-docker.sh | 20 +++++++++++++------- 2 files changed, 34 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index a8060e91349..ceb8f2c9dfb 100644 --- a/README.md +++ b/README.md @@ -165,15 +165,27 @@ OpenChannel needs a host that speaks it. LightNVM was removed from Linux in ## Running it in a container -`femu-scripts/femu-docker.sh` is the path below done in a container, for hosts -where the path below cannot run. Two steps of it need root: `pkgdep.sh` -installs packages, and `run-blackbox.sh` launches QEMU under `sudo` because -FEMU pins its memory backend, which needs `RLIMIT_MEMLOCK` raised past the -device size. A host that allows the usual 64 MiB cannot start a 64 GiB device -at all. - -The container gets `IPC_LOCK` and an unlimited memlock without the host -granting root to anyone. The steps map one to one: +`femu-scripts/femu-docker.sh` is the path below done in a container. Two +separate things push it there. + +**The host cannot run the path below.** `pkgdep.sh` installs packages, and +`run-blackbox.sh` launches QEMU under `sudo` because FEMU pins its memory +backend, which needs `RLIMIT_MEMLOCK` raised past the device size. A host +allowing the usual 64 MiB cannot start a 64 GiB device at all, and raising it +needs root. The container gets `IPC_LOCK` and an unlimited memlock without the +host granting root to anyone. + +**Inside the container that argument stops applying, and a different one +starts.** The container runs as root, so `sudo` would be moot there -- it is not +even installed. What makes `run-blackbox.sh` unusable in it is that the SSD +layout is written into the file: `pgs_per_blk=256`, `luns_per_ch=8`, `nchs=8`, +`ssd_size=12288`, a fixed `u20s.qcow2`. It reads no environment, so there is no +way to hand it this fork's geometry, a different guest disk, a payload disk or +a counter path per run. `docker/femu-run` is that same script with those +constants lifted out into environment variables; the steps it performs are +unchanged. + +The steps map one to one: | this README | container | |---|---| diff --git a/hw/femu/scripts/femu-docker.sh b/hw/femu/scripts/femu-docker.sh index 0f2bdd4a46b..b2c78e6b6d1 100755 --- a/hw/femu/scripts/femu-docker.sh +++ b/hw/femu/scripts/femu-docker.sh @@ -2,14 +2,20 @@ # The README's build-and-run path, done in a container. # # The README tells you to install dependencies with sudo, compile into -# build-femu/, and launch with run-blackbox.sh. Step one does not work on the -# host this fork is measured on -- there is no passwordless sudo -- and step -# three would not either: run-blackbox.sh launches QEMU under sudo, and FEMU -# pins its memory backend, which needs RLIMIT_MEMLOCK raised. This host allows -# 64 MiB against a 64 GiB device. +# build-femu/, and launch with run-blackbox.sh. That path does not run on the +# host this fork is measured on: there is no passwordless sudo for pkgdep.sh, +# and FEMU pins its memory backend, which needs RLIMIT_MEMLOCK raised past the +# device size -- this host allows 64 MiB against 64 GiB. # -# So the same three steps happen in a container, which gets IPC_LOCK and an -# unlimited memlock without the host granting root to anyone: +# Inside the container neither of those bites: it runs as root and compose +# gives it IPC_LOCK with an unlimited memlock. run-blackbox.sh still is not +# what runs there, for an unrelated reason -- it writes the SSD layout into +# itself (pgs_per_blk=256, luns_per_ch=8, nchs=8, ssd_size=12288, a fixed +# u20s.qcow2) and reads no environment, so a run cannot be given this fork's +# geometry, its guest disk, its payload disk or its counter path. femu-run is +# that script with those constants lifted into environment variables. +# +# The steps themselves are unchanged: # # README here # sudo ./pkgdep.sh docker/Dockerfile, builder stage From d03878ec33eb4a8c5441a7243a771c35f3917083 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Sun, 13 Sep 2026 23:45:19 +0900 Subject: [PATCH 16/17] femu-scripts: let run-blackbox.sh take its configuration from the environment The SSD layout was written into the file, so running a second configuration meant editing the script or keeping a forked copy of it per device. That is why this fork ended up launching through a separate program instead: not because the script did the wrong thing, but because there was no way to hand it anything. Every constant is now an environment override with its former value as the default, using the same names the container path already uses, so one description of a run works whichever way it is started. Added the same way, appended only when set: the NAND cell type, the read-energy coefficients, any further device options, and extra read-only drives -- a replay payload is far too large for a cloud-init seed and has to arrive as a disk. Checked rather than assumed: with an empty environment the composed device string is byte-identical to the one this script produced before. With this fork's settings it matches what the container path builds, down to the energy coefficients and op_pcent=7. Two things that were not configuration but blocked reuse. sudo is how an ordinary user reaches KVM and raises RLIMIT_MEMLOCK for the pinned backend; in a container the process is already root and sudo is usually not installed, so it is skipped when the caller is root. And the binary is looked up on PATH when the build-femu copy is not beside the script. One interaction worth recording: pg_rd_lat stays in the command line for compatibility, but setting a cell type turns the flat timing off (ftl-media.c:147) and the per-page-class table replaces it, so on QLC the flat latency is inert. Co-Authored-By: Claude Opus 5 --- hw/femu/scripts/run-blackbox.sh | 101 +++++++++++++++++++++++++------- 1 file changed, 80 insertions(+), 21 deletions(-) diff --git a/hw/femu/scripts/run-blackbox.sh b/hw/femu/scripts/run-blackbox.sh index c6184fe939b..741a828cbbe 100755 --- a/hw/femu/scripts/run-blackbox.sh +++ b/hw/femu/scripts/run-blackbox.sh @@ -1,31 +1,46 @@ #!/bin/bash # Huaicheng Li # Run FEMU as a black-box SSD (FTL managed by the device) +# +# Every setting below can be overridden from the environment, and each default +# is the value this script used when they were constants. Set nothing and the +# command line is what it always was. +# +# The reason for the change: the layout was written into the file, so running +# two devices, or running one under a harness, meant editing the script or +# keeping a forked copy per configuration. The names match the ones the +# container path uses, so a run is described the same way whichever way it is +# started. # image directory -IMGDIR=$HOME/images +IMGDIR=${FEMU_GUEST_DIR:-$HOME/images} # Virtual machine disk image -OSIMGF=$IMGDIR/u20s.qcow2 +OSIMGF=${FEMU_IMAGE:-$IMGDIR/${FEMU_IMAGE_NAME:-u20s.qcow2}} # Configurable SSD Controller layout parameters (must be power of 2) -secsz=512 # sector size in bytes -secs_per_pg=8 # number of sectors in a flash page -pgs_per_blk=256 # number of pages per flash block -blks_per_pl=256 # number of blocks per plane -pls_per_lun=1 # planes per LUN -luns_per_ch=8 # number of chips per channel -nchs=8 # number of channels -ssd_size=12288 # in megabytes, if you change the above layout parameters, make sure you manually recalculate the ssd size and modify it here, please consider a default 25% overprovisioning ratio. +secsz=${FEMU_SECTOR_SIZE:-512} # sector size in bytes +secs_per_pg=${FEMU_SECTORS_PER_PAGE:-8} # number of sectors in a flash page +pgs_per_blk=${FEMU_PAGES_PER_BLOCK:-256} # number of pages per flash block +blks_per_pl=${FEMU_BLOCKS_PER_PLANE:-256} # number of blocks per plane +pls_per_lun=${FEMU_PLANES_PER_LUN:-1} # planes per LUN +luns_per_ch=${FEMU_LUNS_PER_CHANNEL:-8} # number of chips per channel +nchs=${FEMU_CHANNELS:-8} # number of channels +ssd_size=${FEMU_SSD_SIZE_MB:-12288} # in megabytes, if you change the above layout parameters, make sure you manually recalculate the ssd size and modify it here, please consider a default 25% overprovisioning ratio. # Latency in nanoseconds -pg_rd_lat=40000 # page read latency -pg_wr_lat=200000 # page write latency -blk_er_lat=2000000 # block erase latency -ch_xfer_lat=0 # channel transfer time, ignored for now +pg_rd_lat=${FEMU_PAGE_READ_LATENCY:-40000} # page read latency +pg_wr_lat=${FEMU_PAGE_WRITE_LATENCY:-200000} # page write latency +blk_er_lat=${FEMU_BLOCK_ERASE_LATENCY:-2000000} # block erase latency +ch_xfer_lat=${FEMU_CHANNEL_TRANSFER_LATENCY:-0} # channel transfer time, ignored for now # GC Threshold (1-100) -gc_thres_pcent=75 -gc_thres_pcent_high=95 +gc_thres_pcent=${FEMU_GC_THRESHOLD:-75} +gc_thres_pcent_high=${FEMU_GC_THRESHOLD_HIGH:-95} + +# Guest resources and the port forwarded to its sshd +vm_cpus=${FEMU_CPUS:-4} +vm_memory=${FEMU_MEMORY:-4G} +ssh_port=${FEMU_GUEST_SSH_PORT:-8080} #----------------------------------------------------------------------- @@ -48,8 +63,38 @@ FEMU_OPTIONS=${FEMU_OPTIONS}",ch_xfer_lat=${ch_xfer_lat}" FEMU_OPTIONS=${FEMU_OPTIONS}",gc_thres_pcent=${gc_thres_pcent}" FEMU_OPTIONS=${FEMU_OPTIONS}",gc_thres_pcent_high=${gc_thres_pcent_high}" +# Appended only when asked for, so an unset environment reproduces the command +# line this script produced before any of this was configurable. The flat +# pg_rd_lat above is what a cell type replaces: set FEMU_NAND_CELL_TYPE and the +# device reads its per-page-class table instead. +[ -n "${FEMU_NAND_CELL_TYPE:-}" ] && + FEMU_OPTIONS=${FEMU_OPTIONS}",nand_cell_type=${FEMU_NAND_CELL_TYPE}" +for c in 0 1 2 3; do + v=FEMU_E_READ_C$c + [ -n "${!v:-}" ] && FEMU_OPTIONS=${FEMU_OPTIONS}",e_read_c${c}_mpj=${!v}" + v=FEMU_E_ARRAY_C$c + [ -n "${!v:-}" ] && FEMU_OPTIONS=${FEMU_OPTIONS}",e_array_c${c}_mpj=${!v}" +done +[ -n "${FEMU_E_XFER:-}" ] && FEMU_OPTIONS=${FEMU_OPTIONS}",e_xfer_mpj=${FEMU_E_XFER}" +[ -n "${FEMU_STATS_FLUSH_MS:-}" ] && + FEMU_OPTIONS=${FEMU_OPTIONS}",stats_flush_ms=${FEMU_STATS_FLUSH_MS}" +# Anything else the device takes, comma separated: op_pcent=7, buffer_size=... +[ -n "${FEMU_EXTRA_DEVICE_OPTS:-}" ] && + FEMU_OPTIONS=${FEMU_OPTIONS}",${FEMU_EXTRA_DEVICE_OPTS}" + echo ${FEMU_OPTIONS} +# Extra read-only guest disks, ';' separated -drive specs. A replay payload is +# far too large for a cloud-init seed, so it arrives as a disk the guest copies +# onto the emulated SSD from inside. +EXTRA_DRIVES=() +if [ -n "${FEMU_EXTRA_DRIVES:-}" ]; then + IFS=';' read -r -a specs <<< "${FEMU_EXTRA_DRIVES}" + for spec in "${specs[@]}"; do + [ -n "$spec" ] && EXTRA_DRIVES+=(-drive "$spec") + done +fi + if [[ ! -e "$OSIMGF" ]]; then echo "" echo "VM disk image couldn't be found ..." @@ -59,20 +104,34 @@ if [[ ! -e "$OSIMGF" ]]; then exit fi -sudo FEMU_EXP_LOG=${FEMU_EXP_LOG} \ +# sudo is how an ordinary user reaches KVM and raises RLIMIT_MEMLOCK for the +# pinned memory backend. In a container the process is already root and sudo is +# often not installed, so asking for it there fails for want of a binary. +SUDO=sudo +[ "$(id -u)" = 0 ] && SUDO= +# Built in build-femu/ by femu-compile.sh, which is where this script is run +# from; a packaged build puts it on PATH instead. +QEMU=${FEMU_QEMU_BIN:-./qemu-system-x86_64} +[ -x "$QEMU" ] || QEMU=$(command -v qemu-system-x86_64) || { + echo "qemu-system-x86_64 not found; build it or set FEMU_QEMU_BIN"; exit 1; } + +$SUDO FEMU_EXP_LOG=${FEMU_EXP_LOG} \ FEMU_SECRET=${FEMU_SECRET} \ FEMU_DUMP_LPN=${FEMU_DUMP_LPN} \ - ./qemu-system-x86_64 \ + FEMU_QLC_STATS_PATH=${FEMU_QLC_STATS_PATH} \ + FEMU_ALLOW_UNPINNED=${FEMU_ALLOW_UNPINNED} \ + "$QEMU" \ -name "FEMU-BBSSD-VM" \ -enable-kvm \ -cpu host \ - -smp 4 \ - -m 4G \ + -smp ${vm_cpus} \ + -m ${vm_memory} \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-hd,drive=hd0 \ -drive file=$OSIMGF,if=none,aio=native,cache=none,format=qcow2,id=hd0 \ + "${EXTRA_DRIVES[@]}" \ ${FEMU_OPTIONS} \ - -net user,hostfwd=tcp::8080-:22 \ + -net user,hostfwd=tcp::${ssh_port}-:22 \ -net nic,model=virtio \ -nographic \ -qmp unix:./qmp-sock,server,nowait 2>&1 | tee log From e30e50215e75d67181afb031041c60eee1923db0 Mon Sep 17 00:00:00 2001 From: Github_Woong-DoubleK Date: Mon, 14 Sep 2026 00:15:26 +0900 Subject: [PATCH 17/17] docs: write down the commands that bring this device up The container path existed but the way to use it did not, beyond a compose invocation with six environment variables buried in a conversation. RUNNING.md is the two entry points with their actual output: a device to poke at, and a measurement. Every command in it was run on the measurement host and the quoted output is what came back, including the guest reporting nvme0n1 as 59.8G -- which is the 64 GiB device with op_pcent=7, and the quickest way for a reader to tell they got the right one. The failures worth naming are named, because each cost time here first. The guest's login prompt appears minutes before ssh works, since cloud-init installs the key afterwards; the seed builder needs pycdlib, which the system interpreter has and a venv usually does not; and image refuses to overwrite an instance's disk rather than silently discarding a guest. The blanket *.md in .gitignore swallowed this file too, the same way it nearly swallowed the harness README. Root-level markdown is now excepted rather than force-added, so the next document beside README.md does not vanish without a word. docker-data/, which the container writes into beside the checkout, is named as ignored instead of sitting untracked. Co-Authored-By: Claude Opus 5 --- .gitignore | 7 +++ README.md | 4 ++ RUNNING.md | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 157 insertions(+) create mode 100644 RUNNING.md diff --git a/.gitignore b/.gitignore index 9215d642714..f1f16cc0012 100644 --- a/.gitignore +++ b/.gitignore @@ -137,4 +137,11 @@ build/ # drops documentation. The harness's own docs are the part a reader needs # most, so they are excepted rather than force-added one at a time. !moe-harness/**/*.md +# and the documentation at the root, which is the first thing a reader opens. +# README.md predates the rule above and is tracked; without this the next +# document beside it would be dropped without a word, as RUNNING.md was. +!/*.md +# What the container writes: counter CSVs, console logs, the QMP socket. The +# compose default puts it beside the checkout, so it needs naming here. +/docker-data/ subprojects/.wraplock diff --git a/README.md b/README.md index ceb8f2c9dfb..532a2f4de2e 100644 --- a/README.md +++ b/README.md @@ -165,6 +165,10 @@ OpenChannel needs a host that speaks it. LightNVM was removed from Linux in ## Running it in a container +Commands, with their real output, are in [RUNNING.md](RUNNING.md). The short +version and the reasoning follow. + + `femu-scripts/femu-docker.sh` is the path below done in a container. Two separate things push it there. diff --git a/RUNNING.md b/RUNNING.md new file mode 100644 index 00000000000..eeccd0a0daa --- /dev/null +++ b/RUNNING.md @@ -0,0 +1,146 @@ +# Running this FEMU + +Every command below was run on the measurement host and its output is what is +quoted. Two entry points, because they answer different questions. + +| you want | go to | +|---|---| +| a VM with this project's SSD attached, to poke at | [A device](#a-device) | +| the placement measurement, end to end | [A measurement](#a-measurement) | + +The device is the same either way, and no geometry argument is needed for it: +`compose.yaml` defaults to the configuration the measurements were taken on -- +64 GiB over 2 channels x 4 LUNs, 512 pages per block, QLC, `op_pcent=7`. + +## Why a container + +The [README](README.md)'s path builds on the host and launches with +`run-blackbox.sh`. That does not work here. `pkgdep.sh` installs packages, and +FEMU pins its memory backend, which needs `RLIMIT_MEMLOCK` raised past the +device size -- this host allows 64 MiB against 64 GiB. Both need root, and +there is no passwordless sudo. + +The container is given `IPC_LOCK` and an unlimited memlock, so it pins without +the host granting root to anyone. The steps are the README's: + +| README | here | +|---|---| +| `sudo ./pkgdep.sh` | `docker/Dockerfile`, builder stage | +| `./femu-compile.sh` | `docker/Dockerfile`, builder stage | +| `./qemu-system-x86_64 -device femu,help` | `femu-docker.sh verify` | +| `./run-blackbox.sh` | `femu-docker.sh run` | + +Inside the container that argument no longer applies -- it runs as root -- and +`run-blackbox.sh` is usable there too: it now reads the same environment +variables. What it could not do before was take any configuration at all; the +layout was written into the file. + +## A device + +```bash +cd /data/kwkim02/MoE_FEMU + +export FEMU_GUEST_DIR=/data/kwkim02/images # where guest disks live +export FEMU_DATA_DIR=$PWD/docker-data # container's /data +export FEMU_INSTANCE=demo # names disk, seed, container +``` + +**Build.** Dependencies and compile happen inside; the host gets nothing. + +```bash +./femu-scripts/femu-docker.sh build +``` + +**Check the device registered.** + +```bash +./femu-scripts/femu-docker.sh verify +# femu options: +# acl= ... blks_per_pl= ... nand_cell_type= ... +``` + +**Make this instance's guest disk.** A copy-on-write overlay plus a cloud-init +seed carrying your ssh key. The base image is never written, and the cloud +image ships no password, so without the seed there is no way in. + +```bash +./femu-scripts/femu-docker.sh image +# overlay /data/kwkim02/images/femu-root-demo.qcow2 +# seed /data/kwkim02/images/seed-demo.iso +``` + +**Start it.** Holds the terminal; Ctrl-C detaches and leaves the container up. + +```bash +./femu-scripts/femu-docker.sh run +# FEMU mode=bbssd, NAND cell type=4, image=/guest/femu-root-demo.qcow2 +# Guest SSH is forwarded to container port 2222 +``` + +**Get in.** Boot takes three to four minutes. The `femu login:` prompt appears +well before you can log in -- cloud-init installs the key after it. Wait for +`Cloud-init ... finished` on the console. + +```bash +./femu-scripts/femu-docker.sh ssh +# femu@femu:~$ lsblk -dno NAME,SIZE /dev/nvme0n1 +# nvme0n1 59.8G +``` + +59.8 G rather than 64 is `op_pcent=7`: the over-provisioning every layout here +is planned against. + +**Stop.** + +```bash +./femu-scripts/femu-docker.sh stop +``` + +**What am I about to get.** + +```bash +./femu-scripts/femu-docker.sh status +# cell=4 size=65536MB 2ch x 4LUN 512pg/blk 1024blk/pl opts=op_pcent=7 +``` + +## A measurement + +A device with nothing on it measures nothing. A run also needs a payload image, +a compiled trace and the placement checks, which is a different entry point: + +```bash +FEMU_PROJECT_ROOT=/data/kwkim02/MoE_SSD \ + bash /data/kwkim02/MoE_FEMU/moe-harness/exp/moe_bcq/femu_run/run_device.sh \ + DEVICE_TAG IMAGE_BASENAME IMAGE_PAGES SPECFILE +``` + +`FEMU_PROJECT_ROOT` is where the data lives. The harness resolves its own code +from where it sits and the data from there, so the two need not be together -- +the images, payload packages and records are tens of gigabytes and are +distributed separately from this repository. + +It boots a device, fills it, asserts the placement landed, replays each trace +in the spec file, and writes `groups.jsonl.gz` and `replay.csv` per run under +`/exp/moe_bcq/femu_run/records//`. + +See [moe-harness/README.md](moe-harness/README.md) for what a spec file is, what +the fill contract is, and why a matching read-back hash does not mean the +placement is right. + +## If something goes wrong + +**`no python3 with pycdlib`** — the seed builder needs it. `pip install --user +pycdlib`, or set `PYTHON` to an interpreter that has it. + +**`kex_exchange_identification: Connection closed`** — the guest is up but +cloud-init has not installed the key yet. Wait for `Cloud-init ... finished`. + +**`already exists: .../femu-root-demo.qcow2`** — `image` refuses to overwrite an +instance's disk. Delete it to start over, or use another `FEMU_INSTANCE`. + +**A device that is not 59.8 G** — something is overriding the compose defaults. +`femu-docker.sh status` prints what the instance would actually get. + +**Port 2222 already bound** — another instance is running. `docker ps`, then +`FEMU_INSTANCE= ./femu-scripts/femu-docker.sh stop`, or set +`FEMU_SSH_PORT` for this one.