Skip to content

A load platform that drives the store without Accumulate - #95

Open
PaulSnow wants to merge 1 commit into
put-never-compactsfrom
load-platform
Open

PaulSnow wants to merge 1 commit into
put-never-compactsfrom
load-platform

Conversation

@PaulSnow

Copy link
Copy Markdown
Contributor

Stacked on #93 (its spec text refers to #92).

cmd/bdbench drives one or more sharded stores the way accumulate/pkg/database/keyvalue/bcdb does, without Accumulate: one block per interval of dynamic rewrites over a hot key set, permanent appends of new keys, lookups across all ages, then a seal; maintenance on the adapter's cadence (Compress, MergeFinalized, PackFinalized) in its own goroutine, one pass in flight per store, skipped not queued. Defaults are the 2026-09-16 soak's BVN store at 500 tps (~11k dyna puts, ~8k perm puts, ~40k lookups per 1 s block). Flags only (1.10).

Once a minute it reports what 1.2 governs, so a cost that grows with the age of the store is a column that climbs: blocks over the interval; block and seal p50/p90/max; put and read p99; passes and time per maintenance kind; the process's disk read/write rate; store size and file count; history segments per layer; resident filter memory; and reads that returned anything but the last written value, which fail the run. -http serves a live page (cards, charts against age, the per-minute table); -stores N runs several stores on one disk in one process; -pprof serves profiles.

First result (one store, defaults, 30 min on the soak machine's NVMe): seal p50 ~51 ms and block p50 ~210 ms flat from minute 1 to minute 18, put/read p99 under 10 µs, compaction 0.7 s per pass, store 3.8 GB at 18 min, zero wrong answers. A store alone did not knee where the soak's nine did; the shared fsync queue is the next run (-stores 9).

Spec: 2.9 rewritten to the adapter as it is (sharded, seal limit 12,500, window 20, maintenance every 20 commits, packs every 1000; #62 and #47 closed in the register); 2.11 defines the platform and what a healthy store must show on it; #94 enters the register.

🤖 Generated with Claude Code

https://claude.ai/code/session_015wUZmWRdAPtcfFgYrLJCc5

cmd/bdbench drives one or more sharded stores the way the adapter
does: a block per interval of dynamic rewrites over a hot key set,
permanent appends, lookups across all ages, then a seal; maintenance
on the adapter's cadence in its own goroutine, one pass in flight per
store.  Defaults are the 2026-09-16 soak's BVN store at 500 tps.  It
reports once a minute what spec 1.2 governs -- seal and block against
the interval, put and read p99, each maintenance kind's passes and
time, the process's disk traffic, store size, history segments,
resident filter memory -- and fails a run whose reads return anything
but the last written value.  The report is a live page (-http) that a
run is launched by opening; -stores puts several stores on one disk,
the way a validator's partitions and a test network's nodes are.

SPEC 2.9 now describes the adapter as it is (sharded, seal limit
12,500, window 20, maintenance every 20 commits, packs every 1000);
the register closes #62 and #47 and carries #94; 2.11 defines the
platform and what a healthy store must show on it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015wUZmWRdAPtcfFgYrLJCc5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant