fs-correctness

POSIX correctness testing for a mounted filesystem on macOS, built around a differential baseline.

Written to test Space Drive - an FSKit filesystem (spacefs) mounted at /Volumes/Tigris-Space - but nothing here is specific to it. Point SPACE_MOUNT at any mount.

Why a baseline

pjdfstest fails a few hundred assertions on any macOS filesystem. Some tests require root. Some encode FreeBSD/UFS expectations. APFS itself does not implement everything the suite checks. A raw pass/fail count is close to meaningless.

The same is true of fsx, in a less obvious way. Its -r 4096 -w 4096 -t 4096 configuration fails on APFS with Mapped Write: non-zero data past EOF - a mapped write extending a file trips fsx's past-EOF check on macOS generally, not on any particular filesystem. Reported without a baseline, that reads as data corruption in the filesystem under test.

So every suite runs twice: once against the mount, once against a scratch APFS directory, on the same machine, in the same privilege context, in the same run. The signal is the difference, not the absolute numbers. The tooling reports "fails here but passes on APFS" as its headline and files everything else under noise.

This matters more than it sounds. In practice the diff is what separates a real defect from a cascade - one unsupported operation early in a test file makes every later assertion in that file fail on a missing path, and those cascades can outnumber genuine findings ten to one.

Requirements

macOS with Xcode command line tools, plus autoconf/automake for the pjdfstest build:

xcode-select --install
brew install autoconf automake

Python 3 and Perl come with macOS.

Setup

The vendored suites are not committed — they carry their own licenses and build to native binaries. Fetch and build them first:

./build-tools.sh

This clones pjdfstest and secfs.test into tools/ and compiles them. Re-run with REFRESH_SOURCES=1 to pull newer upstream revisions.

Usage

./run-correctness.sh smoke        # subset of pjdfstest + short fsx/fsstress; minutes
sudo ./run-correctness.sh full    # all 238 pjdfstest files, long fsx, fstorture; hours

variable	default	meaning
SPACE_MOUNT	/Volumes/Tigris-Space	filesystem under test
BASELINE_ROOT	$TMPDIR	where the APFS baseline is created
SKIP_BASELINE	0	set to 1 to skip the baseline (output becomes hard to read)
PJD_GROUPS	profile-dependent	space-separated pjdfstest groups, e.g. "rename link"
FSX_OPS	profile-dependent	fsx operations per configuration
FSSTRESS_OPS	profile-dependent	fsstress operations per process; 4 processes run, so full's 5000 is 20,000 operations total
FSX_SEED	7	fixed so target and baseline run identical operation sequences
STEP_TIMEOUT	3600	per-step wall-clock cap in seconds

Two standalone scripts are useful on their own:

./probe-semantics.sh /some/mount   # capability sheet as TSV, no build required
./rerun-fsstress.sh                # re-run just the fsstress step, verbose

Run it as root

pjdfstest emits not ok N not root for every test needing privilege, which is a large fraction of chown, chmod and link. The baseline diff absorbs most of that noise, but an unprivileged run genuinely covers less ground — permission enforcement in particular is untestable without dropping privilege to a second user, which pjdfstest only does as root.

sudo needs a real TTY. It will not work from an agent shell or an editor's integrated terminal that does not allocate one.

Reading the output

Each run writes results/<timestamp>-fs-correctness-<profile>/:

summary.md                 start here
summary.json               same, machine-readable
diff/pjdfstest-diff.txt    FAIL-HERE-ONLY first, then failures shared with APFS
diff/fsx-diff.txt          per-config exit codes, both sides
diff/probe-diff.txt        semantics table, divergences marked
target/  baseline/         raw TAP and tool logs
run.log  status.tsv        per-step timing and status

FAIL-HERE-ONLY is the section that matters. Everything under FAIL-BOTH is root-only tests, FreeBSD-specific expectations, or APFS gaps.

Two cautions when triaging:

• Cascades inflate the count. If a test file's setup step fails - say the filesystem does not support FIFOs - every later assertion in that file fails on a missing path. Group failures by test file and look for a poisoned setup step before counting anything as a distinct defect.
• fsstress exiting 0 is not automatically a pass. Check that its workers reached the requested operation count. Stopping early means they ran out of tree, which is itself a finding.

What the suites cover

tool	what it finds
pjdfstest	POSIX metadata semantics: permissions, ownership, link/rename/unlink error codes, timestamps
fsx	data path: overlapping reads/writes, mmap, truncate, verified against an in-memory model
fsstress	operation mixing across concurrent processes; good at directory-loop and rename bugs
fstorture	Apple's concurrent stressor across two directories (macOS only, full profile)
probe-semantics.sh	readable capability sheet: hard links, FIFOs, xattrs, locking, timestamp granularity, pathconf
workloads	git commit + fsck, concurrent SQLite in WAL mode, tar round-trip + diff -r

The workloads exist because synthetic suites miss integration-level breakage. SQLite in WAL mode exercises POSIX advisory locking and fsync ordering together, which is where network filesystems usually fail first.

fsrand.py ships in secfs.test but is Python 2 and is skipped unless a python2 is on PATH.

Linux

The heavy artillery - xfstests, LTP, CrashMonkey, Metis - is Linux-only, and xfstests' fsx.c no longer builds on Darwin (it needs linux/mman.h, statx.h, copy_file_range). If the filesystem under test gains a Linux client, running xfstests' generic/ group there covers substantially more than anything here.

Licensing

The scripts in this repository are Apache-2.0; see LICENSE.

The suites build-tools.sh fetches are third-party and carry their own licenses, including GPL v2. They are deliberately not vendored here. See THIRD-PARTY.md.