test(fuzz): add race fuzz targets, make fuzz and race pattern docs by samber · Pull Request #440 · samber/ro

samber · 2026-10-04T16:29:46Z

Adds native Go fuzz targets that hunt races between subscribe, unsubscribe, in-flight messages, multiple subscriptions and sync/async sources, across the core package and the plugins with custom subscribe logic.

Layout: core targets live in fuzz/, one file per source file (fuzz/operator_filter_fuzz_test.go, ...), like bench/. Plugin targets sit next to the code they test, named after the source file (plugins/cron/source_fuzz_test.go, ...).
Readable targets: each target takes typed, named inputs (items uint8, asyncSource bool, unsubscribeAfter uint8...) and keeps its scenario inline, with a doc comment stating the invariant and the seeds. Shared building blocks are small purpose-named files in fuzz/ (source_test.go, recorder_test.go, guards_test.go, waits_test.go...), indexed in fuzz/doc.go. No bitmask decoding, no enum scenarios. fuzz/main_test.go runs goleak for the whole package.
One counter: internal/xfuzz reads RO_FUZZ_ITERATIONS (default 100, 10 under -short). Every target seeds itself with it, so go test -race, make fuzz and CI use the same knob. The 4 hand-rolled round loops (Connectable, Zip, ToChannel, GroupBy) are now native FuzzXxx targets on it.
make fuzz runs every target (RO_FUZZ_ITERATIONS=1000 by default). CI runs it on the stable Go version only. make test still runs each target on its seeds; a comment in the Makefile says to use -skip '^Fuzz' once Go < 1.20 is dropped.
Sync and async: each target runs against both a synchronous source (the teardown is registered after subscribe returns, so it cannot stop the source) and an asynchronous one.
Docs: contributing.md gets a "Race condition patterns" section: 15 generic patterns, each with a bad and a good example and how to test it. The rules: a native fuzz target per applicable pattern, sync and async sources, always -race. hacking.md and CLAUDE.md link to it.

Bugs proven on `main`

155 targets are added. 68 of them reproduce a bug on main and are skipped with f.Skip("race: <id>; remove when fixed"), so CI stays green and each fix PR removes its own skip. The other 87 pass at 2000 iterations and show the race is absent for that operator.

Families with at least one reproduced bug:

Subjects: re-entrant IsClosed/Next/Subscribe deadlock, UnicastSubject closed-subscriber and replay deadlock, long lock while broadcasting.
Connectable/Share: stale subject after a synchronous completion, re-entrant Connect deadlock, ShareReplay never releases its source.
Higher-order and loops: Catch/StartWith over Merge (overlapping Next), Race winner leak, Retry/While/DoWhile/RepeatWith/OnErrorResumeNextWith ignore a closed downstream, FlatMap blocks its producer.
Buffer, window, ordering: BufferWithCount data race, BufferWhen and BufferWithTime lost buffers, WindowWhen lost items and open windows, Zip out of order, CombineLatest stale or duplicated tuples.
Time and async: ObserveOn send on closed channel, SubscribeOn infinite hang, Delay deadlock, FromChannel lost items, Future panic.
Short-circuit: Contains/Find/First keep calling the predicate, Average on an empty source emits twice.
Plugins: iter (ToSeq), stdio and csv readers and writers, http/client shared request, websocket/client, cron.

Fixes follow in separate PRs, one per family.

plugins/iter and plugins/cron are outside go.work, so make fuzz does not run them: use GOWORK=off.

samber

✅ 4 commits: counter + make fuzz + CI, core fuzz targets, plugin fuzz targets, docs
⚠️ 60 of 99 new targets are skipped on purpose: each proves a bug on main
ℹ️ Fix PRs follow, one per family, each removing its skips

samber · 2026-10-04T16:30:10Z

+// a synchronous source emits inside Subscribe (teardown is registered only after it returns),
+// while an asynchronous source emits from its own goroutine (Next races Unsubscribe and Complete).
+// Derive async from a bit of the fuzz input so the corpus covers both.
+func fuzzSource(seed int64, n int, async bool) Observable[int] {


✅ Every target must use both kinds. A synchronous source emits before the teardown exists, an asynchronous one races Unsubscribe.

codecov-commenter · 2026-10-04T16:36:57Z

Codecov Report

❌ Patch coverage is 98.41270% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 68.61%. Comparing base (0b38de7) to head (0b13640).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/xfuzz/fuzz.go	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #440      +/-   ##
==========================================
+ Coverage   68.05%   68.61%   +0.56%     
==========================================
  Files          96       98       +2     
  Lines        8207     8279      +72     
==========================================
+ Hits         5585     5681      +96     
+ Misses       2622     2598      -24

Flag	Coverage Δ
unittests	`68.61% <98.41%> (+0.56%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Every fuzz target seeds itself through internal/xtest, so one variable, RO_FUZZ_ITERATIONS, controls the number of interleavings explored by go test, make fuzz and CI. CI runs make fuzz on the stable Go version.

Targets cover subjects, connectable and share, higher-order operators, buffer and window operators, Zip and CombineLatest ordering, time-based operators and short-circuit operators, each with synchronous and asynchronous sources. Targets that prove a bug on main are skipped with the bug id, so each fix PR removes its own skip. The four hand-rolled round loops become native fuzz targets on the shared counter.

…et and cron Plugins are separate modules and cannot import internal/xtest, so each carries a small helper that reads the same RO_FUZZ_ITERATIONS variable. Targets that prove a bug are skipped with the bug id.

List the 15 race patterns found in the codebase, with how to test each one. New and changed operators must ship a native fuzz target that covers synchronous and asynchronous sources, and tests must always run with -race.

Core fuzz targets live in the fuzz package, one file per source file (fuzz/operator_filter_fuzz_test.go, ...), like bench/. Plugin fuzz targets sit next to the code they test, named after the source file. The shared iteration helpers move from internal/xtest to internal/xfuzz and plugins import them directly. The fuzz make target is phony, because a fuzz directory now exists, and a comment records that the test target can skip fuzz targets once Go 1.20 is the minimum version.

…ples Each of the 15 patterns gets its own section with a generic bad and good example, how to test it, and the library facts behind it. The section no longer references pull requests or specific operators, and documents where fuzz targets and helpers live.

Docusaurus fails the build on broken anchors. Link to the page and name the section in the text instead.

Wait helpers, panic recovery, leak assertion, standard seeds and the failing writer were duplicated across plugin fuzz tests. They now live in internal/xfuzz with their own unit tests. The remaining per-plugin helper files are named fuzz_test.go.

Helpers and targets carried cryptic family prefixes. They now describe what they do, for example runWithRawSink, rawNotificationSink and predicateCallCounter, and targets are named after the operator under test. Duplicate min helpers are merged and stale file references are removed from skip comments.

…e inputs Targets take typed, named inputs and carry a doc comment with the scenario and the invariant, instead of decoding a seed and a bit mask. Generic building blocks (sources, recorder, guards, waits, stop helpers) live in small purpose-named files. The conventions are written in fuzz/doc.go.

Each target takes typed, named inputs and documents its scenario and invariant. Targets whose scenarios read differently are split, for example the self and outside unsubscribe variants of the broadcast target.

…e inputs Targets that mixed several scenarios in one input are split by scenario, for example plain, take and unsubscribe variants of the higher-order operators. Boundary, ordering and higher-order checks are written as named expectations instead of mode switches.

…ld helpers Sink, creation, utility, error handling and upstream targets take typed named inputs and use the shared fuzz primitives. The old enum-driven helper file is removed and the contributor docs describe the new primitives.

The ./bench directory shares the target name, so make reported it as up to date.

samber commented Oct 4, 2026

View reviewed changes

samber changed the title ~~test: add race fuzz targets, shared RO_FUZZ_ITERATIONS and race pattern docs~~ test(fuzz): add race fuzz targets, make fuzz and race pattern docs Oct 4, 2026

samber marked this pull request as ready for review October 4, 2026 18:07

samber added 14 commits October 6, 2026 23:37

test(fuzz): add shared iteration counter, make fuzz and CI step

1b3b06b

Every fuzz target seeds itself through internal/xtest, so one variable, RO_FUZZ_ITERATIONS, controls the number of interleavings explored by go test, make fuzz and CI. CI runs make fuzz on the stable Go version.

test(fuzz): add race fuzz targets for iter, stdio, csv, http, websock…

44ffa9a

…et and cron Plugins are separate modules and cannot import internal/xtest, so each carries a small helper that reads the same RO_FUZZ_ITERATIONS variable. Targets that prove a bug are skipped with the bug id.

docs: drop anchor links that break the docusaurus build

e9737f6

Docusaurus fails the build on broken anchors. Link to the page and name the section in the text instead.

test(fuzz): rewrite subject and connectable targets with readable inputs

3c99500

Each target takes typed, named inputs and documents its scenario and invariant. Targets whose scenarios read differently are split, for example the self and outside unsubscribe variants of the broadcast target.

build: mark bench target phony

01d47cd

The ./bench directory shares the target name, so make reported it as up to date.

samber force-pushed the test/race-fuzz branch from bd6085b to 01d47cd Compare October 7, 2026 07:40

Navigation Menu

Repository navigation

test(fuzz): add race fuzz targets, make fuzz and race pattern docs - #440

test(fuzz): add race fuzz targets, make fuzz and race pattern docs#440
samber wants to merge 14 commits into
mainfrom
test/race-fuzz

samber commented Oct 4, 2026 •

edited

Loading

Uh oh!

samber left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samber Oct 4, 2026

Uh oh!

Uh oh!

codecov-commenter commented Oct 4, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

samber commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bugs proven on main

Uh oh!

samber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samber Oct 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov-commenter commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

samber commented Oct 4, 2026 •

edited

Loading

Bugs proven on `main`

codecov-commenter commented Oct 4, 2026 •

edited

Loading