Skip to content

fix: seeded mock generation now gives the same output on every run - #654

Merged
daveshanley merged 1 commit into
pb33f:mainfrom
tx3stn:more-mock-determinism
Oct 10, 2026
Merged

daveshanley merged 1 commit into
pb33f:mainfrom
tx3stn:more-mock-determinism

Conversation

@tx3stn

@tx3stn tx3stn commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Another mock determinism fix to plug the gaps still left over to make pb33f/doctor#119 changes deterministic.

Seeded mock generation now gives the same output on every run. Dates and times come from a fixed UTC instant instead of the clock, UUIDs come from the seed instead of crypto/rand, and XML children flattened with nodeType: none follow the schema's order instead of Go's random map order.

I built one spec that hits every branch of the renderer:

  • every string format (date-time, date, time, email, hostname, ipv4, ipv6, uri, uri-reference, uuid, byte, password, binary, bigint, decimal), plus pattern, enum and length bounds
  • every number format
  • arrays, $ref, oneOf, anyOf, allOf and additionalProperties
  • every XML node type: attribute, legacy attribute, text, none, wrapped and namespaced
version lines that changed between runs
v0.41.5 13 of 194
these changes 0 of 194

Each of the 13 changed lines on v0.41.5 was one of the three fixes: dateTime, time, uuid (including a uuid nested inside allOf), or a flattened nodeType: none child that moved position. Nothing else changed.

Code audit:

  • libopenapi renderer: no goroutines and no reflection over maps. With a seed set, it never reads the clock. The remaining map loops only copy values and don't draw random numbers. The render cache is rebuilt on every call, so one operation can't affect another's body.
  • reggen draws everything from the generator it's given, and the renderer seeds that.
  • frank has no clock, randomness or environment reads. Every loop goes over a slice or an ordered map, and the Go maps it uses are only for lookups.

@codecov

codecov Bot commented Oct 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (60b177e) to head (3f53470).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #654   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          309       309           
  Lines        39208     39213    +5     
=========================================
+ Hits         39208     39213    +5     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@daveshanley daveshanley left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daveshanley
daveshanley merged commit c4c4be7 into pb33f:main Oct 10, 2026
4 checks passed
@tx3stn
tx3stn deleted the more-mock-determinism branch October 11, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants