Skip to content

Wait for timers due within the current millisecond on epoll, kqueue and IOCP - #254

Open
endel wants to merge 1 commit into
mitchellh:mainfrom
endel:timer-timeout-spin
Open

endel wants to merge 1 commit into
mitchellh:mainfrom
endel:timer-timeout-spin

Conversation

@endel

@endel endel commented Oct 21, 2026

Copy link
Copy Markdown

The epoll, kqueue and IOCP backends work out how long tick should wait for the next timer from both times truncated to whole milliseconds, floor(next_ms) - floor(now_ms). A timer due later in the current millisecond therefore gets a timeout of 0: tick calls epoll_wait, kevent or GetQueuedCompletionStatusEx without waiting, re-reads the clock, and does it again, until the timer is due. A loop with a timer pending busy-polls for up to a millisecond before it fires, instead of sleeping.

We hit this in a WebTransport game server built on quic-zig, which re-arms a 1 ms timer. On Linux (aarch64, in a VM) with 512 clients connected, it made ~236,700 epoll_pwait calls a second, nearly all with a zero timeout, against ~4,200 recvmmsg calls. With this change it makes ~1,960 a second, and the server's CPU use fell by about a third: 32.7% → 21.6% at 512 clients and 63.4% → 42.7% at 1,024 (median of 4 alternating runs), with latency unchanged at 512 and lower at 1,024. clock_gettime is a real syscall in that VM, which makes each wasted iteration dearer; on macOS (kqueue), the same server's CPU use fell by ~3%.

Fix

Each backend computes the wait in its own tick, for a syscall that takes a different type, so the fix is a small helper in each:

  • epoll: epoll_wait takes whole milliseconds, so the wait rounds up. It is also capped at maxInt(i32); a timer more than ~24.8 days out failed the @intCast to i32 before.
  • kqueue: kevent takes a timespec, so the wait is exact to the nanosecond and there is nothing to round. Seconds are capped at maxInt(i32) (about 68 years).
  • IOCP: GetQueuedCompletionStatusEx takes whole milliseconds, so the wait rounds up, as on epoll. A wait too long for a DWORD still falls back to INFINITE - 1.

Rounding up means a timer can fire up to 1 ms late on epoll and IOCP, where it used to fire on time at the cost of spinning. On Windows, how long a 1 ms wait lasts depends on the system timer resolution, as it already did for any nonzero wait (#70). epoll_pwait2 would allow an exact wait, but needs Linux 5.11. io_uring (native timeouts) and wasi_poll (an absolute deadline) aren't affected.

Test

Each helper has a unit test: a timer later in the same millisecond, due and overdue timers, and a timer too far off for the syscall's type; the epoll and IOCP tests also check the rounding at whole milliseconds. With the old truncating arithmetic in place of the helpers, the epoll and kqueue tests fail.

  • macOS (aarch64, kqueue): zig build test, 60 passed
  • Linux (aarch64, epoll and io_uring): 144 passed, 3 skipped
  • zig build -Demit-example -Demit-bench, and zig build for every target in the CI matrix, which compiles the IOCP test

Windows and x86_64 tests are left to CI. CI on main has failed since #249 because the workflow still installs Zig 0.16 (#250 moves it to 0.17), so this PR's checks will fail the same way until then.

🤖 Generated with Claude Code

The wait for the next timer was the difference of both times truncated to
whole milliseconds, so a timer due later in the current millisecond got a
timeout of 0: tick called epoll_wait, kevent or GetQueuedCompletionStatusEx
without waiting, over and over, until the timer was due. A server re-arming
a 1 ms timer made ~237,000 epoll_pwait calls a second.

epoll_wait and GetQueuedCompletionStatusEx take whole milliseconds, so their
wait rounds up instead, and a timer can fire up to 1 ms late. kevent takes a
timespec, so it waits exactly. The epoll wait is also capped at maxInt(i32)
ms, where a timer more than ~24.8 days out failed an @intcast.

Assisted-by: Claude Opus 5.5
endel added a commit to endel/quic-zig that referenced this pull request Oct 23, 2026
libxev computed a timer's wait as floor(next_ms) - floor(now_ms), 0 for
one later in the current millisecond, so the loop polled without waiting
until it came. netcode's 1 ms room timer made that constant: 240k
epoll_pwait/s against 4k recvmmsg at 512 clients.

libxev is repinned to quic-zig-2026-10-21, which carries the fix
(mitchellh/libxev#254), and quic-zig's own timeout rounds up for the same
reason. Server CPU on the Linux VM: -34%.

Assisted-by: Claude Opus 5.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant