Skip to content

io: wait for writable sockets before retrying sync writes#12116

Open
stondo wants to merge 2 commits into
fluent:masterfrom
stondo:fix/sync-write-poll-ready-v3
Open

io: wait for writable sockets before retrying sync writes#12116
stondo wants to merge 2 commits into
fluent:masterfrom
stondo:fix/sync-write-poll-ready-v3

Conversation

@stondo

@stondo stondo commented Jul 19, 2026

Copy link
Copy Markdown

Summary

Replace the one-second sleep used by synchronous nonblocking socket writes
after EAGAIN/EWOULDBLOCK with a bounded POLLOUT wait.

Fixes #12115

Root cause

fd_io_write() currently sleeps for one full second whenever send() or
sendto() temporarily fills a nonblocking socket buffer. A large response can
hit that path repeatedly, adding one second per backpressure event even when
the socket becomes writable much sooner.

Changes

  • wait for POLLOUT for at most one second before retrying;
  • use WSAPoll on Windows, matching the existing network implementation;
  • do not count an interrupted POSIX wait as one of the bounded retries;
  • preserve the existing 30-attempt limit and partial-write reporting;
  • add deterministic internal tests for prompt wakeup and peer-close error
    propagation using small nonblocking socket pairs.

Compatibility

There is no configuration or API change. The one-second upper bound and
30-attempt limit remain unchanged. The writer now resumes promptly when the
socket becomes writable instead of always waiting the full second.

The regression tests are POSIX-only because they use socketpair() and
pthreads. The Windows source path uses the same WSAPoll pattern present in
src/flb_network.c.

Testing

Commands:

cmake -S . -B build-sync-final \
  -DFLB_TESTS_RUNTIME=On \
  -DFLB_TESTS_INTERNAL=On \
  -DFLB_EXAMPLES=Off \
  -DCMAKE_CXX_FLAGS=-mavx2
cmake --build build-sync-final -j8
ctest --test-dir build-sync-final --output-on-failure \
  -R '^flb-(rt-in_tcp|rt-out_tcp|it-network|it-http_client|it-http_server)$'
ctest --test-dir build-sync-final --output-on-failure \
  --repeat until-fail:10 \
  -R '^flb-it-network$'

The tests ran in the documented Debian 12 rootless Podman environment. The
explicit AVX2 flag is a GCC 12 compatibility workaround for bundled simdutf,
not part of this change.

Results:

  • full runtime/internal build: passed;
  • five focused network, HTTP, and TCP tests: passed;
  • flb-it-network: passed 10 consecutive iterations;
  • negative control with the test and old writer: failed in 2.02 seconds at the
    elapsed-time assertion, as expected;
  • patched test: completes in approximately 20-30 ms;
  • peer-close test: preserves EPIPE, ECONNRESET, or ENOTCONN instead of
    returning with stale EAGAIN;
  • focused regression under Valgrind: 0 bytes live at exit and 0 errors;
  • full-range commit-prefix validation: passed;
  • git diff --check: passed;
  • DCO: signed.

Live syscall evidence from the original reproduction:

before: 16 EAGAIN results, 16 one-second sleeps, 16-17 s response
after:  23 EAGAIN results, 23 POLLOUT waits, 0 sleeps, 0.252 s response

Enter [N/A] in the box, if an item is not applicable to your change.

Testing

  • [N/A] No configuration surface is changed
  • Runtime and syscall output are included above
  • Focused Valgrind run shows no leaks or memory errors

Packaging

  • [N/A] No packaging or container-output change

Documentation

  • [N/A] No user-facing configuration or API change

Backporting

  • Candidate for a separate 5.0 backport after the master change is accepted

Fluent Bit is licensed under Apache 2.0. By submitting this pull request I
understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of non-blocking socket writes by waiting for socket writability (poll-based) rather than using fixed sleep retries.
    • Enhanced handling of interrupted waits and improved behavior under peer disconnects.
    • Added Windows-compatible readiness waiting for socket operations.
  • Tests
    • Added non-Windows coverage to verify non-blocking write completion within a time bound.
    • Added validation that early peer close correctly propagates expected write failures.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2c4d162f-402c-47e9-a876-141ee98cfb90

📥 Commits

Reviewing files that changed from the base of the PR and between abced0f and 9b88c34.

📒 Files selected for processing (1)
  • tests/internal/network.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/internal/network.c

📝 Walkthrough

Walkthrough

Synchronous socket writes now use poll/WSAPoll to wait for POLLOUT after EAGAIN or WOULDBLOCK, with bounded retries and poll-error handling. Non-Windows socket tests cover completion under backpressure and peer-close error propagation.

Changes

Synchronous socket write wait

Layer / File(s) Summary
Poll-based write retry
src/flb_io.c
Adds Windows WSAPoll support and replaces one-second sleeps after temporary send failure with bounded POLLOUT polling, including interruption and poll-error handling.
Writability and peer-close regression tests
tests/internal/network.c
Adds non-Windows socketpair tests that verify timely completion while a reader drains data and failure propagation when the peer closes early.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: edsiper

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: synchronous writes now wait for socket writability before retrying.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f56c5c1e48

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/flb_io.c Outdated
Synchronous writes sleep for one second when send returns EAGAIN. This adds
avoidable latency whenever a nonblocking socket buffer fills.

Wait for POLLOUT before retrying so writes resume as soon as the socket can
accept more data. Retry the write after poll wakes so peer-close errors reach
the existing connection error handling. Preserve the one-second wait bound and
30-attempt limit.

Signed-off-by: Stefano Tondo <stondo@gmail.com>
@stondo
stondo force-pushed the fix/sync-write-poll-ready-v3 branch from f56c5c1 to abced0f Compare July 19, 2026 23:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/internal/network.c`:
- Around line 324-395: Update
test_nonblocking_socket_write_propagates_peer_close to block or ignore SIGPIPE
around the flb_io_fd_write call, restoring the prior signal state afterward so
the test process survives the peer-close condition and can execute its existing
error assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 60941f93-f271-452b-8cd1-4667ab831957

📥 Commits

Reviewing files that changed from the base of the PR and between f56c5c1 and abced0f.

📒 Files selected for processing (2)
  • src/flb_io.c
  • tests/internal/network.c

Comment thread tests/internal/network.c
Use small nonblocking socket pairs to exercise temporary backpressure and a
peer closing while the writer waits. Verify writes resume promptly and closed
connections preserve a critical socket error.

Signed-off-by: Stefano Tondo <stondo@gmail.com>
@stondo
stondo force-pushed the fix/sync-write-poll-ready-v3 branch from abced0f to 9b88c34 Compare July 20, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Synchronous socket writes add one-second delays after EAGAIN

1 participant