Repository navigation
Conversation
Cancel the streaming request for a tracked container when its display entry is removed, so subsequent start events do not accumulate collectors for the same container ID. Keep event-driven collectors out of the initial-sample wait group, synchronize reads of the tracked ID, and allow decoder notifications to stop on context cancellation. Signed-off-by: a.p.matveev <a.p.matveev@tmn4.esoft.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Running
docker statswithout container arguments opens another stats stream each time a container restarts. Thediehandler removes the container from the display, but leaves its request open. The nextstartcreates a new collector for the same ID, so connections and collectors accumulate even after the restart loop ends.Cancel the collector when removing its display entry, and cancel outstanding requests when the command returns. Let the decoder stop on cancellation too. Only initial collectors use the startup wait group, and container-ID lookups take the stats lock to avoid races during restarts.
Testing
TestRunStatsRestartchecks that each exit closes the old HTTP request and the next start opens a new one. It fails on unmodified master and passes with this patch:I also ran 20 container restarts against Engine 29.8.1. With CLI 29.9.3, socket FDs rose from 24 to 44 and stayed there after a pause. With this patch, they stayed at 24. The separate mock-API test with
--allalso kept one active stream.The unit suite and 20 runs of the stats tests under
-racepassed. The container package passed under-racewithTestInitTtySizeErrorsskipped; that test has a buffer race that I also reproduced on unmodified master. The full daemon-backed e2e suite was not run.Manual reproduction on Linux
Save this as
repro-real-engine.sh. It uses a local Docker Engine, creates one temporary Alpine container, restarts it 20 times, then removes it and stops its stats client. Use a test daemon; the script may pull Alpine 3.22.Run against the same Engine with each client:
dockeron PATH handles the container lifecycle;CLIselects the stats binary. SetDOCKER_HOSTto a different Unix socket if needed. Keep the stats command without container arguments so it follows start/die events.Before the fix, socket FDs should grow by roughly one per restart and remain elevated. After the fix, they should settle near the initial count. The count includes the events connection and idle HTTP connections, so exact totals vary.
Release notes