Bound the GPFIFO-space wait in nvWriteGpEntry()#1192
Open
Lumzdas wants to merge 2 commits into
Open
Conversation
cf7592c to
2fc027e
Compare
With nvWriteGpEntry() bounded, the same stalled-channel scenario spins in __nvPushMakeRoom() instead, one call earlier on the same path. Bound it with the same NV_PUSH_NOTIFIER_SHORT_TIMEOUT / noTimeout logic. Since this function returns void and callers write methods unconditionally after it returns, on timeout wrap the write pointer back to the buffer base and mark the whole buffer free so the writes stay in bounds; the following kickoff then fails and the caller gives up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bound the GPFIFO and pushbuffer waits in nvidia-push
Problem
nvWriteGpEntry() waits for a free GPFIFO entry with no timeout. __nvPushMakeRoom() waits for pushbuffer space with no timeout. Both loops exit only when the GPU makes progress or a channel error is flagged. If a channel stalls without a flagged error, the loop spins forever in kernel context. When the spinning code holds the nvkms lock, every later modeset blocks in D state and the desktop is unusable until reboot.
I hit this reproducibly with DIFR prefetch after resume from suspend. Same trace as #1167 and #1177.
Environment: RTX 5090, driver 595.71.05 open modules, NVreg_PreserveVideoMemoryAllocations enabled, kernel 7.0.3, KDE Plasma on Wayland, S3 suspend.
The nvidia-modeset kthread spins on one core:
and the compositor blocks on the nvkms lock held by that thread:
Fix
Commit 1 bounds nvWriteGpEntry() the same way IdleChannel() already does: poll nvPushImportGetMilliSeconds(), honor the channel noTimeout flag, log an error and return FALSE after NV_PUSH_NOTIFIER_SHORT_TIMEOUT. Kickoff() already handles the FALSE return.
Commit 2 bounds __nvPushMakeRoom() with the same timeout. This function returns void and callers write methods unconditionally after it returns, so on timeout it wraps the write pointer back to the buffer base and marks the whole buffer free. Each method write is asserted smaller than the buffer, so the writes stay in bounds, and they land only in the channel's own pushbuffer. The following kickoff then fails and the caller gives up. Commit 2 is needed because with only commit 1 applied the same stall spins in __nvPushMakeRoom() instead, one call earlier on the same DIFR path. Observed on my machine.
DIFR prefetch already has a failure path for a kickoff that never completes: the semaphore wait in PrefetchSingleSurface() times out and reports NV2080_CTRL_LPWR_DIFR_PREFETCH_FAIL_CE_HW_ERROR. That path was unreachable only because the waits above it could spin first.
Channels created with noTimeout = TRUE keep the old behavior.
Results
Both commits, rebased onto 595.71.05, have been running on my machine since June 25. The stall occurred six times since then, mostly within seconds of resume. Each time the desktop kept working and the driver logged: