Skip to content

[337] Add logic to run child processes at lower priority than parent#142

Merged
rkoster merged 19 commits into
developfrom
337-nice-child-scripts-v2
Jun 4, 2026
Merged

[337] Add logic to run child processes at lower priority than parent#142
rkoster merged 19 commits into
developfrom
337-nice-child-scripts-v2

Conversation

@neddp
Copy link
Copy Markdown
Member

@neddp neddp commented May 29, 2026

Description

This PR continues the work from #133 (by @ionphractal) to add support for spawning child processes with a lower scheduling priority than the parent process.

Motivation

Related to cloudfoundry/bosh-agent#337. When the BOSH agent spawns child scripts (drain, pre-start, etc.), they can starve the agent itself of CPU time. By running child processes at a lower priority (nice +5 on Linux, BelowNormal on Windows), the agent remains responsive even under heavy child workload.

Changes

  • Added SpawnWithLowerPriority field to system.Command struct
  • Implemented lowerProcessPriority in execCmdRunner that:
    • On Linux/Unix: sets child nice value to min(parent_nice + 5, 19)
    • On Windows: sets child process priority class to BelowNormal
  • Priority is applied after Start() in both RunComplexCommand and RunComplexCommandAsync
  • Added integration tests for both Linux and Windows
  • Inlined platform-specific priority logic into system/process_priority_unix.go and system/process_priority_windows.go (inspired by github.com/hekmon/processpriority, MIT, Copyright 2024 Edouard Hur) to avoid the external dependency
  • On Windows, OpenProcess uses minimum required access rights (PROCESS_QUERY_INFORMATION for get, PROCESS_SET_INFORMATION for set) following the principle of least privilege
  • Unix tests invoke bash <script> instead of executing the temp file directly, to work correctly on systems with noexec /tmp mounts

Original author

Commits authored by Michael Both (@ionphractal)

Changes compared to original PR #133

The following changes address review feedback and fix issues found during testing:

  • Use BelowNormal instead of Idle for Windows - Idle is too aggressive (only runs when system is idle); BelowNormal better matches Linux nice +5
  • Use consistent log tag - Pass command name instead of os.Args[0] so logs match the calling RunComplexCommand
  • Make lowerProcessPriority unexported - The receiver is unexported, no reason to export the method; remove duplicate error logging from caller
  • Include pid and error details in log messages - Log pid in all messages; use Error level when getting parent priority fails
  • Cap test priority expectation at 19 - Match the clamping logic so tests pass even when parent nice >= 15
  • Fix test flakiness - Add sleep in test scripts to ensure priority is set before querying (priority applied after Start())
  • Support SpawnWithLowerPriority in RunComplexCommandAsync - The async path is used for drain scripts, errands, and cert manager ops
  • Improve SpawnWithLowerPriority documentation - Document exact platform behavior and clarify clamping edge case
  • Inline priority logic, drop external dependency - Per reviewer feedback (@rkoster), removed the github.com/hekmon/processpriority dependency and inlined the relevant logic into platform-specific files to eliminate supply chain risk
  • Narrow Windows OpenProcess access masks - Use PROCESS_QUERY_INFORMATION / PROCESS_SET_INFORMATION instead of PROCESS_ALL_ACCESS
  • Use bash to run test scripts - Avoids failures on noexec /tmp mounts

bosh-agent changes: cloudfoundry/bosh-agent/pull/438

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 29, 2026

Review Change Stack

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f546d88c-4b0a-4c79-b313-d0bc6b1ca6a7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

Adds an exported Command field SpawnWithLowerPriority and updates go.mod to directly require golang.org/x/sys. The command runner conditionally calls a new execCmdRunner.lowerProcessPriority after spawning a child process. Platform-specific implementations adjust nice values on Unix and set BelowNormal priority on Windows. New tests (Unix and Windows) verify the lowered priority for synchronous and async runs.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding logic to run child processes at lower priority than the parent process.
Description check ✅ Passed The description is detailed and directly related to the changeset, explaining the motivation, specific implementation details across platforms, and test coverage.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 337-nice-child-scripts-v2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented May 29, 2026

CLA Signed
The committers listed above are authorized under a signed CLA.

@neddp
Copy link
Copy Markdown
Member Author

neddp commented May 29, 2026

@coderabbitai review

@neddp neddp requested a review from Copilot May 29, 2026 09:21
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 29, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@system/exec_cmd_runner.go`:
- Around line 57-59: RunComplexCommandAsync currently ignores
Command.SpawnWithLowerPriority; update RunComplexCommandAsync to mirror
RunComplexCommand by checking cmd.SpawnWithLowerPriority immediately after
starting the process (after invoking Start()) and calling
r.lowerProcessPriority(cmd.Name, process.cmd.Process.Pid) (same nolint:errcheck
handling if present). Ensure you reference the same symbols:
RunComplexCommandAsync, Command.SpawnWithLowerPriority, lowerProcessPriority,
and the started process at process.cmd.Process.Pid so async execution respects
the lower-priority flag.
- Around line 31-40: The Windows branch currently hardcodes
processpriority.BelowNormal which can be >= parent when the parent is Idle;
instead derive the child's priority class from the parent's current class and
clamp at Idle so the child is strictly lower. In the Windows branch (where
runtime.GOOS == "windows"), read the parent's priority class (use the variable
or API you already obtain for the parent), compute the next-lower Windows class
(e.g., map parent -> next lower: High->AboveNormal, AboveNormal->Normal,
Normal->BelowNormal, BelowNormal->Idle, Idle->Idle), clamp to
IDLE_PRIORITY_CLASS, then call processpriority.Set(processPid, computedClass)
rather than using the hardcoded processpriority.BelowNormal; keep logging via
r.logger.Debug and preserve SpawnWithLowerPriority semantics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 38e47492-53c3-41a7-8a93-3fcd67d0d9b3

📥 Commits

Reviewing files that changed from the base of the PR and between 3956c21 and ea0ce9a.

⛔ Files ignored due to path filters (33)
  • go.sum is excluded by !**/*.sum
  • vendor/github.com/hekmon/processpriority/.gitignore is excluded by !vendor/**
  • vendor/github.com/hekmon/processpriority/LICENSE is excluded by !vendor/**
  • vendor/github.com/hekmon/processpriority/README.md is excluded by !vendor/**
  • vendor/github.com/hekmon/processpriority/priority.go is excluded by !vendor/**
  • vendor/github.com/hekmon/processpriority/priority_unix.go is excluded by !vendor/**
  • vendor/github.com/hekmon/processpriority/priority_win.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/aliases.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/dll_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/env_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/eventlog.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/exec_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/memory_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/mkerrors.bash is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/mkknownfolderids.bash is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/mksyscall.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/race.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/race0.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/security_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/service.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/setupapi_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/str.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/syscall.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/syscall_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/types_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/types_windows_386.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/types_windows_amd64.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/types_windows_arm.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/types_windows_arm64.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/zerrors_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/zknownfolderids_windows.go is excluded by !vendor/**
  • vendor/golang.org/x/sys/windows/zsyscall_windows.go is excluded by !vendor/**
  • vendor/modules.txt is excluded by !vendor/**
📒 Files selected for processing (4)
  • go.mod
  • system/cmd_runner_interface.go
  • system/exec_cmd_runner.go
  • system/exec_cmd_runner_test.go

Comment thread system/exec_cmd_runner.go Outdated
Comment thread system/exec_cmd_runner.go
@github-project-automation github-project-automation Bot moved this from Inbox to Waiting for Changes | Open for Contribution in Foundational Infrastructure Working Group May 29, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an opt-in mechanism in bosh-utils/system to spawn child processes at a lower scheduling priority than the parent process, to prevent child scripts from starving the agent of CPU time.

Changes:

  • Adds SpawnWithLowerPriority to system.Command to request reduced scheduling priority for spawned processes.
  • Implements priority lowering in execCmdRunner using github.com/hekmon/processpriority (nice adjustment on Unix; priority class adjustment on Windows).
  • Vendors new dependencies and adds integration tests validating the effective priority of spawned processes on Unix and Windows.

Reviewed changes

Copilot reviewed 4 out of 37 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
go.mod Adds github.com/hekmon/processpriority dependency.
go.sum Adds checksums for github.com/hekmon/processpriority.
system/cmd_runner_interface.go Extends Command with SpawnWithLowerPriority.
system/exec_cmd_runner.go Applies post-Start() priority lowering when requested.
system/exec_cmd_runner_test.go Adds OS-specific integration tests asserting lowered child priority.
vendor/modules.txt Updates vendoring metadata to include new dependency and Windows sys package.
vendor/github.com/hekmon/processpriority/README.md Vendors processpriority documentation.
vendor/github.com/hekmon/processpriority/priority.go Vendors processpriority cross-platform API.
vendor/github.com/hekmon/processpriority/priority_unix.go Vendors Unix nice-based priority implementation.
vendor/github.com/hekmon/processpriority/priority_win.go Vendors Windows priority-class implementation.
vendor/github.com/hekmon/processpriority/LICENSE Vendors processpriority license.
vendor/github.com/hekmon/processpriority/.gitignore Vendors processpriority gitignore.
vendor/golang.org/x/sys/windows/aliases.go Vendors x/sys/windows support code (required by processpriority on Windows).
vendor/golang.org/x/sys/windows/dll_windows.go Vendors x/sys/windows support code.
vendor/golang.org/x/sys/windows/env_windows.go Vendors x/sys/windows support code.
vendor/golang.org/x/sys/windows/eventlog.go Vendors x/sys/windows support code.
vendor/golang.org/x/sys/windows/exec_windows.go Vendors x/sys/windows support code.
vendor/golang.org/x/sys/windows/memory_windows.go Vendors x/sys/windows support code.
vendor/golang.org/x/sys/windows/mkerrors.bash Vendors x/sys/windows generation script.
vendor/golang.org/x/sys/windows/mkknownfolderids.bash Vendors x/sys/windows generation script.
vendor/golang.org/x/sys/windows/mksyscall.go Vendors x/sys/windows generation driver.
vendor/golang.org/x/sys/windows/race.go Vendors x/sys/windows race build helper.
vendor/golang.org/x/sys/windows/race0.go Vendors x/sys/windows non-race build helper.
vendor/golang.org/x/sys/windows/security_windows.go Vendors x/sys/windows security APIs.
vendor/golang.org/x/sys/windows/service.go Vendors x/sys/windows service APIs.
vendor/golang.org/x/sys/windows/setupapi_windows.go Vendors x/sys/windows setupapi bindings.
vendor/golang.org/x/sys/windows/str.go Vendors x/sys/windows helper code.
vendor/golang.org/x/sys/windows/syscall.go Vendors x/sys/windows syscall helpers.
vendor/golang.org/x/sys/windows/types_windows_386.go Vendors x/sys/windows types for 386.
vendor/golang.org/x/sys/windows/types_windows_amd64.go Vendors x/sys/windows types for amd64.
vendor/golang.org/x/sys/windows/types_windows_arm.go Vendors x/sys/windows types for arm.
vendor/golang.org/x/sys/windows/types_windows_arm64.go Vendors x/sys/windows types for arm64.
vendor/golang.org/x/sys/windows/zknownfolderids_windows.go Vendors generated known-folder IDs for Windows.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread system/exec_cmd_runner.go Outdated
Comment thread system/exec_cmd_runner.go
Comment thread system/exec_cmd_runner_test.go
Comment thread system/cmd_runner_interface.go Outdated
ionphractal and others added 14 commits May 29, 2026 12:46
Idle priority is too aggressive and can cause process starvation
since it only runs when the system is idle. BelowNormal provides
a closer behavioral match to the Linux nice +5 implementation.
Pass the command name as a logTag parameter instead of using
os.Args[0]. This ensures logs from LowerProcessPriority use
the same tag as the calling RunComplexCommand.
The receiver type (execCmdRunner) is unexported, so there is no
need to export LowerProcessPriority. Also remove duplicate error
logging from the caller since the function already logs errors.
Log the process pid in all debug/error messages to aid
troubleshooting. Use Error level (not Debug) when getting
the parent priority fails since it is an operational failure.
Match the implementation's clamping logic in the test. If the
parent process runs with nice >= 15, rawParentPrio+5 exceeds 19
and the test would fail even though the implementation is correct.
Since priority is set after Start(), a short-lived script may
read its nice value before the parent has applied the change.
Add a brief sleep in test scripts to ensure the priority is
set before querying it. Also fix typo in comments.
The async path is used for drain scripts, errands, and cert
manager operations - the primary use cases that benefit from
lower priority execution.
Document the exact behavior on each platform and clarify that
the child may run at the same priority if the parent is already
at the minimum.
@neddp neddp force-pushed the 337-nice-child-scripts-v2 branch from ba45651 to 244989b Compare May 29, 2026 09:46
@neddp
Copy link
Copy Markdown
Member Author

neddp commented May 29, 2026

Had to force-push to remove the inception@anynines.com email from some of the commits. This was causing the EasyCLA to fail.

@neddp neddp marked this pull request as ready for review May 29, 2026 09:52
Verify that RunComplexCommandAsync also applies lower priority
to spawned processes on both Linux and Windows.
@github-project-automation github-project-automation Bot moved this from Waiting for Changes | Open for Contribution to Pending Merge | Prioritized in Foundational Infrastructure Working Group May 29, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 37 changed files in this pull request and generated 2 comments.

Comment thread system/exec_cmd_runner.go Outdated
Comment thread system/exec_cmd_runner.go
Comment thread system/exec_cmd_runner_test.go Outdated
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Waiting for Changes | Open for Contribution in Foundational Infrastructure Working Group Jun 4, 2026
@neddp neddp requested a review from rkoster June 4, 2026 14:20
@neddp
Copy link
Copy Markdown
Member Author

neddp commented Jun 4, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 4, 2026

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@system/exec_cmd_runner_test.go`:
- Around line 190-211: The test is executing the temp script file directly which
fails on mounts with noexec; modify the test to invoke the script via the shell
instead of executing the file: keep creating/writing/chmod the temp script, but
construct the Command passed to runner.RunComplexCommand with Name set to "bash"
(or the system shell) and pass the temp file path as the first argument (e.g.,
Args containing tmpFile.Name()), preserving SpawnWithLowerPriority on the
Command; update any expectations that reference the command invocation if
needed.

In `@system/process_priority_windows.go`:
- Around line 15-22: The code currently defines processAllAccess with a broad
0xffff mask; replace it with the minimal Windows rights used for priority
operations: use the narrower constants PROCESS_QUERY_INFORMATION for
GetPriorityClass and PROCESS_SET_INFORMATION for SetPriorityClass (or define a
combined mask PROCESS_QUERY_INFORMATION|PROCESS_SET_INFORMATION if a single mask
is needed). Update the constant(s) (e.g., replace processAllAccess) and any
callers that pass it to OpenProcess so they request only the required rights
instead of full processAllAccess.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 54506463-2ec4-4823-b5c4-acf0b161afe4

📥 Commits

Reviewing files that changed from the base of the PR and between ea0ce9a and 8294573.

⛔ Files ignored due to path filters (1)
  • vendor/modules.txt is excluded by !vendor/**
📒 Files selected for processing (6)
  • go.mod
  • system/cmd_runner_interface.go
  • system/exec_cmd_runner.go
  • system/exec_cmd_runner_test.go
  • system/process_priority_unix.go
  • system/process_priority_windows.go

Comment thread system/exec_cmd_runner_test.go
Comment thread system/process_priority_windows.go
neddp added 2 commits June 4, 2026 17:38
Use PROCESS_QUERY_INFORMATION for GetPriorityClass and
PROCESS_SET_INFORMATION for SetPriorityClass instead of
PROCESS_ALL_ACCESS.
Avoids failures on systems where /tmp is mounted noexec.
Also removes the now-unnecessary os.Chmod calls.
@github-project-automation github-project-automation Bot moved this from Waiting for Changes | Open for Contribution to Pending Merge | Prioritized in Foundational Infrastructure Working Group Jun 4, 2026
@rkoster rkoster merged commit 2093ad1 into develop Jun 4, 2026
11 checks passed
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Done in Foundational Infrastructure Working Group Jun 4, 2026
@neddp neddp deleted the 337-nice-child-scripts-v2 branch June 4, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

4 participants