Skip to content

fix(taskfiles): Always default to a special JOBS variable for concurrency related variables to simplify user control; Tasks that run the concurrent tools default to nprocs.#117

Open
davidlion wants to merge 2 commits into
y-scope:mainfrom
davidlion:concurrency-ctrl
Open

Conversation

@davidlion

@davidlion davidlion commented Jun 30, 2026

Copy link
Copy Markdown
Member

Description

Currently, concurrency variables are not handled consistently so a user cannot easily control the concurrency from the command line. The goal of this PR is to create a consistent flow for users to easily control concurrency in resource limited scenarios.

For example, most tasks use the variable JOBS, but cmake also has a task that uses CMAKE_JOBS and passes that value as a JOBS argument. If the user sets JOBS it will not be respected by the task using CMAKE_JOBS. This is further complicated as top-level projects (e.g. CLP) end up using their own ad-hoc variables to control concurrency.

With this PR, we define how the default behaviour (no explicit value set) should work for concurrency related variables.

  1. Low-level tasks that actually run a tool with concurrency (e.g. boost or cmake) default to the number of logical processors.
  2. Any tasks propagating concurrency related variables should always default to JOBS to respect the user's value.

If the concurrency level is explicitly set by a task it will take priority over any user overrides, but this should only be used in niche cases.

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Manually tested with the boost test and prints.

Summary by CodeRabbit

  • New Features

    • Build tasks now automatically use all available CPU cores by default when no parallelism value is provided.
    • Parallel build settings can now inherit from a shared default across related workflows.
  • Bug Fixes

    • Improved consistency in how build concurrency is handled, reducing cases where builds fell back to tool-specific defaults.
    • Updated related task defaults so build and install steps behave more predictably together.

… set by a user for concurrency related variables; Tasks that use the concurrency value default to nprocs.
@davidlion davidlion requested a review from a team as a code owner June 30, 2026 22:37
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR adds a G_NPROCS variable to misc.yaml computed via getconf _NPROCESSORS_ONLN, and updates boost.yaml and cmake.yaml to default JOBS/CMAKE_JOBS concurrency variables using G_NPROCS or inheritance from JOBS, rather than defaulting to empty values. Documentation comments were adjusted accordingly.

Changes

JOBS Concurrency Defaulting

Layer / File(s) Summary
Add shared processor-count variable
exports/taskfiles/utils/misc.yaml
Adds a top-level G_NPROCS variable computed via `getconf _NPROCESSORS_ONLN`.
Boost JOBS defaulting
exports/taskfiles/utils/boost.yaml
Includes misc.yaml, updates JOBS documentation, defaults build-and-install's JOBS var to G_NPROCS when unset, and removes the empty-string default for JOBS in download-and-install so it relies on the called task's defaulting.
CMake JOBS/CMAKE_JOBS defaulting
exports/taskfiles/utils/cmake.yaml
Updates build task docs and defaults its JOBS var to G_NPROCS when unset; updates install-remote-tar docs and defaults CMAKE_JOBS to inherit from JOBS when unset.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant DownloadAndInstall
  participant BuildAndInstall
  participant InstallRemoteTar
  participant BuildTask
  participant GNPROCS

  Caller->>DownloadAndInstall: invoke (JOBS optional)
  DownloadAndInstall->>BuildAndInstall: forward JOBS
  BuildAndInstall->>GNPROCS: default JOBS to G_NPROCS if unset
  BuildAndInstall->>BuildAndInstall: run build with resolved JOBS

  Caller->>InstallRemoteTar: invoke (CMAKE_JOBS optional)
  InstallRemoteTar->>BuildTask: default CMAKE_JOBS to JOBS if unset
  BuildTask->>GNPROCS: default JOBS to G_NPROCS if unset
  BuildTask->>BuildTask: run cmake build with resolved jobs
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the core change: standardizing concurrency-related taskfile defaults so JOBS uses nprocs where appropriate.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
exports/taskfiles/utils/boost.yaml (1)

154-160: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Keep a local JOBS default here. download-and-install now passes {{.JOBS}} straight through to build-and-install; if it’s unset, Task will expand that to <no value>, which prevents the child task’s default .G_NPROCS .JOBS fallback from doing its job and can surface as an invalid -j value on the default path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@exports/taskfiles/utils/boost.yaml` around lines 154 - 160, The boost task
variables block is missing a local JOBS default, so when download-and-install
forwards {{.JOBS}} into build-and-install it can expand to <no value> and bypass
the child task’s fallback. Update the variables section in boost.yaml near
BUILD_DIR and BUILD_AND_INSTALL_ARGS to define JOBS with a default that falls
back to .G_NPROCS when .JOBS is unset, so build-and-install can safely receive a
valid jobs value.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@exports/taskfiles/utils/boost.yaml`:
- Around line 154-160: The boost task variables block is missing a local JOBS
default, so when download-and-install forwards {{.JOBS}} into build-and-install
it can expand to <no value> and bypass the child task’s fallback. Update the
variables section in boost.yaml near BUILD_DIR and BUILD_AND_INSTALL_ARGS to
define JOBS with a default that falls back to .G_NPROCS when .JOBS is unset, so
build-and-install can safely receive a valid jobs value.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 22618304-b326-4ec4-9998-487ac7d8c79e

📥 Commits

Reviewing files that changed from the base of the PR and between 7947642 and b8491ae.

📒 Files selected for processing (3)
  • exports/taskfiles/utils/boost.yaml
  • exports/taskfiles/utils/cmake.yaml
  • exports/taskfiles/utils/misc.yaml

@davidlion davidlion requested a review from kirkrodrigues July 2, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant