Skip to content

fix: lock CPU power history swap - #1335

Open
davidberenstein1957 wants to merge 2 commits into
masterfrom
fix/cpu-power-history-race
Open

fix: lock CPU power history swap#1335
davidberenstein1957 wants to merge 2 commits into
masterfrom
fix/cpu-power-history-race

Conversation

@davidberenstein1957

Copy link
Copy Markdown
Collaborator

Fixes a lost-update race on CPU._power_history.

What changed

CPU._power_history is appended to by the 1 Hz monitor scheduler thread (CPU.monitor_power) and drained by the measure_power_secs scheduler thread (CPU.total_power) — two distinct PeriodicScheduler timer threads (codecarbon/emissions_tracker.py:345-352). The read-then-rebind in total_power was not atomic, so any sample appended between the list comprehension and self._power_history = [] went into the discarded list.

  • Added a threading.Lock to CPU, held only for the O(1) list swap. _get_power_from_cpus() is called outside the lock in both paths, so a slow backend (e.g. the IntelPowerGadget subprocess) never blocks the monitor thread.
  • Removed the unreachable if not power_history_in_W: branch — a sample is unconditionally appended before the average, so the list is never empty.

Behaviour is otherwise unchanged, including the extra synchronous sample taken at measurement time.

Why

In cpu_load mode the reported cpu_power is the mean of the buffered samples. The samples dropped in the swap window are exactly those taken while a slow measurement is in flight, so the resulting error is small but systematic and silent.

How it was verified

New test tests/test_cpu_load.py::TestCPULoad::test_cpu_total_power_keeps_samples_added_while_draining makes the race deterministic (a list subclass that fires monitor_power() at iteration exhaustion, i.e. inside the lost-update window) and asserts the injected sample survives. It fails on master and passes with this change. uv run pytest tests/test_cpu_load.py -q → 9 passed.

Note: uv run task format reformats a large number of unrelated files on the current tree, so only the two touched files are included here.

Closes #1315

🤖 Generated with Claude Code

The 1 Hz monitor scheduler thread appends to CPU._power_history while
the measurement scheduler thread drains it. The read-then-rebind in
total_power() is not atomic, so any sample appended between the list
comprehension and the rebinding was written to the discarded list and
lost, biasing the reported cpu_power.

Take the swap under a lock, held only for the O(1) rebinding so a slow
_get_power_from_cpus() backend never blocks the monitor thread. Also
drop the unreachable empty-history branch: a sample is always appended
before the average.

Closes #1315

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.48%. Comparing base (065d0e6) to head (d70b9ae).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1335      +/-   ##
==========================================
+ Coverage   91.39%   91.48%   +0.08%     
==========================================
  Files          49       49              
  Lines        5056     5059       +3     
==========================================
+ Hits         4621     4628       +7     
+ Misses        435      431       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 12, 2026 19:14
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 12, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU power samples are silently dropped by a race in CPU.total_power()

1 participant