Skip to content

feat: add codecarbon doctor command - #1360

Open
davidberenstein1957 wants to merge 3 commits into
masterfrom
feat/codecarbon-doctor
Open

feat: add codecarbon doctor command#1360
davidberenstein1957 wants to merge 3 commits into
masterfrom
feat/codecarbon-doctor

Conversation

@davidberenstein1957

@davidberenstein1957 davidberenstein1957 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What this adds

codecarbon doctor, a command that tells the user, per power component, whether CodeCarbon measures it or estimates it, why a better method was not used, and the concrete fix.

Today the fallbacks are invisible: on a Linux box where /sys/class/powercap/intel-rapl is root-only the CPU silently drops to a load-over-TDP model, an unrecognised CPU model silently uses a generic TDP constant, RAM is always modelled — and codecarbon detect prints a correct hardware list that suggests all is well. This is a disclosure problem, not a measurement problem.

No new dependency, no new measurement code: it reads the mode each hardware object landed in (CPU._mode, CPU._is_generic_tdp) and reuses the existing availability checks (powermetrics.is_powermetrics_available, windows_emi.is_emi_available) to explain the failure.

User-facing surface

codecarbon doctor [--json] [--strict]
  • default: a per-component report with MEASURED / ESTIMATED / UNAVAILABLE, the method, a Why: line and a Fix: line, plus a one-line summary;
  • --json: the same report as JSON, for CI checks and bug reports;
  • --strict: exit 1 when any component is ESTIMATED (an absent GPU is UNAVAILABLE and does not fail, since "this machine should have had a GPU" is guesswork);
  • codecarbon detect gains a closing pointer to doctor.

Public API: codecarbon.diagnostics.ComponentDiagnostic and diagnose(hardware) -> list[ComponentDiagnostic].

Sample:

CPU  12th Gen Intel(R) Core(TM) i7-1260P
    ESTIMATED - CPU load model over a 28 W TDP
    Why: /sys/class/powercap/intel-rapl exists but its energy counter is not
         readable by this user (permission denied)
    Fix: sudo chmod -R a+r /sys/class/powercap/intel-rapl

The reason distinguishes "no RAPL interface on this platform" from "RAPL is there but unreadable" — that distinction is the point of the command; without it the user is back on the RAPL how-to page guessing which stanza applies.

How it was verified

  • tests/test_diagnostics.py: 10 tests covering measured/estimated CPU classification, the RAPL absent-vs-unreadable distinction against a fake powercap tree in tmp_path, generic-TDP flagging, RAM always estimated, GPU present/absent, the JSON schema, and --strict exit codes via CliRunner.
  • uv run pytest tests/test_diagnostics.py tests/cli tests/test_package_integrity.py -q → 82 passed.
  • Ran the real command on an Apple Silicon machine without passwordless sudo; both text and --json output are correct.
  • black and ruff check run scoped to the touched files (the repo-wide task format / task lint reformat ~120 unrelated files, so they were not run); remaining ruff findings in these files are the same pre-existing style class as the rest of cli/main.py.

Deliberately left out

These change behaviour for every existing user and belong in their own PR with their own release note:

  • the start-of-run warning from EmissionsTracker.__init__ when a component is estimated;
  • the measurement_quality provenance field on EmissionsData / the CSV — the vocabulary is settled here (measured / estimated / unavailable) so a follow-up can adopt it;
  • per-backend why_unavailable() methods on rapl.py, powermetrics.py, windows_emi.py, gpu_*.py; the reason is derived from the existing availability checks instead, which covers the common Linux and macOS cases without touching six modules;
  • a cloud/location component (needs an IMDS probe with a timeout budget, worth its own discussion);
  • doctor running privileged fix commands itself — it prints them.

Closes #1350

🤖 Generated with Claude Code


Review follow-up

  • --strict could never pass. RAM is ESTIMATED on every machine (no platform exposes a DRAM energy counter), and --strict failed on any ESTIMATED component, so it exited 1 everywhere. --strict now fails only on components that could have been measured, via diagnostics.strict_failures(); RAM is exempt for the same reason an absent GPU is UNAVAILABLE rather than a failure — the user has nothing to fix. test_doctor_strict_passes_on_a_machine_with_measured_cpu_and_gpu asserts exit 0 with a measured CPU + GPU and an estimated RAM, and test_strict_failures_ignores_ram_but_not_the_cpu asserts an estimated CPU still fails.
  • isinstance instead of type(hw).__name__. The module already reads those classes' private attributes, so it now imports them and dispatches on isinstance; a rename fails at import instead of silently dropping a component from the report. The test fixtures build real (bare) hardware instances accordingly.
  • allow_multiple_runs=True. Without it, a config with allow_multiple_runs = false plus a live run makes __init__ return early, leaving _hardware_initialized unset and _ensure_hardware_ready() raising AttributeError — exactly when someone would reach for doctor. Covered by test_doctor_allows_multiple_runs.
  • Quiet start-up. The tracker's init chatter and the "Multiple instances allowed" warning are suppressed around the probe and the previous log level is restored afterwards, so doctor does not reconfigure a caller's logger.

Report per component whether power is measured from a hardware energy
counter or estimated from a model, why a better method was not used, and
the concrete fix. --json for CI and bug reports, --strict to fail a job
when a machine silently falls back to estimation.

Closes #1350

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.66%. Comparing base (065d0e6) to head (ad62825).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1360      +/-   ##
==========================================
+ Coverage   91.39%   91.66%   +0.26%     
==========================================
  Files          49       50       +1     
  Lines        5056     5195     +139     
==========================================
+ Hits         4621     4762     +141     
+ Misses        435      433       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

davidberenstein1957 and others added 2 commits August 12, 2026 19:11
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- --strict ignored nothing, and RAM is ESTIMATED on every machine, so it
  exited 1 everywhere. Exempt RAM (no platform exposes a DRAM counter),
  the same rationale as an absent GPU being UNAVAILABLE, via
  diagnostics.strict_failures().
- dispatch on isinstance instead of type(hw).__name__, so a rename of the
  hardware classes fails at import instead of silently dropping components.
- pass allow_multiple_runs=True: with a live run, __init__ returns early
  and _ensure_hardware_ready() blows up on unset attributes.
- quiet the tracker's start-up logs around the report, restoring the
  caller's log level afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 12, 2026 19:14
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 12, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature proposal: codecarbon doctor to report measurement quality

1 participant