Skip to content

docs: correct the estimation chain reference - #1377

Open
davidberenstein1957 wants to merge 4 commits into
docs/traction-batchfrom
docs/methodology-rewrite
Open

docs: correct the estimation chain reference#1377
davidberenstein1957 wants to merge 4 commits into
docs/traction-batchfrom
docs/methodology-rewrite

Conversation

@davidberenstein1957

@davidberenstein1957 davidberenstein1957 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Rewrites docs/explanation/methodology.md against the source, and splits the bibliography and the dashboard equivalences into their own pages.

The page is the one a skeptical evaluator reads before deciding whether to trust CodeCarbon's numbers, and several of its central claims did not match the code. Every constant now carries the file.py:LINE it comes from so a reviewer can spot-check.

Factual corrections

1. The CPU fallback ladder was wrong (resource_tracker.py:249-279)

  • Before: "platform interface → TDP registry → global constant → psutil/cpu_load → 50% of TDP"
  • After: an 11-row ordered table. force_cpu_power comes first and skips every platform backend (:255-260, guard at :272); then force_mode_cpu_load (:263-270); then the platform backend (:221-247); then _setup_fallback_tracking (:159-219), where cpu_load is preferred over constant whenever psutil is present.

2. The 50%-of-TDP figure was attributed to the wrong mode

  • Before: presented as how CodeCarbon estimates from TDP and CPU load.
  • After: CONSUMPTION_PERCENTAGE_CONSTANT = 0.5 (hardware.py:26) applies only in the no-psutil "constant" mode (hardware.py:363). cpu_load mode does something else entirely — see the next item.

2b. cpu_load has TWO power models, selected by tracking_mode (hardware.py:274-352)

  • Before: undocumented in either form. The first revision of this PR documented only the machine-mode cubic, which was arguably worse than documenting neither: it invited a reader comparing a machine-mode run against a process-mode run to attribute the whole divergence to attribution scope and conclude the tool is inconsistent.

  • After: both documented side by side, keyed to tracking_mode, in a new The two cpu_load models section:

    tracking_mode="machine" tracking_mode="process"
    Load source psutil.cpu_percent() (:280-282) per-process CPU-time deltas over wall clock (:294-330)
    Normalisation none divided by core count (:345)
    Curve cubic, 0.1 + 0.9 × (load/100)³ (:287-288) linear, tdp × load_norm / 100 (:346)
    Idle floor 10% of TDP none

    With a worked divergence: at 50% utilisation machine mode gives 21% of TDP, process mode gives 50% — more than double, from the same underlying load. Flagged in an admonition so a user seeing that gap knows it is two models, not a bug.

    The exponent, the 0.1 floor, and the absence of a floor in process mode are all stated as uncited — same treatment as the RAM multipliers; none has a comment, citation or fitting procedure anywhere in the module.

2c. accuracy.md deviation figures were unscoped, and its curve description was wrong

  • Before: the CPU-load-vs-TDP deviation table (−60% to +90%) was presented without naming a tracking mode, and the prose read "CodeCarbon interpolates linearly between idle and TDP" — true of process mode only.
  • After: compare_cpu_load_and_RAPL.py:289-292 constructs its tracker with force_mode_cpu_load=True and no tracking_mode argument, so every figure was gathered in the default machine mode. Stated in an admonition, with an explicit note that the numbers do not transfer to process mode, which is uncharacterised. The "interpolates linearly" sentence now names both curves correctly.

3. Apple Silicon was documented backwards

  • Before: "tracks Apple Silicon using powermetrics… if you do not want to give sudo rights, CodeCarbon will fall back to constant mode", with sudoers instructions.
  • After: _setup_cpu_load_fast is tried before powermetrics (resource_tracker.py:228-232), so with psutil installed — the normal case — cpu_load wins and powermetrics is effectively unreachable. Stated explicitly.

4. DEFAULT_POWER_PER_CORE = 4 was undocumented

  • Before: absent; the docs implied the 85 W global constant covered unknown CPUs.
  • After: a table of the three cases. Registry hit → registry TDP. Model detected but absent from the registry, psutil present → threads × 4 W (cpu.py:29, used :1024) — the actual common case. No model or no psutil → POWER_CONSTANT = 85 (hardware.py:23, applied :457).

5. Equivalences cited a file that does not exist in this repo

  • Before: 0.12 kgCO₂/km, 138 Wh/day TV, 13.3 t/yr, sourced to webapp/src/helpers/constants.ts — there is no webapp/ directory here.
  • After: deleted, replaced by explanation/equivalences.md documenting what actually ships in codecarbon/viz/data.py: 0.409 kg CO₂e/mile (:65), 0.097 kg CO₂/hour (:76), 160.58 kg CO₂/week (:96), with the EPA derivations from their docstrings, and a note that the TV figure has no source in the code at all.

6. Carbon intensity resolution order was scattered and incomplete

  • Before: cloud or country, then a 475 g world average. Electricity Maps was absent entirely despite being the highest-priority non-forced source.
  • After: the full six-level table — forced → cloud region (impact.csv) → Electricity Maps (emissions.py:162-177) → regional US/CAN/Nordic (:182-195, :234-290) → country (:292-326) → 475 gCO₂eq/kWh world average. Plus an explicit warning that a cloud-region miss falls back silently to country then world average (emissions.py:76-97, :303-315) with nothing in the output recording which level answered.

7. The RAM model was presented as empirically motivated

  • Before: "this approach significantly improves the accuracy… a more reasonable estimate."

  • After: stated as a two-stage heuristic, with both stages tabulated, and: "Neither the 5 W figure nor any of the 0.9/0.8/0.7 multipliers has a source in the code." The only citation in the module is a pre-v3 Crucial FAQ for the replaced rule (ram.py:20-22). force_ram_power is pointed at prominently.

    Two worked examples were also wrong: 128 GB and 1 TB were given as ~40 W; the model yields 38 W (5×4 + 5×0.9×4).

8. PUE applies to the per-component columns

  • Before: undocumented.
  • After: energy *= self._pue runs inside the per-hardware measurement loop (emissions_tracker.py:1194), so cpu_energy, gpu_energy and ram_energy are already inflated. Do not multiply again.

9. GitHub issue #457 was cited as if it were a source

  • Before: cited at three places as a reference for Intel Power Gadget's discontinuation.
  • After: relabelled as known limitations, with a note that the issue is the tracking record, not a source.

10. Bibliography moved to explanation/references.md

Every entry verified against the actual paper before pasting. Corrections made against the draft: Henderson et al. is JMLR 21(248):1–43, 2020 (confirmed at jmlr.org/papers/v21/20-312.html); Khan et al. is TOMPECS 3(2), Article 9, 1–26, 2018 (doi 10.1145/3177754); Lacoste et al. and Patterson et al. are listed as arXiv preprints, since neither arXiv record carries the workshop/venue the draft asserted. Author lists, years and arXiv IDs for Strubell, Lottick and Luccioni all check out as drafted. Ends with a link to CITATION.cff.

Also

Verification

uv run task docs passes — build clean, link check clean (4898 attributes).

Not verified

  • The 0.097 kg CO₂/hour TV factor cannot be reproduced from anything in the repo; documented as unsourced rather than given a source.
  • RAM_SLOT_POWER_X86 = 5, the 1.5 W ARM figure and the 0.9/0.8/0.7 multipliers have no citation anywhere in the codebase; documented as asserted.
  • The cubic cpu_load curve's exponent, its 10% floor, and process mode's lack of a floor are likewise uncited in the code.
  • Process-mode cpu_load accuracy has never been profiled; only machine mode has.
  • External URLs are not fetched by the link checker (239 external URLs not checked), though every new citation URL was opened during drafting.

🤖 Generated with Claude Code

The documented CPU fallback ladder did not match resource_tracker.py,
the equivalence constants were sourced to a file absent from the repo,
Electricity Maps was undocumented, and the RAM heuristic was presented
as empirically motivated. Rewrite methodology.md against the source and
split the bibliography and equivalences into their own pages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tracking_mode silently swaps the power model on the estimation path:
cubic with a 10% floor in machine mode (hardware.py:287-288), linear
with no floor in process mode (hardware.py:346). Documenting only the
cubic invited readers to blame the divergence on attribution scope.

Also scope accuracy.md's deviation figures to machine mode, which is
what the profiling script actually measured.

Refs #1378

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The overview page states the formula PUE-inflated; this page stated
E x C and introduced PUE later, so the two read as disagreeing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 13, 2026 05:05
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 13, 2026 05:05
The GPU row named only NVML, but AMD devices are read through AMDSMI
in codecarbon/core/gpu_amd.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant