Skip to content

Carbon-aware scheduling: codecarbon wait and a @carbon_aware decorator #1356

Description

@davidberenstein1957

Feature proposal.

The problem

CodeCarbon answers "how much did that job cost the atmosphere?" It cannot answer "when should I have run it?"

For jobs that are not latency-sensitive — nightly retrains, hyperparameter sweeps, batch inference, backfills, cron CI — the largest reduction available is usually not making the code faster, it is moving it a few hours. Grid carbon intensity in a typical European or US zone swings by a factor of two to four within a day. Deferring a multi-hour training job from an evening peak to an overnight wind trough is a large cut for a one-line change.

This is the question users ask after their first month with the package: "fine, I know it's 4 kg — now what?"

Proposed design

Three layers, each usable on its own.

1. Forecast retrieval. A new codecarbon/core/intensity_forecast.py exposing IntensityPoint / Forecast dataclasses and get_forecast(geo, token=..., horizon_hours=48) -> Forecast | None. It reuses the existing geolocation (GeoMetadata lat/lon or country_2letter_iso_code) and the already-supported electricitymaps_api_token config key, so existing users need no new credential. When no provider is configured it returns None and every caller degrades to "run now" — a job must never block on a missing credential.

2. Window selection. best_window(forecast, duration, deadline) -> (start, mean_intensity). A sliding mean over the forecast points restricted to windows finishing before the deadline. Pure function, no I/O, trivially testable. It returns "now" whenever nothing better exists, so the run-immediately path is the default rather than a special case.

3. User-facing surfaces.

A codecarbon wait CLI command, alongside codecarbon monitor:

# print the recommendation and exit
$ codecarbon wait --dry-run --deadline 24h --duration 90m
Best start: 2026-08-13 03:00 UTC  (112 gCO2e/kWh, now: 341)  -> saves ~67%

# block until the greenest window, then run under measurement
$ codecarbon wait --deadline 12h --duration 2h -- python train.py

The blocking form delegates to the existing run_and_monitor, so measurement, CSV output and exit-code propagation are unchanged.

And a decorator / context manager in codecarbon/carbon_aware.py:

@carbon_aware(deadline="6h", duration="45m")
def nightly_retrain():
    ...

All options resolve through the existing config hierarchy (.codecarbon.config / CODECARBON_*), so carbon_aware_deadline, carbon_aware_duration and carbon_aware_threshold_g behave like every other key.

Hard dependency: pluggable intensity providers with forecast support

Today the only intensity source that could serve a forecast is Electricity Maps, via codecarbon/core/electricitymaps_api.py. That module is a single hardcoded .../carbon-intensity/latest endpoint whose one function returns emissions for an amount of energy, not intensity — the carbonIntensity value is fetched, converted, and immediately multiplied by energy.kWh. Nothing exposes the raw intensity and nothing looks forward in time.

So this feature realistically depends on the pluggable intensity-provider work: without a provider layer, carbon-aware scheduling can only talk to Electricity Maps, and only for users who already hold a paid token. That is a real but narrow slice of the audience and it caps the value the feature can deliver. Once providers land, get_forecast should become an optional forecast() method on the provider protocol rather than a second HTTP client. If any part of this lands first, the HTTP should be kept in one place so the provider work can absorb it cleanly.

Scope boundary

Explicitly not in scope:

  • A scheduler. This is a time.sleep, not a daemon. It does not fork, daemonise or persist. A process that must survive a reboot belongs to cron, systemd or Airflow, and the docs should say so.
  • A static fallback diurnal profile for users with no provider token. It would broaden reach enormously but risks systematically wrong advice — solar-heavy zones trough at midday, wind-heavy zones trough at night. Better to return "run now" than confident wrong advice.
  • Marginal intensity. Load shifting properly responds to marginal generation; Electricity Maps' average figure is a proxy, not the same thing. The honest framing is "we optimise against average intensity". A marginal-signal mode belongs with the provider work.
  • Region shifting. For cloud jobs the greenest choice is often where, not when. That is a natural sequel and a much larger scope.

Open questions

  • Should EmissionsTracker gain a carbon_aware=True parameter that defers start()? A tracker that silently blocks for six hours is a surprising API; leaning towards keeping deferral explicit at the call site.
  • Recording what a deferral avoided (deferred_seconds, avoided_emissions on EmissionsData) is what makes the feature demonstrable, but should only ship alongside a blocking path — two always-zero CSV columns are a schema change for no reader.
  • A waiting process must not hold codecarbon/lock.py for the duration of the wait.
  • Re-evaluate mid-wait as the forecast updates, or commit to the initial decision? Suggested: re-check coarsely, and only ever move the start time earlier.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions