Feature proposal: scheduler job metadata on the emissions record
The problem
HPC users are among CodeCarbon's most motivated adopters — academic and national centres have sustainability reporting obligations and already account every job by user and project — but the unit of work on a cluster is a SLURM job, not a Python function. Today there is no way to say "this row came from job 1234567 on node nid001, charged to account xyz".
The workaround people actually use is smuggling the job id into the project name:
export CODECARBON_PROJECT_NAME=$SLURM_JOB_ID
That works, but it overloads a field that means something else, and it gives you only one of the several identifiers a centre needs. Our own material shows the gap: docs/how-to/slurm.md is a long hand-rolled recipe, and examples/slurm_rocm/ ships four batch scripts that each re-solve the same tagging problem. The information CodeCarbon needs is already sitting in the environment — SLURM exports it into every job step for free — we simply do not read it.
The payoff for reading it is that the emissions CSV becomes directly joinable against sacct -j <id>, which is what turns a pile of per-run rows into a cluster report.
Proposed design
New optional fields on EmissionsData, all defaulting to "" so nothing changes for non-HPC users:
scheduler: str = "" # "slurm"
job_id: str = ""
job_name: str = ""
job_user: str = ""
job_account: str = ""
job_partition: str = ""
node_name: str = ""
Populated automatically in EmissionsTracker._prepare_emissions_data() from the environment. Under SLURM these come from SLURM_JOB_ID, SLURM_JOB_NAME, SLURM_JOB_ACCOUNT, SLURM_JOB_PARTITION, SLURM_JOB_USER and SLURMD_NODENAME. No user action, no new API — an existing batch script starts producing tagged rows the moment it upgrades.
A generic escape hatch so other schedulers do not need code: any field can be set or overridden with a CODECARBON_JOB_* environment variable (CODECARBON_JOB_ID, CODECARBON_JOB_ACCOUNT, …). PBS, LSF and OAR sites map their own variables onto that contract in two lines of shell, and we publish the contract as documentation rather than shipping a backend per scheduler.
Why it fits the existing extension points
_prepare_emissions_data() is already the single place where run context is assembled onto the record, alongside cloud and geography metadata, so this is one more source of context in the place that collects context. The CSV writer already detects a header change and backs up the previous file, so the schema addition is handled by machinery that exists. The API output path lists its fields explicitly in ApiClient.add_emission, so the new columns land in the CSV without altering the API contract. Prometheus labels are an explicit allowlist and are likewise unaffected. There are no new dependencies — it is os.environ and a dataclass.
Scope boundary
Deliberately out of scope for now:
- A
codecarbon slurm wrapper command. codecarbon monitor -- <cmd> already wraps an arbitrary command, and with automatic metadata it needs nothing SLURM-specific. A dedicated command should wait until there is a job the generic one cannot do.
- Prolog/epilog integration. Reading energy counters at job start and end from SLURM's prolog/epilog would give a centre cluster-wide coverage with no user changes, but it means CodeCarbon code running as root on every compute node. That is a much larger blast radius and deserves its own proposal and its own security review.
- One-tracker-per-node rank election. Only meaningful once something is launching trackers on our behalf. Related to the same problem in distributed training, and should be solved once, generically, when there is a caller for it.
- Per-job attribution on shared nodes. When several jobs share a node, node-level power cannot be split between them. This needs a decision before any site-wide path ships; it does not block simply labelling the rows.
- PBS/LSF backends, superseded by the
CODECARBON_JOB_* contract above.
Testing
Entirely testable with monkeypatch.setenv — no cluster required. A fake SLURM environment produces populated metadata, an empty environment stays inert, and CODECARBON_JOB_* takes precedence.
Implementation of the scoped slice is ready; PR to follow.
Feature proposal: scheduler job metadata on the emissions record
The problem
HPC users are among CodeCarbon's most motivated adopters — academic and national centres have sustainability reporting obligations and already account every job by user and project — but the unit of work on a cluster is a SLURM job, not a Python function. Today there is no way to say "this row came from job 1234567 on node nid001, charged to account xyz".
The workaround people actually use is smuggling the job id into the project name:
That works, but it overloads a field that means something else, and it gives you only one of the several identifiers a centre needs. Our own material shows the gap:
docs/how-to/slurm.mdis a long hand-rolled recipe, andexamples/slurm_rocm/ships four batch scripts that each re-solve the same tagging problem. The information CodeCarbon needs is already sitting in the environment — SLURM exports it into every job step for free — we simply do not read it.The payoff for reading it is that the emissions CSV becomes directly joinable against
sacct -j <id>, which is what turns a pile of per-run rows into a cluster report.Proposed design
New optional fields on
EmissionsData, all defaulting to""so nothing changes for non-HPC users:Populated automatically in
EmissionsTracker._prepare_emissions_data()from the environment. Under SLURM these come fromSLURM_JOB_ID,SLURM_JOB_NAME,SLURM_JOB_ACCOUNT,SLURM_JOB_PARTITION,SLURM_JOB_USERandSLURMD_NODENAME. No user action, no new API — an existing batch script starts producing tagged rows the moment it upgrades.A generic escape hatch so other schedulers do not need code: any field can be set or overridden with a
CODECARBON_JOB_*environment variable (CODECARBON_JOB_ID,CODECARBON_JOB_ACCOUNT, …). PBS, LSF and OAR sites map their own variables onto that contract in two lines of shell, and we publish the contract as documentation rather than shipping a backend per scheduler.Why it fits the existing extension points
_prepare_emissions_data()is already the single place where run context is assembled onto the record, alongside cloud and geography metadata, so this is one more source of context in the place that collects context. The CSV writer already detects a header change and backs up the previous file, so the schema addition is handled by machinery that exists. The API output path lists its fields explicitly inApiClient.add_emission, so the new columns land in the CSV without altering the API contract. Prometheus labels are an explicit allowlist and are likewise unaffected. There are no new dependencies — it isos.environand a dataclass.Scope boundary
Deliberately out of scope for now:
codecarbon slurmwrapper command.codecarbon monitor -- <cmd>already wraps an arbitrary command, and with automatic metadata it needs nothing SLURM-specific. A dedicated command should wait until there is a job the generic one cannot do.CODECARBON_JOB_*contract above.Testing
Entirely testable with
monkeypatch.setenv— no cluster required. A fake SLURM environment produces populated metadata, an empty environment stays inert, andCODECARBON_JOB_*takes precedence.Implementation of the scoped slice is ready; PR to follow.