Feature proposal.
Problem
Every online EmissionsTracker probes the cloud instance metadata service (IMDS) at startup to find out whether it runs on AWS, Azure or GCP. get_env_cloud_details() in codecarbon/core/cloud.py walks CLOUD_METADATA_MAPPING sequentially, with a 1 s timeout per provider. All three entries live on the same link-local address 169.254.169.254, so on a laptop, an on-prem server or any machine where that address is simply not routed, we pay up to three consecutive timeouts before concluding "not on a cloud". That is a guaranteed-failing, guaranteed-slow startup cost for every non-cloud user, and it grows linearly with each provider we might add later.
Some users cannot pay it at all: in air-gapped or tightly egress-filtered environments the probe is not just slow, it is noise in security monitoring, and there is currently no way to turn it off. The only escape today is to use OfflineEmissionsTracker, which gives up much more than cloud detection.
Proposed design
Two small, independent changes to the same code path:
-
cloud_detection switch. A new tracker option, following the existing configuration conventions, so it works as a constructor argument, a .codecarbon.config key and an environment variable:
[codecarbon]
cloud_detection = false
EmissionsTracker(cloud_detection=False)
CODECARBON_CLOUD_DETECTION=false
Default stays true, so behaviour is unchanged for everyone who does not set it. When disabled, EmissionsTracker._get_cloud_metadata() returns an empty CloudMetadata without issuing any HTTP request, and the tracker takes the normal non-cloud geolocation path.
-
Concurrent IMDS probing. Replace the sequential loop in get_env_cloud_details() with a bounded concurrent.futures.ThreadPoolExecutor fan-out over the same mapping. Total detection wall time becomes roughly one timeout instead of one per provider, and stays constant if the mapping grows. Provider selection stays deterministic: results are resolved in CLOUD_METADATA_MAPPING order, not in completion order, so a machine that somehow answers on two entries always reports the same provider it does today.
Neither change alters the public return shape of get_env_cloud_details() or the CloudMetadata dataclass, and no new dependency is involved — concurrent.futures is stdlib.
Why it fits the existing extension points
CLOUD_METADATA_MAPPING is already the single source of truth for what gets probed and how; the concurrency change is confined to the loop that consumes it, and the per-entry postprocess_function hook (used by GCP to strip attributes, which carries Kubernetes config and secrets) is preserved unchanged. The configuration switch uses _set_from_conf, the same mechanism as every other tracker option, so it inherits config-file and environment-variable support for free.
Scope boundary
This proposal deliberately covers only the two items above. Explicitly out of scope, and better argued separately:
- Support for additional providers (OVH, Scaleway, Hetzner, OCI). Adding entries is cheap, but each needs a payload parser and a verification predicate — several vendors answer on the same link-local address, and the generic OpenStack metadata path answers on any OpenStack private cloud, so a naive entry risks mislabelling a machine's region. A confidently wrong region is worse than no region.
- Region-level carbon intensity in
codecarbon/data/cloud/impact.csv. The schema already supports it; the real cost is data curation and provenance (location-based vs market-based figures, sourcing, staleness), which deserves its own discussion.
- A
carbon_intensity_source field on EmissionsData recording which fallback rung produced the intensity.
Happy to open follow-ups for those once the plumbing here is in place.
Feature proposal.
Problem
Every online
EmissionsTrackerprobes the cloud instance metadata service (IMDS) at startup to find out whether it runs on AWS, Azure or GCP.get_env_cloud_details()incodecarbon/core/cloud.pywalksCLOUD_METADATA_MAPPINGsequentially, with a 1 s timeout per provider. All three entries live on the same link-local address169.254.169.254, so on a laptop, an on-prem server or any machine where that address is simply not routed, we pay up to three consecutive timeouts before concluding "not on a cloud". That is a guaranteed-failing, guaranteed-slow startup cost for every non-cloud user, and it grows linearly with each provider we might add later.Some users cannot pay it at all: in air-gapped or tightly egress-filtered environments the probe is not just slow, it is noise in security monitoring, and there is currently no way to turn it off. The only escape today is to use
OfflineEmissionsTracker, which gives up much more than cloud detection.Proposed design
Two small, independent changes to the same code path:
cloud_detectionswitch. A new tracker option, following the existing configuration conventions, so it works as a constructor argument, a.codecarbon.configkey and an environment variable:Default stays
true, so behaviour is unchanged for everyone who does not set it. When disabled,EmissionsTracker._get_cloud_metadata()returns an emptyCloudMetadatawithout issuing any HTTP request, and the tracker takes the normal non-cloud geolocation path.Concurrent IMDS probing. Replace the sequential loop in
get_env_cloud_details()with a boundedconcurrent.futures.ThreadPoolExecutorfan-out over the same mapping. Total detection wall time becomes roughly one timeout instead of one per provider, and stays constant if the mapping grows. Provider selection stays deterministic: results are resolved inCLOUD_METADATA_MAPPINGorder, not in completion order, so a machine that somehow answers on two entries always reports the same provider it does today.Neither change alters the public return shape of
get_env_cloud_details()or theCloudMetadatadataclass, and no new dependency is involved —concurrent.futuresis stdlib.Why it fits the existing extension points
CLOUD_METADATA_MAPPINGis already the single source of truth for what gets probed and how; the concurrency change is confined to the loop that consumes it, and the per-entrypostprocess_functionhook (used by GCP to stripattributes, which carries Kubernetes config and secrets) is preserved unchanged. The configuration switch uses_set_from_conf, the same mechanism as every other tracker option, so it inherits config-file and environment-variable support for free.Scope boundary
This proposal deliberately covers only the two items above. Explicitly out of scope, and better argued separately:
codecarbon/data/cloud/impact.csv. The schema already supports it; the real cost is data curation and provenance (location-based vs market-based figures, sourcing, staleness), which deserves its own discussion.carbon_intensity_sourcefield onEmissionsDatarecording which fallback rung produced the intensity.Happy to open follow-ups for those once the plumbing here is in place.