Skip to content

feat: add cloud_detection switch and probe providers concurrently - #1359

Open
davidberenstein1957 wants to merge 2 commits into
masterfrom
feat/cloud-detection
Open

feat: add cloud_detection switch and probe providers concurrently#1359
davidberenstein1957 wants to merge 2 commits into
masterfrom
feat/cloud-detection

Conversation

@davidberenstein1957

Copy link
Copy Markdown
Collaborator

What this adds

Two small changes to the cloud metadata detection path.

Concurrent probing. get_env_cloud_details() walked CLOUD_METADATA_MAPPING sequentially with a 1 s timeout per provider. All three entries live on 169.254.169.254, so any machine where that address is not routed paid up to three consecutive timeouts at tracker startup before concluding it is not on a cloud. The loop now fans out over a ThreadPoolExecutor (stdlib, no new dependency) and detection costs about one timeout regardless of how many providers are in the mapping. Results are resolved in mapping order rather than completion order, so the provider reported for a machine that answers on more than one entry is the same as before.

A cloud_detection switch. New tracker option, wired through _set_from_conf like every other one, so it works three ways:

EmissionsTracker(cloud_detection=False)
[codecarbon]
cloud_detection = false
export CODECARBON_CLOUD_DETECTION=false

Default is true, so nothing changes for existing users. When disabled, EmissionsTracker._get_cloud_metadata() returns an empty CloudMetadata without any HTTP call and the tracker takes the normal geolocation path. This is aimed at air-gapped and egress-filtered environments, where the probe today is a guaranteed-failing request with no way to opt out short of switching to OfflineEmissionsTracker.

The public return shape of get_env_cloud_details() and the CloudMetadata dataclass are unchanged, as is the GCP postprocess_function that strips attributes (Kubernetes config and secrets).

Verification

uv run pytest tests/test_cloud.py tests/test_geography.py tests/test_config.py tests/test_emissions_tracker.py passes. Four new tests in tests/test_cloud.py, none of them touching the network:

  • all providers are probed and detection stays deterministic when several answer (guards the concurrency change),
  • no request is issued when cloud_detection=False,
  • detection still runs by default.

The existing per-provider responses-based tests are untouched and still pass, which covers the refactor of the probe itself.

black --check is clean on the four touched files. uv run task lint was not run: it currently reports pre-existing findings across the whole repository unrelated to this change.

Deliberately left out

The proposal this came from also covered broader vendor support and region-level intensity. Both are better argued separately and neither is needed for the wins above:

  • New providers (OVH, Scaleway, Hetzner, OCI). Each needs a payload parser and a verification predicate — several vendors share the same link-local address and the generic OpenStack metadata path answers on any OpenStack private cloud, so a naive mapping entry risks mislabelling a machine. A confidently wrong region is worse than no region.
  • Region-level carbon intensity in codecarbon/data/cloud/impact.csv. The schema already supports it; the cost is data curation and provenance, not code.
  • A carbon_intensity_source field on EmissionsData recording which fallback rung produced the intensity.
  • track_emissions decorator: the new option is not exposed as a decorator keyword, since the config file and environment variable already cover it.

Closes #1352

🤖 Generated with Claude Code

Probe the cloud metadata services concurrently instead of one after
another, so detection costs one timeout rather than one per provider on
machines that are not on a cloud. Resolution stays in mapping order, so
the detected provider does not depend on which probe returns first.

Add a `cloud_detection` option (constructor, config file and
CODECARBON_CLOUD_DETECTION) to skip the probe entirely, for air-gapped
and egress-filtered environments.

Closes #1352

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.45%. Comparing base (065d0e6) to head (0ad25e9).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1359      +/-   ##
==========================================
+ Coverage   91.39%   91.45%   +0.06%     
==========================================
  Files          49       49              
  Lines        5056     5069      +13     
==========================================
+ Hits         4621     4636      +15     
+ Misses        435      433       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 12, 2026 19:14
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 12, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a cloud_detection switch and probe cloud metadata concurrently

1 participant