Skip to content

feat(telemetry): support env var override - #2157

Open
Hweinstock wants to merge 7 commits into
aws:refactorfrom
Hweinstock:feat/telemetry-disabled-env
Open

feat(telemetry): support env var override#2157
Hweinstock wants to merge 7 commits into
aws:refactorfrom
Hweinstock:feat/telemetry-disabled-env

Conversation

@Hweinstock

Copy link
Copy Markdown
Contributor

Problem

CI is emitting telemetry, and adding noise to the metrics.

Solution

  • Add a AGENTCORE_TELEMETRY_DISABLED environment variable for disabling telemetry without config writes.
  • Set that env var to true in the relevant CI jobs.

Testing

[ starting with a config.json with telemetry.enabled = true]
> AGENTCORE_TELEMETRY_DISABLED=1 agentcore config
{
  "telemetry": {
    "enabled": false,
    "audit": false,
    "endpoint": "https://telemetry.agentcore.aws.dev"
  },
  "installationId": "e92dbb9b-cd4f-4b9f-94cc-ef69d95087cb"
}

> agentcore config
{
  "telemetry": {
    "enabled": true,
    "audit": false,
    "endpoint": "https://telemetry.agentcore.aws.dev"
  },
  "installationId": "e92dbb9b-cd4f-4b9f-94cc-ef69d95087cb"
}

@github-actions github-actions Bot added the size/s PR size: S label Sep 1, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added claude-security-reviewing Claude Code /security-review in progress agentcore-harness-reviewing AgentCore Harness review in progress labels Sep 1, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Changes requested

Thanks for adding the kill switch — the env-var handling, casing/trimming, and use of real temp dirs in tests all look good. One correctness concern:

AGENTCORE_TELEMETRY_DISABLED leaks into the persisted config

src/globalConfig/config.tsx (lines 25–39): the env-var override is applied inside applyOverrides, which is called by DefaultGlobalConfigAccessor.get(). This means the overridden value flows through any subsequent set() call and gets diffed against DEFAULT_GLOBAL_CONFIG before being written to disk (accessor.tsx, set()diff(newConfig, DEFAULT_GLOBAL_CONFIG)).

Repro scenario:

  1. User's on-disk config has telemetry.enabled: true (or is absent, and default is true).
  2. User runs, e.g., AGENTCORE_TELEMETRY_DISABLED=1 agentcore config set telemetry.audit true — or any other command that ends up writing the global config.
  3. The handler reads config via get()telemetry.enabled is false (env override). It sets audit=true and writes the whole object back.
  4. set() diffs against defaults and persists telemetry: { enabled: false, audit: true } to disk.
  5. Next run without the env var set: telemetry.enabled is now false on disk, so telemetry stays disabled. The env var — which is documented as a transient override — has permanently flipped the user's config.

This is especially likely because the PR sets AGENTCORE_TELEMETRY_DISABLED=1 in the CI workflows; any test/CI path that writes the global config will now bake enabled: false into whatever it wrote.

Options to fix:

  • Move the env-var check out of applyOverrides and into the telemetry consumer (e.g., wherever telemetry.enabled is actually read to decide whether to emit). That keeps the persisted config untouched.
  • Keep the env-var check where it is, but have applyOverrides remember the "raw" pre-override value (or have set() re-read the on-disk value for telemetry.enabled when the env var is active) so the write path doesn't observe the override.
  • Have set() explicitly strip the env-var-influenced fields before diffing/writing.

A regression test for "env var is set → run a config set on an unrelated key → on-disk file's telemetry.enabled is unchanged" would be worth adding alongside the fix.

Minor (non-blocking)

  • Consider documenting AGENTCORE_TELEMETRY_DISABLED in the top-level README.md so users can discover it.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 1, 2026
@codecov-commenter

codecov-commenter commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.22%. Comparing base (d304147) to head (d9e8563).
⚠️ Report is 2 commits behind head on refactor.

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2157   +/-   ##
=========================================
  Coverage     97.22%   97.22%           
=========================================
  Files           507      507           
  Lines         33809    33812    +3     
=========================================
+ Hits          32872    32875    +3     
  Misses          937      937           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Sep 1, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@Hweinstock

Copy link
Copy Markdown
Contributor Author

Addressed harness reviewer comments by moving to telemetry. It would be great to handle this in the global config, but handling the edge case described introduces a ton of complexity.

@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Sep 1, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@Hweinstock
Hweinstock marked this pull request as ready for review September 1, 2026 14:17

@jariy17 jariy17 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants