feat(telemetry): support env var override - #2157
Conversation
|
Claude Security Review: no high-confidence findings. (run) |
There was a problem hiding this comment.
AgentCore Harness Review
Verdict: Changes requested
Thanks for adding the kill switch — the env-var handling, casing/trimming, and use of real temp dirs in tests all look good. One correctness concern:
AGENTCORE_TELEMETRY_DISABLED leaks into the persisted config
src/globalConfig/config.tsx (lines 25–39): the env-var override is applied inside applyOverrides, which is called by DefaultGlobalConfigAccessor.get(). This means the overridden value flows through any subsequent set() call and gets diffed against DEFAULT_GLOBAL_CONFIG before being written to disk (accessor.tsx, set() → diff(newConfig, DEFAULT_GLOBAL_CONFIG)).
Repro scenario:
- User's on-disk config has
telemetry.enabled: true(or is absent, and default istrue). - User runs, e.g.,
AGENTCORE_TELEMETRY_DISABLED=1 agentcore config set telemetry.audit true— or any other command that ends up writing the global config. - The handler reads config via
get()→telemetry.enabledisfalse(env override). It setsaudit=trueand writes the whole object back. set()diffs against defaults and persiststelemetry: { enabled: false, audit: true }to disk.- Next run without the env var set:
telemetry.enabledis nowfalseon disk, so telemetry stays disabled. The env var — which is documented as a transient override — has permanently flipped the user's config.
This is especially likely because the PR sets AGENTCORE_TELEMETRY_DISABLED=1 in the CI workflows; any test/CI path that writes the global config will now bake enabled: false into whatever it wrote.
Options to fix:
- Move the env-var check out of
applyOverridesand into the telemetry consumer (e.g., wherevertelemetry.enabledis actually read to decide whether to emit). That keeps the persisted config untouched. - Keep the env-var check where it is, but have
applyOverridesremember the "raw" pre-override value (or haveset()re-read the on-disk value fortelemetry.enabledwhen the env var is active) so the write path doesn't observe the override. - Have
set()explicitly strip the env-var-influenced fields before diffing/writing.
A regression test for "env var is set → run a config set on an unrelated key → on-disk file's telemetry.enabled is unchanged" would be worth adding alongside the fix.
Minor (non-blocking)
- Consider documenting
AGENTCORE_TELEMETRY_DISABLEDin the top-levelREADME.mdso users can discover it.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## refactor #2157 +/- ##
=========================================
Coverage 97.22% 97.22%
=========================================
Files 507 507
Lines 33809 33812 +3
=========================================
+ Hits 32872 32875 +3
Misses 937 937 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…in persisted config
|
Addressed harness reviewer comments by moving to telemetry. It would be great to handle this in the global config, but handling the edge case described introduces a ton of complexity. |
|
Claude Security Review: no high-confidence findings. (run) |
|
Claude Security Review: no high-confidence findings. (run) |
Problem
CI is emitting telemetry, and adding noise to the metrics.
Solution
AGENTCORE_TELEMETRY_DISABLEDenvironment variable for disabling telemetry without config writes.Testing