Otel bump v0.150.0 - #2222
Draft
sky333999 wants to merge 30 commits into
Draft
Conversation
Contributor
|
This PR was marked stale due to lack of activity. |
Squashed application of the aws-sdk-v2 migration (feature/aws-sdk-v2) onto latest main. Replaces the SDKv1 credential chain, service clients, retryers, and request handlers with SDKv2 equivalents, and removes the hand-vendored SDKv1 cloudwatch/cloudwatchlogs services (~44k lines). Source PRs (Jeffrey Chien): - #1981 Support credential chain for AWS SDK Go v2 - #1985 Migrate CloudWatch exporter to SDKv2 - #1989 Migrate CloudWatch logs output plugin to SDKv2 - #1992 Migrate EC2 metadata to SDKv2 - #1994 Migrate ECS service discovery to SDKv2 - #1995 Migrate wizard, downloader, and translator to SDKv2 - #2001 Remove SDKv1 direct dependencies - Use a shared HTTP client to reduce file descriptor usage aws-sdk-go (v1) is no longer a direct dependency. cfg/aws now exposes the v2 credential chain (CredentialsConfig.LoadConfig), STS regional->partitional fallback, smithy custom-header middleware, and a shared BuildableClient.
Resolve conflicts where main landed features on files the migration rewrote: - plugins/outputs/cloudwatch: fold #2073 DowngradeErrors logging onto the v2 smithy.APIError error path, and #2052 config-driven retry count / backoff / concurrency onto the v2 PutMetricData loop. Tests use the v2 client wiring with createDefaultConfig so the new Config fields are populated. - extension/entitystore: keep main's #2072 LeaseWriter + AvailabilityZone, drop the now-dead v1 getEC2Provider (the migration's newServiceProvider sources ASG/service name via IMDS instance tags, not the EC2 DescribeTags API). - translator/tocwconfig: keep main's added env setup, use the v2 retryer const (DefaultMetadataRetries).
Two files added to main after the migration's merge-base still used the SDKv1 aws/endpoints package for partition DNS-suffix lookup. They merge cleanly so conflicts don't surface them; convert them to the vendored awsrulesfn: - applicationsignals/translator.go: endpoints.PartitionForRegion -> awsrulesfn.GetPartition(region).DnsSuffix (#2111) - service/configprovider/otlphttp_validator.go: endpoints.DefaultPartitions() -> awsrulesfn.PartitionDNSSuffixes() (#2062) Add sdk/endpoints/awsrulesfn/suffixes.go (non-generated helper exposing all partition DNS suffixes; survives `make copy-aws-sdk-partition` regeneration). After this commit, no first-party package imports aws-sdk-go v1.
- amazon-contributing fork pin -> v0.0.0-20260607233959-ee44579b1ae3 (otel-bump-v0.150.0 @ ee44579b) - collector-core: stable v1.56.0, unstable v0.150.0; contrib v0.150.0 - semconv frozen at last-available v0.128.1-0.20250610090210-188191247685 (removed upstream) - awscloudwatchlogsprovisionerextension + attributestocontextprocessor pinned to old fork commit 0e486aa (not yet on the v0.150 fork branch) - disable containerlog stanza blank import (not yet on the v0.150 fork branch; TODO to restore)
go.opentelemetry.io/collector/confmap changed KeyDelimiter from a const to a package var, so it can no longer initialize a local const.
…p rename - ClientConfig.Auth is now configoptional.Optional[configauth.Config] (configauth.Authentication removed) -> wrap with configoptional.Some - otlphttp exporter type renamed to otlp_http upstream; update ID assertion
… default drift
- ClientConfig.Auth -> configoptional.Some(configauth.Config{...})
- testdata fixture: headers no longer initialized to {}; force_attempt_http2 defaults true; protobuf_message defaults prometheus.WriteRequest (RW v1)
…TLS Optional - otlpreceiver GRPC/HTTP are now configoptional.Optional[T]: disable via None[T](), populate via GetOrInsertDefault() (Default flavor has HasValue()=true but Get()=nil) - confighttp.ServerConfig endpoint moved under NetAddr (confignet.AddrConfig) - configgrpc/confighttp TLS is now configoptional.Optional[configtls.ServerConfig]
The awsproxy Config renamed its TLSSetting field to TLS.
… TLS; Reload for loaded flag - TargetAllocator is now configoptional.Optional[targetallocator.Config] (HasValue/Get); TLS field renamed from TLSSetting - set TA TLS defaults only when unset; drop CollectorID>0 guard (D5) — CollectorID is populated later (POD_NAME/Validate) - Reload() after manual prom-config copy: sets the loaded flag GetScrapeConfigs requires in prometheus v0.308.1+; applied to both the receiver/prometheus translator and the opentelemetry prometheus pipeline translator - test builds expected via promconfig.Load() to match Reload() defaults
…Telemetry factory
- telemetry config types moved from service/telemetry to service/telemetry/otelconftelemetry; Service.Telemetry is now component.Config (use &otelconftelemetry.Config{...})
- Factories() must set a non-nil Telemetry factory (otelconftelemetry.NewFactory()); the config provider / agent startup requires it.
…cator migration - metrics_receiver: labels.Labels is now a struct (Len()/Range()); add new AppendSTZeroSample / AppendHistogramSTZeroSample appender methods - metrics_type_handler: scrape.Target.DiscoveredLabels(*labels.Builder) - start: promslog.AllowedLevel/Format -> Level/Format; scrape.NewManager gains an AppendableV2 arg (nil) - target_allocator: promslog.Level, TargetAllocator configoptional.Optional, TLS field, 3-arg tamanager.NewManager - tests: labels.FromStrings, 4-arg scrape.NewTarget, MetricMetadata.MetricFamily
Upstream renamed several component types to snake_case (otlphttp->otlp_http, tcplog->tcp_log, udplog->udp_log, filelog->file_log, k8sattributes->k8s_attributes, metricstarttime->metric_start_time, signaltometrics->signal_to_metrics), registering both the canonical and a deprecated-alias type. - translator-output assertions use the canonical names (factory.Type()), including the database-insights pipeline (count/signal_to_metrics) and k8s_attributes translators - defaultcomponents factory-map test expects both canonical + deprecated aliases - jmx: pipeline.MustNewIDWithName removed -> NewIDWithName(pipeline.SignalMetrics, ...)
- debug exporter: build expected from CreateDefaultConfig() + translator overrides (debugexporter QueueConfig has non-zero defaults a hardcoded struct missed) - filterprocessor: compare exported config fields individually; Config now holds unexported ottl.Factory pointer maps that differ by address per CreateDefaultConfig() - journald exporter: LogRetention is int32 under v0.150 cwlogs (SDK v2); cast retention and update the test expectation
Regenerated the sampleConfig golden YAMLs for v0.150 output: component-id renames (otlp_http/tcp_log/udp_log), sending_queue block_on_overflow (was blocking), force_attempt_http2/protobuf_message defaults, cumulativetodelta max_staleness 0->1h, resourcedetection resource_attributes serialization, and empty-map->null changes.
…_errors Upstream made resourcedetection detector errors propagate unconditionally (propagateerrors feature gate promoted to Stable/always-on in contrib v0.146.0, gate code removed in v0.150.0), so a detector failure now aborts collector startup instead of degrading silently. Set ignore_detector_errors: true on both embedded configs so detector failures (EC2 partial IMDS, ECS task-metadata timeout, DescribeTags/DescribeInstances denials) log and continue. Set eks node_from_env_var: HOST_NAME on the default config so the EKS detector's API fallback path can resolve the node name on EKS Auto/Fargate where IMDS is unavailable. HOST_NAME is provided via the downward API in the agent's K8s deployments; it is unset (no-op) on EC2/ECS.
Upstream graduated the pkg.translator.prometheus.NormalizeName gate to Stable (contrib v0.130) and dropped it from the code path. The prometheusremotewrite exporter's factory default add_metric_suffixes is true, which now appends type/unit suffixes (_total, _ratio, unit names) to AMP metric names -- a backward-incompatible change for existing dashboards and queries. Set cfg.AddMetricSuffixes = false in the translator to preserve the prior names, and update the exporter testdata and the six affected sample-config goldens. With that same gate gone, the pkg/translator/prometheus replace (which pinned the fork to revert PR #20519's gate-enable) no longer diverges from upstream -- the fork copy is byte-identical to v0.150.0 -- so drop the replace and resolve the package to upstream.
…figs
model.ValidationScheme is an int enum whose zero value (UnsetValidation) panics
in IsValidLabelName ("Invalid name validation scheme requested: unset"). The 3
relabel.Config structs built programmatically in relabelScrapeConfigs() and
metricNameRelabelConfigs bypass Prometheus's YAML unmarshal + ScrapeConfig.Validate()
pipeline, so their NameValidationScheme stays at the zero value. At scrape time
PopulateLabels -> relabel.ProcessBuilder runs the Replace action on these configs
and panics in the scrape pool goroutine, killing the EMF/CloudWatch prometheus
pipeline (zero metrics).
Set NameValidationScheme: model.UTF8Validation on all 3 configs.
…atch OTLP The prometheus receiver stopped stamping StartTimeUnixNano on cumulative sums/histograms once the RemoveStartTimeAdjustment feature gate was promoted to stable upstream (contrib v0.142.0), delegating that to the standalone metricstarttime processor. CloudWatch's OTLP ingestion requires a start time for sums and histograms and drops data points without it, so prometheus counters silently disappeared from the opentelemetry.collect.prometheus pipeline while gauges were unaffected. Add the metricstarttime processor (default true_reset_point strategy) as the first processor in the EC2 prometheus pipeline to restore the start time before export. Bundle the translator unit test and the affected tocwconfig goldens.
The collector-core v0.138.0 default sending_queue enables an exporter-level batcher (flush 200ms / min 8192 items) with no partitioner. For headerssetter- routed logs exporters it merges requests across distinct aws.log.group.name / aws.log.stream.name values and resets the merged batch context to context.Background(), dropping the client.Info metadata. The headers then come out empty and awscloudwatchlogsprovisioner silently skips creating the stream, so one of the DBI (server-logs/raw-events) and AppSignals dynamic-logs streams is intermittently never created. Add WithSendingQueueBatchMetadataKeys to the otlphttp translator and set sending_queue.batch.partition.metadata_keys on the base-logs and AppSignals dynamic-logs exporters so the batcher partitions per (group,stream) and preserves each partition's context. Metrics and traces exporters are unchanged.
CWA's routing translator left Table[].Action unset, emitting `action: ""` in the generated config and relied on the connector's Config.Validate() to default empty to "move". Contrib v0.145.0 introduced an Action.UnmarshalText that rejects the empty string at config decode time (only "move"/"copy" are valid). Decode runs before Validate(), so the collector aborts loading before the default can apply -- and Validate()'s empty->Move default is on a range copy anyway, so it never wrote back. This crashed collector startup for every config using the app_signals routing connectors, taking the whole agent down. Default empty actions to Move at translate time so the emitted config is decodable. Surfaced as uniform failures of the mac and windows feature integ tests (which bundle app_signals) and app_signals_service_events on linux.
- cloudwatchlogs deadlock (#2190) part-2 adapted to aws-sdk-go-v2 (sharedRetryer/sharedClient) + v1->v2 tests - windowseventlog translator: component id windowseventlog -> windows_event_log (v0.150 factory type) - regenerate main's new/updated OTel-section goldens for v0.150 (default_otel_config*, files_config, host_metrics azure/aks, prometheus pipelines)
The 3 windows-tagged tocwconfig goldens (complete_windows_config, opentelemetry/ default_otel_config_windows, opentelemetry/windows_events_config) were not regenerated during the v0.150 sync because their tests are //go:build windows and don't run on Linux. Regenerated on a Windows Server 2022 host and verified the windows-tagged tests pass there. Changes are the v0.150 schema deltas only (error_output_paths, propagators, exclude_service_attributes, profile_statements, keep_alives_enabled, sending_queue batch/storage restructure, otlp_http rename).
v0.150 prometheusremotewrite adds otel_scope_name/otel_scope_version to every exported series by default. otel_scope_version carries the agent build version, so each agent upgrade would churn every AMP series. Symmetric with the existing AddMetricSuffixes=false. Regenerates the 6 affected sampleConfig goldens + translator testdata.
The agent config schema uses -1 for "never expire"; the exporter's retention validator rejects -1 and expects 0 for the same meaning, so a schema-valid journald config aborted the agent at startup. Pre-existing since the journald pipeline landed (also broken against v0.124).
sky333999
force-pushed
the
otel-bump-v0.150.0
branch
from
August 18, 2026 05:40
ebc64a2 to
beb4ef3
Compare
sky333999
force-pushed
the
otel-bump-v0.150.0
branch
from
August 18, 2026 21:04
beb4ef3 to
ea58720
Compare
Two regressions on the EKS target allocator path inherited with the OTel bump: - Start panics: the TA manager now dereferences the host when building its HTTP client, and CWA passed nil. Pass a nop host; the TA client config uses no extensions. - Start stalls: the initial sync retries with backoff for up to ~15 minutes on an unreachable allocator and only stops on context cancellation, blocking startup and shutdown. Bound it at 2 minutes and cancel on Shutdown; the manager soft-fails and the periodic re-sync recovers once the allocator is reachable. The context stays live after a successful start (reused for TLS-rotation rebuilds). Adds unreachable-allocator regression tests for both.
…w-list The jmxreceiver hash-gates the metrics-gatherer JAR at runtime; a JAR absent from its supported_jars.go silently breaks JMX (failed launch + ~5s restart loop). The update script previously downloaded the latest upstream release unguarded. Verify the downloaded jar's sha256 against the allow-list of the go.mod-pinned receiver before installing it.
Port the fork awsutil's shared-credentials hardening onto the agent's parallel copy in cfg/aws: - an explicitly configured credentials file is authoritative: the SDK no longer merges the default shared config file (~/.aws/config), and a missing file or profile fails loudly - a keyless profile errors instead of returning empty credentials that fail later at signing - an empty filename resolves via AWS_SHARED_CREDENTIALS_FILE before the default path, restoring v1 provider behavior - an empty profile resolves via AWS_PROFILE then "default"; the v2 SDK's LoadSharedConfigProfile rejects an empty profile name
sky333999
force-pushed
the
otel-bump-v0.150.0
branch
from
August 18, 2026 22:21
ea58720 to
bb6a179
Compare
Contributor
Binary Size Reportlinux/amd64
Notable changes:
linux/arm64
windows/amd64
Investigating size changesUse go-size-analyzer to compare binaries: GOEXPERIMENT=jsonv2 go install github.com/Zxilly/go-size-analyzer/cmd/gsa@latest
gsa diff --old <baseline-binary> --new <new-binary> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of the issue
Describe the problem or feature in addition to a link to the issues.
Description of changes
How does this change address the problem?
License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Tests
Describe what tests you have done.
Requirements
Before commiting your code, please do the following steps.
make fmtandmake fmt-shmake lintIntegration Tests
To run integration tests against this PR, add the
ready for testinglabel.