Dual-path retry: exponential backoff + CDN-driven httpConfig support - #143
Dual-path retry: exponential backoff + CDN-driven httpConfig support#143MichaelGHSeg wants to merge 4 commits into
Conversation
- HTTPClient: dual-path retry loop (429+Retry-After vs counted exponential backoff), configurable retry properties (MaxRetries, MaxTotalBackoffDuration, MaxRateLimitDuration, BackoffEnabled, RateLimitEnabled, BaseBackoffMs, MaxBackoffMs, StatusCodeOverrides), X-Retry-Count header on retries - Configuration: add MaxRetries, MaxTotalBackoffDuration, MaxRateLimitDuration constructor params - EventPipeline: wire config retry params into HTTPClient at construction; expose _httpClient as internal - SegmentDestination: parse httpConfig from CDN settings response and apply to HTTPClient at runtime - UnityHTTPClient: update DoPost signature to match new abstract base - Tests.csproj: add net10.0 target for local dev (net6.0 runtime no longer installed)
HTTPClient.cs: - Rewrite Upload() with dual-path retry: 429+Retry-After uses uncounted rate-limit path; other retryable errors use counted exponential backoff - Add configurable properties: MaxRetries, MaxRateLimitRetries, BackoffEnabled, RateLimitEnabled, BaseBackoffMs, MaxBackoffMs, MaxRetryAfterCapSeconds, StatusCodeOverrides - Return true (discard) on all terminal failure paths so EventPipeline removes the batch file instead of re-uploading indefinitely - Accept Retry-After: 0 as valid (sleep 0ms, still counts as rate-limit) SegmentDestination.cs: - Read httpConfig from CDN settings and apply backoffConfig/rateLimitConfig - Use CultureInfo.InvariantCulture for double parsing - Map maxRetryInterval to MaxRetryAfterCapSeconds (per-request cap) e2e-cli: - Add CapturingLogger that only captures final-failure messages - Use enrichment callbacks for userId instead of pre-identify - Add AUTO_SETTINGS 2s wait for settings to arrive before first upload - Enable retry-settings test suite
…ides - Clamp ParseRetryAfter minimum to 1 second to prevent busy-loop when server sends Retry-After: 0 or MaxRetryAfterCapSeconds is misconfigured - Replace hardcoded status code list in ParseStatusCodeOverrides with dynamic key enumeration (matches analytics-kotlin approach), so CDN overrides for any valid HTTP status code (100-599) are respected
|
Closing this as superseded, with the surviving piece carried forward in #147. Context. This branch and #144 were never stacked — both branched independently off What #144 already covers. #144 reimplemented this ground more thoroughly (1,929 insertions vs 326 here): the full What was still missing, and where it went. #144 left no user-facing entry point — #147 closes the gap in the Kotlin/Swift shape instead: it makes Nothing here is lost — the branch remains if we want to revisit the per-knob API separately. |
Summary
Rewrites
HTTPClient.Upload()with a structured dual-path retry system and adds support for reading retry configuration from CDN settings (integrations["Segment.io"].httpConfig).Retry logic (
HTTPClient.cs)MaxRateLimitRetriesandMaxRateLimitDuration(default 12h).Retry-After: 0is treated as a 1-second floor to prevent busy-loops.MaxRetriesandMaxTotalBackoffDuration(default 12h).trueon all terminal failure paths soEventPipelineremoves the batch file instead of re-uploading indefinitely.MaxRetries,MaxRateLimitRetries,BackoffEnabled,RateLimitEnabled,BaseBackoffMs,MaxBackoffMs,MaxRetryAfterCapSeconds,StatusCodeOverrides.X-Retry-Countheader on retry attempts.CDN-driven config (
SegmentDestination.cs)httpConfig.backoffConfigandhttpConfig.rateLimitConfigfrom CDN settings and applies them to the HTTP client after settings arrive.statusCodeOverridesuses dynamic key enumeration (matching the analytics-kotlin approach) instead of a hardcoded status code list — any valid HTTP status code (100–599) in the CDN config is respected.CultureInfo.InvariantCulturefordouble.TryParseto handle non-English locales correctly.maxRetryIntervaltoMaxRetryAfterCapSeconds(per-request sleep cap, not total budget).E2E cli (
e2e-cli/)CapturingLoggerfilters to final-failure messages only ("Retries exhausted", "Max total backoff") to avoid false failures from transient per-attempt errors.userIdinjection instead of pre-callingIdentify.new Analytics()whenAUTO_SETTINGS=trueso CDN settings arrive before the first upload.retryandretry-settingstest suites.Test plan
dotnet testpassesbasic,retry,retry-settingssuites pass (68/68)