Skip to content

Keep a transient EDGAR 503 from failing the SEC daily run - #17

Merged
AlexCatarino merged 2 commits into
QuantConnect:masterfrom
AlexCatarino:bug-sec-index-file-retry
Sep 25, 2026
Merged

AlexCatarino merged 2 commits into
QuantConnect:masterfrom
AlexCatarino:bug-sec-index-file-retry

Conversation

@AlexCatarino

@AlexCatarino AlexCatarino commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

The 2026-09-24 daily run failed after EDGAR briefly answered 503 for one CIK's index.json (it served 200 moments later). Errors other than 403/429 were retried with no pause, so all five attempts landed inside the outage, and the final throw ended the download before cik-ticker-mappings.txt was fetched, so the converter failed on every filing.

  • Transient failures (5xx, timeouts, transport errors) on index files now back off 2-16s. A CIK that still fails is logged and skipped; the converter already skips reports without an index file.
  • The CIK-ticker mappings and CIK lookup lists are fetched before the daily loop, with the same backoff, and a failure is now thrown with the URL instead of being dropped. ticker.txt itself answers 503 on about half the requests right now, and those were retried at once and then ignored.

🤖 Generated with Claude Code

AlexCatarino and others added 2 commits September 25, 2026 21:31
… download

EDGAR briefly served a 503 for one CIK's index.json. Non-rate-limit errors
were retried without a pause, then thrown, so the CIK-ticker mappings were
never downloaded and the converter failed for every filing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ticker.txt answers 503 about half the time. Its download ran after the
daily loop, so any failure there skipped it, and a 503 was retried at once
and then dropped silently, leaving the converter without the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AlexCatarino AlexCatarino changed the title Skip an SEC index file that keeps failing instead of ending the day's download Keep a transient EDGAR 503 from failing the SEC daily run Sep 25, 2026
@AlexCatarino
AlexCatarino merged commit 5f1faeb into QuantConnect:master Sep 25, 2026
1 check passed
@AlexCatarino
AlexCatarino deleted the bug-sec-index-file-retry branch September 25, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant