Skip to content

429 support and other fixes#45

Open
qqmyers wants to merge 12 commits into
gdcc:mainfrom
QualitativeDataRepository:429SupportAndMinorFixes
Open

429 support and other fixes#45
qqmyers wants to merge 12 commits into
gdcc:mainfrom
QualitativeDataRepository:429SupportAndMinorFixes

Conversation

@qqmyers

@qqmyers qqmyers commented Jun 10, 2026

Copy link
Copy Markdown
Member

At QDR, I've been testing with 10K files with a rate AWS limit, trying refresh dataset info, etc. This PR has updates including

  • retry on 429 with a subsequent slowdown of average rate,
  • disabling refresh during uploads,
  • resetting upload counts and file checked states upon refresh,
  • fixing file counting on failures and retries, etc.
    It builds on Add Refresh Capability #40.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant