Skip to content

feat(fetch): retry uncharged busy answers so a brief capacity wait does not fail the command - #64

Draft
gonzalo-zenrows wants to merge 2 commits into
mainfrom
feat/fetch-retry-jitter
Draft

gonzalo-zenrows wants to merge 2 commits into
mainfrom
feat/fetch-retry-jitter

Conversation

@gonzalo-zenrows

@gonzalo-zenrows gonzalo-zenrows commented Oct 16, 2026 •

Copy link
Copy Markdown

Why

zenrows fetch and zenrows extract sent each request once. A brief busy answer (429, 502, 503 or 504) failed the command, although Zenrows does not charge these answers. A 503 RESP009 told agents to rerun with --manual --js-render --premium-proxy. That is the most expensive configuration, and it does not help.

This PR applies the retry policy that every Zenrows Fetch and Extract client shares.

What changes

  1. Retried statuses. The CLI retries 429, 502, 503 and 504 up to --retries <n> times. The flag is new on fetch and extract. The default is 2, and 0 sends the request once.
  2. Retried network errors. The CLI retries a network error only when the request never reached the API. That means a refused connection, a name that did not resolve, a connect timeout or an unreachable network.
  3. Not retried. The CLI never retries its own timeout, or a connection that closed after the request was sent. In both cases the API may still finish the request and charge it.
  4. 429 by code. A 429 with AUTH004 or AUTH014 means the account or key is out of credits, so the CLI sends it once. Every other 429 is retried, including a page that is not JSON.
  5. Target statuses. Under --original-status, or when allowed_status_codes lists the status, the 429 or 5xx may be the target site's page. The CLI sends it once.
  6. Delay. Retry-After counts when it is whole digits. The CLI waits that many seconds, at least 0.25 s and at most 30 s. Otherwise it waits uniform(0.25 s, min(30 s, 1 s × 2^attempt)), where attempt counts retries from 0.
  7. Budget. No retry starts after 90 s, counted from the command's first request. Extract's fallback to Autoparse shares that clock. The budget does not stop a running request, so a command can take longer than 90 s.
  8. Timeout. The default --timeout rises from 120 s to 200 s. The API's own budget is 180 s, and the public docs tell clients to wait at least that long. A test again guards that the default stays above the budget.
  9. SERVICE_BUSY. A 503 RESP009 now returns this new error. It says the request was not charged. It quotes the API's own title, which tells the two Extract meanings apart. The next action says to wait and run the same command again.
  10. The same command. SERVICE_BUSY and the final error after a retried 429, 502, 503 or 504 suggest the command that ran. The suggestion keeps the extract method and every flag, quoted for a shell. These errors no longer advise --js-render or --premium-proxy.
  11. Attempts. With --json, the result has a new attempts field. It counts the HTTP requests the command sent, so 1 means no retry.
  12. Shared helper. src/core/backoff.ts holds the delay schedules. Crawl keeps its own schedule and its own reading of Retry-After, so its behaviour does not change.
  13. Docs. The fetch and extract help, the README Fetch section, and the protected-fetch and trace-debug skills describe all of the above.

Behaviour change

Agents that branched on FETCH_FAILED for a 503 RESP009 now see SERVICE_BUSY. A command that hits a busy API now takes longer before it fails, but no retry starts after 90 s. A slow request now waits up to 200 s before the CLI gives up, instead of 120 s. A connection that closes after the request was sent now fails at once instead of being retried. The --json result gains an attempts field, and no existing field changes. Exit codes stay the same.

Not covered

A 429 page that is not JSON and mentions "upgrade" or "credit" is now retried. After the retries, the existing quota check still reports it as POLICY_MAX_CREDITS_EXCEEDED. That check also serves zenrows usage, so changing it belongs in a separate PR. Under --original-status, a RESP009 from Zenrows itself is now sent once, because the CLI cannot always tell it from a target page. Error envelopes other than --json results do not report the attempt count. The CLI has no cancel option of its own; Ctrl-C ends the command, including a wait between retries. I did not run any of this against the live API.

Testing

  1. npm ci, npm run typecheck and npm test pass: 505 tests, 0 failures, on Node 26 locally. CI runs Node 20 and 22.
  2. tests/fetch-retry.test.ts holds the shared conformance tables for Retry-After, jitter bounds and the retry decision. It also checks the 90 s budget on both paths and exact waits with a fixed random source.
  3. Two tests use real node:http sockets. A refused port gets three attempts. A server that closes the connection after reading the request gets one.
  4. tests/fetch-retry-command.test.ts runs the fetch and extract commands. It checks the suggested command for each extract method, the Autoparse fallback and a manual fetch. It also checks extract --retries 0 and the attempts field.
  5. I ran the built CLI against a local mock API. A 503 RESP009 made three requests and suggested the exact command. A connection closed after the request made one request.
  6. I reverted each fix in turn on a copy of the tree. All 26 mutations made at least one test fail.

…es not fail the command

Fetch and Extract sent each request once. A 429, 502, 503 or 504 (none of
which the gateway charges) or a network error failed the command, and 503
RESP009 fell into the generic branch that advised the most expensive manual
configuration.

- Retry those answers up to --retries times (default 2; 0 sends once). Each
  wait honours Retry-After in whole seconds, capped at 30 s, or is full
  jitter: uniform(0.25 s, min(30 s, 1 s x 2^attempt)). No retry starts once
  the time spent plus the next wait would pass 90 s.
- Never retry the client timeout: the API keeps working and may still charge
  the request. Never retry other 4xx, or a 429 that says the account is out
  of credits. The Extract-not-enabled fallback to autoparse is unchanged.
- 503 RESP009 is now SERVICE_BUSY: not charged, retry shortly, no escalation.
- The gateway budget is 180 s for the whole request, not 90 s. The client
  timeout stays at 120 s; the timeout error now says the API may still
  finish and charge the request, and suggests a value above the budget.
- Retry-After parsing and the delay schedules move to src/core/backoff.ts;
  Crawl keeps its own schedule unchanged.
…back to the command they ran

The retry change resent requests the API may already be working on, retried
pages the target site sent, and told agents to run a different command than
the one that failed.

- Retry a network error only when the request never reached the API
  (refused connection, DNS failure, connect timeout, unreachable network).
  A connection that closes after the request was sent is not retried: the
  API may still finish and charge it, as with a timeout.
- Do not retry a 429, 502, 503 or 504 under original_status, or when
  allowed_status_codes covers it: that status may be the target site's page.
- Decide a 429 by its error code: AUTH004 and AUTH014 are not retried, any
  other 429 is, including a non-JSON page.
- Read Retry-After as whole digits clamped to 0.25-30 s, so 0 cannot drive a
  tight loop. Crawl keeps its own reading. --retries takes whole decimal
  digits only, so 1e1, 0x2 and padded values are usage errors.
- Extract's Autoparse fallback shares the first request's 90 s budget.
- Raise the default client timeout to 200 s, above the API's 180 s budget,
  as the public docs advise, and restore the test that guards it.
- SERVICE_BUSY and the final error after a retried 429, 502, 503 or 504
  suggest the command that ran, with its flags and extract method, and no
  longer advise the most expensive configuration. On Extract, SERVICE_BUSY
  quotes the API's own title, which says which of RESP009's two meanings
  applies.
- --json results report attempts, the number of requests sent.
@gonzalo-zenrows
gonzalo-zenrows force-pushed the feat/fetch-retry-jitter branch from 0df80be to 4104b09 Compare October 19, 2026 21:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant