Skip to content

Poll jobs in short intervals: long blocks are unreliable - #185

Merged
vahid-ahmadi merged 2 commits into
mainfrom
fix/job-poll-interval
Aug 26, 2026
Merged

Poll jobs in short intervals: long blocks are unreliable#185
vahid-ahmadi merged 2 commits into
mainfrom
fix/job-poll-interval

Conversation

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Found by running the real thing: a headless Claude Code session against the deployed server, scoring 1p on the UK basic rate through start_job.

It got the right answer — costings 6.463…7.378, cumulative GDP −£10.713bn, matching a local run to the pound — but needed nine polls, and reported:

several polls hit transport timeouts and one Modal InternalFailure; wait_seconds: 30 polled reliably where 60–120 did not

So staying under the 150s ceiling is necessary but not sufficient. The flakiness tracks how long the connection is held open, not how close it gets to the ceiling — it's the same intermittent InternalFailure: Server has lost track of input that shows up on long direct calls.

Changes

  • Default wait 60 → 30, cap 120 → 60, so a caller asking for a long block is capped well below the ceiling rather than at it.
  • Both tool descriptions now say a poll that errors is not a job that failed: the job runs on regardless of the connection watching it, so the right response is to call again with the same job_id. An agent that reads a transport error as a failed score would throw away a running job and start another.
  • The remote smoke test polls at 30s too, so CI exercises the interval users actually get.

Testing

293 integration tests pass (16 in test_jobs.py). Site contract consistent.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lcj9DDqam9KmVCfhEdnJcJ

@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
macromod Ready Ready Preview Aug 26, 2026 11:42am

Request Review

vahid-ahmadi and others added 2 commits August 26, 2026 12:41
Found by running the real thing: a headless Claude Code session against
the deployed server, scoring 1p on the UK basic rate through start_job.
It got the right answer -- costings 6.463..7.378, cumulative GDP
-10.713bn, matching a local run to the pound -- but needed NINE polls,
and reported that "several polls hit transport timeouts and one Modal
InternalFailure; wait_seconds: 30 polled reliably where 60-120 did not".

So staying under the 150s ceiling is necessary but not sufficient. The
flakiness tracks how long the connection is held open, not how close it
gets to the ceiling -- the same intermittent `InternalFailure: Server has
lost track of input` that shows up on long direct calls.

Default wait drops 60 -> 30 and the cap 120 -> 60, so a caller asking
for a long block is capped well below the ceiling rather than at it.

Both descriptions now say a poll that ERRORS is not a job that failed:
the job runs on regardless of the connection watching it, so the right
response is to call again with the same job_id. An agent that reads a
transport error as a failed score would throw away a running job and
start another.

293 integration tests pass (16 in test_jobs).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lcj9DDqam9KmVCfhEdnJcJ
Repairs main. The vintage refresh (#177) generated twelve release-note
pages under notes/releases/2026-08-26-* without running site_nav.py, so
the Site provenance job has been failing on main since that merge with
"FAIL stale navigation" on each new page.

My miss: I checked #177 against the site test suite, site_contract.py
and its own no-edited-vintages invariant, all of which passed, and
merged it while only the Vercel checks had reported. site_nav.py is a
separate gate and I did not run it.

Mechanical regeneration -- `python3 site_nav.py --write`, now clean on
110 pages.
@vahid-ahmadi
vahid-ahmadi force-pushed the fix/job-poll-interval branch from eba5bf3 to 765ccb0 Compare August 26, 2026 11:42
@vahid-ahmadi
vahid-ahmadi merged commit aab0827 into main Aug 26, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant