Skip to content
This repository was archived by the owner on Aug 26, 2026. It is now read-only.

fix(flaresolverr): schedule on the always-on amd64 node and raise the CPU cap - #759

Merged
ExtraToast merged 1 commit into
mainfrom
fix/flaresolverr-schedule-on-t1000
Aug 16, 2026
Merged

fix(flaresolverr): schedule on the always-on amd64 node and raise the CPU cap#759
ExtraToast merged 1 commit into
mainfrom
fix/flaresolverr-schedule-on-t1000

Conversation

@ExtraToast

Copy link
Copy Markdown
Owner

Symptom

Prowlarr reported the FlareSolverr indexer proxy as unreachable:

Unable to connect to proxy: Http request timed out. Check the log surrounding this error for details

The service looked healthy from every angle that gets checked first: the pod was 1/1 Running, and /health answered {"status": "ok"} in milliseconds, so Gatus stayed green and there were no restarts or probe failures to notice.

Root cause

The Deployment pinned only to personal-stack/site: enschede, which matches all six Enschede nodes, and the pod had landed on enschede-pi-1 — an arm64 Raspberry Pi.

Solving a Cloudflare Turnstile challenge runs headless Chrome. On the Pi that does not finish inside Prowlarr's 60 s proxy timeout. The container log shows the split cleanly: requests where no challenge appears return 200 in 12-41 s, while every request that actually has to solve one runs 70-85 s and fails:

2026-08-16 16:00:23 INFO  Incoming request => POST /v1 body: {'maxTimeout': 60000, 'cmd': 'request.get', 'url': 'https://prowlarr.servarr.com/v1/ping', ...}
2026-08-16 16:01:37 ERROR Error: Error solving the challenge. Timeout after 60.0 seconds.
2026-08-16 16:01:37 INFO  Response in 74.044 s
2026-08-16 16:01:37 INFO  POST .../v1 500 Internal Server Error

Only /v1 was ever affected, which is why /health never reflected the fault.

The single-core CPU limit compounded it. Chrome saturated the cap during a solve, and the container's cgroup recorded nr_throttled 17928 with throttled_usec 1101692394 — roughly 1100 s of throttling.

Change

The pod now pins to enschede-t1000-1 (amd64, 12 cores, currently 14 % CPU).

It stays in Enschede deliberately: a residential IP draws far fewer Cloudflare challenges than a datacenter address, so moving the solver to Frankfurt would increase the number of challenges it has to solve rather than reduce them. Among the amd64 Enschede nodes, enschede-gtx-960m-1 and enschede-rx7900xtx-1 are desktops that power off when not in use, leaving the always-on t1000 as the only stable target.

The CPU limit goes from 1000m to 3000m so a solve can burst. The request stays at 500m, since the process idles between indexer queries and a larger request would consume scheduling budget it does not need.

Validation

  • kubectl kustomize platform/cluster/flux/apps/stateless renders the intended nodeSelector and resources
  • cd platform/tests && node --test — pass

Solve latency will be re-checked against the live Prowlarr indexer test after this reconciles.

… CPU cap

The Deployment pinned only to personal-stack/site: enschede, so the pod
landed on enschede-pi-1. Solving a Cloudflare Turnstile challenge runs
headless Chrome, which the arm64 Pis cannot finish inside Prowlarr's 60 s
proxy timeout: challenge-solving requests took 70-85 s and returned 500
"Error solving the challenge. Timeout after 60.0 seconds.", surfacing in
Prowlarr as "Unable to connect to proxy: Http request timed out".

/health kept answering 200 in milliseconds throughout, so the container
stayed Ready and the failure never appeared as a restart or a probe
error.

The pod now pins to enschede-t1000-1. It stays in Enschede because a
residential IP draws far fewer challenges than a datacenter address; the
other amd64 Enschede nodes are desktops that power off when not in use,
so the always-on t1000 is the only stable amd64 target.

The CPU limit goes from 1000m to 3000m. Chrome saturated the single-core
cap during a solve, accumulating ~1100 s of cgroup throttling. The
request stays at 500m because the process idles between indexer queries.
@ExtraToast ExtraToast added the bug Something isn't working label Aug 16, 2026
@ExtraToast ExtraToast self-assigned this Aug 16, 2026
@ExtraToast
ExtraToast merged commit 13877ac into main Aug 16, 2026
22 checks passed
@ExtraToast
ExtraToast deleted the fix/flaresolverr-schedule-on-t1000 branch August 16, 2026 19:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant