feat: capture partial page content on timeout for JS-heavy sites

[Oaklight]

Summary

• When page.goto() times out on the final render attempt, catch the timeout and extract whatever HTML has rendered so far via page.content(), instead of returning a 502 error
• Added capture_partial parameter to _do_render — enabled only on last-resort attempts (tier-1 fallback or single-worker mode)
• Tier-0→tier-1 fallback logic is completely unchanged
• Catches both Python's TimeoutError (from asyncio.wait_for) and Playwright's TimeoutError (from page.goto)

Before: JS-heavy sites (Reddit, HuggingFace Spaces, Discourse, Finnhub) → both tiers timeout → 502 with no content (4 of 6 prod failures)

After: Timeout on final attempt → return partial DOM content (200 with status_code: 0 in metadata). Tested with finnhub.io/docs/api/etfs-holdings which previously failed — now returns 1.4MB HTML / 167KB markdown.

Closes #57

Test plan

• ruff check and ruff format pass
• ty check passes
• Normal URLs render fully with correct status_code (200)
• JS-heavy URL with short timeout returns partial content (200, status_code: 0) instead of 502
• Partial response includes title, HTML, markdown, readability, and links

[milo-oaklight]

Good improvement — partial content beats a useless 502 for JS-heavy sites. Code is clean, scoped to the right paths (tier-1 fallback and single-worker), and tier-0→tier-1 fallback is untouched. Approved.

Two non-blocking notes:

1. status_code: 0 is ambiguous — the normal path already sets status_code = 0 when response is None (e.g. about:blank, certain redirects). After this PR, timeout-partial also produces status_code: 0. Callers can't distinguish "no network response but page loaded fully" from "page timed out, content is partial." A dedicated field (partial: true) or a distinct sentinel (e.g. status_code: -1) would make the signal unambiguous.

2. page.content() / page.title() after timeout have no timeout themselves — these are usually fast CDP calls, but if the browser context is unhealthy after a goto timeout, they could hang. Probably fine in practice, but worth a brief comment or a short asyncio.wait_for wrapper as defense.

[elena-oaklight]

Approved ✅

Smart pragmatic fix — returning partial DOM on timeout is strictly better than a 502 with nothing, and the implementation is clean. Tier-0 still raises (preserving the fallback path), only the final-attempt renders capture partial content. The outer asyncio.wait_for in _render_pipeline still guards against page.content() hanging on a half-loaded page.

One substantive suggestion:

status_code: 0 is now overloaded — it previously meant "goto returned no response object" and now also means "goto timed out, captured partial DOM." Callers can't distinguish the two. Consider adding a field to the response metadata to signal partial capture:

# in the timeout branch, set a flag
partial = True

# later, include in the response JSON
"partial": partial,  # or "timed_out": True

This lets callers decide whether to trust the content or show a warning, without breaking existing consumers that ignore unknown fields.

Minor note:

Inside _do_render, page.goto(timeout=t) should only raise Playwright's TimeoutError — Python's built-in TimeoutError would come from asyncio.wait_for, which wraps the outer _render_pipeline(), not page.goto() directly. Catching both is harmless (and defensive if patchright's TimeoutError inherits from the builtin), just worth a comment clarifying the intent.

[clementine-oaklight]

Good change — the key design decision is right: only the last-resort attempt captures partial content, so tier-0→tier-1 fallback stays intact.

One non-blocking note:

status_code: 0 is now overloaded

Before this PR, status_code: 0 meant "navigation completed but response was None." Now it also means "timed out, partial DOM captured." Consumers can't distinguish between the two. A metadata field like "partial": true or a response header (X-Veilrender-Partial: true) would let callers know they're getting a best-effort capture vs. a full render. Not urgent — 0 already signals "something unusual" — but worth considering if callers start caring about the distinction.

Rest looks solid:

• Catching both TimeoutError and PlaywrightTimeoutError is defensive and correct — Patchright's TimeoutError may or may not subclass Python's built-in
• page.content() / page.title() remain valid on the Playwright page object after a goto timeout — the DOM has whatever was loaded
• The outer asyncio.wait_for timeout raises at the call site, not inside _do_render, so the inner catch doesn't accidentally swallow it

[Oaklight]

Thanks for the reviews!

Both non-blocking notes addressed in 3635944:

@milo-oaklight —

1. Added partial: true to response metadata (only present when true, won't break existing consumers). status_code: 0 is no longer ambiguous.
2. Re: page.content()/page.title() hanging — the outer asyncio.wait_for(_render_pipeline(), timeout=settings.request_deadline) already guards this: if the post-timeout CDP calls hang, the 45s deadline kills the whole pipeline. Adding another inner timeout would be redundant.

@elena-oaklight — same partial flag addresses your suggestion. Re: catching both TimeoutError types — agreed it's defensive; Playwright's TimeoutError doesn't inherit from the builtin, so both catches are needed for different code paths (goto vs asyncio.wait_for).