Skip to content

fix(catalog): let the CDN cache anonymous catalog pages#656

Closed
tiankaima wants to merge 2 commits into
mainfrom
fix/catalog-anonymous-cdn-cache
Closed

fix(catalog): let the CDN cache anonymous catalog pages#656
tiankaima wants to merge 2 commits into
mainfrom
fix/catalog-anonymous-cdn-cache

Conversation

@tiankaima

@tiankaima tiankaima commented Jul 25, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #654.

Anonymous GETs for public catalog pages (/catalog, /catalog/*) previously fell through to the blanket Cache-Control: no-store in hooks.server.ts, so every request — including the heavy AI-crawler traffic from #652 — ran the full Worker pipeline (hooks, Postgres queries, SSR) with zero edge caching.

Change

src/hooks.server.ts + src/lib/public-cache-control.ts:

  • Anonymous GET requests to /catalog / /catalog/* that don't already set Cache-Control now get the new PUBLIC_LOCALE_CATALOG_PAGE_HEADERS:
    • Cache-Control: public, max-age=0 — browsers must revalidate every navigation (no stale-while-revalidate; see below)
    • Cloudflare-CDN-Cache-Control: public, max-age=60, stale-while-revalidate=300 — the edge keeps a short TTL + SWR window
    • Vary: Accept-Language, Cookie
  • Catalog requests carrying an auth signal (better-auth/session cookie or bearer header, via the existing hasRequestAuthSignal check) get PRIVATE_LOCALE_CATALOG_HEADERS instead: private, no-store + Cloudflare-CDN-Cache-Control: no-store, so personalized renders can never be cached.
  • Non-catalog routes keep the no-store default; routes that set their own Cache-Control (robots.txt, sitemap.xml, API routes) are untouched.
  • Scope is deliberately limited to HTML page responses; JSON/__data requests are unaffected.

Why no stale-while-revalidate for the browser

The first version of this PR sent public, max-age=0, stale-while-revalidate=300 to browsers and failed the catalog E2E shards deterministically: Chromium may serve an SWR-cached document without revalidating, so flows that mutate state and expect a fresh SSR on the next navigation (locale switch, login, subscribe/unsubscribe, preference autosave) acted on stale pages. SWR now lives only on Cloudflare-CDN-Cache-Control, where it is safe.

Deploy note (infra step, not in this PR)

Cloudflare does not cache extensionless HTML by default, so the Cloudflare-CDN-Cache-Control header only takes effect once a zone Cache Rule makes these paths eligible for cache, e.g.:

  • Expression: http.host eq "life-ustc.tiankaima.dev" and starts_with(http.request.uri.path, "/catalog") and not http.cookie contains "better-auth." and not http.cookie contains "session"
  • Action: eligible for cache, Edge TTL = respect origin

The cookie guard in the rule mirrors the origin-side auth check, so logged-in users can never be served cached anonymous HTML. Until the rule exists this PR is a no-op at the CDN (headers only).

Tests

  • New tests/unit/catalog-cache-control.test.ts: anonymous catalog → public CDN headers; authenticated catalog → private/no-store; non-catalog → no-store; explicit route Cache-Control preserved.
  • Full unit suite, biome check, tsc (typecheck + tests configs), and catalog E2E shards — see CI.

Expected impact

Crawler/bot traffic on catalog pages becomes mostly edge cache hits (~0 ms Worker CPU) instead of full SSR renders.

Anonymous catalog HTML fell through to the blanket no-store default, so
every crawler hit paid for a full SSR render. Emit the existing
PUBLIC/PRIVATE_LOCALE_CATALOG_HEADERS based on the request auth signal
instead: anonymous GETs become briefly CDN-cacheable while authenticated
renders stay private and uncacheable.

Closes #654
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 25, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
life-ustc f412c8d Commit Preview URL

Branch Preview URL
Jul 25 2026, 06:46 PM

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

Browsers may serve an SWR-cached document without revalidating, which
breaks flows that mutate state and expect a fresh render on the next
navigation (locale switch, login, subscribe) and failed the catalog E2E
suites. Keep SWR only on the CDN-facing header.
@tiankaima

Copy link
Copy Markdown
Member Author

Closing as unsafe to merge and superseded by #657. Production-style probes against this branch returned CDN HIT responses for anonymous HTML even when the follow-up request carried Cookie or Bearer credentials. The cached response also reused locale-dependent HTML, the CSP nonce, and x-request-id, while the cache key did not represent those request differences. That creates auth-boundary, privacy, and response-integrity risks. Safe shared-page caching should stay behind the gateway architecture tracked in #507 and #573; #657 removes the public preview ingress responsible for the dominant Worker CPU load without caching user-facing HTML.

@tiankaima tiankaima closed this Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anonymous catalog pages are never cached at the Cloudflare edge

1 participant