From a54104e509261e539429c70898247bb3a3ee2593 Mon Sep 17 00:00:00 2001 From: Ned Twigg Date: Wed, 4 Nov 2026 16:55:47 -0800 Subject: [PATCH] Say what chat's rate limit and questions really came to Co-Authored-By: Claude Opus 5.5 --- docs/specs/chat.md | 2 +- docs/specs/chat.rationale.md | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/specs/chat.md b/docs/specs/chat.md index 597f2e1..dafd41a 100644 --- a/docs/specs/chat.md +++ b/docs/specs/chat.md @@ -13,7 +13,7 @@ Source of truth: `ui/src/worker/chat.ts`, `compare` in `ui/src/worker/github.ts` - **Must answer anyone, signed in or not**, since every project here is public (rationale). - **Must keep `pageHeaders` and `sameOriginWrites` on it**, as on any review route (`docs/specs/cloud.md` → Pages). -- **Must take at most 10 questions a minute from one address**, by the `CHAT_LIMIT` binding, and answer 429 `too-many-questions` past that (rationale). +- **Must ask the `CHAT_LIMIT` binding, set to 10 questions a minute from one address, and answer 429 `too-many-questions` when it refuses.** It's a loose limit, and never a cap on spending (rationale). - **Must take at most 40 messages, with at most 4,000 characters in a question and 40,000 in an answer, in at most 1 MB**, and answer 400 `bad-request` otherwise. - **Must offer it only on a hosted story that isn't unchanged.** `selfie ui` has no chat. Any other story is 404 `unknown-story`. diff --git a/docs/specs/chat.rationale.md b/docs/specs/chat.rationale.md index aedc1fb..e466a44 100644 --- a/docs/specs/chat.rationale.md +++ b/docs/specs/chat.rationale.md @@ -6,10 +6,14 @@ **Anyone can ask.** Ned's call on 2026-11-04, for a service that's only up for the hackathon. Everything it reads is already public, so the only thing at risk is spending. Before this stays up longer, or takes a private repository, this should need a person signed in. -**Ten questions a minute.** Without a limit, anyone could use this as a free model, since a page supplies its own conversation. The limit is per address and per Cloudflare location, and it's only approximate, so it slows someone down and doesn't cap the bill. A spend limit in Meta's dashboard is what does that. +**Ten questions a minute.** Without a limit, anyone could use this as a free model, since a page supplies its own conversation. Cloudflare's limiter is per location and eventually consistent, and it's loose in practice. + +**Measured 2026-11-04** on the deployed Worker, from one address, with a story that isn't in the batch, so that nothing reached the model: 14 requests in a burst were all let through, and then only 3 of 45 at about one a second were refused. So someone can ask far more than 10 a minute, and at the costs below, one script could spend several dollars a minute. Only sign-in, or something outside this Worker, would really bound that. + +**What a question costs.** Measured 2026-11-04, from the usage each question logs, at `muse-spark-1.3`'s prices: 15,864 tokens in and 859 out for a 16-file range, $0.02, and 60,476 in and 1,110 out for a 148-file range, $0.08. Reasoning was more than half of what came out. A question repeated on the same range showed no cached tokens. The first text took 11 to 13 s, of which about 4 s is before the model is asked. ## What it may do -**Standard, not contributor.** Contributor models cost about 8 to 20 times less, and they let Meta train on what's sent. The code here is public, but that's the repository owner's to agree to, and at about $0.04 a question there's nothing to save. They're also limited to 150 requests a minute, against 3,000. +**Standard, not contributor.** Contributor models cost about 8 to 20 times less, and they let Meta train on what's sent. The code here is public, but that's the repository owner's to agree to, and at a few cents a question there's little to save. They're also limited to 150 requests a minute, against 3,000. **No tools.** Anyone who can push a branch to a hosted repository can put words in front of the model, in code or in a rendered story. The instructions tell it to ignore them, which isn't a boundary. Having no tools and no links is, since the worst it can then do is say something wrong to the person reading, who's been told it can be wrong and decides for themselves.