Code and data for the experiments on @tschmitztschmitz.
Every number that appears in a video comes from a file in this repository. If a video states a figure, the notebook that produced it is here, along with the raw responses it was computed from — including the runs that went against what I expected.
One folder per experiment.
| Folder | Question | Short answer |
|---|---|---|
long-context-rulebook |
Does a 200,000-token policy document still govern an agent? | Mostly yes — and the one dramatic failure I found did not replicate |
Each folder has a notebook you can run yourself and a data/ directory with the
verbatim model responses behind every chart. Re-running costs a few dollars of
API credit and takes minutes, not hours.
If you get a different number than I did, that is worth knowing. Open an issue with your output.
- No figure appears in a video unless it comes out of
data/. - Model responses are quoted verbatim, never tidied up to sound worse.
- When a result contradicts the hypothesis, the video says so.
- A finding that does not survive a re-run does not become a headline.