You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Star0 (0)You must be signed in to star a repository
About
Open, reproducible benchmark of ways to get a YouTube transcript: success rate, latency and failure modes from different networks. Sample frozen before the first result.
An open, reproducible benchmark of ways to get a YouTube transcript: how often each method works, how fast it is, and how it fails, measured from different networks. The sample was frozen before the first result and everything, including the raw data, is in this repository.
The short version, which the tables below generate from the raw results: where you run matters at least as much as which method you pick. A method that works from a home connection can fail on its first request from a cloud server. Each result states the network it came from.
Kept for transparency. These were made before the harness setup was corrected (see "Changes after the first run" in METHODOLOGY.md), so they are not comparable with the runs above.
Written by hand on 2026-10-18 from the tables above. One day, two networks, 54 videos: patterns, not rankings.
From a residential connection, all three methods mostly work. The hosted API and the library each returned a transcript for 44 of 44 videos with captions. yt-dlp returned 39 of 44: five HTTP 429 errors on subtitle downloads, concentrated in Shorts and long videos with automatic captions. Read the library's 44 of 44 with the caveat below.
From a cloud runner, the free tools were blocked at once. On a GitHub-hosted runner both youtube-transcript-api (RequestBlocked) and yt-dlp (bot check) failed all of their first 5 attempts and stopped, with 49 videos not attempted. The hosted API returned 43 of 44 from the same runner and failed once, with an HTTP 504 after 25 seconds. That failure is ours and is counted.
Speed is similar for the median and not for the tail. Median latency for successful, uncached requests: hosted API 0.9 s from the cloud runner and 1.4 s from India (distance counts), library 1.6 s, yt-dlp 3.8 s (it starts a process each time). The hosted API's slow tail is real: its 95th percentile is 11 to 12 seconds, against about 2 s for the library, and it is not explained by video length (a 42-second video took 12 s). We do not know why from outside and will look at it.
Edge cases agree. Every method reports videos with captions disabled, age-restricted videos and unavailable videos as failures, never as transcripts. The library and yt-dlp name them differently; the hosted API answers all three with the same HTTP 404.
Caveats that change how to read this
The library's score is partly circular. The caption labels used to build the sample came from youtube-transcript-api, from a home connection. A video was only accepted into a "has captions" category if the library could list its tracks. That favours the library and says nothing for or against the other two methods.
One network per type, one day. A different home connection, country or week can give different results. Cloud addresses are not all treated alike either.
Each method ran the way its documentation recommends, after we corrected our own setup three times. The corrections and why are in METHODOLOGY.md. The earlier runs are in the results folder, marked superseded.
How to read these: success is counted over attempted requests on videos that have captions. Requests a method never attempted because its network was blocked are counted separately, not as failures. Latency is for successes only and, for the hosted API, for uncached responses only. 54 videos support broad patterns, not small differences. Read METHODOLOGY.md for the details and the limits of the sample before quoting any number.
Run it yourself
pip install youtube-transcript-api "yt-dlp[default]"# plus Node.js, which yt-dlp uses as its JavaScript runtime
python harness/run.py --env "home network, <country>" --env-slug home --methods ytapi,ytdlp
python harness/analyze.py
The hosted API method runs only when GYT_API_KEY is set. Requests are sequential and gentle; after five blocks in a row a method stops. Send your results file (results/<date>-<env-slug>.jsonl) as a pull request to add your network.
Adding a method
Write an adapter in harness/adapters.py using only what the service documents publicly, register it, and open a pull request. Vendors are welcome to add and correct their own adapter. See CONTRIBUTING.md.
Who maintains this
The team behind GetYouTubeTranscript, one of the three methods tested. We publish unfavourable results like any other, and the methodology says how we limit our own influence. MIT licensed.
About
Open, reproducible benchmark of ways to get a YouTube transcript: success rate, latency and failure modes from different networks. Sample frozen before the first result.
