Repository navigation

Train a random-init six-layer BERT-style MLM with native TorchTPU - #178

Draft

charlesmartin14 wants to merge 3 commits into

mainfrom

codex/distilbert-scratch-torchtpu

Draft

Train a random-init six-layer BERT-style MLM with native TorchTPU#178
charlesmartin14 wants to merge 3 commits into
mainfrom
codex/distilbert-scratch-torchtpu

Commits on Oct 20, 2026
- Add isolated random-init DistilBERT MLM pilot on native TorchTPU
  charlesmartin14
  committed
  896fa64
Commits on Oct 21, 2026
- Audit BERT-6 pretraining and make validation-based calibration reproducible
  charlesmartin14
  committed
  c251149
- Fix TorchTPU RNG compatibility and interrupted-run recovery
  charlesmartin14
  committed
  a6ae88b