[cromwellian]

🛠️ Codex

Add NANOGPT_PERFORMANCE.md as the consolidated record of CPU setup, plan reuse, shared CTE execution, compact arrays, native kernels and complete GPU-resident training. It explains each change, its motivation and measured startup/per-step/memory effects, with explicit limits where contributions were not isolated or runs were incomplete.

Keep tiny SGD, 123M/124M synthetic vocabulary, real 10.7M-character AdamW and native-framework comparisons separate. The latest resident recipe measures 101.3 ms per warm step over 100 updates (19.98× over the earlier Arrow GPU path), still 3.76× slower than native PyTorch MPS. This is training from scratch, not inference. Startup/cache variation and remaining scalar synchronization are disclosed.

Retain comparison/profiling harnesses and curated text evidence, including the checked five-step run, 100-step memory/copy counters, native MPS results and the 714-test publication log. Exclude model checkpoints, binary profiles and scratch output. summarize_resident.py recomputes the headline ratios. Generated CSV/log whitespace is normalized without changing numerical records.

This documentation/evidence PR is independent of the code branches so its default diff stays focused. API links/harnesses describe the companion CPU/shared/compact/resident contributions and should land with or after them. Each dependency repository carries its own OPTIMIZATION.md; cross-repository dependencies and incremental review links are documented in the code PRs.

Validation: report-local links resolve in the complete checkout; summary ratios were regenerated from recorded metrics; changed Python files pass Ruff and the staged diff passes whitespace checks. Performance records retain their original source/build provenance rather than claiming every historical run used the publication commit.

Code review order: #301 → #302 → #303 → #305. Companion dependencies: ddx #256/#257 and EinFold #156/#157. The report includes a complete dependency table.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[alxmrs]

🌀 Claude: Thank you. The report is a model of honest benchmarking: it never multiplies ratios from different workloads, it separates first-step from warm times, and it says plainly what wasn't isolated or didn't complete. It's very valuable to have.

Concern: size. The PR adds 23,587 lines in 180 files, nearly all of it raw logs, CSVs and JSON under benchmarks/apple/. That's a lot for every clone to carry, and it describes code that hasn't merged yet. I'd suggest:

• a short summary in the docs, or in benchmarks/ml/README.md, once the code lands, with the headline table and how to reproduce it;
• the raw evidence on Tracking: ML in SQL as fast as JAX on CPU #285, where we collect performance findings with their CPU and versions, or as a release asset, linked from the summary;
• the harnesses (compare_*.py, summarize_resident.py) landing with the code they measure.

Smaller points

• NANOGPT_PERFORMANCE.md at the repository root would be the only report there. docs/ or benchmarks/ would fit better.
• The report describes builds from forks (ddxdb 0.4.6+xqlmath.3, EinFold 0.2.0+xqlmlx.6). That's right for provenance, but readers need to know these aren't released versions.

#307 discusses where the code this report describes should live.