Use hosted NCU profiling by default without provider setup - #80
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
popcorn submit submission.py --profilenow submits through the authenticated GPU Mode API and downloads NCU reports. Users need only normal Popcorn registration: no Modal SDK, provider account, credentials, or profiling URL. B200 is the default when no GPU flag or directive is supplied.--profile-brevexplicitly selects the existing Brev service. Errors never switch providers, and--localevaluation remains separate. Both profiling paths accept benchmark selection and kernel-name/launch-count filters. Hosted results include.ncu-rep, text/CSV detail exports, and a manifest with selected benchmark specs, capture options, system information, and a configuration fingerprint. Failed or empty captures return errors.Deployment dependency: merge and deploy gpu-mode/kernelbot#530 (API and GPU runners) before releasing this CLI. It adds the
/profile/{leaderboard}/{gpu}endpoint and pins the working server-side NCU version. The new-problem guide is gpu-mode/reference-kernels#172. No production deployment is included here.Validation:
cargo fmt --check,cargo clippy --all-targets -- -D warnings, and release build passed.problems/linalg/qr_v2, benchmark 0 (batch=20, n=32, cond=1, seed=43214), reference-kernels51e22db671d36c1c76091c43c36a44546ba324a1,regex:geqr2, demangled names, launch count 1. NCU 2025.2.1 collected 39 passes and valid occupancy, throughput, cache, and instruction counters. Six ctc__ metrics were unavailable. Other tasks require NCU-compatible evaluators and their own GPU validation.docs/modal-ncu-validation.mdrecords the integration boundary and the NaN-counter issue observed with NCU 2026.2, which motivated the backend's version pin.