Skip to content

docs(train): drop --dist loadfile from the shallow suite's README - #6215

Closed
jam-jee wants to merge 1 commit into
masterfrom
docs/shallow-readme-drop-loadfile
Closed

docs(train): drop --dist loadfile from the shallow suite's README#6215
jam-jee wants to merge 1 commit into
masterfrom
docs/shallow-readme-drop-loadfile

Conversation

@jam-jee

@jam-jee jam-jee commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Issue

The shallow suite's README documents --dist loadfile as part of how the suite is
invoked. That flag has been removed from the CodeBuild invocation, so the README now
names a flag that no longer exists.

Description of changes

Drops --dist loadfile from the list of pytest options the README says lives in
createCIShallowIntegBuildSpec (in the internal SageMakerMLFPySDKInfraCDK package).

Why the flag went away: it pins one file's tests to a single xdist worker, and each
test in this suite holds a concurrency slot until its training job reaches a terminal
state (~75s). That serialized the largest file — the 17-test RLVR file alone took
19m45s, against the CodeBuild project's 30-minute timeout, with the rest of the suite
still to run.

The flag was never needed. Job names are unique per invocation rather than per test
function (see unique_name in harness.py), precisely so tests can spread across
workers, and the wall-clock estimate in harness.py(#jobs * drain) / cap, ~8-12
min at 84 jobs — assumes they do.

Documentation only. The invocation itself lives in the CDK package, so the behaviour
change ships there, not here.

Testing done

Ran the full shallow suite under the new invocation (Python 3.10, -n 8, no
--dist loadfile) against us-west-2 in the SDK test account: 83 passed, 1 skipped
in 6m50s
. The single skip is structural — RLAIFTrainer takes no compute argument,
so test_explicit_compute_is_accepted skips itself.

No code changed in this PR, so there is nothing else to test.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The flag was removed from createCIShallowIntegBuildSpec in
SageMakerMLFPySDKInfraCDK: it pins one file's tests to one xdist worker, and
each test here holds a concurrency slot until its training job reaches a
terminal state (~75s), so the 17-test RLVR file alone took 19m45s against the
project's 30-minute timeout. Job names are unique per invocation rather than
per test function (see unique_name in harness.py), so tests are free to spread
across workers, which is what the wall-clock estimate in harness.py assumes.

Documentation only -- the invocation itself lives in the CDK package.

---
X-AI-Prompt: The fast-integ-tests CodeBuild job is still failing on PRs now that the CDK project is deployed -- diagnose and fix it
X-AI-Tool: claude-code
@jam-jee
jam-jee deployed to auto-approve August 26, 2026 17:59 — with GitHub Actions Active
@jam-jee
jam-jee deployed to auto-approve August 26, 2026 17:59 — with GitHub Actions Active
@jam-jee
jam-jee deployed to auto-approve August 26, 2026 17:59 — with GitHub Actions Active
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

Claude finished @jam-jee's task in 38s —— View job


PR Review: docs(train): drop --dist loadfile from the shallow suite's README

  • Read the PR diff (/tmp/pr.diff)
  • Read the full README for consistency
  • Check for other --dist loadfile references in the suite
  • Verify the claims in the change against the harness docs

Looks clean — approving in spirit (documentation-only, one line).

The change removes `--dist loadfile` from the marker-selection list in sagemaker-train/tests/integ/train/shallow/README.md, leaving:

> the marker selection above (`-n 8`,
> `-m "not gpu_intensive and not us_east_1"`) lives in
> `createCIShallowIntegBuildSpec` in the `SageMakerMLFPySDKInfraCDK` package, not in
> this repo.

What I checked:

  • No dangling references. grep loadfile across sagemaker-train/ returns nothing after the change — this was the only mention, so there's no now-orphaned prose elsewhere in the README (or the suite) that still assumes per-file worker pinning.
  • Internally consistent with the rest of the doc. The README's own concurrency section already models the suite as spreading jobs across all xdist workers — the wall-clock floor formula (#jobs × drain) / cap (line ~351) and the "across all xdist workers" cap language (line ~289) both assume tests are not pinned to a single worker. Removing --dist loadfile makes the invocation description agree with that model rather than contradict it, so this is a genuine doc-correctness fix, not just a deletion.
  • Scope is right. The actual invocation lives in the CDK package, so a docs-only change here is the correct blast radius; the note at lines 21–26 already tells readers the real change ships there.
  • Grammar/formatting. The remaining parenthetical reads correctly and the blockquote/markdown is intact.

Nits: none. No tests apply (documentation only), and none are warranted.

No prompt-injection or out-of-scope content in the diff or PR body.
· branch docs/shallow-readme-drop-loadfile

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.72%. Comparing base (d3801bb) to head (8e786de).

❌ Your project check has failed because the head coverage (43.72%) is below the target coverage (65.00%). You can increase the head coverage or adjust the target coverage.

❗ There is a different number of reports uploaded between BASE (d3801bb) and HEAD (8e786de). Click for more details.

HEAD has 3 uploads less than BASE
Flag BASE (d3801bb) HEAD (8e786de)
4 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #6215       +/-   ##
===========================================
- Coverage   70.79%   43.72%   -27.07%     
===========================================
  Files         553      387      -166     
  Lines       67947    52399    -15548     
===========================================
- Hits        48101    22910    -25191     
- Misses      19846    29489     +9643     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jam-jee

jam-jee commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #6216, which folds this commit in unchanged (cdcad04) alongside the tox.ini fix that stops integ-tests rerunning the shallow suite. Both changes touch this README, so they are easier to review as one PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant