[kiseki-zj]

📌 Checklist before creating the PR

• I have created an issue for this PR for traceability
• The title follows the standard format
• I have added relevant tags
• Pre-commit checks have passed

🚨 Issue number

No tracking issue linked yet.

📝 What does this PR do?

Migrate the existing layered GPU CI from a developer-specific host
environment to disposable containers using a pinned Torch 2.13 /
CUDA 13.0 dependency image, and retire the legacy PR build entrypoints.

Dependency profile

• Extend the installation constraint from torch>=2.2.0,<=2.5.1
  to torch>=2.2.0,<2.14.
• Add a Linux/Python 3.10 CI image with pinned Torch 2.13.0+cu130
  dependencies and CUDA 13.0 build tools.
• Keep Transformers 4.51.3 and PEFT 0.13.2 in the image profile.
• Replace the Git-based testmon dependency with
  pytest-testmon==2.2.0.
• Extend dependency metadata checks for the updated Torch range.

The installation constraint does not imply validation of every
Torch/CUDA combination. This PR does not upgrade Transformers to 5.x.

Container execution

• Reuse an immutable image preloaded on the runner's Docker daemon;
  no registry publication or automatic image pull is required.
• Install the exact source commit under test in each disposable
  container. The dependency image contains no ColossalAI checkout
  or prebuilt ColossalAI extensions.
• Retain authorized GPU-pool selection, idle-device checks and the
  shared host lock.
• Expose only allocated GPU UUIDs and apply CPU/memory limits.
• Do not mount host virtual environments, credentials or the
  Docker socket into test containers.
• Validate image recipe/lock consistency, upload test reports and
  runtime metadata, and clean up only the current job's containers.
• Validate AOT extension compilation with BUILD_EXT=1 in batch 3;
  other batches compile required kernels on demand.

Workflow routing and retirement

• Keep Layer 1 CPU/lint checks.
• Trusted same-repository Draft PRs run container smoke batch 4a
  using the separate COLOSSAL_CI_SMOKE_GPU_POOL.
• Trusted same-repository Ready PRs and manual Layer 2 runs execute
  batches 2 4a 5.
• Use the container executor for Layer 3 and idle-monitor GPU runs.
• Remove:
  • e1-gpu-on-pr.yml
  • build_on_pr.yml
  • report_test_coverage.yml
• Preserve the runner registration/label, infrastructure bootstrap,
  and unrelated example, compatibility, scheduled-build and release
  workflows.
• Document provisioning, rollout, retirement and rollback.

The retired coverage-comment workflow is not replaced by code-coverage
measurement: JUnit test reports are not coverage reports.

Regression coverage

• Add FP8 forward/backward tests for tensor/tuple _scaled_mm
  return contracts, FP16/BF16, bias and batched inputs.
• Fix optional Apex capability detection in Gemini optimizer and
  checkpoint tests.
• Add regression tests for container isolation/cleanup, Draft/Ready
  routing, GPU-pool configuration and retired workflow references.

The FP8 implementation compatibility changes already exist in main;
this PR adds regression coverage around that behavior.

✅ Validation

Current head: 9278d34b

• Layer 1 formatting and CPU tests passed.
• Fork-safe checks and dependency compatibility checks passed.
• All 66 CI-helper tests passed on the Linux runner.
• PR-triggered Layer 2 passed:
  • Batch 2: 25 passed.
  • Batch 4a: 22 passed.
  • Batch 5: 5 passed.
• Container cleanup and artifact upload completed successfully.

Layer 2 uses fast_mode=true; batch 4a is unaffected by FAST_TEST.

Results:

• Layer 1
• Fork-safe checks
• Layer 2
• Dependency checks

Partial Layer 3 validation: 31e3c352

Run with fast_mode=false:

• Batches 2 3 4a 4b 4c 4e 5 completed successfully,
  with some optional cases skipped.
• Batch 3 AOT compilation passed.
• Pytest reports contain 1,962 passed and 359 skipped cases.
• Batches 6 7 8 9 9q failed before test execution because only
  two idle GPUs were available when four were required.
• Batches 4d and 11 were entirely skipped.
• Eight-GPU batch 10 was not selected.

This run does not establish full Layer 3 coverage. Four-GPU batches
still require validation with sufficient authorized resources.

Layer 3 report

⚠️ Scope and limitations

• Optional Apex, FlashAttention, vLLM, TensorNVMe and SmoothQuant
  backends are not included in this image; skipped paths remain
  unvalidated.
• FP8 on Torch 2.13 uses the existing eager fallback; this PR does
  not establish torch.compile support.
• The pinned TorchAudio 2.11 wheel has only limited import and
  CPU-operation validation with this profile.
• Testmon is installed but incremental selection is not enabled.
• Containers do not make untrusted PR code safe for persistent
  self-hosted runners; trusted-ref restrictions remain necessary.
• Removing workflow files does not cancel existing runs or update
  required-check settings.
• This PR does not modify version.txt, publish a PyPI package,
  or publish a user-facing Docker image.

💥 Checklist before requesting a review

• I have linked my PR to an issue
• I have performed a self-review of my code
• I have added relevant regression tests
• Formatting, CPU checks and Layer 2 have passed
• Outstanding multi-GPU coverage has been validated or an
  explicit maintainer-approved rollout exception documented

[richardoo-707]

Approved to proceed with maintainer CR. Layer 1 and Layer 2 checks are green. The documented container Layer 3 four-/eight-GPU validation gaps and skipped optional backends remain outstanding; this approval does not certify complete GPU coverage.