Skip to content

[WIP] Add initial Cambricon MLU provider under backends/mlu - #3

Draft
Nebulixeay wants to merge 4 commits into
InfiniTensor:masterfrom
Nebulixeay:feat/mlu-provider-initial
Draft

Nebulixeay wants to merge 4 commits into
InfiniTensor:masterfrom
Nebulixeay:feat/mlu-provider-initial

Conversation

@Nebulixeay

Copy link
Copy Markdown

This PR adds an initial Cambricon MLU provider under backends/mlu, using InfiniTrain's PrivateUse1 interface and the repository's provider build entry point. It includes runtime integration, BANG C kernels, a GPT-2 example adapter and small operator regression tools. The root dispatcher, MACA provider and pinned framework revision are unchanged.

Work in progress: full GPT-2 single-card training for 10 steps and the requested end-to-end precision targets have not been verified. GEMM still uses a CPU fallback; BF16 training and multi-card support remain unfinished.

Build and validation

  • Validated on MLU590 with GCC 13.4.0, cncc 5.7.6, CMake 4.4.5 and Ninja 1.13.2.
  • Fresh configuration and builds of mlu_gpt2 and mlu_op_test succeeded against the pinned InfiniTrain v0.7.0 content (c90338c); all nine BANG C sources compiled.
  • The 35 small-shape operator comparisons, host comparator self-checks and device-math tests passed. Finite/NaN injection tests returned the expected failure code; the controlled fatal test aborted with the expected diagnostic. Linear tests validate the host fallback.
  • gpt2 --help successfully printed usage. No model training step was run during this validation.
  • The remote staging tree used an archive of the fixed framework tree and shallow retrieval of the four dependencies at their pinned commits. Review compared 2,687 tracked files with the corresponding Git objects: content matched after CRLF/LF normalization. This validates the staged build content, not the standard recursive-submodule download procedure.
  • No MACA or CUDA build regression was performed. Leaving those provider files unchanged is not a claim of cross-platform regression coverage.

Current limitations

  • Device GEMM integration is pending. Optional CNNL handle creation/queue binding is available, but current operators do not compute through CNNL.
  • Several kernels use serial execution; large training shapes and full-model backward remain unverified.
  • BF16 Embedding accumulation still needs correction; no BF16 accuracy result is claimed.
  • CNCL/multi-card support and shared InfiniTrain MLU test-suite integration are pending. BUILD_TEST=ON is explicitly rejected; use the included MLU regression tools.
  • Strict end-to-end loss acceptance tooling is deferred to a separate change.

Model/data assets, build products, internal review scripts and development logs are excluded from this PR. Details and commands are in backends/mlu/README.md and backends/mlu/docs/validation.md.

Register a PrivateUse1 backend named "mlu" through the provider layout that
the repository already uses for external backends: backends/mlu supplies the
runtime, the BANG C kernels, the host kernel integration, the GPT-2 example and
its own pre-project SDK checks, so no InfiniTrain source file is modified.

The backend keeps the existing semantics: FP32 accumulation inside kernels,
POD metadata passed by value, deterministic gather for broadcast backward and a
host GEMM fallback for linear/matmul (NOTE(MLU-FALLBACK)). Device GEMM, BF16,
multi-card collectives and full GPT-2 training are not implemented here; see
backends/mlu/docs/validation.md.

The pinned framework revision does not declare DeviceGuardImpl::ExchangeStream,
so the MLU guard no longer overrides it; the previous implementation had no
caller.
mlu_op_test compares the MLU kernels against the CPU backend (35 small-shape
FP32 cases, pow domain handling, controlled fatal path and probe cases) and
exits non-zero on any mismatch. tests/math_selftest keeps its standalone
cncc + g++ build for the device math functions.

Both are opt-in for the provider: INFINITRAIN_MLU_BUILD_TEST_TOOLS=ON builds the
operator tool, and BUILD_TEST=ON is rejected on purpose so CPU suites cannot
stand in for MLU validation.
VERBATIM escapes the inner quotes of -I"...", so cncc received an include
path carrying literal quotation marks and could not find
kernels/mlu_kernel_api.h. The whole argument is quoted instead, which still
keeps a path containing spaces intact.
List the 2026-11-03 fresh build and small-test results for the provider
layout, keep the pre-migration observations as historical, and state the open
items explicitly: the standard recursive-submodule download was not exercised,
no MACA/CUDA regression was run, and full GPT-2 10-step precision is still
unverified. Restore the original README.zh-CN section heading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant