Skip to content

Fix GPU single-precision DAV convergence, stress accuracy and meta-GGA XC contributions - #8204

Open
dyzheng wants to merge 29 commits into
deepmodeling:developfrom
dyzheng:develop-single-precision-fixes
Open

dyzheng wants to merge 29 commits into
deepmodeling:developfrom
dyzheng:develop-single-precision-fixes

Conversation

@dyzheng

@dyzheng dyzheng commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Fix GPU single-precision DAV convergence, stress accuracy and meta-GGA XC contributions

Summary

Fixes #8127

This PR fixes single-precision correctness and stability issues in PW DFT:

  • GPU-single Davidson restart, orthogonalization and reduced-solve failures
  • single-precision stress and density accumulation errors
  • single-precision non-collinear (nspin=4) force/stress crashes
  • missing meta-GGA XC precision handling in the shared CPU/GPU path
  • Pyabacus build breakage from the new DiagoDavid interface

Wavefunctions and FFTs remain single precision. Selected reductions and the
existing reduced eigensolve are promoted to double precision. No scientific
INPUT semantics or convergence criteria are changed.

Revision

  • Base: deepmodeling/develop = 370ba685e6707b3625c5580ff9d02ef0afb248e8
  • Head: 9d41dad98314f9fcd8739246b5150d958d7bc831
  • Diff: 34 files, +739 / -139
  • Local CPU test binary SHA256:
    beaf19006cd800989109a763a8f889a476bc67a04487a45805325ce828d33ccc
  • Local CUDA single-precision test binary SHA256:
    5556da1f72ee5a67183ea67acc9a2766f6b8f5d44722bc512720a68f10c16598

Changes

Davidson solver

  • Reset rejected/degenerate residual directions safely.
    • SchmidtOrth returns whether a direction was accepted.
    • Rejected columns are removed and the accepted columns are compacted
      before cal_elem/H/S are applied, so zero columns no longer enlarge the
      reduced basis or introduce spurious zero Ritz values.
    • The initial basis aborts with a clear diagnostic if the trial
      wavefunctions are linearly dependent.
  • CUDA float QR path:
    • Check the R diagonal for finite, full-rank blocks.
    • If the projected block is rank deficient, restore the unprojected block
      and fall back to Schmidt orthogonalization.
  • Diago_DavSubspace:
    • Compact individual accepted directions instead of only handling an
      all-zero block.
    • Reject non-finite norms explicitly, avoiding singular generalized
      eigenproblems in mixed warm starts.
  • Use Parallel_Common::bcast_data for the Davidson broadcasts.
  • Pass use_uspp explicitly through the DiagoDavid constructor; remove the
    new PARAM.globalv dependency and module_parameter/parameter.h include.

Single-precision accuracy

  • Accumulate PW stress contributions in double precision:
    • NLCC isotropic stress
    • kinetic stress
    • mGGA stress
    • Ewald/local/nonlocal stress and dylmr2 finite differences
  • Accumulate GPU-single k-point rho/tau in double host buffers.
  • Block long single-precision CGEMM A^H B reductions (k > 2048), with
    partial final blocks.
  • Solve GPU-single DAV reduced matrices in double precision and cast the
    requested eigenpairs back to single precision.
  • Build CUDA float Davidson bases with a single-pass QR where valid.

Single-precision nspin=4

  • Keep double deeq_nc/qq_so buffers valid for the double force/stress
    driver:
    • CPU points the double buffers at the host arrays.
    • GPU single allocates and synchronizes device double buffers.
  • Use FPTYPE* for the DeltaSpin stress lambda in kernel declarations.
  • Do not instantiate float GPU onsite projectors, which are still
    double-typed in this tree.

XC

  • grad_wfc uses precision-matched gcar/kvec_c buffers instead of
    reading a double tensor as float.
  • Added a grad_wfc overload accepting precomputed Real buffers. The mGGA
    stress path now prepares the gcar/kvec buffers once per stress evaluation
    and reuses them across bands.

Pyabacus

  • Update the DiagoDavid adapter to pass the new explicit use_uspp
    argument (false for the LCAO/HSolver adapter).

Tests

Builds

Configuration Result
CPU, LCAO + ELPA, LibXC, ENABLE_FLOAT_FFTW=ON pass
CUDA, LCAO + ELPA, LibXC, ENABLE_FLOAT_FFTW=ON pass
Pyabacus wheel build (pip install .[test]) pass

Unit tests

Passed:

  • MODULE_HSOLVER_dav
  • MODULE_HSOLVER_dav_float
  • MODULE_HSOLVER_LCAO
  • MODULE_HSOLVER_empty_blocks (new exact/approximate warm-start test for
    DiagoDavid and Diago_DavSubspace)
  • MODULE_BASE_KERNELS_Unittests (CPU)
  • CUDA TestModuleHsolverMathKernel.gemm_op_cgemm_blocked_gpu
    (k = 2050 and k = 4101, short final block, padded leading dimensions,
    nonzero beta, double reference)

Command examples:

cmake -B build_dev -DENABLE_LCAO=ON -DENABLE_ELPA=ON -DENABLE_LIBXC=ON \
  -DENABLE_FLOAT_FFTW=ON -DBUILD_TESTING=ON
cmake --build build_dev -j
ctest --test-dir build_dev -R "MODULE_HSOLVER|MODULE_BASE_KERNELS_Unittests"
cmake -B build_cuda -DUSE_CUDA=ON -DENABLE_LCAO=ON -DENABLE_LIBXC=ON \
  -DENABLE_FLOAT_FFTW=ON -DBUILD_TESTING=ON
cmake --build build_cuda --target MODULE_BASE_KERNELS_Unittests -j
cd build_cuda/source/source_base/kernels/test
./MODULE_BASE_KERNELS_Unittests \
  --gtest_filter=TestModuleHsolverMathKernel.gemm_op_cgemm_blocked_gpu

Pyabacus

cd python/pyabacus
python -m pip install .[test]
cd tests
pytest -q
# 64 passed, 5 skipped

Single-precision non-collinear (nspin=4)

Custom PW cases derived from 140_PW_15_SO, SOC and non-SOC, with
cal_force=1, cal_stress=1:

  • CPU single: cg and dav_subspace complete SCF + force + stress
  • CUDA single: cg and dav_subspace complete SCF + force + stress
  • dav_subspace: CPU single vs CPU double energy difference = 0.02 meV
    (-1678.3522966 vs -1678.3523159 eV); GPU single agrees
  • cg: single precision converges to a different magnetic local minimum
    (~13 meV from double cg); this is a CG convergence characteristic, not
    an arithmetic error. Use dav_subspace when tight single-precision
    non-collinear energies are required.

Single-precision PW vs double CPU

Functional Energy error Max force error Max stress error
PBE 0.147 meV/atom 0.006 eV/A 0.24 kbar
SCAN 0.135 meV/atom 0.0097 eV/A 0.59 kbar

PBE energy/stress improve relative to the previous LTS-style single-precision
accumulation. SCAN energy improves; the maximum stress deviation is about
0.005% of the ~12,500 kbar stress.

Not included / known limitations

  • The earlier Davidson vcc reset is not part of this revision.
  • The meta-GGA zero-gradient XC correction is not part of this revision.
  • Main-precision force/stress (Forces<Real> / Stress_PW<Real>) is not
    changed. Some float stress branches therefore remain dormant under the
    current double force/stress driver.
  • Single-precision SCAN force/stress still hits the pre-existing
    double-driver/single-basis mismatch; it fails on the unmodified base as
    well and is not fixed here.
  • GPU Cgemm unit tests require CUDA; ROCm was not tested.
  • Large-system performance was not benchmarked in this PR.

@dyzheng dyzheng added Bugs Bugs that only solvable with sufficient knowledge of DFT Performance Issues related to fail running ABACUS GPU & DCU & HPC GPU and DCU and HPC related any issues labels Oct 23, 2026
@dyzheng
dyzheng force-pushed the develop-single-precision-fixes branch from 693b7b7 to de17ec2 Compare October 23, 2026 03:58
Comment thread source/source_hsolver/diago_david.cpp Outdated
@mohanchen

Copy link
Copy Markdown
Collaborator

Issues to address before merge

  1. Branch hygiene : The merge-base is stale and the diff contains 3 already-upstream commits by other authors (CMake: Prepare test scripts and data at configure time #8161, Precommit: Prettify source_main and source_psi #8191, Refactor DFT+U onsite Hamiltonian assembly #8195), contributing ~1000 lines of unrelated changes (source_psi refactor, CMake test scripts). Please rebase/merge onto the latestdevelop so the diff shows only this PR's own work.
  2. Governance rule violations :diago_david.cpp newly includesmodule_parameter/parameter.h and branches onPARAM.globalv.use_uspp . This increases the globalPARAM dependency budget (rule 1) and hides a workflow switch.use_uspp should be passed explicitly through the constructor or a setter instead.
  3. Duplicated#if defined(__CUDA) blocks : Theblock_qr predicate and branch appear twice inside the generic template function. Consider extracting the condition into a named helper and keeping the CUDA-specific path isolated; the current form hurts readability for CPU/ROCm builds.
  4. Mixed scope : Compilation fixes, crash fixes, precision policy, and an algorithm replacement (QR vs Schmidt) are bundled together. Splitting into at least two PRs ("crash/compile fixes" and "single-precision accuracy policy") would improve reviewability and rollback granularity.
  5. Silent skip : Swallowingpsi_norm <= 0 silently may also mask genuine numerical failures. Emit at least a one-time warning log when a direction is skipped.
    Conclusion : The technical content is solid and most changes are correct, necessary bug fixes; the mixed-precision policy is physically reasonable. But please (a) rebase to drop unrelated commits, (b) eliminate the newPARAM.globalv dependency via explicit parameter passing, and (c) ideally split the PR by topic before merging.

dyzheng and others added 17 commits October 23, 2026 14:40
grad_wfc mapped gcar/kvec_c as DT_DOUBLE but accessed them with
data<Real>() (float in single precision), which triggers the Tensor data
type check and aborts single-precision meta-GGA stress. Build
Real-precision host copies from the PW_Basis_K vectors and upload them,
consistent with the SCF meta-GGA operator. Also instantiate
grad_wfc<complex<float>, GPU, float>.

Validation: precision single + mGGA stress, CPU/GPU.
Use a double scalar for the NLCC diagonal stress reduction to avoid
single-precision accumulation error in the isotropic contribution.

Previously compiled and validated on POSCAR 0224272 against CPU double;
pressure error fell from approximately 0.403 to 0.00099 kbar, with
essentially unchanged energy, forces and observed runtime. Additional
11 MPI x 1 OMP checks on POSCAR 0743323 and 0251308 reduced stress and
pressure errors while preserving the original scientific inputs.

This commit records the tested one-line source change only.
Keep the nine kinetic-stress accumulators in double precision to avoid
rounding loss when summing contributions over bands and k-points.
Wavefunctions and per-band GPU dot products retain their chosen precision.

Validated POSCAR 0159503 with DAV, GPU 1 MPI x 1 OMP and unchanged INPUT,
using the converged CPU-double 11 MPI x 1 OMP result as the reference.
Maximum stress error decreases from 1.31174 to 0.14036 kbar and pressure
error from 0.37485 to 0.13039 kbar. The SCF energy/density history and final
energy are identical. Both runs converge in 24 steps; wall time is
522.92 versus 518.23 seconds, with no observed performance regression.

Rebuilt all three consumers of the changed header before runtime validation.
Accumulate per-k-point rho and kinetic density into the existing double
host fields for GPU single precision. Keep wavefunctions, FFTs and each
k-point density in single precision, reuse one host buffer and clear the
device density workspace after each k point.

Validated structure 0317955 with unchanged INPUT and solver stopping
conditions, GPU 1 MPI x 1 OMP, 18 converged SCF steps and all 100 k points
free of occupied zero states. Reuse the converged CPU-double reference.
Maximum stress error decreases from 0.56409 to 0.10767 kbar and pressure
error from 0.52878 to 0.09912 kbar. Energy and forces remain comparable.
Wall time is effectively unchanged (483.15 to 483.49 s), about 12.1x the
recorded CPU-double speed. Saved-density electron-count error improves
about 72x; rho/tau L2 differences against auxiliary GPU double improve
about 6.7x/20.9x.
Split complex<float> GPU CGEMM A^H B reductions longer than 2048 into
blocks, preserving alpha and the original beta on the first block.
Subsequent blocks accumulate with beta=1. Other paths retain their
existing call and single-precision storage is unchanged.

Validated with a complete 17-step SCF run of 1762323 on L20 using
1 MPI x 1 OMP, dav and unchanged INPUT/stopping criteria. No occupied
anomalous zero states were found. Reused the converged CPU-double
11 MPI reference.

Against CPU double, energy error falls from 3.0136 to 0.0376 meV/atom
and maximum stress error from 0.02996 to 0.01653 kbar. Force improves
slightly and pressure error stays approximately unchanged. Wall time
rises from 315.3 to 345.9 s (+9.7%), retaining 23.3x CPU-double speed.
Solve non-USPP GPU-float reduced matrices once in double precision and
return the requested eigenpairs in single precision. Large wavefunction
operations and stopping thresholds retain their existing precision policy.

Replace the float-prefix Jacobi policy while retaining the preceding six
validated repairs. The previous dav tip is retained on a protection branch.

Validated on L20 with GPU 1 MPI x 1 OMP, unchanged INPUT and reused CPU
double references (11 MPI x 1 OMP):
- 0056851: recover convergence in 68 SCF steps, 1140.5 s, ~14.0x CPU.
- 0186922: 73 -> 33 steps, 1080.2 -> 566.3 s, ~16.7x CPU.
- 1729347: 22 -> 21 steps, 262.9 -> 279.2 s, ~8.9x CPU.

CPU-relative energy errors are 0.04733, 0.07405 and 0.09601 meV/atom,
respectively. Force, stress and pressure were independently checked against
native CPU output. This is a local precision-policy correction; it does not
establish a defect in the vendor's single-precision algorithm.

Measured binary SHA256:
7aec892171fb7e401b572d17ae372c6922461c279cf54ac6fc1eb51ecc7211f1
Replace the initial and expansion Schmidt normalization paths with the
existing float Householder QR kernel for norm-conserving CUDA float
calculations with one rank per diagonalization pool.

A frozen 0058587 projection exposed a 3.53e-5 relative error in the
subtracted norm estimate versus 3.15e-8 in the direct float norm. Native
float replay and complete GPU 1-MPI tests validate the replacement without
an added orthogonalization pass or double reduced eigensolve.

0058587 converges in 39 SCF / 348.6 s, 9.6% less time than the prior converged
extra-orthogonalization version. CPU-relative energy/force/stress/pressure
errors are 0.05674 meV/atom, 0.004476 eV/A, 0.01737 kbar, 0.00894 kbar.
0030651 converges in 13 SCF / 295.8 s, 28.2% less than the retained-five-fix
baseline; energy/stress/pressure errors rise slightly to 0.07688 meV/atom,
0.04274 kbar, 0.01662 kbar. Neither failed nor incomplete runs are used
as timing denominators. INPUT, internal stopping and compiler flags stay
fixed. CPU 11-MPI references are reused; actual GPU tests use 1 MPI x 1 OMP.

CUDA preprocessor guards preserve the existing CPU/ROCm code paths. A
recompile of this guarded source produces the byte-identical executable
used in both completed tests. Incremental build, not a clean full build.
Multi-pool/ROCm runtime and 1729347 coverage are not established here.
The accel branch supports `precision single` on GPU, but several
explicit `<float, base_device::DEVICE_GPU>` instantiations were missing
and the DeltaSpin device kernels used `double*` for the lambda array,
so a GPU build failed to compile/link.

- stress_op.{cu,hip.cu}: use FPTYPE* for the DeltaSpin lambda argument
- cal_mw_from_lambda.cpp: diagonalize into a temporary Real buffer and
  copy back to the double-precision pelec->ekb (both GPU paths)
- onsite_proj.cpp / onsite_proj_tools.cpp: instantiate the float GPU
  OnsiteProjector / Onsite_Proj_tools
- force_op.{cu,hip.cu}: instantiate saveVkbValues<float> and
  revertVkbValues<float>
- xc_grad.cpp: instantiate grad_wfc<complex<float>, GPU, float>
The single-precision residual H*c - lambda*S*c suffers catastrophic
cancellation and collapses to numerical noise with a zero/negative norm.
DiagoDavid::SchmidtOrth and DavSubspace::normalize_op (CPU/CUDA/ROCm)
asserted psi_norm > 0.0, aborting single-precision non-collinear (nspin=4)
calculations. Treat such directions as converged and skip normalization
instead of aborting.
The subspace eigenvectors/eigenvalues broadcast in diag_zhegvx used
hardcoded MPI_DOUBLE_COMPLEX / MPI_DOUBLE, which for single-precision
(complex<float>/float) read and write past the buffer (16 vs 8 bytes per
element) and corrupt the eigenvectors. Use Parallel_Reduce::MPI_Type<T>
/ MPI_Type<Real> so float uses MPI_C_FLOAT_COMPLEX / MPI_FLOAT.
In single precision (has_float_data only), z_deeq_nc and z_qq_so were left as
nullptr, but the force/stress driver (Forces<double> / Stress_PW<double>)
always reads the double-precision deeq_nc via get_deeq_nc_data<double>().
Point them at deeq_nc.ptr / qq_so.ptr unconditionally (mirroring d_deeq and
d_qq_nt) so the double force/stress path does not segfault under precision
single.
The single-pass QR path was written against the older spsi_func
callable. Develop now passes an HSOperator, so use op.spsi() for the
batched basis update.
- Use FPTYPE* for the DeltaSpin stress lambda in the kernel declarations,
  matching the FPTYPE lambda buffers built by Onsite_Proj_tools.
- Do not instantiate float GPU OnsiteProjector/Onsite_Proj_tools: that
  path is still double-typed in develop and fails to compile.
The develop VNL initialization only allocated and filled the double
deeq_nc/qq_so buffers when has_double_data was set. Single-precision GPU
runs therefore passed null device pointers to the double force/stress
kernel. Allocate and synchronize those buffers unconditionally on GPU,
and keep the CPU path's host pointers valid as before.
Remove the new module_parameter/parameter.h dependency from
diago_david.cpp and stop reading PARAM.globalv.use_uspp inside the solver.

- Add a use_uspp constructor argument and member.
- Pass this->use_uspp from HSolverPW and false from the LCAO LR solver.
- Update unit-test and mock call sites.
- Extract the repeated CUDA float QR predicate into use_block_qr().

This keeps the workflow switch explicit and avoids increasing the global
dependency budget.
@dyzheng
dyzheng force-pushed the develop-single-precision-fixes branch from de17ec2 to add1a06 Compare October 23, 2026 06:43
Address the review concern about silently swallowing non-positive norms.

- Emit a one-time warning in DiagoDavid::SchmidtOrth when a direction is
  skipped.
- In Diago_DavSubspace::cal_grad, replace the psi_norm abort with a
  warning and treat an all-degenerate block as converged, matching the
  behavior already used by the single-precision path.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static source/algebra review of 663b14c30b0b235d055379c6e10a5b059fc42331 against 370ba685e6707b3625c5580ff9d02ef0afb248e8 (31 files). No build, ABACUS/GPU/MPI execution, benchmark or CI diagnosis was performed. The three inline rank-handling concerns require attention; this is a COMMENT, not patch acceptance.

  • Non-blocking validation/scope note: please refresh the body for this exact revision (31 files/+335/-108 versus the stated 6/+101/-6) and identify its build/test commands and source/executable SHAs. The claimed full vcc reset and zero-gradient XC repair are absent from this diff, while several float stress branches remain dormant under the current double stress driver. Separate earlier cumulative campaign/reference results from verification of the submitted tree; unchanged base issues are not being filed as new defects.

  • Non-blocking test coverage: beyond the rank-handling tests above, please add/check CUDA Cgemm cases with k>2048, a partial last block, padded leading dimensions and nonzero beta, plus multik/spin density against a double reference. The changed solver tests only adapt the constructor, and their CMake configuration disables CUDA/ROCm.

  • Non-blocking hygiene: git diff --check reports one trailing-whitespace line at source/source_estate/elecstate_pw.cpp:171.

Comment thread source/source_hsolver/diago_david.cpp Outdated
Comment thread source/source_hsolver/diago_david.cpp Outdated
Comment thread source/source_hsolver/diago_dav_subspace.cpp Outdated
Comment thread source/source_hamilt/module_xc/xc_grad_wfc.cpp
Comment thread source/source_hsolver/diago_david.cpp Outdated
DiagoDavid now requires an explicit use_uspp constructor argument. The
pyabacus adapter still used the old five-argument constructor, which
broke the PyTest CI build. Pass false for the LCAO/HSolver adapter.

Validation: python -m pip install .[test]; pytest -> 64 passed, 5 skipped.
Replace the explicit MPI_Bcast / MPI_Type calls with the communicator-aware
Parallel_Common::bcast_data wrappers already used by Diago_DavSubspace.
Address the rank-handling review concerns for the single-precision
Davidson paths.

- SchmidtOrth now returns whether the direction was accepted. The initial
  basis aborts with a clear diagnostic if a trial vector is linearly
  dependent instead of silently appending a zero column.
- In cal_grad, rejected directions are removed and the accepted columns
  are compacted before H/S are applied, so cal_elem() no longer enlarges
  the reduced basis with zero columns.
- For the CUDA float QR path, inspect the R diagonal and fall back to
  Schmidt orthogonalization when the projected block is rank deficient or
  non-finite, restoring the unprojected block before the fallback.
The zero-norm handling previously only declared the whole block converged
when every direction was degenerate. In a mixed block, zero-norm columns
were still appended to scc and could make the generalized eigenproblem
singular.

Track and compact the accepted directions individually, reduce notconv to
the accepted count, and reject non-finite norms explicitly. H/S are then
applied only to the compacted block.
Exercise DiagoDavid and Diago_DavSubspace with one exact eigenvector and
one approximate direction. The test checks the returned energies, norms
and physical residuals, covering the mixed warm-start path used by the
rejected-direction handling.
Add a CUDA Cgemm regression with k > 2048, a short final block, padded
leading dimensions and nonzero beta. Compare against a double-precision
reference.
The precision-matched gcar/kvec host buffers and their device tensors
were rebuilt for every band in the mGGA stress loop. Split grad_wfc into
an overload that accepts precomputed Real buffers, prepare them once per
stress evaluation, and reuse them for all bands.
Module_HSOLVER_empty_blocks exercises ranks with dim==0. SchmidtOrth must
still reach the pool reduction on those ranks; document this invariant so
the zero-projection path cannot return early and desynchronize the
collective sequence.

Also relax the new warm-start regression tolerances slightly to keep it
robust across BLAS/compiler roundoff.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static incremental review of 812c285b6710219416c6b2b57b43d4e4607e687b against 370ba685e6707b3625c5580ff9d02ef0afb248e8; the remaining DAV issues are updated in their existing threads. No product build/test, GPU/MPI execution or CI diagnosis was performed.

Non-blocking: the new warm-start and Cgemm tail/stride/beta oracles improve coverage. The distinct multik/spin rho/tau comparison against a double reference remains useful follow-up coverage for ElecStatePW::psiToRho.

Comment thread source/source_hsolver/test/test_diago_david.cpp Outdated
Skip H/S application when every new Davidson direction is rejected, and capture the diagonal of R before orgqr so the CUDA float rank check no longer inspects Q. Also raise the serial warm-start test dimension above nbase_x.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bugs Bugs that only solvable with sufficient knowledge of DFT GPU & DCU & HPC GPU and DCU and HPC related any issues Performance Issues related to fail running ABACUS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dav_subspace aborts on a small correction direction after normalization (FeRh GPU reproducer)

4 participants