Skip to content

Fix 8084: DFT+U refactor step 12 - #8077

Open
mohanchen wants to merge 30 commits into
deepmodeling:developfrom
mohanchen:2026-10-05-line1
Open

mohanchen wants to merge 30 commits into
deepmodeling:developfrom
mohanchen:2026-10-05-line1

Conversation

@mohanchen

@mohanchen mohanchen commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Fix #8084

  • Add out_freq_ion/out_freq_elec INPUT parameters to write numbered occupation-matrix snapshots dm_onsiteg{istep+1}.txt, with one section per recorded electronic step containing energy, magnetism and status
  • Rebuild the SOC occupation matrix in spinor space from the four Pauli blocks and diagonalize it with zheev; the previous code read the charge block four times and reported identical eigenvalues and magnetism
  • Write dm_onsite.txt in spin-basis layout for nspin=4 so that it matches read_occup_m() on init_chg=file restart
  • Add unit tests for the base IO helpers and update INPUT documentation

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
  • I have requested any needed governance exception below.

Linked Issue

Fix #

Unit Tests and/or Case Tests for my changes

  • Commands run:
  • Result summary:
  • Checks not run, with reason:

What's changed?

  • Example: brief summary of the user-visible or developer-facing change.

Governance Notes

  • INPUT/docs changes:
  • Core module impact:
  • Exceptions requested:

…utput

- Add out_freq_ion/out_freq_elec INPUT parameters to write numbered
  occupation-matrix snapshots dm_onsiteg{istep+1}.txt, with one section
  per recorded electronic step containing energy, magnetism and status
- Rebuild the SOC occupation matrix in spinor space from the four Pauli
  blocks and diagonalize it with zheev; the previous code read the charge
  block four times and reported identical eigenvalues and magnetism
- Write dm_onsite.txt in spin-basis layout for nspin=4 so that it matches
  read_occup_m() on init_chg=file restart
- Add unit tests for the base IO helpers and update INPUT documentation
@mohanchen
mohanchen requested a review from lanshuyue October 5, 2026 10:03
@mohanchen mohanchen added DFT+U Issues related to DFT plus U function Refactor Refactor ABACUS codes labels Oct 5, 2026
abacus_fixer added 8 commits October 5, 2026 21:23
….cpp

The anonymous-namespace helper extract_soc_pauli_blocks() lives at file
scope, so the enum and its enumerators from namespace DFTU_BASE require
explicit qualification.

Verified: make module_pwdft -j4 builds successfully.
write_occup_m() now prefixes every atom block with "<Element> <index>"
and appends the Cartesian position, so dm_onsite.txt and the
dm_onsiteg{#}.txt snapshots share the identical header line, e.g.
  Fe 1 Atom=1 L=2      0.00000000      0.00000000      0.00000000

read_occup_m() skips tokens up to "Atom=", accepts both the compact
"L=2" and the legacy split "L= 2", and treats "ORBITAL=" as optional
(zeta defaults to 0), so legacy dm_onsite.txt files still parse.

Verified: make module_pwdft -j4 builds successfully.
…site.txt with g files

- Move dm_onsite.txt writing from iter_init to iter_finish so each
  electronic-step snapshot records the actual charge-density residual
  drho alongside the configured threshold scf_thr.
- Share one section writer between dm_onsite.txt and dm_onsiteg{#}.txt:
  provenance header (version, timestamp, RELAX STEP), then
  "# Electronic step N / # scf_thr / # drho" plus the compact matrix body.
- Snapshot " N/A" instead of a silent zero matrix at PW istep 0 / iter 1
  when no occupation matrix exists yet (loaded-from-file runs keep the
  real matrix via is_occmat_ready); LCAO always writes the real matrix.
- read_occup_m skips the new comment lines and the N/A token, so
  restart parsing is unaffected.
- Replace the scf_dftu_spin2/spin4 integration cases with
  relax_dftu_spin2/spin4 (relax_nmax 5, scf_nmax 5) and register them
  in CASES_CPU/GPU.txt; add AppendSnapshotNAPlaceholderAndReady unit
  test for the N/A placeholder behavior.

Verification: incremental build clean; ctest
MODULE_PW_dftu_base_test + MODULE_PW_test_dftu_pw_tools 2/2 pass;
relax_dftu_spin2 run exit=0, dm_onsite.txt identical to the last
dm_onsiteg5.txt section; drho sequence varies per step as expected;
nspin4 SOC run + restart (init_chg=file) exit=0; governance check
exit=0 with global-dependency budget net_delta=0.
…matrix output

out_occ_mat (Boolean, default 1) decouples the DFT+U occupation-matrix
files (dm_onsite.txt and dm_onsiteg{#}.txt) from out_chg: users no longer
need to pay the charge-density cube IO to obtain occupation matrices.

The switch is carried by OccmatOutputCfg::out_occ_mat and gates all
writers in one place each: write_latest_occmat() (dm_onsite.txt),
prepare_ion_step_file() via output()/iter_init_dftu_pw() (numbered-file
header), and append_ion_step_snapshot() (numbered-file sections). The
bool out_chg parameters of output(), iter_init_dftu_pw() and
finish_dftu_lcao() served only occupation-matrix output and are removed.

Side fix: with out_chg=0 and out_freq_ion>0, append_ion_step_snapshot()
previously created headerless orphan files; the new gate prevents that.
Global dependency budget is migration-neutral (net_delta=0).
…{#}.txt

dm_onsite.txt and dm_onsiteg{#}.txt are renamed to occ_mat.txt and
occ_matg{#}.txt; gen_ion_step_dm_onsite_filename() is renamed to
gen_ion_step_occ_mat_filename() accordingly. read_occup_m() error
messages now report the actual probed filename.

init_chg=file first probes occ_mat.txt and falls back to the legacy
dm_onsite.txt, so output directories written by older versions can
still be used for restarts. dm_onsite_ini.txt (occ_mat_ctrl input)
keeps its name.

The omc parameter note now points to occ_mat.txt and out_occ_mat=1
(the default) as the way to produce it.

Verification: MODULE_PW_dftu_base_test and MODULE_DFTU_base_io pass;
PW DFT+U SCF writes occ_mat.txt + occ_matg{1,3,5}.txt; NSCF with
init_chg=file reads occ_mat.txt and, after renaming it to
dm_onsite.txt, produces the identical total energy via the fallback;
with both files absent the run quits with a clear error.
band.md and dos.md now reference occ_mat.txt in the NSCF workflow;
run_scf_nscf.sh copies occ_mat.txt (the new output name) from the SCF
output; the dm_onsite.txt mentions in tests/17_DS_DFTU/README.md are
updated accordingly.

Verification: bash -n run_scf_nscf.sh passes; no non-legacy dm_onsite
references remain in docs/advanced/elec_properties or tests/17_DS_DFTU.
- Add init_occ_mat (Integer, 0/1/2) as the primary occupation-matrix
  initialization mode parameter.
- Keep omc as a legacy alias; when both are set, init_occ_mat takes
  precedence (same pattern as out_mat_hs / out_mat_hs2).
- Rename occ_mat_ctrl member/getter to init_occ_mat across dftu_base,
  esolver call sites, and tests.
- Regenerate docs/parameters.yaml and input-main.md.
- init_occ_mat 1/2 now searches read_file_dir for dm_onsite_ini.txt,
  then falls back to occ_mat.txt, then the legacy dm_onsite.txt, so a
  previously output occ_mat.txt can be reused without renaming.
- The init_chg=file branch reuses the same helper.
- Move find_first_existing_file into DFTU_BASE namespace and add a
  unit test for candidate priority.
- Update init_occ_mat description and regenerate docs.
@mohanchen mohanchen changed the title DFT+U refactor step 12 Fix 8084: DFT+U refactor step 12 Oct 6, 2026
abacus_fixer and others added 17 commits October 7, 2026 21:45
- PW DFT+U: when init_occ_mat=2, rebuild pot_onsite and E_U from the
  fixed file-loaded occupation matrix at every electronic step
  (including the first); the matrix itself is still neither
  reprojected from psi nor mixed. Previously the whole cal_occ_pw
  was skipped, leaving V_U and E_plusU at zero for the entire run.
- Remove the per-type U listing and per-atom occupation matrices
  from the running log; the same data is available through the
  out_occ_mat-gated occ_mat.txt / occ_matg{#}.txt files and the log
  block floods the output for large cells.
- Print the full energy table at every electronic step for DFT+U
  calculations so E_plusU can be monitored.
- Convert relax_dftu_spin2 into an init_occ_mat=2 regression case
  (KPT 2x1x1, tighter SCF) with an occ_mat.txt fixture.
Add AGENTS.md rule 17: unit tests must not create or delete files or
directories via std::system("mkdir/rm"), std::remove, or
std::filesystem::remove_all; use committed fixtures or
testing::TempDir() instead.

Fix dftu_base_test.cpp to comply:
- Replace std::system("mkdir -p/rm -rf") calls in
  FindFirstExistingFileTest and InitBaseReadsOccMatFileOnlyOnce with
  testing::TempDir(); use unique fixture names so leftover files from
  a previous run do not affect the "no candidate" case.
- Simulate "file no longer present" on the second ionic step by passing
  a non-existent readin dir instead of deleting the file.
- Add a custom main() that calls MPI_Init/Finalize so the binary runs
  under MPI builds where local_occup_bcast() broadcasts on COMM_WORLD.
Rule 17 now forbids only shell-based file/dir IO
(std::system("mkdir/rm")) in unit tests. Direct library calls
(std::remove, std::filesystem::remove_all) on test-written files
are acceptable since they do not go through a shell and cannot
delete directory trees, so there is no injection or runaway-delete
risk.
… in locals

Pass nspin as an explicit parameter to iter_init_dftu_pw instead of reading
PARAM.inp.nspin inside the function body. In both esolver iter_finish
functions, read PARAM.globalv.{global_out_dir,npol,gamma_only_local} once
into locals so the repeated DFT+U call sites do not each re-enter the
global dependency surface. PR net_delta drops from +5 (BLOCK) to -4 (WARN).
write_latest_occmat and append_ion_step_snapshot only checked
out_occ_mat (default true), not dft_plus_u. Non-DFT+U LCAO/PW runs
dereferenced an empty l_channel vector in has_l_channel() during
iter_finish, segfaulting at GE1 with a nil address.

Add dft_plus_u to OccmatOutputCfg and early-out when dft_plus_u <= 0,
honouring the documented contract that out_occ_mat only takes effect
for DFT+U calculations. Add DftPlusUDisabledWritesNothing regression
test using a default-constructed Plus_U_Base (empty l_channel).

Verified: abacuslite scf.py converges (Si LCAO, E=-210.6232 eV);
MODULE_PW_dftu_base_test 11 existing tests pass; new regression test
compiled locally (user runs final ctest confirmation).
…1/3)

Introduce a dormant scf_rerun_ member in ESolver_KS and a rerun hook at
the bottom of ESolver_KS::runner's SCF loop. When a subclass sets
scf_rerun_, the loop clears the flag and restarts from iter=1. This
replaces the historical iter=0 restart-signal trick that crashed DFT+U
occupation-matrix writers validating iter>=1.

Behavior is unchanged in this step: scf_rerun_ is never set yet, so the
hook is dormant and EXX continues to rely on ++iter to turn iter=0 back
into iter=1. Build + full link pass in build_max_para_test.
EXX no longer mutates the caller's iter. Change Exx_HelperBase and
Exx_Helper signatures: iter_finish and exx_after_converge now take
iter by value. Drop the "iter = 0" write in exx_after_converge that
used to trigger an SCF rerun via ++iter in the for-loop.

The SCF rerun is now requested explicitly: ESolver_KS_PW::iter_finish
captures conv_esolver before the EXX call and, when SCF had converged
but EXX overrode conv_esolver to false, sets ESolver_KS::scf_rerun_.
ESolver_KS::runner (updated in step 1) then restarts the loop at
iter=1. This keeps iter strictly 1-based for every downstream reader
(check_deltaspin_oscillation, write_latest_occmat,
append_ion_step_snapshot, ctrl_iter_pw) and removes the crash in
append_ion_step_snapshot's iter>=1 validation. Build + full link pass.
… 3/3)

Move the dft_plus_u/out_occ_mat guard to the top of
append_ion_step_snapshot, before the nspin/istep/iter/frequency
validation. When DFT+U is off this function is a no-op, so an invalid
iter (defense in depth against any future upstream bug that might
reintroduce an iter=0 sentinel) no longer crashes the iter>=1
validation. Mirrors the guard-first pattern already used by
write_latest_occmat.

Validation still runs in full when DFT+U is on (the only case where
this function does real work), so the existing contract is preserved.
MODULE_PW_dftu_base_test passes; full abacus build + link pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DFT+U Issues related to DFT plus U function Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DFT+U: E_plusU always 0 in PW path; unify occ_mat output filename and decouple its I/O control

1 participant