Unify output directory structure with out_app_flag=2 by mohanchen · Pull Request #8159 · deepmodeling/abacus-develop

mohanchen · 2026-10-16T09:23:23Z

Summary

Resolves #8151. This PR unifies the output directory structure by extending out_app_flag from bool to int with three modes, and routes all output files to consistent uppercase subdirectories when enabled.

Problem

In RT-TDDFT and MD calculations, output files (.cube charge densities, WFC files, matrices, etc.) accumulate in the OUT.<suffix>/ root directory, making it difficult to navigate and manage results.

Solution

`out_app_flag` semantics (backward compatible)

Value	Behavior	Directory
`0`	No append, files get step suffix	Root `OUT.xxx/`
`1` (default)	Append to same file	Root `OUT.xxx/`
`2` (new)	No append, files get step suffix	Unified subdirectories

Unified subdirectories (only created when `out_app_flag=2` and the output is requested)

Directory	Contents
`PSI/`	Wave functions (PW and LCAO unified, replaces `WFC/`)
`CHG/`	Charge density (`chg`, `tau`)
`POT/`	Potential (`pot`, `potes`)
`ELF/`	Electron localization function
`DOS/`	DOS, PDOS, LDOS
`DIP/`	Dipole
`MAT/`	Matrices (replaces `matrix/`, includes `dmk`, `lx/ly/lz`, `rr`, `tr`)

Key changes

Parameter: out_app_flag changed from bool to int with validation {0,1,2}
Directory variables: 7 new System_para members (global_psi_dir, global_chg_dir, etc.) with MPI broadcast
On-demand creation: Subdirectories only created when the corresponding output flag is enabled
Path routing: All output paths updated to use the new directory variables
Backward compatibility: Default out_app_flag=1 preserves existing behavior (root directory, append mode)

Verification

Build: make -j 30 passes on build_max_para_test
Governance: Global dependency budget net decrease of 1 (no new PARAM/GlobalV cross-layer control)
Tests: Updated 4 unit test files and 1 integration test script for new semantics
Docs: Updated 4 documentation files describing out_app_flag behavior

Files changed

source/source_io/module_parameter/ — input parameter, read logic, global directory setup
source/source_io/module_ctrl/ — output path routing for chg/pot/elf/dos/dip/wfc
source/source_io/module_dm/ — dmk path routing
source/source_io/module_hs/ — matrix output path routing (hsr, pos_op, dhs, vxc)
source/source_base/module_out/ — filename_output signature update
tests/ — unit and integration test updates
docs/ — documentation updates

…unified output directories - Change out_app_flag from bool to int with three states: 0: no append, no subdirectory (overwrite) 1: append to same file (default, backward compatible) 2: write to unified subdirectories - Add new global directory variables in System_para: global_psi_dir, global_chg_dir, global_pot_dir, global_elf_dir, global_dos_dir, global_dip_dir, global_mat_dir - Initialize directories in set_global_dir(): out_app_flag==2 -> OUT.suffix/{PSI,CHG,POT,ELF,DOS,DIP,MAT}/ otherwise -> all point to global_out_dir (backward compatible) - Map legacy WFC->PSI, matrix->MAT - Update all out_app_flag usages from bool to int semantics - Broadcast new directory variables via Parallel_Common::bcast_string Issue: deepmodeling#8151

- ctrl_output_fp.cpp: route chg/tau to global_chg_dir, pot/potes to global_pot_dir, ELF to global_elf_dir, dipole to global_dip_dir - ctrl_output_pw.cpp: route wfc to global_psi_dir, dos to global_dos_dir, ldos to global_dos_dir - ctrl_scf_lcao.cpp: route dos to global_dos_dir - ctrl_runner_lcao.cpp: route ldos to global_dos_dir - read_input.cpp: create subdirectories (PSI/CHG/POT/ELF/DOS/DIP/MAT) when out_app_flag==2 - Update docs: out_app_flag semantics from bool to int (0/1/2) Issue: deepmodeling#8151

When out_app_flag defaults to 1 (not 2), all new unified directories (PSI, CHG, POT, ELF, DOS, DIP, MAT) map to global_out_dir for backward compatibility. Update io_sys_var_test.cpp expectations: - global_matrix_dir: "OUT.test/matrix/" -> "OUT.test/" - Add assertions for all 7 new directory variables

- read_input.cpp: change !out_app_flag to out_app_flag == 0 for matrix/WFC legacy directory decisions - Update three unit tests to use int out_app_flag (0/1) instead of bool (false/true), matching the new function signatures: test_cal_dos.cpp, test_write_dos_pw.cpp, write_wfc_nao_test.cpp - Update two comments to describe 0/1 semantics

Replace repeated PARAM.globalv.global_*_dir accesses with the local const variables already declared at function top, eliminating the net +4 PARAM global dependency introduced by the path routing change.

With out_app_flag changed from bool to int, the legacy WFC/ subdirectory is no longer created when out_app_flag=0 (now maps to global_out_dir). Update the multi-k wfc reference path accordingly.

- read_input.cpp: only create PSI/CHG/POT/ELF/DOS/DIP/MAT subdirectories when the corresponding output flag is enabled (out_app_flag=2) - ctrl_scf_lcao.cpp: route write_dmk and angular momentum calculator output to global_mat_dir instead of global_out_dir - hsr_legacy.cpp: route tr_nao.csr to global_matrix_dir - pos_op_writer.cpp: route rr_nao.txt to global_matrix_dir

mohanchen · 2026-10-16T09:23:56Z

@ABACUS-PR-BOT please help review the PR

ABACUS-PR-BOT

Static review of 00ece15 with independent verification. Five implementation issues and three non-blocking suggestions are noted inline. Whitespace checks passed; the pinned governance check reported warnings only. No build or runtime tests were run, and CI diagnostics were unavailable.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

+
+    /// Map legacy directory names to new unified names
+    /// WFC -> PSI, matrix -> MAT
+    sys.global_wfc_dir = sys.global_psi_dir;


[P1] Keep the TDDFT restart reader aligned with the writer

This moves non-appended LCAO snapshots from WFC/ to the root (mode 0) or PSI/ (mode 2), but read_wfc_nao still appends WFC/ for the nonnegative step supplied by TDDFT restart. A restart with init_wfc file therefore misses snapshots written by this revision, or reads stale files from an old WFC folder. Please update the writer and restart reader together.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

+            {param.input.out_chg[0] > 0, param.globalv.global_chg_dir},
+            {param.input.out_pot[0] > 0 || param.input.out_elf[0] > 0, param.globalv.global_pot_dir},
+            {param.input.out_elf[0] > 0, param.globalv.global_elf_dir},
+            {param.input.out_dos > 0 || !param.input.out_band.empty() && param.input.out_band[0] > 0, param.globalv.global_dos_dir},


[P2] Create directories for independently enabled outputs

With mode 2, out_ldos enabled and out_dos/out_band disabled writes into DOS/ without creating it; out_mat_l alone has the same problem with MAT/. Both writers can silently lose the requested files in a fresh output directory. Please make the creation predicates cover every writer using each directory.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z


        ModuleIO::write_hsr(hr_vec, sr, &ucell, inp.out_hsr[0], precision, pv,
-                            out_app_flag, gamma_only, ucell.get_iat2iwt(), ucell.nat, istep,
+                            (out_app_flag == 1), gamma_only, ucell.get_iat2iwt(), ucell.nat, istep,


[P2] Route the remaining matrix outputs through MAT

In mode 2, this write_hsr call still receives global_out_dir, as do H(k), T(k), Vxc and the NPZ outputs; their writers do not redirect that path. Positive-step r(R)/T(R) output during relax also stays in the root because its MAT selection is restricted to MD. Please apply the selected matrix directory consistently across these producers.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

        item.unit = "";
        item.set_availability("basis_type==lcao and gamma_only==0");
-        read_sync_bool(input.out_app_flag);
+        read_sync_int(input.out_app_flag);


[P2] Preserve the previously accepted Boolean spellings

Existing INPUT files containing out_app_flag true or false now reach std::stoi and throw before validation. The previous parser also accepted case-insensitive yes/no, on/off and dotted Boolean forms. Please retain those aliases as modes 0/1 while adding numeric mode 2.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

        const Parallel_Orbitals &pv,
        const bool gamma_only,
-        const bool out_app_flag,
+        const int out_app_flag,


[P2] Convert the integer mode at bool append boundaries

The save_mat calls below still pass this int to a bool append argument, so mode 2 becomes true. When a step file already exists and istep > 0, a rerun or replay appends another record instead of replacing the independent snapshot; clean, distinct-step files hide this. Please pass out_app_flag == 1 here and in write_hk_common.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

-        item.default_value = "true";
+        item.type = "Integer";
+        item.description = R"(Controls output file organization during multi-step calculations (MD, relax, RT-TDDFT).
+* 0: Write to OUT directory directly, no append (files overwritten each step)


Non-blocking: please synchronize the mode help and directory guidance. Mode 0 gives wavefunction/matrix files step suffixes, the new routing also applies to PW/gamma-only, and related WFC/position documentation still promises the old directories. Correct the C++ metadata and prose, then verify generated help/docs; the ignored generated files need not be committed.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

        EXPECT_EQ(sys.global_out_dir, "OUT.test/");
        EXPECT_EQ(sys.global_stru_dir, "OUT.test/STRU/");
-        EXPECT_EQ(sys.global_matrix_dir, "OUT.test/matrix/");
+        // out_app_flag defaults to 1, so matrix dir maps to global out dir


Non-blocking: the updated tests still select only modes 0/1, and these directory assertions cover the default mode. Please add focused mode-2 cases for isolated output flags, repeated snapshot replacement and TDDFT readback; the example INPUT does not verify those behaviors.

ABACUS-PR-BOT · 2026-10-16T10:10:50Z

+        const std::vector<std::pair<bool, std::string>> subdir_flags = {
+            {param.input.out_wfc_lcao > 0 || param.input.out_wfc_pw > 0, param.globalv.global_psi_dir},
+            {param.input.out_chg[0] > 0, param.globalv.global_chg_dir},
+            {param.input.out_pot[0] > 0 || param.input.out_elf[0] > 0, param.globalv.global_pot_dir},


Non-blocking: ELF-only output creates an unused POT/, while band-only output creates DOS/ although write_bands still uses the root. Please make creation match the actual destinations. This empty-directory cleanup can follow the correctness fixes.

abacus_fixer added 8 commits October 16, 2026 14:47

chore: remove accidentally committed quality report file

5adf04b

fix(io): use local cached directory variables in ctrl_output_fp

e734403

Replace repeated PARAM.globalv.global_*_dir accesses with the local const variables already declared at function top, eliminating the net +4 PARAM global dependency introduced by the path routing change.

test(integrate): fix WFC file path in props_cube.sh for out_app_flag=0

9254e95

With out_app_flag changed from bool to int, the legacy WFC/ subdirectory is no longer created when out_app_flag=0 (now maps to global_out_dir). Update the multi-k wfc reference path accordingly.

mohanchen added Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes labels Oct 16, 2026

ABACUS-PR-BOT reviewed Oct 16, 2026

View reviewed changes

Navigation Menu

Repository navigation

Unify output directory structure with out_app_flag=2 - #8159

Unify output directory structure with out_app_flag=2#8159
mohanchen wants to merge 8 commits into
deepmodeling:developfrom
mohanchen:2026-10-16-b

mohanchen commented Oct 16, 2026

Uh oh!

mohanchen commented Oct 16, 2026

Uh oh!

ABACUS-PR-BOT left a comment

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

ABACUS-PR-BOT Oct 16, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mohanchen commented Oct 16, 2026

Summary

Problem

Solution

out_app_flag semantics (backward compatible)

Unified subdirectories (only created when out_app_flag=2 and the output is requested)

Key changes

Verification

Files changed

Uh oh!

mohanchen commented Oct 16, 2026

Uh oh!

ABACUS-PR-BOT left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`out_app_flag` semantics (backward compatible)

Unified subdirectories (only created when `out_app_flag=2` and the output is requested)