[Bug] v3.11.0-beta10: esolver_type=dp forces deviate from model predictions (MDCell neighbor-list path) #8071

Activity

1. mohanchen

   assigned

   19hello

   on Oct 5, 2026
2. mohanchen

   added

   MD & LAMMD and Larege Atomic ModelsMD and Larege Atomic Models

   BugsBugs that only solvable with sufficient knowledge of DFTBugs that only solvable with sufficient knowledge of DFT

   on Oct 5, 2026
3. 19hello commented on Oct 9, 2026

   

   19hello

   on Oct 9, 2026

   Collaborator

   More actions

   Could you provide the minimal reproducible INPUT, STRU, DeePMD model (.pth/.pb), MDSTEP: 0 forces from MD_dump, and the corresponding Python DeepPot.eval() output? I need these files to compare the native DeePMD neighbor list, the external InputNlist, and the post-accumulation ABACUS forces.
4. MarkYoungIV commented on Oct 9, 2026

   

   MarkYoungIV

   on Oct 9, 2026

   Author

   More actions

   Thanks for looking into this! I've prepared a full reproduction package:

   Repro package (secret gist): https://gist.github.com/MarkYoungIV/9fe10e29d2fedee0fa7d22e87d3c9e66
   The gist contains issue8071_repro.zip.b64 (base64 of the zip, base64 used because gists don't take binaries):

   base64 -d issue8071_repro.zip.b64 > issue8071_repro.zip && unzip issue8071_repro.zip
   

   • INPUT, STRU (Li20Ge2P4S24, 50 atoms), frozen se_e2_a model (frozen_seed1.pth)
   • MD_dump from a 10-step MD run (MDSTEP 0 forces)
   • esolver_dp_dump.txt — the exact neighbor list ABACUS passes to DeePMD (owned+ghost atoms, per-atom rows)
   • compare_issue8071.py — reproduces everything below in one command

   Results (matches the report):

   ABACUS MD_dump (step 0) vs native DeepPot.eval:
     rmse = 1.659e-02   mean|dF| = 6.9e-03   max|dF| = 7.0e-02  eV/A
   

   Additional findings:

   1. ABACUS's force plumbing is exact: the MD_dump forces are bit-identical (rmse 1.8e-10) to running the model through deepmd's external-nlist path (forward_lower) with ABACUS's own list and folding the ghost force entries back onto the owned atoms (DomainDecomposition::accumulate_ghost_forces). So the framework accumulation and unit conversions are fine.
   2. The neighbor sets are identical to the native ones: vesin (full_list=True, periodic) returns the same 2280 pairs; per-atom (j, shift) sets match 50/50 atoms.
   3. The difference is the per-atom neighbor order:
      • ABACUS's MDCell neighbor search emits neighbors in allocator order;
      • deePMD's native path (and the LAMMPS deepmd pair style) emits them distance-sorted; se_e2_a-type descriptors report need_sorted_nlist_for_lower() == True.
      • With the raw ABACUS list the descriptor's sel-truncation/padding picks a different neighbor subset for atoms whose counts approach sel (atoms delete snap_psialpha #24/refactor potential and add DELETE_MUL_PTR() #25 are P with 28 S-neighbors vs sel_S = 26; the other large deviations are on the high-coordination P/S atoms), which produces exactly the reported residual.
      • Distance-sorting each atom's neighbor row before constructing the InputNlist removes the discrepancy: rmse 1.65e-05 / max 5.5e-05 eV/A (see script output).

   Suggested fix: in source/source/source_esolver/esolver_dp.cpp runner() (MDCell branch), sort each owned atom's neighbor indices by distance when building deepmd::InputNlist, matching the LAMMPS deepmd convention. We have a small patch for this if useful.

   One related, more severe finding while testing this path with message-passing descriptors (DPA-2/repformers): the same InputNlist construction omits DeePMD's comm setup (nswap/sendlist/sendproc/world). Consequently the border_op ghost-embedding exchange never happens (forward exchange: energies off by 20–37 eV, forces ~0.5 eV/A off), and with MPI initialized MPI_Comm_rank(NULL) aborts the run (MPI_ERR_COMM: invalid communicator). Passing nswap=1 with a ghost→owned self-exchange sendlist and world = &MPI_COMM_WORLD (while letting the framework keep doing the reverse force accumulation) makes DPA-2 MD agree with the native API to ~2e-05 eV/A. Happy to file that as a separate issue with a patch if you're interested.
5. 19hello commented on Oct 9, 2026

   

   19hello

   on Oct 9, 2026

   Collaborator

   More actions

   Thanks for providing the complete reproduction package. I have reproduced the issue and confirmed the suspected root cause.

   The ABACUS force plumbing and ghost-force accumulation are correct. Using ABACUS's original InputNlist with DeepMD's external-nlist path gives the same forces as those written to MD_dump, while the neighbor sets themselves match the native DeepMD neighbor sets.

   The discrepancy comes from the ordering of the neighbors:

   • ABACUS's MDCell neighbor list is emitted in allocator order.
   • DeepMD's native path expects distance-sorted neighbors for this se_e2_a descriptor.
   • Since the descriptor applies sel-based neighbor truncation, different neighbor ordering can result in a different subset of neighbors being selected, even though the complete neighbor sets are identical.

   In my reproduction, the force difference was:

   Original ABACUS neighbor order:
   RMSE = 1.659e-02 eV/A
   MAX  = 7.001e-02 eV/A
   
   After distance-sorting each neighbor row:
   RMSE = 1.839e-05 eV/A
   MAX  = 6.071e-05 eV/A
   
   I opened PR #8104 to address this issue:
   
   https://github.com/deepmodeling/abacus-develop/pull/8104
   
   The patch sorts each owned atom's MDCell neighbor row by physical distance before constructing DeePMD's InputNlist, while keeping the sorted storage alive until the DeePMD calculation finishes. It also adds a focused unit test for the sorting behavior.
   
6. MarkYoungIV commented on Oct 10, 2026

   

   MarkYoungIV

   on Oct 10, 2026

   Author

   More actions

   Great, thanks for the quick turnaround! The numbers in your reproduction (RMSE 1.839e-02 -> 1.839e-05 after sorting) match our independent verification exactly, and the PR looks right to us — keeping the sorted storage alive across the DeePMD call is exactly what our testing showed is needed.

   As for the separate message-passing issue you mentioned (missing nswap/sendlist/sendproc/world), I've now filed it with a full reproduction package and a patch:

   #8108

   It covers both failure modes (the MPI_Comm_rank(NULL) abort at MD step 1 on MPI builds, and the wrong energies/forces without MPI), plus the fix that makes DPA-2 MD agree with the native API to ~2e-05 eV/A. The patch touches the same runner() function but a different hunk than #8104, so they should coexist.
7. mohanchen

   closed this as completedin #8104on Oct 10, 2026