Repository navigation
fix(pt_expt): preserve numb_dos for exported DOS models - #6041
Arthur031221 wants to merge 2 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
njzjz-bot
left a comment
There was a problem hiding this comment.
Reviewed 2a82d16f; the model getter, serialized metadata, and metadata-only inference fallback consistently preserve the DOS width. The change also keeps the existing zero fallback for older metadata-only archives and avoids adding DOS metadata to energy models.
Ran source/tests/pt_expt/infer/test_deep_eval_dos.py against this PR's source with Python 3.10 and PyTorch 2.13.0+cpu: 4 tests and 8 subtests passed, covering full and metadata-only .pte/.pt2 archives, DOS evaluation shapes, and the non-DOS case. No blocking findings.
Coding agent: Codex
Codex version: codex-cli 0.160.0
Model: gpt-6-astra
Reasoning effort: xhigh
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6041 +/- ##
==========================================
- Coverage 77.74% 77.61% -0.14%
==========================================
Files 1155 1170 +15
Lines 139649 140921 +1272
Branches 5056 5057 +1
==========================================
+ Hits 108573 109372 +799
- Misses 29191 29668 +477
+ Partials 1885 1881 -4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
wanghan-iapcm
left a comment
There was a problem hiding this comment.
Reviewed at 2a82d16.
Verification:
- I ran the new test
source/tests/pt_expt/infer/test_deep_eval_dos.pyagainst the head and against the base versions of the three source files. On head it passes: 4 passed. On pre-fix code it fails: 5 failures and 4 errors at subtest level (0 != 2andcannot reshape array of size 2 into shape (1,0)). The test covers.pteand.pt2, full and metadata-only. - With only
deepmd/pt_expt/utils/serialization.pyreverted, exactly the four metadata-only subtests fail, as the PR description says. - The
DOSModel.get_numb_dosoverride matches dpmodel (deepmd/dpmodel/model/dos_model.py) and pt (deepmd/pt/model/model/dos_model.py). This completes for pt_expt what #5722 did for dpmodel and JAX. - No C++ reader (
source/api_cc,source/api_c,source/lmp) usesnumb_dos, so the new metadata key affects only the PythonDeepEval. - Related tests pass on head:
test_deep_eval_dos.py,test_deep_eval_metadata_only.py,pt_expt/model/test_dos_model.py,pt_expt/model/test_dos_graph.pyandconsistent/model/test_dos.pygive 75 passed and 7 skipped.
No blocking findings. Two non-blocking suggestions are inline.
| get_numb_dos = getattr(self._dpmodel, "get_numb_dos", None) | ||
| return int(get_numb_dos()) if get_numb_dos is not None else 0 |
There was a problem hiding this comment.
Non-blocking: get_numb_dos always exists on the dpmodel here. The make_model base class defines it (returning 0 for models without a DOS output, deepmd/dpmodel/model/make_model.py:1148), and SpinModel.__getattr__ forwards to the backbone. So the getattr probe and its else 0 branch cannot be reached. The dpmodel and pt DeepEval call it directly (deepmd/dpmodel/infer/deep_eval.py:169, deepmd/pt/infer/deep_eval.py:459); return int(self._dpmodel.get_numb_dos()) would match them.
| return int(get_numb_dos()) if get_numb_dos is not None else 0 | ||
| # Metadata-only mode: `numb_dos` is written by `_collect_metadata` | ||
| # for models with a `dos` output; other models read as 0. | ||
| return int(self.metadata.get("numb_dos", 0)) |
There was a problem hiding this comment.
Non-blocking: the DOS width is already in the metadata. _collect_metadata writes every fitting output into fitting_output_defs with its shape, and for a DOS model the dos entry has shape [numb_dos] (I checked a numb_dos=3 archive: the dos shape is [3], the same as the new key). In metadata-only mode DeepEval already rebuilds self._model_output_def from that list (deep_eval.py:443-460), so get_numb_dos could read the width from the dos output def instead of a separate numb_dos key. That keeps a single source for the width, drops the metadata.get("numb_dos", 0) fallback that silently returns 0 when the key is missing, and also works for DOS archives exported before this PR. The numb_dos key added in serialization.py would then be unnecessary.
There was a problem hiding this comment.
Took both suggestions in a new commit on top, so the reviewed commit is unchanged. The first one is your line as written, so the commit carries a Co-authored-by trailer for you.
DeepEval.get_numb_dos now calls self._dpmodel.get_numb_dos() directly. In metadata-only mode it reads the width from the shape of the dos entry in the output defs rebuilt from fitting_output_defs, and returns 0 when there's no dos output. The numb_dos key in _collect_metadata is gone, so DOS archives exported before this change work too.
The test now checks that an energy archive reports 0 in both full and metadata-only form, instead of checking a metadata key. test_deep_eval_dos.py, test_deep_eval_metadata_only.py and pt_expt/model/test_dos_model.py give 17 passed and 10 subtests passed. With deep_eval.py put back to the base version, test_deep_eval_dos.py gives 8 failures.
The description is now stale in two places: the second bullet (the numb_dos metadata key) and the test paragraph's check on that key, the serialization revert and the counts. The numbers above replace them.
Address review: call get_numb_dos directly on the dpmodel, and in metadata-only mode take the width from the shape of the dos output definition already stored in fitting_output_defs. This drops the separate numb_dos metadata key, and also covers DOS archives exported before this change. Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Fixes #5949.
A user with a pt_expt DOS model configured with
numb_dos: 2, exported to.pteor.pt2, getsValueError: cannot reshape array of size 2 into shape (1,0)fromDeepDOS.eval, because the exported model reports a DOS width of 0.Two gaps caused it:
deepmd/pt_expt/model/dos_model.py::DOSModelhad noget_numb_dos, so it fell back to the base default of 0. It now returns the fitting net'sdim_out, as the dpmodel and pt DOS models do.deepmd/pt_expt/infer/deep_eval.py::get_numb_dosreturned 0 unconditionally. It now asks the reconstructed dpmodel, and for metadata-only archives (noextra/model.json) it reads a newnumb_doskey. That key is written tometadata.jsononly for models with adosoutput, and archives without it read as 0 as before.Test:
source/tests/pt_expt/infer/test_deep_eval_dos.pyexports anumb_dos=2se_e2_a DOS model to.pteand.pt2, in full and metadata-only form, and checksget_numb_dos()and the shapes returned byDeepDOS.eval. It also checks that an energy archive gets nonumb_doskey. With the three source files reverted to the base commit, pytest reports 9 failures (0 != 2and the reshape error above); with the fix, 4 tests and 8 subtests pass. Reverting only the serialization change fails the four metadata-only subtests.Summary by CodeRabbit