Repository navigation
jax: align training/model support with argcheck schema #5756
Copy link
Copy link
Open
0 / 100 of 10 issues completedOpen
0 / 100 of 10 issues completed
Copy link
Labels
Description
Summary
While comparing the unified input schema in deepmd/utils/argcheck.py with the JAX backend implementation, I found several schema-exposed options that are not implemented or not wired up in deepmd/jax yet.
This issue tracks the gaps so we can either implement them for JAX or make the schema/docs explicitly say that these options are unavailable in the JAX backend.
Gaps
1. Non-energy fittings/models are registered, but JAX training only accepts loss.type: ener
Schema/backend registrations:
deepmd/utils/argcheck.pyexposes fitting types includingener,dos,property,polar, anddipole.- The JAX backend registers matching fitting/model classes for
dos,property,dipole, andpolarunderdeepmd/jax/fitting/fitting.pyanddeepmd/jax/model/*.py.
Implementation:
deepmd/jax/train/trainer.py_build_losses()only acceptsloss_type == "ener"; any other loss raisesRuntimeError("unknown loss type ...").
Impact:
- JAX can construct non-energy model/fitting objects, but JAX training cannot train the corresponding DOS/property/tensor tasks because the loss dispatch rejects non-energy losses.
2. model.spin is accepted by the common schema, but JAX explicitly rejects spin models
Schema/docs:
deepmd/utils/argcheck.pyexposes the top-levelmodel.spinblock.
Implementation:
deepmd/jax/model/model.pyraisesNotImplementedError("Spin model is not implemented yet.")whenmodel.type == "standard"andspinis present.
Impact:
- Schema-valid spin model configs fail at JAX model construction time.
3. Learning-rate schema exposes exp, cosine, and wsd, but JAX only implements exp
Schema/docs:
deepmd/utils/argcheck.pyregisterslearning_rate.typevariantsexp,cosine, andwsd.
Implementation:
deepmd/jax/train/trainer.py_get_lr_and_coef()dispatches onlytype == "exp"toLearningRateExp; all other types raiseRuntimeError("unknown learning_rate type ...").
Impact:
learning_rate.type=cosineorlearning_rate.type=wsdis schema-exposed but unavailable in JAX training.
4. The optimizer schema is not wired into JAX training
Schema/docs:
deepmd/utils/argcheck.pyexposes anoptimizerblock and optimizer variants.
Implementation:
deepmd/jax/train/trainer.pybuilds an Optax chain directly withoptax.scale_by_adam()and_scale_by_global_learning_rate(); it does not readconfig["optimizer"]or dispatch optimizer variants/parameters.
Impact:
- Optimizer options accepted by the common input schema have no effect in JAX training. At minimum this should be documented as unsupported; ideally JAX should consume the optimizer config it can support.
5. Several model-level schema variants are not present in the JAX backend registry
Schema/docs:
deepmd/utils/argcheck.pyexposes model variants such asfrozen,pairtab, andlinear_enerin the common model schema.
Implementation observation:
- JAX model registrations are limited to the task models (
ener,dos,dipole,polar,property) pluszbl; I did not find JAXBaseModel.register(...)implementations forfrozen, top-levelpairtab, orlinear_ener. model.type: dpa4/dpa4_enerare marked PyTorch-only in the schema, so those are not counted here as JAX implementation gaps.
Impact:
- Users can discover these model variants from the unified schema, but JAX cannot construct them unless there is a separate compatibility path.
Expected resolution
Either:
- implement/wire these options in the JAX backend, or
- update
argcheck.py/docs to mark them as unsupported for JAX so schema-valid configurations do not fail later in backend-specific code.
Filed by OpenClaw 2026.6.11 (e085fa1), model: custom-chat-jinzhezeng-group/gpt-5.5.
Implementation checklist
Model, loss, and optimizer support
- Train non-energy DOS, property, dipole, and polar models with matching losses (feat(jax): train non-energy task models with matching losses #5865)
- Implement JAX spin models or reject them during backend-aware validation (feat(jax): support spin models or add backend-aware validation #5866)
- Support
learning_rate.type: cosineandlearning_rate.type: wsd(fix(train): dispatch registered learning rate schedules #5776) - Consume the optimizer schema and support the applicable optimizer options (feat(jax): consume the common optimizer configuration #5867; Add HybridMuon optimizer support to JAX and TF2 for DPA4 training #5771 tracks DPA4
HybridMuon) - Implement or explicitly reject/document
frozen,pairtab, andlinear_enermodel variants (feat(jax): support or reject common composite model variants #5868)
Training support
- Derive training steps from
training.numb_epochand its aliases (feat(jax): support epoch-based and multi-task training horizons #5869) - Support multi-task
training.num_epoch_dict(feat(jax): support epoch-based and multi-task training horizons #5869) - Honor
training.stat_filewhen computing or loading statistics (fix(jax): honor training.stat_file for model statistics #5870) - Implement
training.profilingandtraining.enable_profiler(feat(jax): implement profiler lifecycle and trace output #5871) - Implement
training.tensorboard*output (feat(jax): implement TensorBoard training output #5872) - Apply
training.mixed_precisionto training (feat(jax): apply training.mixed_precision to the JAX train step #5873) - Apply
training.change_bias_after_trainingafter the final step (fix(train): share post-training bias adjustment #5745)
Reactions are currently unavailable
Activity
njzjz-bot commentedon Jul 9, 2026
Addendum: training option coverage
I re-checked the
training_data_args/validation_data_args/mixed_precision_args/training_argspath against the JAX backend on currentmaster(f73de32). Additional training-option gaps:training.numb_epoch/ aliases and multi-tasktraining.num_epoch_dict: the schema allows epoch-based training, but the shared trainer config still falls back toint(training_params["numb_steps"]), and the JAX path constructs/runs the trainer without an epoch -> step conversion. I did not find the legacy-TF-style conversion usingtotal_numb_batch, nor the multi-tasknum_epoch_dict-> model probabilities / total steps derivation.training.stat_file:TrainingTaskConfigcarriesstat_file, butdeepmd/jax/entrypoints/train.py's task factory returns(train_data, valid_data, None), and JAXcompute_or_load_stat(...)is called without astat_file_path. So a configuredstat_fileappears to be ignored in the JAX training path.training.profiling,training.enable_profiler, andtraining.tensorboard*:deepmd/jax/train/trainer.pyreads these fields into attributes, but I did not find profiler or TensorBoard writer hooks using them during training.training.mixed_precision: the trainer readsself.mixed_prec = tr_data.get("mixed_precision", None), but I did not find it applied to the model, train step, optimizer, or loss computation.training.change_bias_after_training: the trainer reads the flag, but I only found bias changes in fine-tuning setup; I did not find a post-training branch analogous to the documented “after the last training step” behavior.Addendum by OpenClaw 2026.6.11 (e085fa1), model: custom-chat-jinzhezeng-group/gpt-5.5.
fix(train): dispatch registered learning rate schedules (deepmodeling…
docs(argcheck): document backend support matrices (deepmodeling#5915)