Skip to content

refactor(ase): consume external abacuslite plugin - #8175

Open
QuantumMisaka wants to merge 19 commits into
deepmodeling:developfrom
QuantumMisaka:integrate/external-abacuslite
Open

QuantumMisaka wants to merge 19 commits into
deepmodeling:developfrom
QuantumMisaka:integrate/external-abacuslite

Conversation

@QuantumMisaka

@QuantumMisaka QuantumMisaka commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
  • If this changes interfaces/ASE_interface/, source/source_relax/socket_*, or the socket metadata schema, I linked the abacuslite counterpart and updated the compatibility contract.
  • I have requested any needed governance exception below.

Linked Issue

Fix #8165. This PR proposes fully external ASE-plugin integration without a submodule. Long-term maintenance arrangements are being discussed separately; this change does not appoint maintainers or transfer repository ownership. LTS workflow adoption remains a separate follow-up.

Plugin counterpart: abacuslite PR #11, released as 1.2.2. See its public API contract, socket contract, and maintenance options.

Unit Tests and/or Case Tests for my changes

  • Current follow-up tested with installed PyPI abacuslite==1.2.2 (Python 3.10 / ASE 3.28):
    • cd interfaces/ASE_interface/tests && python -m unittest test_external_provenance test_harness_facts test_native_md_fixture test_portable_runner -v: 20 passed.
    • Parse .github/workflows/ase_plugin_test.yml with yaml.safe_load; run bash -n on each embedded shell step: passed.
    • Compare the workflow wheel SHA256 with PyPI metadata and the verified release artifact: matched.
    • python3 tools/03_code_analysis/agent_governance_check.py --staged: no findings; checking the full follow-up from 1bf602a to the new head also reports no findings.
    • git diff --check: passed.
  • Installed-package and paired-engine release evidence: abacuslite v1.2.2 tag CI passed all eight jobs, including 9/9 real checks on develop 1bf602a and 9/9 on LTS 725bf475, using public-interface harness 7efe6422e. This establishes the released plugin/harness pair; it is separate from the updated PR's checks.
  • Current PR head ac64e83f7 passed all reported hosted checks, including GNU/Intel/CUDA builds, CPU/GPU tests, PyTest, governance, pre-commit and both Read the Docs builds. The ASE job passed 20 offline harness checks and all nine real-engine cases with published 1.2.2, including magnetic and socket handoff. It tested PR merge checkout a410a97e32ade33831d7240b0a38e2a48c5f13cb (containing head ac64e83f7); the installed wheel identity and 142 retained raw-output hashes were verified after download. No additional local engine rebuild was needed for the version/hash and documentation-only follow-up.
  • Existing engine-side schema test: SocketDriverTest.PropertiesMetadataDeclaresVersionedSchema covers the v1 schema string.

What's changed?

  • Remove the vendored interfaces/ASE_interface/abacuslite package, its pyproject.toml, examples and internal developer guide. The plugin is independently installed; source archives remain complete without a submodule.
  • Keep interfaces/ASE_interface/tests as the engine-owned compatibility harness. It verifies installed distribution provenance and wheel/runtime identity, then checks SCF, ASE relaxation/MD, native MD, bands, magnetic moments and socket handoff against independent raw-output facts.
  • Exercise the plugin's public behavior: no private reader-state probes, asset-discovery calls, ASE server/process manipulation or global magnetic-property patches. Use explicit assets and public socket close, retain whole-case process-group timeouts and treat nonzero client-exit warnings as failures. Report schema v2 records engine version and oracle format; numerical thresholds are unchanged.
  • Default ASE CI to published abacuslite==1.2.2, with wheel SHA256 d4675aae2f70b99a6e18882e2c4ecf7f67f57eae10f5ba3e01622e879c6b8caf. Manual dispatch still accepts a reviewed plugin candidate. The workflow builds ABACUS and requires every interface case; weekly checks supplement PR checks.
  • Document the parser/API ownership boundary and abacus.socket.properties.v1: additive fields stay in v1; breaking wire changes require a new schema version and a transition in both projects. Add kPropertiesSchema and its engine-side contract test.
  • Add a PR-template reminder to link the plugin counterpart for ASE/socket changes; point installation and examples to abacuslite.

Governance Notes

  • INPUT/docs changes: docs/advanced/interface/ase.md, interfaces/ASE_interface/README.md and interfaces/README updated. No INPUT parameter metadata or generated documentation changes.
  • Core module impact: source/source_relax/socket_utils.{h,cpp} only moves the existing schema literal into kPropertiesSchema; wire values and numerical behavior are unchanged. No ESolver, HSolver, ElecState, Hamilt, Operator or Psi changes.
  • The public-interface follow-up and stable wheel pin were independently reviewed before pushing. Maintenance staffing and repository location remain open community decisions.
  • Exceptions requested: none.

@QuantumMisaka QuantumMisaka added Compile & CICD & Docs & Dependencies Issues related to compiling ABACUS Interfaces Interfaces with other packages Tests/Examples Issues/PR related to unit tests and integrate tests labels Oct 18, 2026

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of 6b37c6b1c5e168540540f11f80974298cc64ff42. Two nonblocking follow-ups below. No builds or runtime tests were run, and CI was not independently verified.

Comment thread docs/advanced/interface/ase.md Outdated
Comment thread interfaces/ASE_interface/tests/run_engine_compatibility.py
mohanchen and others added 4 commits October 19, 2026 08:45
The retained paragraph claimed that starting MD from Python is unsupported, but the engine harness now requires native_md.py (engine runs its own MD, plugin reads the trajectory) alongside the ASE-driven Langevin case. State both validated paths and keep calculation types outside the plugin surface out of scope.
capture() only signalled the isolated group on timeout. When a case exited nonzero while a descendant stayed in the group, the runner returned and later cases could start beside the leftover. Check the group after any exit, escalate SIGTERM to SIGKILL, and reap it. Add a regression where the leader exits nonzero before a SIGTERM-ignoring child.
@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

@Growl1234 Can you have a review? thanks !

@Growl1234

Copy link
Copy Markdown
Collaborator

Ah OK, I will have a look soon. From the first glance I don't see problem.

@kirk0830

Copy link
Copy Markdown
Collaborator

I'm concerned that this PR transfers the maintenance responsibility from the original maintainers to you. This positions you as a central point of contact for all future updates and bug fixes. Do you think this is always a sustainable approach?

@Growl1234 Growl1234 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, downstream code and external tests should never rely on the private implementation details of an external package. Once those internals change, the mechanism is broken. Please double-check your abacuslite code and clarify which interfaces are intended to be public and stable.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

@Growl1234 Following up on your review, this PR now includes the public-interface follow-up and uses published abacuslite 1.2.2 with its verified wheel SHA256. The updated head is ac64e83f7.

The plugin now owns an explicit public API and compatibility contract, covering supported calculator/profile, I/O and k-point entry points, units, atom ordering, missing values and deprecation rules. Its public-boundary tests exercise real parsers and calculator handoffs, including mixed develop/LTS formats and noncollinear results.

On the ABACUS side, the tests no longer inspect private reader switches or ASE server/process objects, use asset-discovery helpers, or patch ASE's global magnetic-property definitions. They retain independent raw-output assertions and numerical thresholds, with public socket close covered by the existing process-group timeout and nonzero-exit checks. Parser ownership and the canonical contract are linked from interfaces/ASE_interface.

Removing the magnetic registry patch exposed a result-projection defect, which was fixed in abacuslite 1.2.2 with a regression. Validation of this PR update: 20 offline harness tests passed against the published package, and the current PR ASE CI passed all nine real-engine cases, including magnetic and socket checks. The job tested the PR merge checkout a410a97e containing this head; its reported wheel identity and 142 raw-output hashes were verified. Separately, the 1.2.2 tag CI passed both installed-package/consumer lanes and all nine real-engine checks on each pinned develop/LTS snapshot.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

@kirk0830 Regarding your maintenance concern, I agree that a personal repository alone does not establish sustainable maintenance. Alongside the published API contract and tests, I have added a concrete options document for discussion:

  1. Keep the current repository temporarily, with willing additional maintainers, agreed review/release access and a documented handover route.
  2. Transfer the independent package to a community repository if accepted, preserving its package identity and independent release cadence.
  3. Maintain abacuslite as an ABACUS monorepo subproject, with its own Python package versions and PyPI release pipeline.

All three keep one authoritative implementation and pip install abacuslite without requiring an ABACUS build. The API contract, plugin-owned behavioral tests and ABACUS-owned real-engine checks are already implemented and can be retained under any of these arrangements.

My suggested interim step is shared maintenance in the current repository, if volunteers accept concrete roles; community hosting or a monorepo subproject can then be chosen on its practical merits. No appointment or community commitment is assumed. Repository access, PyPI ownership and publishing permissions are separate responsibilities. Before describing the arrangement as sustainable, we should identify an available alternate and demonstrate that someone other than the principal contributor can prepare and validate a release.

Could we discuss which arrangement the participating maintainers are willing to operate, who can accept review/triage/release responsibilities, and what evidence is needed before removing the embedded source? I would like to settle that maintenance arrangement explicitly before upstream adoption.

@Growl1234 Growl1234 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good from integration side.

@kirk0830

Copy link
Copy Markdown
Collaborator

@QuantumMisaka My suggestion would be:

  1. transferring this Python package to be under the deepmodeling community
  2. rename it as ase-abacus-lite (following other repo conventions)
  3. managed still in the open-source way, invite @mohanchen @dyzheng @PeizeLin and other developers as the maintainer

How do you think?

@kirk0830

kirk0830 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

I'm concerned that this PR transfers the maintenance responsibility from the original maintainers to you. This positions you as a central point of contact for all future updates and bug fixes. Do you think this is always a sustainable approach?

Plus, if the abacuslite is moved out from this repo at the time after which it is still possible to have refactor on i/o, the CI/CD workflow of abacuslite in this repo will block the progress of the refactor and suppress the efficency on refactor, because there should always be a person to stay near the "merge" button in the abacuslite individual repo to wait for the update due to the change from ABACUS side. What's worse, there will also be some delay after the whl is updated on Pypi.org, if the CI/CD workflow in this repo have a step like pip install abacuslite.

I think this PR can only be considered after the i/o of ABACUS is stable, no further frequent breaking changes on format is made. Otherwise there will be a time the abacuslite blocks the develop of ABACUS too much and its related CI/CD workflow will be removed and, further destroys the synchronicity between abacuslite and ABACUS.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

@kirk0830 Thanks for the suggestions. Community hosting under deepmodeling, the proposed ase-abacus-lite name, and inviting the original developers to participate in maintenance all make sense to me.

I also understand your concern about the timing of the extraction. While breaking output-format changes are still frequent, moving the parser into a separate repository introduces additional coordination for review, merging and releases. Community ownership would help with shared responsibility, but would not by itself remove that coordination cost.

There is no urgency to merge this PR. I suggest keeping it open for discussion and revisiting the extraction when the output formats are more stable and we have agreed on a practical shared-maintenance workflow. In particular, we should avoid making routine ABACUS development depend on an external maintainer being immediately available or on a new PyPI release.

I’m happy to continue discussing the repository organization and maintenance arrangements here.

@mohanchen

Copy link
Copy Markdown
Collaborator

Maybe we can discuss this PR after the release of ABACUS 3.11 version

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Compile & CICD & Docs & Dependencies Issues related to compiling ABACUS Interfaces Interfaces with other packages Tests/Examples Issues/PR related to unit tests and integrate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

abacuslite should be independent repo as submodule in abacus/interfaces

5 participants