Skip to content

openvmm: pin the refusal of earlier snapshot formats by their version - #1417

Merged
Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
agents/issue-1210-manifest-alignment
Nov 9, 2026
Merged

Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
agents/issue-1210-manifest-alignment

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Part of #1210

What and why

#1210 asked to move the snapshot manifest's state_size_bytes off field 8,
which upstream's SaveVM and RestoreVM (microsoft/openvmm#4553) had taken for
their snapshot ID, to bump the manifest's version, and to refuse NVX's version
6 manifests with an E_SNAPSHOT_VERSION recapture diagnostic before decoding
their integer field 8 as a GUID. Two things changed after it was filed:

So the manifest keeps version 2 and every field. The diagnostic that #1210
asked for was still missing, though, for a reason that the renumbering
created. Restore decoded the whole manifest (decode_snapshot_manifest in
openvmm_helpers/src/snapshot/restore.rs) before it checked the version and
magic, and earlier formats give some field numbers other types, which mesh
refuses before any check runs: version 6 keeps its machine contract, a
message, in field 11, where version 2 keeps the saved-state schema version, a
varint, and the first manifests of version 2 keep artifact digests in fields 9
and 10, where version 2 now keeps its machine contract and magic. A version 6
snapshot that NVX release v0.1.0-dev.304f8e1686b4 captured therefore failed to
restore with failed to decode snapshot manifest: decoding failed: expected a varint, rather than with the E_SNAPSHOT_VERSION that doc/run.md and
OpenVMM's snapshot format guide promise. nvx-rs, which reads a template's
manifest itself (check_format in nvx-rs/src/host/template/manifest.rs),
already checks the version and magic first, so only direct restores were
affected: openvmm --restore-snapshot and the management RPC's restore.

microsoft/openvmm#4994, two commits on ed8fc0713, the head of
feature/microvm/dev:

  • openvmm_helpers: refuse earlier snapshot formats before decoding them
    decodes the version alone, field 1 in every format, upstream's included,
    then the magic alone, field 10, whose type only the formats of version 2
    agree on, and refuses any other with [E_SNAPSHOT_VERSION] ... recapture the snapshot before it decodes the rest. This is the version-first decode that
    snapshot: align NVX manifest with upstream SaveVM/RestoreVM #1210 asked for, and it mirrors what nvx-rs's check_format does.
  • openvmm_helpers: reserve the manifest fields that upstream assigns keeps
    fields 18, 19, and 20 free, in a comment where SnapshotManifest declares
    its fields and in the Guide's rule for extending the format.

Decisions:

  • No version bump and no recapture. No field moves, so existing snapshots
    of version 2 stay restorable, and snapshots of earlier releases need
    recapture as before; doc/run.md documents that, and no conversion or
    backward-compatible restore is added.
  • Fields 9 and 10 stay reused. snapshot: align NVX manifest with upstream SaveVM/RestoreVM #1210 wanted them retired; the v4.0
    reorganization gave them to the machine contract and the magic when it
    restarted the manifest. Renumbering again would invalidate every current
    snapshot for nothing: the version-then-magic check refuses the first
    manifests of version 2, which used them for digests, by their missing magic
    before it decodes those fields.
  • No snapshot_id yet. ttrpc: add SaveVM and RestoreVM APIs openvmm#4553 is still open, so the chain
    does not take its field ahead of upstream; the roadmap now notes what a
    rebase onto it needs.

Snapshot invalidation: this change invalidates no snapshot, so #1180's note
that #1210 changes the manifest's layout and version, and that other changes
that invalidate snapshots should land with it, no longer applies. Of the open
issues queued after it, #995 admits more tokens on a platform snapshot's
command line, which leaves existing snapshots valid (its new CaptureClass
encoding only retires nvx-rs's existing templates, which nvx-rs captures
again), and #1361, #1364, #1373, #1154, #1157, and #565 only add checks or
reorder restore steps. The saved-state changes of #1182 (#1141, #1160, #1368,
#1158, #1161, #1169, #1369, #1171, #1359, and #1367, depending on its fix)
and #1184's #1163 can invalidate snapshots: they should share one compatibility
policy and one saved-state schema bump, as #1182's notes ask, so that
snapshots are recaptured once, and that batch should make a schema mismatch,
which validate_manifest_header reports today without a code, report
E_SNAPSHOT_VERSION and ask for a recapture.

Left open in #1210, once microsoft/openvmm#4553 merges and the chain rebases
onto it:

  • adopt upstream's snapshot_id (field 18) in the chain's SnapshotManifest,
    with upstream's policy for a missing ID (a zero GUID means none), and tests
    of a manifest without an ID and with a malformed one;
  • in the same pin promotion, let nvx-rs's reader of a template's manifest,
    which refuses fields that it does not know, accept field 18.

Commits:

  • openvmm: pin the refusal of earlier snapshot formats by their version
    moves the gitlink from ed8fc0713 to
    98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7, the head of
    [DO NOT REVIEW] openvmm_helpers: refuse earlier snapshot formats before decoding them openvmm#4994. The protocol vectors and CPU profiles that nvx-rs
    copies are unchanged.
  • doc: say that OpenVMM refuses earlier manifests before decoding them says
    in doc/run.md's "Snapshots of an earlier OpenVMM" that OpenVMM refuses the
    earliest manifests of version 2 by their missing magic, and every earlier
    manifest before it decodes the rest.
  • doc: map the refusal of earlier snapshot formats in the OpenVMM roadmap
    maps positions 230 and 231, unassigned, 100 commits past the mapped tip,
    with the mapped work that they change: the manifest decoding and header
    check of PR11's position 031, which PR12's position 035 moves to the opened
    snapshot directory, and their tests use the machine contract of PR17's
    position 074. They conflict with upstream main in all four of their files,
    so they follow PR11, PR12, and PR17 upstream. It also notes
    ttrpc: add SaveVM and RestoreVM APIs openvmm#4553 under the open upstream pull requests.

98e5b3ca5 is patch-equivalent to a851d870b, the same two commits on
99aac19b3, which the first validation round below tested (identical
range-diff).

OpenVMM's tests: two new tests in openvmm_helpers::snapshot::restore, and no
change to an existing test. This pull request changes no NVX test.

Landing order: land microsoft/openvmm#4994 first, by fast-forwarding
feature/microvm/dev to 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7
(OpenVMM's merge button squashes, which would leave the pin off the branch);
then merge this pull request.

Validation

Reproduction, before and after:

Check Profile Backend Host type Before After
read_snapshot_refuses_a_version_6_manifest_by_its_version local workstation Windows - 99aac19b3: fails, expected a varint 98e5b3ca5: passes
read_snapshot_refuses_an_early_version_2_manifest_by_its_magic local workstation Windows - 99aac19b3: fails, unknown wire type 3 98e5b3ca5: passes
Restore of a version 6 snapshot that release v0.1.0-dev.304f8e1686b4 captured, which that release's OpenVMM restores nvx-kvm-test kvm virtual machine ed8fc0713 and 99aac19b3: failed to decode snapshot manifest (expected a varint) 98e5b3ca5 and a851d870b: [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported (expected 2); recapture the snapshot
The same nvx-mshv-test mshv virtual machine the same the same
The same nvx-whp-test whp virtual machine the same the same

Snapshot scenarios, with OpenVMM built at the pin on each host
(python3 scripts/nvx.py test-microvm --backend BACKEND --scenario ... --processors 1): snapshot-core, run-console-restore, snapshot-tiers,
scratch-snapshot, console-snapshot, filesystem-snapshot, network-snapshot,
smp-snapshot, restore-processors, restore-memory, snapshot-rollback,
restore-downtime, platform-restore-processors, platform-zero-fill,
platform-network-policy, untiered-network-policy, and sandbox-blocks.

Profile Backend Host type At a851d870b At this head
nvx-kvm-test kvm virtual machine 17 pass 17 pass
nvx-mshv-test mshv virtual machine 17 pass 17 pass
nvx-whp-test whp virtual machine 17 pass 17 pass

The guest artifacts were those of release v0.1.0-dev.da327b5bf6d1 on KVM and
MSHV, and of v0.1.0-dev.7714de16388c on WHP, whose guest inputs match this
head's.

OpenVMM package checks for openvmm_helpers at 98e5b3ca5: on Windows,
cargo clippy --all-targets, cargo doc --no-deps, and cargo nextest run --profile agent pass (122 tests), and cargo xtask fmt --fix is clean; on
Linux (Azure Linux 3.0, nvx-kvm-test), the same pass (128 tests).

NVX gates (python3 scripts/local_gates.py --base origin/dev --run):
test-changes, markdown-links, test-layout, openvmm-vectors, and
openvmm-cpu-profiles pass on Windows; the Linux nvx-cli gate's tests pass
on nvx-kvm-test (2188 unit tests, 43 skipped, and 84 adversarial tests),
without its pip install step; openvmm-ci waits for microsoft/openvmm#4994's
CI. python3 scripts/openvmm_pin.py check-branch --topic, check-roadmap --worktree, and
python3 scripts/rebase_check.py verify pass.

Left to CI: the full NVX matrix, including the hypervisor suites on the CI
runners, and OpenVMM's CI of the pin.

Restoring a snapshot of an earlier OpenVMM directly, with
`openvmm --restore-snapshot` or the management RPC, fails with a
decoding error rather than with E_SNAPSHOT_VERSION, which run.md
promises. Manifest version 6, which OpenVMM wrote until its v4 microVM
chain restarted the manifest at version 2, gives fields 11 and 13 other
types than version 2 does, and the first manifests of version 2 give
fields 9 and 10 other types, but restore decoded the whole manifest
before it checked the version and magic:

    failed to decode snapshot manifest: decoding failed: expected a
    varint

nvx-rs, which reads a template's manifest itself, already checks the
version and magic first, so only a direct restore was affected.

Pin microsoft/openvmm#4994, 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7,
two commits on feature/microvm/dev ed8fc0713:

- openvmm_helpers: refuse earlier snapshot formats before decoding
  them. Restore decodes the version alone, then the magic alone, and
  refuses any other with E_SNAPSHOT_VERSION before it decodes the rest.
- openvmm_helpers: reserve the manifest fields that upstream assigns.
  Upstream's SaveVM and RestoreVM (microsoft/openvmm#4553) moved their
  snapshot ID from field 8, which this manifest gives to the length of
  state.bin, to field 18, and reserve fields 19 and 20, which no field
  of this manifest takes.

No field moves, so the format stays manifest version 2, and every
existing snapshot stays restorable. The protocol vectors and CPU
profiles that nvx-rs copies are unchanged.

On KVM and MSHV, on Azure Linux 3 virtual machines, and on WHP, on a
Windows virtual machine, a microVM snapshot that the OpenVMM of NVX
release v0.1.0-dev.304f8e1686b4 captured, of manifest version 6, which
that OpenVMM restores, fails to restore with the error above at the
previous pin, and with

    [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported
    (expected 2); recapture the snapshot

at this one. With this pin, the snapshot-core, run-console-restore,
snapshot-tiers, scratch-snapshot, console-snapshot, filesystem-snapshot,
network-snapshot, smp-snapshot, restore-processors, restore-memory,
snapshot-rollback, restore-downtime, platform-restore-processors,
platform-zero-fill, platform-network-policy, untiered-network-policy,
and sandbox-blocks microVM scenarios pass on all three backends.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Snapshots of an earlier OpenVMM in run.md said that OpenVMM refuses
their manifest by its version with E_SNAPSHOT_VERSION. Until the
previous commit's pin, it did so only once it had decoded the whole
manifest, which fails first for manifest version 6 and the first
manifests of version 2, whose fields have other types than version 2
gives them. With the pin, OpenVMM decodes the version and then the
format magic before the rest, so it refuses every earlier manifest
with E_SNAPSHOT_VERSION: by its version, or, for the earliest manifests
of version 2, which have no magic, by the magic that they lack.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Map positions 230 and 231, the two commits of microsoft/openvmm#4994,
which NVX now pins at 98e5b3ca5, 100 commits past the mapped tip:

- 230, openvmm_helpers: refuse earlier snapshot formats before decoding
  them. Restore decodes the manifest's version and then its magic,
  each alone, and refuses any other with E_SNAPSHOT_VERSION before it
  decodes the rest, since positions 195 and 205 restarted the manifest
  at version 2 and numbered its fields without gaps.
- 231, openvmm_helpers: reserve the manifest fields that upstream
  assigns. A comment and the snapshot format guide keep fields 18-20
  for upstream's SaveVM and RestoreVM.

Both are unassigned, and neither is a fixup of a mapped PR: they change
the manifest decoding and header check of PR11's position 031, which
PR12's position 035 moves to the opened snapshot directory, and their
tests use the machine contract of PR17's position 074. They conflict
with upstream main in all four of their files, so they follow PR11,
PR12, and PR17 upstream.

Under the open upstream pull requests, note microsoft/openvmm#4553,
which proposes no chain commit: it records its snapshot ID in field 18,
moved from field 8 after #1210, and reserves fields 19 and
20, so a rebase onto it adds the field to the chain's manifest, and
nvx-rs's template-manifest reader must accept it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the documentation updates covering snapshot-format rejection and the OpenVMM pin.

No open issues 🎉

What changed in this PR

Documents the OpenVMM pin update addressing earlier snapshot-format rejection from #1210, without changing the current manifest layout or invalidating current snapshots.

Changes:

  • Clarifies that earlier manifests are rejected by version or missing format magic before full decoding.
  • Maps the two new OpenVMM commits and their upstream dependencies.
  • Records upstream’s reserved manifest fields and the future snapshot_id integration requirements.
File Description
doc/​run.md Clarifies early rejection and diagnostics for older snapshot formats.
doc/​openvmm-upstream-roadmap.md Updates the pin, commit mapping, and upstream manifest integration notes.

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 9, 2026
Merged via the queue into dev with commit 8ebadbd Nov 9, 2026
47 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1210-manifest-alignment branch November 9, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants