Repository navigation
Conversation
This comment has been minimized.
This comment has been minimized.
1642d27 to
f27c1ef
Compare
This comment has been minimized.
This comment has been minimized.
f27c1ef to
8b0f611
Compare
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
UsingOrdering::Relaxedto publishHEAP_EXECUTABLEis risky because the value is consumed from… · New UsingOrdering::Relaxedto publishHEAP_EXECUTABLEis risky because the value is consumed from… · New The variable nameinit_data_offsethere is misleading: this value is the offset after pushing the… · New
What changed in this PR
Removes the legacy fixed-size guest heap region from snapshot memory and switches the guest to mapping heap pages from scratch on first touch, reducing snapshot size and improving performance.
Changes:
- Removes heap-size configuration and updates tests/fixtures/benchmarks to size by scratch instead.
- Bumps snapshot ABI to 8 and config schema media type to v5, updating goldens and versioning docs.
- Updates guest fault handling to map heap pages from scratch on demand and propagates heap executability via a PEB flag.
| File | Description |
|---|---|
| src/tests/rust_guests/simpleguest/src/main.rs | Updates guest OOM/exhaustion behavior to reflect scratch-backed heap semantics. |
| src/hyperlight_testing/src/lib.rs | Renames benchmark/test size constants from heap sizes to scratch sizes and adjusts types. |
| src/hyperlight_host/tests/snapshot_goldens/goldens_version.rs | Bumps goldens version to v8.0 for the new snapshot ABI/schema. |
| src/hyperlight_host/tests/snapshot_goldens/fixtures.rs | Removes heap sizing from golden snapshot config fixture. |
| src/hyperlight_host/tests/sandbox_host_tests.rs | Updates leak-related tests to constrain scratch rather than heap. |
| src/hyperlight_host/tests/integration_test.rs | Updates integration tests to assert scratch exhaustion aborts with MallocFailed. |
| src/hyperlight_host/src/sandbox/uninitialized.rs | Updates sandbox/snapshot tests to remove heap sizing and validate scratch-only layout behavior. |
| src/hyperlight_host/src/sandbox/snapshot/tripwires.rs | Bumps expected ABI/schema and updates invariant commentary around layout offsets. |
| src/hyperlight_host/src/sandbox/snapshot/file_tests.rs | Updates snapshot file tests for schema v5 and ABI 8; removes heap layout assertions. |
| src/hyperlight_host/src/sandbox/snapshot/file/mod.rs | Updates loader to accept config v5 only and removes heap from persisted layout reconstruction. |
| src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs | Adds config v5 media type and bumps snapshot ABI version to 8. |
| src/hyperlight_host/src/sandbox/snapshot/file/config.rs | Removes heap_size from config schema/layout and updates schema pins/validation. |
| src/hyperlight_host/src/sandbox/initialized.rs | Removes heap override propagation from snapshots and updates tests to size by scratch. |
| src/hyperlight_host/src/sandbox/config.rs | Removes heap size configuration API; clarifies scratch semantics and updates tests. |
| src/hyperlight_host/src/sandbox/builder.rs | Removes heap_size builder APIs; updates docs to show scratch sizing. |
| src/hyperlight_host/src/mem/mgr.rs | Updates memory manager tests to use scratch sizing constants (no heap sizing). |
| src/hyperlight_host/src/mem/layout.rs | Removes heap region from snapshot memory layout; updates offsets/PEB contents accordingly. |
| src/hyperlight_host/src/hypervisor/mod.rs | Adjusts initialise test parameters now that heap region assumptions are gone. |
| src/hyperlight_host/examples/map-file-cow-test/main.rs | Removes heap sizing from example; relies on scratch sizing only. |
| src/hyperlight_host/benches/benchmarks.rs | Re-keys benchmark configurations to scratch sizes and adds heap_first_touch benchmark. |
| src/hyperlight_guest_bin/src/lib.rs | Claims a fixed virtual heap range and reads heap executability from the PEB. |
| src/hyperlight_guest_bin/src/arch/amd64/layout.rs | Adds a static assert to ensure heap range doesn’t overlap processor control. |
| src/hyperlight_guest_bin/src/arch/amd64/exception/handle.rs | Maps heap pages from scratch on first touch; supports executable heap via PEB flag. |
| src/hyperlight_guest_bin/src/arch/aarch64/exception/handle.rs | Same as amd64: scratch-backed heap faults with executable flag support. |
| src/hyperlight_guest/src/layout.rs | Exposes heap virtual range constants and asserts no overlap with transport/stack. |
| src/hyperlight_guest/src/arch/amd64/layout.rs | Defines 512GiB-aligned heap virtual range for amd64. |
| src/hyperlight_guest/src/arch/aarch64/layout.rs | Defines 512GiB-aligned heap virtual range for aarch64. |
| src/hyperlight_common/src/mem.rs | Replaces PEB heap region field with a heap executability flag. |
| fuzz/fuzz_targets/host_call.rs | Removes heap sizing from fuzz target configuration. |
| docs/snapshot-versioning.md | Documents ABI 8/config v5 and notes heap region removal from snapshot memory. |
| docs/snapshot-oci-format.md | Updates config media type reference to v5. |
| docs/hyperlight-execution-details.md | Updates lifecycle docs to describe scratch-backed stack+heap mapping. |
| CHANGELOG.md | Records breaking changes: ABI/schema bump and removal of heap sizing APIs. |
Review effort: Lite
53bbd23 to
360b772
Compare
This comment has been minimized.
This comment has been minimized.
360b772 to
40f3bec
Compare
This comment has been minimized.
This comment has been minimized.
dummyguest halts only when it receives its expected PEB address and seed. Otherwise it reads 0x8000 to force an exit. The test passed a dummy PEB address and seed, so it relied on 0x8000 being mapped, which held only because that address fell inside the guest heap region. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
A not-present fault in the stack range always maps a fresh scratch page, so the page table walk in virt_to_phys is wasted work. Check the range first and walk only for faults outside it. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
40f3bec to
a7dc630
Compare
The guest heap is a fixed 512 GiB virtual range. Guest init claims the whole range with talc, and the page fault handler maps each heap page from scratch on first touch, the same way it maps stack pages. Heap memory is bounded by scratch rather than a fixed heap size, and untouched heap pages cost nothing. The host publishes the executable heap policy in the scratch-top metadata, and the PEB drops the heap address and size. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
The guest maps heap pages from scratch on first touch, so the host no longer reserves a zero-filled heap region in snapshot memory. This removes SandboxConfiguration::set_heap_size, DEFAULT_HEAP_SIZE, and SandboxBuilder::heap_size and get_heap_size. Scratch is the only guest memory setting. Snapshots no longer carry up to hundreds of MiB of zero pages, which speeds up sandbox creation and snapshot capture, save, and load. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
Snapshot memory no longer contains a heap region, and the PEB drops the heap address and size. The scratch-top metadata carries the executable heap policy. The config schema drops the heap size. This is a hard break. Loaders reject config v4 and older, and the goldens move to v9.0. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
Tests that sized the heap now size scratch. Allocation succeeds beyond scratch, and the guest aborts with MallocFailed when a write needs a page and scratch is full, so heap exhaustion tests expect that abort. FillHeapAndCauseException is removed because the heap can no longer run out before scratch does. The shared sandbox sizes in hyperlight-testing become scratch sizes. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
Benchmark sandbox sizes set scratch, which is the only guest memory setting. The small, medium, and large variants use 8, 64, and 256 MiB of scratch. heap_first_touch measures writes to heap the guest has not touched since the last restore. Each variant writes half its scratch. The default size is skipped because fixed reservations take most of its scratch. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
a7dc630 to
94f50e6
Compare
Benchmark ResultsMeasured commit: kvm / amd (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
kvm / intel (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / amd (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / intel (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / amd (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / intel (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
Reported by |


Remove the fixed-size guest heap region from sandbox memory. It held only zeros, and the first write to each page copied it into scratch anyway. The guest now maps each heap page from scratch on first touch, as it does for stack pages. Snapshots no longer carry the zero-filled region, and the heap is bounded only by scratch.
Each first write to a heap page no longer copies a zero page out of the snapshot, which speeds up all guest workloads that allocate.
Please review commit by commit.
Closes #1697
details
talc. The page fault handler maps a fresh scratch page on each heap page's first touch, so untouched pages cost nothing.MallocFailed.executable_heapstays a host feature. The guest sets the heap's execute permission in its own page tables, as the host did before. The hypervisor never enforced it, so a malicious guest can circumvent it. We could consider removing the feature, since it adds little security. See Consider removingexecutable_heaphost cargo feature #2003.Alternatives considered
ZeroInitpage-table entries for the heap. Rejected for performance, and because it needs a fixed heap size again. The marker exists only on 4 KiB leaf entries, so all of the heap's page tables exist up front, 1/512 of the heap size. Every snapshot captures them and every restore copies them into scratch. First touch is also slower, because the fault handler walks the page tables to find the marker instead of checking a range.Breaking changes
SandboxConfiguration::set_heap_size,SandboxConfiguration::DEFAULT_HEAP_SIZE,SandboxBuilder::heap_sizeandSandboxBuilder::get_heap_sizeare removed. Size scratch for the guest's heap use.MallocFailedon the write that needs a page. Allocations no longer return null, and Rust guests no longer panic withmemory allocation of N bytes failed.HyperlightPEB::guest_heapis removed.Benchmarks
Compared to the PR below. MSHV runs Linux 6.6.137.mshv2-5 and 6.6.100.mshv1-2. KVM runs Linux 6.17.
Each size keeps the PR below's scratch and drops its heap of 128 KiB, 8, 64 or 256 MiB. This PR then redefines the sizes as 8, 64 and 256 MiB of scratch.
heap_first_touchis new. It writes half of 8, 64 or 256 MiB of scratch, and the PR below gets a heap as large as scratch.guest_calls/heap_first_touch/mediumguest_calls/heap_first_touch/largeguest_functions_with_large_parameters/guest_call_with_large_parameterssandboxes/create_initialized/largesandboxes/sandbox_from_snapshot/largesnapshots/create/largesnapshot_files/save_snapshot/largesnapshot_files/cold_start_via_snapshot/large6.6.137 populates the old heap region on first touch and 6.6.100 at sandbox creation, so first touch gains more on 6.6.137 and creation gains more on 6.6.100.
Regressions (not really when compared to main before this stack!)
snapshots/create/defaultsnapshot_files/cold_start_via_snapshot_unverified/defaultWHP and aarch64 were not benchmarked.