Skip to content

fix(nbdkit): disable malloc-debug checking #18705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tobias Brick (tobiasb-ms) wants to merge 1 commit into
base: 4.0
Choose a base branch
from
+110 −2
Open

fix(nbdkit): disable malloc-debug checking #18705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions base/comps/nbdkit/0001-tests-disable-glibc-malloc-check.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Tobias Brick <tobiasb@microsoft.com>
Date: Thu, 3 Sep 2026 20:17:00 +0000
Subject: [PATCH] tests: disable glibc malloc check

The glibc malloc-debug checker interposes allocation APIs through
libc_malloc_debug.so.0. SymCrypt has unversioned allocator references that
can bind aligned_alloc to libc while binding free to the debug allocator,
causing OpenSSL consumers to abort during provider initialization.

Keep malloc perturbation enabled with values in glibc's valid 1-255 range
to retain detection of uninitialized and use-after-free behavior without
enabling the incompatible checked allocator.
---
tests/Makefile.am | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 3af137f..cf49212 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -69,18 +69,15 @@ TESTS_ENVIRONMENT = \
LIBNBD_DEBUG=1 \
$(NULL)

-# Enable malloc-check as a cheap way to find some use-after-free and
-# uninitialized read problems when using glibc, and doesn't affect
-# normal operation or other libc.
+# Perturb allocated and freed memory as a cheap way to find some
+# use-after-free and uninitialized read problems.
-random = $(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))')
+random = $(shell bash -c 'echo $$(( 1 + (RANDOM % 255) ))')
if HAVE_GLIBC_234
TESTS_ENVIRONMENT += \
- LD_PRELOAD="$${LD_PRELOAD:+"$$LD_PRELOAD:"}libc_malloc_debug.so.0" \
- GLIBC_TUNABLES=glibc.malloc.check=1:glibc.malloc.perturb=$(random) \
+ GLIBC_TUNABLES=glibc.malloc.perturb=$(random) \
$(NULL)
else
TESTS_ENVIRONMENT += \
- MALLOC_CHECK_=1 \
MALLOC_PERTURB_=$(random) \
$(NULL)
endif
--
2.55.0
15 changes: 15 additions & 0 deletions base/comps/nbdkit/overlays/0003-disable-glibc-malloc-check.overlay.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SymCrypt's unversioned allocator references can split between libc and
# libc_malloc_debug when glibc's malloc-debug checked allocator is enabled.
# This causes a crash on free() during SymCrypt load.

[metadata]
category = "azl-temp-workaround"
upstream-status = "unknown"
Comment thread
tobiasb-ms marked this conversation as resolved.
bugs = [
{ url = "https://dev.azure.com/mariner-org/mariner/_workitems/edit/23389" },

@reubeno reuben olinsky (reubeno) Sep 4, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question(blocking): Did you mean to leave in a reference to the internal bug here? If there's a GitHub-located bug filed against SymCrypt that might be the more appropriate reference.

@tobiasb-ms Tobias Brick (tobiasb-ms) Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's intentional. There are many other examples of doing this with an internal bug url. There is no SymCrypt issue filed.

@reubeno reuben olinsky (reubeno) Sep 4, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see that, Tobias Brick (@tobiasb-ms), and won't block on this. But also I'm not convinced those are really the intended use of the bugs property here -- these aren't so much bugs as they are some internal tracking tickets for feature work.

If there are no upstream bugs and there's already sufficient explanation of the issue here, I don't know that the persistent reference is too useful. (And if there were additional useful context that's publishable in GitHub, we should see if we could do that instead.)

]

[[overlays]]
description = "Disable incompatible glibc malloc-debug checking in tests"
type = "patch-add"
source = "../0001-tests-disable-glibc-malloc-check.patch"
2 changes: 1 addition & 1 deletion locks/nbdkit.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ version = 1
import-commit = 'e033665ff8146ea184d31f82bc22ec3183bc212b'
upstream-commit = 'e033665ff8146ea184d31f82bc22ec3183bc212b'
manual-bump = 2
input-fingerprint = 'sha256:667190fddaa6c95a055d3d56d748cc0a73691dffa1b945541c1f742434563f9e'
input-fingerprint = 'sha256:32e17ee018de88e27ea8f74118c5458c1bf63a7b416248580a82bf79b7ee7047'
resolution-input-hash = 'sha256:466421704711c4fd3c71f0b2ed715a0e61d49e3e26f3a2637fee755795849c8e'
46 changes: 46 additions & 0 deletions specs/n/nbdkit/0001-tests-disable-glibc-malloc-check.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Tobias Brick <tobiasb@microsoft.com>
Date: Thu, 3 Sep 2026 20:17:00 +0000
Subject: [PATCH] tests: disable glibc malloc check

The glibc malloc-debug checker interposes allocation APIs through
libc_malloc_debug.so.0. SymCrypt has unversioned allocator references that
can bind aligned_alloc to libc while binding free to the debug allocator,
causing OpenSSL consumers to abort during provider initialization.

Keep malloc perturbation enabled with values in glibc's valid 1-255 range
to retain detection of uninitialized and use-after-free behavior without
enabling the incompatible checked allocator.
---
tests/Makefile.am | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 3af137f..cf49212 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -69,18 +69,15 @@ TESTS_ENVIRONMENT = \
LIBNBD_DEBUG=1 \
$(NULL)

-# Enable malloc-check as a cheap way to find some use-after-free and
-# uninitialized read problems when using glibc, and doesn't affect
-# normal operation or other libc.
+# Perturb allocated and freed memory as a cheap way to find some
+# use-after-free and uninitialized read problems.
-random = $(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))')
+random = $(shell bash -c 'echo $$(( 1 + (RANDOM % 255) ))')
if HAVE_GLIBC_234
TESTS_ENVIRONMENT += \
- LD_PRELOAD="$${LD_PRELOAD:+"$$LD_PRELOAD:"}libc_malloc_debug.so.0" \
- GLIBC_TUNABLES=glibc.malloc.check=1:glibc.malloc.perturb=$(random) \
+ GLIBC_TUNABLES=glibc.malloc.perturb=$(random) \
$(NULL)
else
TESTS_ENVIRONMENT += \
- MALLOC_CHECK_=1 \
MALLOC_PERTURB_=$(random) \
$(NULL)
endif
--
2.55.0
3 changes: 2 additions & 1 deletion specs/n/nbdkit/nbdkit.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@

Name: nbdkit
Version: 1.46.2
Release: 5%{?dist}
Release: 6%{?dist}
Summary: NBD server

License: BSD-3-Clause
Expand Down Expand Up @@ -223,6 +223,7 @@ Requires: (%{name}-selinux if selinux-policy-%{selinuxtype})
%endif


Patch0: 0001-tests-disable-glibc-malloc-check.patch
%description
NBD is a protocol for accessing block devices (hard disks and
disk-like things) over the network.
Expand Down
Loading