Skip to content

fix: avoid zero-size realloc in remove_nans - #41

Merged
tvatter merged 3 commits into
mainfrom
fix/realloc-size-zero
Sep 14, 2026
Merged

tvatter merged 3 commits into
mainfrom
fix/realloc-size-zero

Conversation

@tnagler

@tnagler tnagler commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

The problem

CRAN's valgrind check of the R package kde1d reports two errors:

https://www.stats.ox.ac.uk/pub/bdr/memtests/valgrind/kde1d/tests/testthat.Rout

realloc() with size 0
   at 0x484DB80: realloc
   by 0x...: aligned_realloc (Memory.h:221)
   by 0x...: conservativeResize (PlainObjectBase.h:426)
   by 0x...: kde1d::tools::remove_nans(...) (tools.hpp:85)
   by 0x...: kde1d::Kde1d::fit(...)
   by 0x...: fit_kde1d_cpp(...) (kde1d-interface.cpp:38)

remove_nans moves every dropped row to the end and then shrinks both vectors:

size_t last = x.size() - 1;          // unsigned
...
x.conservativeResize(last + 1);
if (weights.size() > 0)
  weights.conservativeResize(last + 1);

When every row is a drop marker -- a NaN observation, a NaN weight or a zero weight -- last wraps to SIZE_MAX and the new size is zero. (An already empty x gets there the same way.) For a dynamic vector, Eigen's conservativeResize reaches aligned_realloc, which calls std::realloc(ptr, 0). That is deprecated in C17 and undefined in C23, so valgrind flags it. Two contexts: one for x, one for weights.

The reported trigger is a zero-inflated fit whose observations are all zero. Kde1d::fit zeroes the weight of every zero observation, marks those rows NaN and calls remove_nans a second time, which empties both vectors. In R that is

kde1d(rep(0, 20), xmin = 0, type = "zero-inflated")

which the R package's test suite exercises directly. The fit paths that drop every row and then throw reach the same call, so kde1d(c(NA, NA)) and all-zero weights hit it too.

Nothing here is a memory error -- no bad access, no leak -- but CRAN treats the valgrind report as a check failure, and the call is undefined under C23 regardless.

The fix

Handle the empty result without asking Eigen to reallocate: swap in a fresh empty vector, which only exchanges storage pointers and releases the old buffer through free. Resulting sizes are unchanged, so every caller sees exactly what it saw before.

Tests

test/test.cpp gains a case covering all the drop-everything paths, including the zero-inflated fit CRAN reported.

CI: memory-checks

A zero-size realloc returns normally, so the suite could never have caught this on its own -- it only fails under a checker, and the suite has never been run under one here. Every existing job passes -DOPT_ASAN=OFF, and OPT_ASAN could not simply have been turned on instead: it only appends to CMAKE_CXX_FLAGS_DEBUG, so it is inert in the release build the tests actually run, and it cannot request anything but the address sanitizer.

So this adds a SANITIZERS cache variable taking an explicit -fsanitize= list, applied to every configuration, with -fno-sanitize-recover=all so a finding exits nonzero -- UBSan otherwise prints a diagnostic and carries on, and CI would pass. OPT_ASAN is left alone.

The new memory-checks job is a matrix of three entries, each building the release suite and running both test binaries:

entry catches
valgrind the zero-size realloc above, which no sanitizer flags
asan+lsan+ubsan GNU stack/heap errors, leaks, signed overflow, bad shifts, invalid casts
asan+lsan+ubsan Clang the same, where GCC and Clang instrument differently

Valgrind and the address sanitizer both instrument allocation and cannot share a process, hence separate entries. The leak sanitizer comes with the address sanitizer on Linux.

All three pass locally on this branch (587 + 721 assertions, no findings). Checked that they fail as intended:

  • on the test commit alone, the valgrind entry reports 7 errors;
  • temporarily introducing a signed overflow and a heap overread into the suite makes both sanitizer entries exit nonzero.

Downstream verification

Rebuilt the R package against these headers and ran its test suite under valgrind, in CRAN's configuration: [ FAIL 0 | WARN 0 | SKIP 0 | PASS 501 ], ERROR SUMMARY: 0 errors from 0 contexts (was 2).

Other sites of the same kind

Checked every other resize/conservativeResize in include/; none can reach a zero-size realloc:

  • stats.hpp:194, tab.conservativeResize(lev, 2) -- lev is 0 for a single observation, but tab is a column-major matrix whose row count changes, and Eigen cannot reallocate in place for that: conservative_resize_like_impl builds a temporary and swaps. Verified clean under valgrind.
  • interpolation.hpp:357, interpolation.hpp:378, kde1d.hpp:1258 -- plain resize, whose DenseStorage path frees and allocates, guarded on a nonzero size. No realloc involved.

tnagler and others added 3 commits September 12, 2026 11:20
`remove_nans` shrinks `x` and `weights` with `conservativeResize`, which for
a dynamic vector reaches Eigen's `aligned_realloc` and calls
`std::realloc(ptr, new_size)` directly. When every row is dropped -- every
observation NaN, every weight a drop marker, or an already empty `x` --
`new_size` is zero. `realloc(ptr, 0)` is deprecated in C17 and undefined in
C23, and valgrind reports it as an error.

CRAN's valgrind check of the R package hit this through a zero-inflated fit
whose observations were all zero: `fit` zeroes the weight of every zero
observation, marks those rows NaN, and calls `remove_nans` a second time,
which empties both vectors.

https://www.stats.ox.ac.uk/pub/bdr/memtests/valgrind/kde1d/tests/testthat.Rout

Nothing in this suite noticed, because a zero-size `realloc` returns
normally -- the tests only fail under a memory checker. So this adds both
the cases and a CI job that runs the suite under
`valgrind --error-exitcode=1`. Both currently fail there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
When every row is a drop marker, `last` wraps to `SIZE_MAX` and the new size
is zero. Eigen's `conservativeResize` reaches `aligned_realloc` for a dynamic
vector, so that became `std::realloc(ptr, 0)` -- deprecated in C17, undefined
in C23, and an error under valgrind.

Swap in a fresh empty vector instead. That only exchanges storage pointers,
and the old buffer is released by `free` when the temporary dies. The
resulting sizes are unchanged, so every caller sees what it saw before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Valgrind catches the zero-size `realloc` this branch fixes, but that is one
narrow class of defect. Signed overflow, bad shifts, invalid casts, stack
errors and leaks are all things it either cannot see or does not report, and
the suite has never been run under a sanitizer in CI -- every existing job
passes `-DOPT_ASAN=OFF`.

`OPT_ASAN` could not have been turned on instead: it only appends to
`CMAKE_CXX_FLAGS_DEBUG`, so it is silently inert in the release build the
tests actually run, and it cannot request anything but the address
sanitizer. So this adds a `SANITIZERS` cache variable taking an explicit
`-fsanitize=` list, applied to every configuration, with
`-fno-sanitize-recover=all` so a finding exits nonzero -- UBSan otherwise
prints a diagnostic and carries on, and CI would pass.

The valgrind job becomes a `memory-checks` matrix: valgrind, and
address+undefined under both GCC and Clang (the leak sanitizer comes with
the address sanitizer on Linux). Valgrind and the address sanitizer both
instrument allocation and cannot share a process, hence separate entries;
GCC and Clang disagree often enough about what UBSan instruments to be worth
both.

All three entries pass locally on this branch: 587 + 721 assertions, no
findings. Verified the sanitizer entries fail as intended by temporarily
introducing a signed overflow and a heap overread into the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tvatter
tvatter merged commit 37a1710 into main Sep 14, 2026
9 checks passed
@tvatter
tvatter deleted the fix/realloc-size-zero branch September 14, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants