Repository navigation
Guard dim_t underflow in shrink - #2128
Merged
Merged
Conversation
Collaborator
|
Could you add tests for shrinking by zero and shrinking an already empty dimension? The description also mentions a shrink/grow round-trip test, but I don’t see it in the diff. |
Contributor
Author
|
Done, added in 0064bb1:
Both cases (empty dim shrunk by zero, empty dim over-shrunk) are also exercised in my local standalone check against master, where the over-shrink surfaces as the old generic "shape contains a negative dimension" error, confirming the guard is what turns it into the clear underflow message. |
Collaborator
|
Thanks so much! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
shrink()is the counterpart ofgrow(), but the guards added togrow()in #2097 have noshrink()equivalent:shrink()still forwards the raw_shape[dim] - sizesubtraction toresize().Verified against current
master, on aStorageView(Shape{4}, DataType::INT32):shrink(0, -1)returns without an error and the shape silently becomes{5}, with the fifth element uninitialized — the exact mirror of the negative-increment bug fixed in Guard dim_t overflow in compute_size, reshape and grow #2097 (a negativegrow()silently shrank the dimension; a negativeshrink()silently grows it and exposes uninitialized memory).shrink(0, INT64_MIN)overflows the subtraction (signed overflow, so formally UB; it wraps in practice), and the wrapped negative dimension then tripscompute_size()'s "shape contains a negative dimension" — an error thrown far from the offending call, and only thanks to the wrap behavior.shrink(0, 5)on a dimension of size 4 also surfaces as "shape contains a negative dimension" fromcompute_size(), with nothing pointing at theshrink()call that caused it.What changed
StorageView::shrink()now mirrorsgrow()'s two guards:size < 0→std::invalid_argument("can't shrink a dimension by a negative size")size > _shape[dim]→std::invalid_argument("new dimension size leads to an underflowing tensor size")sizechecked to be in[0, _shape[dim]], the subtraction can no longer overflow or go negative, soresize()receives a valid size and the rejection happens inshrink()itself, with the storage left untouched.shrink(0, dim)) keeps working.Tests
Three tests in
tests/storage_view_test.cc, mirroring thegrow()tests from #2097:ShrinkRejectsNegativeDecrement—-1andINT64_MINboth rejected with the negative-size message, storage untouched.ShrinkRejectsUnderflow— shrinking by more than the current size rejected with the underflow message, storage untouched.ShrinkDecreasesShape— normal decreases on a{3, 4}view, shrinking a dimension to exactly zero, and a shrink/grow round-trip.Testing
Same approach as #2097, for the same reason: the gtest binary still fails to start in my local environment (0xC0000409, reproduces on unmodified
master), so the CI suite is the authoritative check for the tests.src/storage_view.cc,tests/storage_view_test.cc) compile cleanly with MSVC.ctranslate2library, run in A/B: againstmaster'sstorage_view.ccit reports 5 failures (the silent{4}→{5}growth above, plus the two misleading/guard-less rejections), against this patch 0 failures, with the happy paths (decrease, shrink-to-zero, shrink/grow round-trip) unchanged in both.AI usage disclosure
Per the contribution policy: this fix was developed with AI assistance (analysis of the underflow paths, drafting the guards and tests, and building/verifying locally). I have reviewed every line of the diff, understand the wrap arithmetic and the symmetry with the #2097
grow()guards, and take full responsibility for correctness, performance, and design.