fix: preserve 16-bit depth in LDM3D rgblike_to_depthmap#14211
Closed
sergioperezcheco wants to merge 1 commit into
Closed
fix: preserve 16-bit depth in LDM3D rgblike_to_depthmap#14211sergioperezcheco wants to merge 1 commit into
sergioperezcheco wants to merge 1 commit into
Conversation
The method combined two 8-bit channels into a 16-bit value using int32 arithmetic but then cast the result back to the original uint8 dtype, truncating values > 255. This caused numpy_to_depth to fail with 'buffer is not large enough' when building I;16 PIL images. Fix: return uint16 from both the torch and numpy branches instead of casting back to the input dtype.
Author
|
Closing this as a duplicate of #14200 by @chuenchen309, which was opened ~30h earlier and addresses the same bug from #14206. Both PRs converge on uint16 for the numpy path; the only difference is mine also casts the torch path to uint16 whereas #14200 keeps int32 there. Happy to leave that dtype question to the maintainers rather than fragment review across two PRs. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
VaeImageProcessorLDM3D.rgblike_to_depthmapcombines two 8-bit channels into a 16-bit depth value using widened arithmetic (int32), but then casts the result back to the originaluint8input dtype. This truncates any depth value > 255 to the low byte.Downstream,
postprocess(output_type="pil")callsnumpy_to_depth, which constructs amode="I;16"PIL image expecting 2 bytes per pixel. The truncateduint8buffer is half the expected size, causing:Fix
Both the
torch.Tensorandnp.ndarraybranches now returnuint16instead of casting back to the input dtype. This preserves the full 16-bit depth range [0, 65535] and is consistent with whatnumpy_to_depthexpects.Removed the now-unused
original_dtypevariable in both branches.Test
Added
test_rgblike_to_depthmap_preserves_uint16_rangeverifying:uint8input with high byte=1, low byte=0 →uint16output with value 256Fixes #14206