Skip to content

MDEV-39150 Some data conversion macros fail to use memcpy() - #5824

Open
vaintroub wants to merge 1 commit into
11.4from
11.4-MDEV-39150
Open

vaintroub wants to merge 1 commit into
11.4from
11.4-MDEV-39150

Conversation

@vaintroub

@vaintroub vaintroub commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

https://jira.mariadb.org/browse/MDEV-39150

MDEV-37788 converted the uintNkorr() and intNstore() macros to use memcpy() and byte swap intrinsics. The floating point and short/long macros in big_endian.h, little_endian.h and myisampack.h still accessed data one byte at a time, which compilers may translate to slow byte-at-a-time loads and stores.

Changes

  • Reimplement the macros with memcpy(), and with MY_BSWAP32/MY_BSWAP64 where the byte order differs from the host. The helper macros are MY_SWAPCPY4/8 and MY_COPY_LE4/LE8/BE4/BE8. big_endian.h and little_endian.h no longer differ, so the macros move to my_byteorder.h and the two headers are removed (no longer installed).
  • float4store, float4get, float8store and float8get keep their names and store little-endian on any host. mi_float4store() etc. in myisampack.h use MY_COPY_BE4/BE8 directly and store big-endian. No new public macro names are added. change_double_for_sort() uses mi_float8store().
  • The float*get and mi_float*get macros check the destination size with compile_time_assert().
  • Remove the unused ulongget() macro and the mixed-endian floating point layout (little-endian CPU, big-endian float word order; obsolete ARM FPA) from the macros, change_double_for_sort() and dtoa.c.
  • Reimplement mach_double_read(), mach_double_write(), mach_float_read() and mach_float_write() in InnoDB with float8get(), float8store(), float4get() and float4store().

The stored formats do not change.

Testing

  • unittest/mysys/byte_order-t is extended to check the byte layout of the floating point macros (little-endian float4*/float8*, big-endian mi_float*, aligned and unaligned, -0.0 and NaN payload) and sign/zero extension of the native byte order macros: 108 checks pass on x86, and on s390x under qemu-user.
  • A small program using every macro gives byte-identical output with the old and the new headers, on x86 and on s390x (qemu-user).
  • Debug build on Windows (MSVC) and 23 mtr tests covering float/double, sort order, MyISAM, Aria, dyncol, gis, rtree and innodb_gis pass.
  • Buildbot s390x-ubuntu-2204-debug (big-endian) built this PR and finished with a successful result (build 17388); its mtr step reported warnings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resolve the macro temporary-name collision and correct the byte-order test plan count.

4 open findings
What changed in this PR

Updates floating-point and native byte-order conversions to use memcpy() and shared endian helpers while preserving stored formats.

Changes:

  • Centralizes conversion macros in my_byteorder.h.
  • Updates MyISAM, filesort, InnoDB, and dtoa conversions.
  • Removes obsolete endian-specific headers and expands byte-order tests.
File Change
unittest/​mysys/​byte_order-t.c Expands floating-point and native-order conversion tests.
strings/​dtoa.c Removes obsolete mixed-endian handling.
storage/​innobase/​include/​mach0data.inl Migrates InnoDB conversions to shared macros.
sql/​filesort.cc Uses big-endian floating-point storage.
include/​myisampack.h Adds MyISAM floating-point aliases.
include/​my_byteorder.h Adds centralized endian conversion macros.
include/​little_endian.h Removes obsolete header.
include/​CMakeLists.txt Removes deleted headers from installation.
include/​big_endian.h Removes obsolete header.

🧠 Review effort: Lite


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread include/my_byteorder.h Outdated
Comment thread unittest/mysys/byte_order-t.c
Comment thread include/my_byteorder.h Outdated
Comment thread include/myisampack.h Outdated
@vaintroub
vaintroub force-pushed the 11.4-MDEV-39150 branch 2 times, most recently from bd02033 to 5b9dd95 Compare October 9, 2026 16:09
MDEV-37788 converted the uintNkorr() and intNstore() macros to use
memcpy() and byte swap intrinsics, but the floating point and short/long
conversion macros in big_endian.h and myisampack.h still accessed data
one byte at a time, and those in little_endian.h were a mix of both.
Compilers may emit slow byte-at-a-time loads and stores for such code.

Reimplement the macros with memcpy(), and with MY_BSWAP32/MY_BSWAP64
where the byte order differs from the host. big_endian.h and
little_endian.h no longer differ, so move the macros to my_byteorder.h
and remove the two headers, which are no longer installed. Implement
mi_float4store() etc. in myisampack.h with the same helper macros.

Remove the unused ulongget() macro, and the code for the mixed-endian
floating point layout (a little-endian CPU with big-endian floating
point word order) from the macros, change_double_for_sort() and dtoa.c.
It only applied to the obsolete ARM FPA format.

Reimplement mach_double_read(), mach_double_write(), mach_float_read()
and mach_float_write() in InnoDB with float8get(), float8store(),
float4get() and float4store(), instead of copying bytes in a loop.

The stored formats do not change. The unit test byte_order-t now checks
the byte layout of the floating point macros and the sign extension of
the native byte order macros, so no MTR test is added.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Restore compatibility for the removed public headers and provide the promised _be floating-point entry points.

1 open finding
4 resolved since last review

🧠 Review effort: Lite

Comment thread include/myisampack.h
@vaintroub
vaintroub requested a review from dr-m October 9, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants