Use __atomic builtins as the freelist 128-bit CAS fallback

[phongn]

Summary

ATS does not build on riscv64. The build stops in ink_queue.h with the error "unsupported processor" (#13555). The freelist head needs one of two mechanisms: a 128-bit CAS, or a hand-written pointer layout with version bits. riscv64 has neither mechanism.

This change adds a portable fallback. When the 16-byte __sync builtins are not available, the freelist uses the __atomic builtins. The __atomic builtins lower to libatomic calls. The build then links against libatomic.

Why not wait for Zacas hardware

The Zacas extension does not help here:

• GCC does not emit the 128-bit amocas.q instruction. The initial Zacas support deferred it, and it never landed.
• LLVM refuses to emit amocas.q for ABI-compatibility reasons. See the LLVM RISC-V usage notes.
• Zacas is an optional extension in the ratified RVA23 profile.

Thus, on riscv64 the 16-byte atomic operations always go through libatomic. This is true on current hardware and on future Zacas hardware.

Why not a hand-written pointer layout

A packed 64-bit layout is possible, but it is fragile on RISC-V:

• The virtual address width varies (Sv39, Sv48, Sv57). Linux kernels before the mmap window change (~6.9) returned Sv57 addresses by default. A packed pointer silently corrupts on such kernels.
• An Sv57-safe layout leaves only 6 version bits. LogObject.cc uses the version field as a reference count, so few version bits are not safe.

The __atomic fallback has no address-space assumptions and keeps the full 64-bit version field. A packed riscv64 fast path can come later as an optimization if measurements justify it.

Correctness

On riscv64, libatomic implements the 16-byte operations with internal locks. This is correct for this code because every access to a shared head_p goes through INK_QUEUE_LD and ink_atomic_cas. All of these operations serialize on the same libatomic lock, so the load/CAS retry loops keep their current semantics. The code comments now record this constraint.

The __sync path stays first in the probe order. x86-64 and aarch64 builds produce the same code as before; GCC does not inline the 16-byte __atomic CAS even with -mcx16 (GCC PR80878), so a blanket switch would regress them.

Note: libatomic exports only __atomic_* symbols. The 16-byte __sync builtins emit undefined __sync_*_16 references when the compiler cannot inline them, and nothing provides those symbols. That is why the fallback must use the __atomic builtins, and why the two paths are mutually exclusive.

Changes

• cmake/Check128BitCas.cmake: probe the __atomic builtins when the __sync probe fails, first without and then with -latomic. New variables: TS_HAS_128BIT_CAS_LIBATOMIC, TS_NEEDS_LIBATOMIC_FOR_CAS. GCC >= 14 removed the automatic -latomic on RISC-V, so the explicit link is necessary.
• ink_queue.h: INK_QUEUE_LD becomes a 16-byte __atomic_load on the fallback tier; head_p keeps the plain {pointer, int64 version} layout there.
• ink_atomic.h: an ink_atomic_cas<__int128_t> specialization uses __atomic_compare_exchange_n on the fallback tier.
• src/tscore/CMakeLists.txt: link atomic (PUBLIC) when needed.
• Tests: the old src/tscore/test_atomic.cc stress test was orphaned (no build system referenced it). It is replaced with Catch2 tests in unit_tests/test_InkAtomicList.cc: a concurrent push/pop/popall conservation test with a double-reachability detector, an ink_atomiclist_remove test, and a concurrent InkFreeList new/free test.
• Drive-by: remove the dead INK_QUEUE_NT conditional (ink_queue_nt.c no longer exists) and report the new feature flag in traffic_layout.

Performance

The fallback affects only platforms that had no working build before. A microbenchmark of the freelist access pattern (x86 proxy numbers): the lock-based path is ~1.7x slower than the inline CAS when uncontended (~37 ns vs ~21 ns per op), and equal or slightly faster under contention, because the futex parks waiters instead of burning CAS retries. ProxyAllocator thread caches absorb most freelist traffic, so the global head is not on the per-request fast path.

Testing

• Full build and test_tscore pass on x86-64 (__sync tier, unchanged codegen).
• The fallback tier was exercised on x86-64 with a forced configuration and a lock-based libatomic stand-in; all stress tests pass.
• Verification on real riscv64 hardware would be very welcome. traffic_layout info --features should show TS_HAS_128BIT_CAS_LIBATOMIC: 1 there.

Fixes: #13555

🤖 Generated with Claude Code