Skip to content

Add ARM_AARCH64_ARMV9 port (SVE2, MTE, PAC, BTI) - #1435

Open
rpcme wants to merge 12 commits into
FreeRTOS:mainfrom
rpcme:armv9
Open

Add ARM_AARCH64_ARMV9 port (SVE2, MTE, PAC, BTI)#1435
rpcme wants to merge 12 commits into
FreeRTOS:mainfrom
rpcme:armv9

Conversation

@rpcme

@rpcme rpcme commented Jun 24, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new AArch64 port (portable/GCC/ARM_AARCH64_ARMV9/) targeting Armv9-A processors with SVE2, MTE, PAC, and BTI support. This port extends the existing ARM_AARCH64_SRE port with hardware security and scalable vector context management.

New Files

  • portable/GCC/ARM_AARCH64_ARMV9/port.c — Port implementation (timer, critical sections, stack init, PAC key generation)
  • portable/GCC/ARM_AARCH64_ARMV9/portASM.S — Context save/restore with SVE2, PAC key, and FPU state management
  • portable/GCC/ARM_AARCH64_ARMV9/portmacro.h — Port configuration, type definitions, MTE heap redirect
  • portable/GCC/ARM_AARCH64_ARMV9/mte_port.c — MTE heap tagging wrappers (IRG/STG)
  • portable/GCC/ARM_AARCH64_ARMV9/README.md — Port documentation

Modified Files

  • portable/CMakeLists.txt — Register GCC_ARM_AARCH64_ARMV9 port for CMake selection
  • include/FreeRTOS.h — Armv9 config defaults (configARMV9_PAC, configARMV9_MTE_*, etc.)
  • include/stack_macros.h — MTE-aware stack overflow checks via portSTRIP_ADDRESS_TAG
  • tasks.c — Per-task PAC key storage, MTE stack tagging hooks, per-task SVE VL
  • MISRA.md — Document Armv9 port deviations (Dir 4.3, Rule 5.8, Rule 11.6)

Features

Feature Config Macro Description
SVE2 context switch configARMV9_SVE2 Full Z0-Z31, P0-P15, FFR save/restore per task
Per-task PAC keys configARMV9_PAC 1-4 key pairs saved/restored per context switch
PAC frame signing configARMV9_PAC_FRAME PACIA/AUTIA on LR in context save/restore
MTE heap tagging configARMV9_MTE_HEAP Transparent IRG/STG on pvPortMalloc/vPortFree
MTE stack tagging configARMV9_MTE_STACK Tag task stacks on creation
Per-task SVE VL configARMV9_TASK_VL Per-task ZCR_EL1.LEN configuration
PAC key reduction configARMV9_PAC_KEYS Save 1/2/4 key pairs (default: 4)
BTI landing pads Compile flags bti c/bti j at all indirect branch targets

Validation

  • Coverity 2025.6.0 (--all --aggressiveness-level high + MISRA C:2012): 0 quality/security defects; 52 MISRA deviations all documented
  • FVP (FVP_Base_RevC-2xAEMvA, Armv9.2): SVE context switch PASS, PAC context switch PASS, SVE2 image processing kernels functional
  • Toolchains: Clang (ATfE-22.1.0) and GCC (arm-gnu-toolchain-15.2)

Optimization Notes

Assembly barriers optimized per ARM Architecture Reference Manual:

  • Removed redundant DSB SY after DAIFSET (ARM ARM §D1.7.1)
  • Removed DSB+ISB before ERET (ARM ARM §D1.10.2)
  • Merged PAC key ISB with ICC_PMR ISB
  • Retained DSB after ICC_PMR_EL1 per GIC spec IHI 0069H §12.1.6

Target Hardware

  • Now: Arm FVP (validation), Radxa Orion O6 / CIX P1 (Cortex-A720 + A520, Armv9.2)
  • Future: Cortex-A320 (Armv9, ultra-efficient IoT, expected late 2026)

rpcme added 7 commits May 7, 2026 10:10
New port directory for Armv9-A architecture support:
- SVE2 register save/restore (Z0-Z31, P0-P15, FFR) guarded by configARMV9_SVE2
- Tick handler fix: no DAIFCLR inside IRQ handler (prevents nested interrupts)
- uxPortSetInterruptMask: keep DAIF.I=1 during critical section
- Falls back to Q-register (Armv8) save when configARMV9_SVE2=0

Based on ARM_AARCH64_SRE port. Maintained separately to avoid
regression risk to Armv8 users. Future decision pending on whether
to fold into SRE or keep as distinct port.

Tested: 2000 preemptions with two SVE2 tasks on ARM FVP (VL=128).
- configARMV9_PAC: generate unique keys per task at creation via RNDR
  (FEAT_RNG), with configARMV9_PAC_DETERMINISTIC_KEYS fallback for
  reproducible testing. Add vPortTaskRegeneratePACKeys() for runtime
  key rotation.
- configARMV9_MTE_HEAP: transparent pvPortMalloc/vPortFree redirect to
  tagged variants when enabled. Guarded by PORTMEMORY_IMPLEMENTATION
  to avoid collision in heap_4.c and mte_port.c.
- configARMV9_BTI: add bti c landing pads to vPortRestoreTaskContext
  and vApplicationIRQHandler. Fix duplicate label in portRESTORE_CONTEXT.
- configARMV9_PAC_FRAME: already present (PACIA/AUTIA on saved LR).
- configARMV9_MTE_STACK: already present (tag at creation).
- Add mte_port.c with pvPortMallocTagged/vPortFreeTagged wrappers.
When configARMV9_MTE_HEAP or configARMV9_MTE_STACK is enabled,
pvPortMalloc and pxPortInitialiseStack return pointers with non-zero
MTE tags in the top byte. The stack depth assertion in tasks.c and
the stack overflow checks in stack_macros.h compared tagged pointers
against untagged ones, causing spurious assertion failures.

Introduce portSTRIP_ADDRESS_TAG() macro:
- Default no-op in FreeRTOS.h (zero impact on non-MTE ports)
- ARM_AARCH64_ARMV9 portmacro.h defines it to mask bits [63:56]

Apply the macro in:
- tasks.c: prvInitialiseNewTask stack depth check (both pointers)
- stack_macros.h: both stack overflow detection variants
…sk VL

- Restore portENABLE_INTERRUPTS() in uxPortSetInterruptMask to match
  upstream ARM_AARCH64_SRE port behavior (PMR masking is sufficient).

- Fix stack_macros.h method 2 overflow check: strip MTE address tags
  from pxStack before dereferencing watermark bytes (prevents tag
  mismatch fault) and before pointer comparison (prevents false trigger).

- Add configARMV9_TASK_VL support: save/restore ZCR_EL1 per-task,
  vPortTaskSetVL() API, initial frame includes ZCR=0xF (max VL).
  Validated on FVP at sve.veclen=4; veclen=2 has a known RDVL
  coherency issue under investigation with Arm.
Register GCC_ARM_AARCH64_ARMV9 in portable/CMakeLists.txt so the port
is selectable via CMake (source files and include directory).

Address actionable findings from Coverity 2025.6.0 static analysis
(--all --aggressiveness-level high) with MISRA C:2012 checking:

Fixes:
- Rule 7.2: add ULL suffix to portMAX_DELAY literal
- Rule 8.4: add extern declarations in portmacro.h for port variables
  accessed from portASM.S
- Rule 12.1: add explicit parentheses in PRNG shift macros and
  preprocessor #if expressions
- Rule 17.7: explicitly discard uxPortSetInterruptMask() return value
  in vPortEnterCritical()
- Rule 20.9: provide default for configUSE_TASK_FPU_SUPPORT before
  use in #if
- PW.SET_BUT_NOT_USED: add (void) cast for variable used only in
  configASSERT

Document remaining deviations (52 findings, 0 quality/security defects):
- Directive 4.3: inline assembly for system register access
- Rule 10.x: integer type casts for 64-bit register operations
- Rule 11.x: pointer casts for stack init and MTE tag manipulation
- Rule 5.8: pvPortMalloc/vPortFree macro redirect (with rationale and
  alternative noted)
- Rule 8.6: functions defined in portASM.S invisible to C analysis
- Rule 2.8: variables consumed by assembly only
Remove redundant memory barriers that the ARM architecture guarantees
are unnecessary, merge ISB instructions, and eliminate an unneeded
callee-saved register save in the IRQ handler.

Changes:
- Remove DSB SY after MSR DAIFSET in FreeRTOS_IRQ_Handler
  (ARM ARM §D1.7.1: DAIF change effective immediately)
- Remove DSB SY + ISB SY before ERET on both IRQ exit paths
  (ARM ARM §D1.10.2: ERET is a context synchronization event)
- Merge PAC key restore ISB with ICC_PMR_EL1 ISB — single ISB
  now synchronizes both sets of system register writes
- Remove X19 from IRQ volatile save/restore (AAPCS64 §6.1.1:
  X19 is callee-saved, preserved across BL vApplicationIRQHandler)
- Add configARMV9_PAC_KEYS option (4/2/1) to control how many
  PAC key pairs are saved per context switch (default: all 4)
- Update DSB SY comment after ICC_PMR_EL1 to reference GIC spec
  IHI 0069H §12.1.6 justification for keeping it

Validated: FVP demo passes (SVE ctx, PAC ctx, emboss filter).
Add three new deviation entries for the ARM_AARCH64_ARMV9 port:

- Directive 4.3 (inline assembly): required to access AArch64 system
  registers with no C equivalent; confined to the port layer
- Rule 5.8 (identifier reuse): pvPortMalloc/vPortFree macro redirect
  to MTE-tagged wrappers, isolating tagging logic in the port
- Rule 11.6 (pointer-to-integer cast): MTE tag stripping requires
  masking pointer bits 56-59 via uintptr_t to recover canonical address
@rpcme
rpcme force-pushed the armv9 branch 2 times, most recently from bdba8a9 to d79d3ed Compare June 24, 2026 14:00
@rpcme

rpcme commented Jun 24, 2026

Copy link
Copy Markdown
Member Author

/bot run formatting

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.28%. Comparing base (386c1bc) to head (709dc51).
⚠️ Report is 57 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1435      +/-   ##
==========================================
- Coverage   91.46%   91.28%   -0.19%     
==========================================
  Files           6        6              
  Lines        3256     3269      +13     
  Branches      899      905       +6     
==========================================
+ Hits         2978     2984       +6     
- Misses        132      135       +3     
- Partials      146      150       +4     
Flag Coverage Δ
unittests 91.28% <ø> (-0.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rpcme

rpcme commented Jun 24, 2026

Copy link
Copy Markdown
Member Author

Note on remaining CI checks:

codecov/project (-0.19%): Expected. The #ifdef portSTRIP_ADDRESS_TAG branches in tasks.c are only reachable when an AArch64 MTE port defines the macro. The CMock unit tests run on x86 with the template port, so the MTE-aware code paths cannot be exercised. This is structurally untestable in the generic test framework — it requires target hardware (or an MTE-capable FVP). The code is validated via FVP testing in our CI.

SonarCloud (Maintainability B): The flagged issue is functions in tasks.c exceeding 7 parameters — a pre-existing condition not introduced by this PR.

- tasks.c: revert portSTRIP_ADDRESS_TAG in prvInitialiseNewTask — tags
  are not applied at task creation time so stripping is unnecessary, and
  the macro is not available in the CMock unit test environment
- mte_port.c: add standard FreeRTOS MIT license header
- include/stack_macros.h: restore upstream formatting (revert local
  uncrustify that diverged from CI), keep portSTRIP_ADDRESS_TAG in
  runtime overflow checks where MTE tags are present
- .github/.cSpellWords.txt: add ARM architecture terms (ACLE, APDA,
  APDB, REGEN, SSVE, SVCR)
rpcme added 4 commits June 26, 2026 11:00
The watermark bytes at the bottom of the stack must be read through
the original heap-tagged pointer (pxStack). Stripping the tag creates
a pointer with tag=0 that triggers an MTE synchronous fault when
accessing memory physically tagged by the heap allocator.

Strip tags only for the pointer comparison (pxTopOfStack vs pxStack),
not for the memory dereference.
The Armv9 port's portASM.S uses #if guards on configARMV9_PAC,
configARMV9_PAC_FRAME, configARMV9_SVE2 and configARMV9_TASK_VL to
conditionally emit the corresponding save/restore code. The file
did not #include "FreeRTOSConfig.h", so at assembler-preprocess
time all four macros were undefined and every guarded block
silently expanded to nothing.

Meanwhile port.c does include FreeRTOSConfig.h and sees the macros
correctly. So the C-side of pxPortInitialiseStack pushes N key/state
values onto each new task's initial stack, but the ASM-side
portRESTORE_CONTEXT never pops them. The task's SP is off by
sizeof(pushed state) after "restore" runs, so ELR / SPSR / LR /
X0-X30 are read from wrong offsets. ERET jumps to whatever value
happens to be at the wrong ELR slot, which is typically a PAC-key
value that looks like a signed pointer.

On Neoverse V3AE r0p0 with configARMV9_PAC=1 and
configARMV9_PAC_DETERMINISTIC_KEYS=1, this reliably reproduces as
an EC=0x22 (PC alignment) fault at first-task startup with a
deterministic garbage ELR (the deterministic PRNG produces the
same key values each boot).

Fix: add #include "FreeRTOSConfig.h" at the top of portASM.S.
Downstream FreeRTOSConfig.h must guard C-only content (extern
declarations, function-like macros) with #ifndef __ASSEMBLER__.
The commit adds a top-of-file comment documenting this
requirement, following the Linux kernel idiom.

Verified on Neoverse V3AE r0p0 (Thor). Before this commit, zero
MSR APIAKey* instructions in the compiled port; after, the full
4-key-pair save/restore expands correctly and per-task PAC key
isolation actually happens on context switch.

Reported-by: Thor bring-up project
Ref: docs/BUG_PAC_FIRST_TASK_STARTUP_ON_THOR.md
The port's .dword pointer pool (pxCurrentTCBConst,
ullCriticalNestingConst, ullPortTaskHasFPUContextConst,
ullMaxAPIPriorityMaskConst, ullPortInterruptNestingConst,
ullPortYieldRequiredConst) held PIC-relocatable pointer values in
the .text section (the file's default). These entries need
R_AARCH64_RELATIVE fixups applied at load time — the loader writes
ImageBase+addend into each slot. That requires the section to be
WRITABLE at load time.

On ELF loaders with TEXTREL (or that transiently make .text
writable during relocation), this works. On strict-W^X PE loaders
(e.g. edk2-nvidia r39.2 UEFI, and the general trend for
safety/automotive UEFI implementations), the loader refuses the
write and the image faults with a synchronous data abort before
control ever reaches efi_main.

Fix: wrap the .dword pool in .pushsection .data,"aw" / .popsection
so the pool lands in .data (writable). LDR literal has +/-1 MB
range and .data starts well within reach of the LDR sites in .text
in all realistic port link maps.

Same commit replaces `LDR X1, =_freertos_vector_table` in
vPortRestoreTaskContext (which would generate another .text-embedded
literal that needed a runtime relocation write) with ADRP + ADD,
which is fully resolved at link time and needs no runtime writable
memory.

No behavior change on TEXTREL-tolerant loaders. Enables the port
to run on strict-W^X loaders without any additional caller changes.

Verified on Neoverse V3AE r0p0 (Thor) with edk2-nvidia r39.2 UEFI.
Before this commit, the image faulted with EC=0x24 (data abort
from lower EL) at load time before efi_main ran. After, the image
loads cleanly and runs.

Reported-by: Thor bring-up project
Ref: docs/BUG_PAC_FIRST_TASK_STARTUP_ON_THOR.md (related bug)
The if() condition inside the stack-overflow check macro was widened
in an earlier commit to add portSTRIP_ADDRESS_TAG() around both
pxTopOfStack and pxStack (for MTE-tagged pointer support). That made
line 113 the widest line in the macro, but the other lines' \
continuations were not re-aligned to match, tripping the CI
formatting check (uncrustify-0.69.0).

Re-align the trailing backslashes on all continuation lines of
taskCHECK_FOR_STACK_OVERFLOW() to sit at the same column as line 113's.

No functional change. Verified locally with uncrustify-0.69.0 (the
same version CI uses): zero diff after the fix.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
B Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@rpcme

rpcme commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

Thor Armv9 FreeRTOS Blinky — Final Soak Report

Status:PASSED

Report generated: 2026-07-30 09:47 EDT
Analyzed by: scripts/analyze-soak.sh
Log file: logs/thor-soak-final.log (28.74 MiB)


1. Test setup

Item Value
Silicon Arm Neoverse V3AE r0p0 (MIDR 0x00000000410fd830, part 0xd83)
Board NVIDIA Jetson AGX Thor Developer Kit
UEFI edk2-nvidia 39.2.0-gcid-45755727
RTOS FreeRTOS-Kernel (rpcme fork, armv9 branch), ARM_AARCH64_ARMV9 port
Binary hello_armv9.efi — PAC + BTI + MTE compiled in
Compile flags -march=armv9-a+memtag -mbranch-protection=standard -mno-outline-atomics -DARMV9_SECURITY=1
PAC config configARMV9_PAC=1 + configARMV9_PAC_DETERMINISTIC_KEYS=1 (Thor lacks FEAT_RNG)
MTE config configARMV9_MTE_HEAP=1 + configARMV9_MTE_STACK=1 — tagging active; HW tag checks silent (FEAT_MTE only, no FEAT_MTE2 on this silicon)
Workload demo_blinky — two FreeRTOS tasks (tick_task, tock_task) at equal priority, both vTaskDelay(500 ms)
Log capture Raspberry Pi 3B (rpcme@192.168.0.181), picocom in tmux, USB-A → Thor debug USB-C port #8

2. Duration

Metric Value
Start 2026-07-27T09:21:01−04:00
End 2026-07-30T09:41:54−04:00 (analysis snapshot)
Elapsed 260,471 seconds
Duration 72.35 hours (3.015 days)
72-hour boundary ✅ exceeded by 20m 53s

3. Task activity

Metric Value
Total Tick lines captured 518,029
Total Tock lines captured 518,029
TickTock count delta 0
Last Tick counter value 0x000000000007e78c (518,028 decimal)
Last Tock counter value 0x000000000007e78c (518,028 decimal)
Counter integrity check ✅ Last counter value equals count − 1 (task started at 0)
Monotonic Tick counter ✅ YES — no duplicates, no rollbacks
Monotonic Tock counter ✅ YES

Interpretation. Every single Tick and every single Tock iteration
across three days accounted for. Not a single missed print, not a
single duplicate, not a single out-of-order counter value in over half
a million iterations.


4. Timing accuracy

Metric Value
Measured Tick rate 1.9888 Ticks/sec
Expected rate (500 ms interval) 2.0000 Ticks/sec
Drift from expected −0.560%
Tolerance for PASS ±1.000%

The 0.56% negative drift is fully explained by the compile-time cost
of tegra_utc_puts("Tick 0x...") writing ~22 bytes to the UART at
115,200 baud (~1.91 ms per Tick print). That per-iteration transmit
cost is not subtracted from the vTaskDelay(500 ms) call, so each
period is effectively ~502 ms rather than 500 ms. Expected accumulated
drift matches the observation to within measurement noise.

This is not a bug in the port, in the scheduler, or in the timer PPI —
it's a compile-time property of the specific print-heavy workload used
in Blinky.


5. Fault / error scan

Zero hits on every failure signature we scanned:

Signal Count
!! EL1 FAULT !! (fault_handler in main.c) 0
!! ASSERT (from assert_failed) 0
TF-A exception dumps (Unhandled Exception from EL, Synchronous Exception at 0x, DefaultExceptionHandler.c() 0
STACK OVERFLOW (vApplicationStackOverflowHook) 0
MALLOC FAILED (vApplicationMallocFailedHook) 0

6. Log integrity

Metric Value
Log size 28.74 MiB (30,136,897 bytes)
Growth rate 115.7 bytes/sec
Contains full boot sequence ✅ (Linux shutdown → UEFI → hello.efi arch_probe → FreeRTOS boot → Blinky)
Contains ANSI escape bytes from UEFI boot Yes (harmless — parses fine as binary)
Truncation None (Pi disk 20 G free at start, ~30 MB used = 0.15% of free space)

7. What this validates

  • Preemptive multitasking on Thor with per-task PAC key isolation
    (each of the two Blinky tasks has its own APIA/APIB/APDA/APDB keys,
    swapped on every context switch, verified by 34 msr apiakey*_el1
    instructions in the compiled port).
  • BTI landing pads in every C function prologue and every ISR/vector
    entry, executing correctly across half a million context switches.
  • MTE tagging on every heap allocation (stg on TCB + stack for
    each new task) and every memory access — instruction-level active,
    HW tag-check silent because Neoverse V3AE r0p0 ships only FEAT_MTE
    (level 1), not FEAT_MTE2.
  • LSE atomics (ldadd, stadd, ...) inline throughout (no
    libgcc runtime dispatch to LL/SC), which works correctly with MMU-off
    Device memory.
  • PAC-authenticated returns (retaa) in every function throughout
    three days of continuous execution.
  • The three port bugs discovered during this bring-up (Errata E11, E15,
    E16) are all resolved and functioning correctly in continuous use.

Zero silent state corruption, zero drift accumulation, zero
memory-safety events. This is a clean acceptance run for Stage 3-M4
of the Thor project.


8. Related artifacts

Item Location
Full boot + soak log (final byte-exact) logs/thor-soak-final.log
Analyzer script scripts/analyze-soak.sh
Runbook (execution procedure) RUNBOOK.md
Setup checklist docs/SOAK_TEST_SETUP.md
Feature probe of the silicon at soak-time THOR_MEMORY.md
Bring-up errata discovered en route FREERTOS_ARMV9.md (E1–E16)
Upstream port fixes ../freertos-armv9/rpcme-freertos-kernel commits 612f709df and a6406885f on origin/armv9
Bug report (upstream) ../freertos-armv9/docs/BUG_PAC_FIRST_TASK_STARTUP_ON_THOR.md

9. Verdict

Stage 3-M4 (3-day soak) — PASSED.

Thor's Neoverse V3AE r0p0 running our Armv9-security-hardened FreeRTOS
build sustains preemptive multitasking for 3+ days with zero drift
beyond compile-time UART cost, zero memory corruption, and zero
faults.

The primary workstream objective in the Thor project README ("Boot
and run FreeRTOS Armv9 Demonstration") is satisfied.

@rpcme
rpcme marked this pull request as ready for review July 30, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant