Skip to content

Disable C++ RTTI by default - #1817

Merged
bkaradzic-microsoft merged 4 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:disable-rtti
Aug 7, 2026
Merged

Disable C++ RTTI by default#1817
bkaradzic-microsoft merged 4 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:disable-rtti

Conversation

@bkaradzic-microsoft

@bkaradzic-microsoft bkaradzic-microsoft commented Aug 5, 2026

Copy link
Copy Markdown
Member

Babylon Native contains no dynamic_cast or typeid expressions in Core, Plugins, Polyfills, Embedding or Apps, yet MSVC, GCC and Clang all emit RTTI records (type_info objects and the vtable slots that reach them) for every polymorphic class by default. That metadata is dead weight in the shipped binary.

Change

A single ENABLE_RTTI option, defaulting to OFF, in the top-level CMakeLists.txt:

option(ENABLE_RTTI "Emit C++ RTTI. Required by UBSan's vptr check." OFF)

if(NOT ENABLE_RTTI)
    if(MSVC)
        add_compile_options($<$<COMPILE_LANGUAGE:CXX,OBJCXX>:/GR->)
    else()
        add_compile_options($<$<COMPILE_LANGUAGE:CXX,OBJCXX>:-fno-rtti>)
    endif()
endif()

Notes on the details:

  • ENABLE_RTTI is not a new name. It is the exact variable the sanitizer block already sets via set(ENABLE_RTTI ON CACHE BOOL "" FORCE). Because that FORCE runs after the option(), -DENABLE_SANITIZERS=ON still turns RTTI back on and UBSan's vptr check keeps working. Declaring the option just makes the knob discoverable and gives it a default.
  • COMPILE_LANGUAGE:CXX,OBJCXX guard. Neither /GR- nor -fno-rtti is valid for C sources, so the guard keeps them off the C dependencies built in-tree. OBJCXX is listed explicitly so Objective-C++ (.mm) sources are covered on Apple regardless of how CMake classifies them.
  • Placement. add_compile_options at this point reaches every target added afterwards, which is all of Dependencies, Core, Plugins, Polyfills, Embedding and Apps.
  • Per-target opt-out. Dependencies that genuinely need RTTI can ask for it back. DirectXTK already does exactly this — target_compile_options(${t} PRIVATE /Wall /EHsc /GR), commented "Model uses dynamic_cast, so we need /GR (Enable RTTI)". A directory-scoped flag cannot break it.

Anyone who wants the old behaviour can configure with -DENABLE_RTTI=ON.

Dependencies that need RTTI

Two JsRuntimeHost targets do not build without it. They get it back through a babylon_native_force_rtti() helper, called right after FetchContent_MakeAvailable_With_Message(JsRuntimeHost) — Hermes is fetched from inside JsRuntimeHost, so both targets already exist at that point.

target why
v8inspector Pulls in asio, whose headers call typeid() unconditionally (any_executor.hpp, service_registry.hpp, handler_work.hpp). Clang and GCC reject that outright; MSVC only warns, which is why the Windows V8 job passed and the Android ones did not.
jsi jsi.cpp is the only translation unit that defines typeinfo for facebook::jsi::*. Hermes builds its API/hermes targets with RTTI and their vtables reference those symbols, so compiling jsi without RTTI breaks the link of libhermesvmlean.so. Node-API-JSI uses the same target name for V8JSI, so one call covers both engines.

This is the same escape hatch DirectXTK uses, just applied from the consumer side because these targets live in another repo.

Measurements

Windows x64, RelWithDebInfo, Apps/Playground/Playground.exe:

bytes
before 11,116,032
after 10,927,616
delta -188,416 (-1.70%)

RTTI content in the image, from the linker map (??_R0??_R4 symbols):

bytes % of exe
before 202,000 1.85%
after 20,632 0.19%

Before, that RTTI was spread across the whole tree — Babylon Native itself 144,280 B, DirectXTK/UrlLib/arcana 31,392 B, glslang/SPIRV 14,056 B, bgfx/bimg/bx 12,096 B — which is why a global flag is the right scope rather than a per-target one.

The 20,632 B that remain are ??_R0 type descriptors MSVC emits for throw / catch under /EHsc. That is exception-handling metadata rather than RTTI proper, and cannot be removed while exceptions are enabled.

Validation

Full build of Playground and all dependencies produced zero C4541 ('dynamic_cast' used on polymorphic type with /GR-) and zero C4530 warnings. C4541 is only a warning and not an error, so this is the check that actually proves no dynamic_cast was silently broken.

Playground validation suite on D3D11, run in eight chunks of 100 test indices:

indices ran passed failed
0-99 95 95 0
100-199 34 34 0
200-299 41 41 0
300-399 39 39 0
400-499 12 12 0
500-599 22 22 0
600-699 54 54 0
700-719 4 4 0
total 301 301 0

Only the Windows/MSVC path was built and measured locally. Every other platform, engine and toolchain is covered by CI, which is green across all 34 checks — including Android, Apple, Linux and UWP on JSC, QuickJS, V8, Hermes and JSI, plus the Windows and macOS sanitizer jobs that exercise the ENABLE_SANITIZERS path where RTTI stays on.

Babylon Native contains no `dynamic_cast` or `typeid` expressions in Core,
Plugins, Polyfills, Embedding or Apps, yet MSVC, GCC and Clang all emit RTTI
records for every polymorphic class by default. That metadata is dead weight in
the shipped binary.

Adds an `ENABLE_RTTI` option (default OFF) that passes `/GR-` on MSVC and
`-fno-rtti` elsewhere. The flag is guarded on `COMPILE_LANGUAGE:CXX` so it is
never handed to C sources. `ENABLE_RTTI` is the same variable the sanitizer
block already forces ON, so `-DENABLE_SANITIZERS=ON` keeps RTTI and UBSan's
vptr check keeps working. Dependencies that genuinely need RTTI opt back in per
target -- DirectXTK already does exactly this for its Model class.

Measured on Windows x64 RelWithDebInfo (Playground.exe):

| | bytes |
| --- | ---: |
| before | 11,116,032 |
| after | 10,927,616 |
| delta | -188,416 (-1.70%) |

RTTI content dropped from 202,000 bytes (1.85% of the image) to 20,632 (0.19%).
The remainder is exception-handling type descriptors emitted for `throw` /
`catch` under `/EHsc`, which cannot be removed while exceptions are enabled.

Validated with the Playground validation suite on D3D11: 301 enabled tests ran,
301 passed, 0 failed, across eight chunked runs. The build produces zero C4541
(`dynamic_cast` used with /GR-) and zero C4530 warnings.

Only the Windows/MSVC path was built and tested locally; Apple, Android and
Linux rely on CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f28021aa-1036-4de0-b8ad-ca88efa3429e

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The current COMPILE_LANGUAGE:CXX gating likely misses Objective-C++ (.mm) compilation units (typically OBJCXX), so RTTI may remain enabled for Apple builds despite the new default.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Adds a top-level CMake option to disable C++ RTTI by default (while keeping sanitizer builds able to force RTTI back on), to reduce shipped binary size across Babylon Native and its in-tree dependencies.

Changes:

  • Introduces ENABLE_RTTI CMake option (default OFF) near other top-level build options.
  • Applies /GR- (MSVC) or -fno-rtti (non-MSVC) globally via add_compile_options, scoped to C++ compilation units via COMPILE_LANGUAGE.
File summaries
File Description
CMakeLists.txt Adds ENABLE_RTTI and injects compiler flags to disable RTTI by default (with sanitizer compatibility).
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread CMakeLists.txt Outdated
CI showed two dependency targets cannot be compiled without RTTI:

  * `v8inspector` pulls in asio, whose headers call `typeid()`
    unconditionally (`any_executor.hpp`, `service_registry.hpp`,
    `handler_work.hpp`). Clang and GCC reject that outright; MSVC only warns,
    which is why the Windows V8 job passed and the Android ones did not.
  * `jsi` defines `typeinfo for facebook::jsi::*`. Hermes builds its
    `API/hermes` targets with RTTI and their vtables reference those symbols,
    so compiling jsi without RTTI breaks the link of libhermesvmlean.so.

Adds a `babylon_native_force_rtti()` helper and applies it to both targets
right after JsRuntimeHost is made available -- Hermes is fetched from inside
JsRuntimeHost, so both exist at that point. The same target name `jsi` is used
by Node-API-JSI for V8JSI, so one call covers both engines.

Also extends the language guard from `COMPILE_LANGUAGE:CXX` to
`COMPILE_LANGUAGE:CXX,OBJCXX` so Objective-C++ sources are covered on Apple
regardless of how CMake classifies `.mm` files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f28021aa-1036-4de0-b8ad-ca88efa3429e

@bghgary bghgary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Reviewed by Copilot on behalf of @bghgary]

Two inline.

Comment thread CMakeLists.txt Outdated
Comment thread CMakeLists.txt Outdated
bkaradzic and others added 2 commits August 7, 2026 14:04
ENABLE_RTTI was an option() whose only documented reason to be ON was
UBSan's vptr check, which ENABLE_SANITIZERS already forced. The option
therefore added nothing, and the cache entry it created made the FORCE
sticky: a build directory configured once with -DENABLE_SANITIZERS=ON
kept ENABLE_RTTI:BOOL=ON afterwards, so later configures of the same
directory silently lost the size win. Deriving a plain variable from
ENABLE_SANITIZERS drops the option and the stale-cache bug together.

babylon_native_force_rtti() silently returned for an unknown target,
which is what let Dependencies/CMakeLists.txt call it unconditionally
for v8inspector and jsi even though neither exists in, say, a QuickJS
build. It also meant an upstream rename would surface much later as a
missing-typeinfo link error. Gate each call on the engine that provides
the target and make an unresolved name a configure error.

Verified by configuring with NAPI_JAVASCRIPT_ENGINE=Chakra (neither
target defined, both calls skipped), with V8 (v8inspector generated
with RuntimeTypeInfo=true while Graphics stays false), and by toggling
ENABLE_SANITIZERS ON then OFF in one build directory, which now returns
to RuntimeTypeInfo=false instead of staying enabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 88569c10-a7ff-4373-9a58-afa9c68b8c09
target_compile_options already rejects an unknown target and names it,
so with both call sites gated the explicit check only restated the
error less precisely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 88569c10-a7ff-4373-9a58-afa9c68b8c09
@bkaradzic-microsoft
bkaradzic-microsoft enabled auto-merge (squash) August 7, 2026 23:08
@bkaradzic-microsoft
bkaradzic-microsoft merged commit d5e4268 into BabylonJS:master Aug 7, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants