debugger_linux_lkm

Minimal out-of-tree kernel module scaffold for early arm64 Android GKI debugger experiments.

This repository intentionally starts with a small smoke-test module and a safe-first hidden transport scaffold:

• loads and unloads cleanly
• can optionally export a debugfs status file at /sys/kernel/debug/lkmdbg/status
• can optionally inline-hook /proc/version open as a narrow session bootstrap
• provides a stable place to add probe, breakpoint, and event plumbing next
• on load, best-effort disables the kprobe blacklist and patches common CFI slowpath symbols to simplify inline hooks on Android GKI

Current source layout:

• core/: module init, runtime symbol lookup, protection bypass helpers
• hook/: arm64 inline hook support with trampoline generation and rollback
• transport/: hidden /proc/version transport and session fd handling
• mem/: target memory read/write helpers
• ui/: debugfs status export
• include/: shared kernel/user protocol and internal declarations
• userspace/: supported liblkmdbg C SDK, packaging, and tests
• tools/: host-side CLI tools, examples, and reusable bridge helpers

Current hook support is intentionally minimal:

• single-target arm64 inline hook install/uninstall
• 4-instruction entry patch with a relocated trampoline
• relocation handling for common branch, ADR/ADRP, literal load, and test-branch instructions
• stop-machine patching for install and rollback

The current low-level arm64 hook core is vendored from KernelPatch and adapted into this repository as a kernel-internal backend:

• source basis: kernel/include/hook.h and kernel/base/hook.c
• local adapter keeps a private lkmdbg_hook_* kernel API
• KernelPatch-specific KPM/runtime integration is not included yet
• chain hooks (hook_wrap) and function-pointer hooks are not wired into lkmdbg yet

Local build

Build against the currently selected kernel headers:

make

Build against a specific kernel tree or headers directory:

make KDIR=/path/to/kernel/build

Build the user-space bootstrap test tool:

cmake -S userspace -B build/userspace -G Ninja
cmake --build build/userspace
ctest --test-dir build/userspace --output-on-failure

This produces the supported liblkmdbg.so and liblkmdbg.a libraries. See userspace/README.md for installation and API details. The older tools/driver/bridge_* helpers remain as compatibility code while the command-line tools migrate to the SDK.

Build the legacy user-space tools directly:

cc -O2 -Wall -Wextra -o tools/lkmdbg_open_session tools/lkmdbg_open_session.c tools/driver/bridge_c.c
cc -O2 -Wall -Wextra -o tools/lkmdbg_stealth_ctl tools/lkmdbg_stealth_ctl.c
cc -O2 -Wall -Wextra -pthread -o tools/lkmdbg_mem_test tools/lkmdbg_mem_test.c tools/driver/bridge_c.c tools/driver/bridge_events.c tools/driver/bridge_memory.c tools/driver/bridge_control.c
cc -O2 -Wall -Wextra -o tools/lkmdbg_sysrule_ctl tools/lkmdbg_sysrule_ctl.c tools/driver/bridge_c.c tools/driver/bridge_control.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_session_status tools/examples/example_session_status.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_mem_rw tools/examples/example_mem_rw.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_threads_query tools/examples/example_threads_query.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_regs_fp tools/examples/example_regs_fp.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_stealth_roundtrip tools/examples/example_stealth_roundtrip.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -o tools/examples/example_sysrule_combo tools/examples/example_sysrule_combo.c tools/driver/bridge_c.c tools/driver/bridge_control.c tools/driver/bridge_events.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_vma_page_query tools/examples/example_vma_page_query.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_remote_alloc_rw tools/examples/example_remote_alloc_rw.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_remote_map_rw tools/examples/example_remote_map_rw.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_phys_translate_read tools/examples/example_phys_translate_read.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_perf_baseline tools/examples/example_perf_baseline.c userspace/src/lkmdbg.c
cc -O2 -Wall -Wextra -Iuserspace/include -Iinclude -o tools/examples/example_throughput_compare tools/examples/example_throughput_compare.c userspace/src/lkmdbg.c

tools/examples/ now provides one-file runnable examples for core session-fd flows and is executed in QEMU smoke CI.

Current composable examples:

• example_session_status: session bootstrap + GET_STATUS
• example_mem_rw: target attach + remote memory read/write
• example_threads_query: target attach + thread enumeration
• example_regs_fp: freeze/thaw + arm64 GPR/FP register round-trip
• example_stealth_roundtrip: session stealth get/set/get flow
• example_sysrule_combo: syscall control + rule config + rule events
• example_vma_page_query: VMA walk + page-table view (QUERY_PAGES)
• example_remote_alloc_rw: remote alloc lifecycle + memory read/write
• example_phys_translate_read: target VA->PA translate + physical read
• example_perf_baseline: memory/translation/remote_alloc baseline throughput and latency

tools/lkmdbg_sysrule_ctl provides fast syscall rule config and CRUD helpers:

sudo ./tools/lkmdbg_mem_test sysset <pid> event+control enter+exit
sudo ./tools/lkmdbg_sysrule_ctl cfg <pid> enforce raw+rule
sudo ./tools/lkmdbg_sysrule_ctl add <pid> 173 setret -1 0 100 persistent exit
sudo ./tools/lkmdbg_sysrule_ctl add <pid> 173 rwret -7777 0 120 persistent exit
sudo ./tools/lkmdbg_sysrule_ctl add <pid> 173 rwnr 0 0 130 persistent enter 172
sudo ./tools/lkmdbg_sysrule_ctl list <pid>

add now accepts action combinations joined by +:

• setret
• stop
• rwargs
• rwnr (requires trailing rewrite_nr)
• rwret

Legacy user-space driver code is split under tools/driver/:

• session.* for session open/target/status
• events.* for event config ioctls
• memory.* for memory transfer ioctls
• driver.* as a thin C++ facade
• bridge_c.* for C session/status calls
• bridge_events.* for C event queue helpers
• bridge_memory.* for C memory xfer helpers
• bridge_control.* for C control-path ioctls
• common.hpp for shared logging/error formatting

Load and inspect:

sudo insmod lkmdbg.ko enable_debugfs=1
sudo cat /sys/kernel/debug/lkmdbg/status
sudo rmmod lkmdbg

Enable the hidden /proc/version ioctl transport:

sudo insmod lkmdbg.ko hook_proc_version=1 enable_debugfs=1
sudo cat /sys/kernel/debug/lkmdbg/status

Run the module-local inline hook smoke test on load:

sudo insmod lkmdbg.ko hook_selftest_mode=1 enable_debugfs=1
sudo cat /sys/kernel/debug/lkmdbg/status

Selftest modes:

• hook_selftest_mode=1: prepare the inline hook only
• hook_selftest_mode=2: prepare the exec pool only
• hook_selftest_mode=3: allocate an exec trampoline slot only
• hook_selftest_mode=4: populate the exec trampoline
• hook_selftest_mode=5: install the hook, but do not invoke the hooked function
• hook_selftest_mode=6: install the hook and invoke one module-local test call

The hardening bypasses are now opt-in:

sudo insmod lkmdbg.ko bypass_kprobe_blacklist=1 bypass_cfi=1

Input tracking is optional and now disabled by default unless explicitly requested:

sudo insmod lkmdbg.ko hook_proc_version=1 enable_input_tracking=1

This stage is intentionally conservative:

• normal /proc/version reads stay untouched
• the module hooks the original /proc/version open callback and only swaps file->f_op on matching opened instances
• it does not patch the shared operations table in place
• unload removes the inline hook and restores the original path

The hidden ioctl protocol now uses a bootstrap-plus-session model:

• /proc/version only handles LKMDBG_IOC_OPEN_SESSION
• that ioctl returns an anonymous session fd created with anon_inode_getfd()
• follow-up control ioctls run on the session fd instead of /proc/version
• the session fd now supports read() and poll() for queued events
• stealth control also lives on the session fd; /proc/version remains bootstrap-only

Current stealth controls are intentionally narrow:

• debugfs visibility can be toggled on or off at runtime
• module-list hide only removes lkmdbg from /proc/modules and lsmod
• optional sysfshide removes the module kobject from /sys/module/lkmdbg
• optional ownerprochide hides the owner session process from /proc views
• module-list hide is only accepted when the /proc/version hook is active, so there is still a restore path
• sysfshide is also only accepted when the /proc/version hook is active, so there is still a restore path

ownerprochide currently uses a lookup-only procfs backend:

• hook points: proc_pid_lookup, proc_pid_readdir, and proc_tgid_base_lookup
• cache invalidation: enabling owner hide drops cached /proc/<tgid> dentries with d_drop()
• no proc_pid_permission hook is used in the current design
• direct file paths such as /proc/<tgid>/status, /proc/<tgid>/comm, and /proc/<tgid>/cmdline are blocked via proc_tgid_base_lookup
• owner process self-bypass is intentionally disabled while owner hide is active

Example stealth flow:

sudo insmod lkmdbg.ko hook_proc_version=1
cc -O2 -Wall -Wextra -o tools/lkmdbg_stealth_ctl tools/lkmdbg_stealth_ctl.c
sudo ./tools/lkmdbg_stealth_ctl report
sudo ./tools/lkmdbg_stealth_ctl show
sudo ./tools/lkmdbg_stealth_ctl hide
sudo ./tools/lkmdbg_stealth_ctl restore

report also checks the current user-visible exposure surface, including:

• /proc/modules
• /sys/module/lkmdbg*
• /sys/kernel/debug/lkmdbg*
• /proc/kallsyms
• /proc/bus/input/devices

The current session ioctls include:

• LKMDBG_IOC_GET_STATUS
• LKMDBG_IOC_RESET_SESSION
• LKMDBG_IOC_SET_TARGET
• LKMDBG_IOC_READ_MEM
• LKMDBG_IOC_WRITE_MEM
• LKMDBG_IOC_QUERY_VMAS
• LKMDBG_IOC_QUERY_PAGES
• LKMDBG_IOC_QUERY_THREADS
• LKMDBG_IOC_GET_REGS
• LKMDBG_IOC_SET_REGS
• LKMDBG_IOC_SET_SYSCALL_TRACE
• LKMDBG_IOC_GET_SYSCALL_TRACE
• LKMDBG_IOC_SET_EVENT_CONFIG
• LKMDBG_IOC_GET_EVENT_CONFIG
• LKMDBG_IOC_RESOLVE_SYSCALL
• LKMDBG_IOC_FREEZE_THREADS
• LKMDBG_IOC_THAW_THREADS
• LKMDBG_IOC_GET_STOP_STATE
• LKMDBG_IOC_CONTINUE_TARGET
• LKMDBG_IOC_ADD_HWPOINT
• LKMDBG_IOC_REMOVE_HWPOINT
• LKMDBG_IOC_QUERY_HWPOINTS
• LKMDBG_IOC_REARM_HWPOINT
• LKMDBG_IOC_SINGLE_STEP
• LKMDBG_IOC_REMOTE_CALL
• LKMDBG_IOC_REMOTE_THREAD_CREATE

Generic input transform VM

Input channels are profile-free. Open one channel with LKMDBG_INPUT_CHANNEL_FLAG_CONTROLLER to become the single controller for a device; other channels remain observers. The controller loads a bounded program through the channel fd using LKMDBG_INPUT_IOC_LOAD_PROGRAM and an array of struct lkmdbg_input_vm_insn. Programs can read event context or bounded state, rewrite type/code/value, emit up to eight outputs, pass, or drop. GET_PROGRAM_INFO, RESET_STATE, and UNLOAD_PROGRAM complete the control path. Physical events pass through on VM errors, while injected event write failures are reported to the writer.

The input_event hook is reference-counted: it is installed when the first input channel is opened and removed after the last channel closes, unless enable_input_tracking=1 pins it for the module lifetime. With no channels, the VM, transform state, and observer queues consume no per-device runtime resources beyond the existing device registry.

Syscall tracing now has three session-fd modes:

• EVENT: queue LKMDBG_EVENT_TARGET_SYSCALL records
• STOP: freeze the target on enter/exit and surface a normal stop state
• CONTROL: enter-only fallback-hook mode where user space resolves the syscall before it runs

The control flow is:

• arm LKMDBG_IOC_SET_SYSCALL_TRACE with LKMDBG_SYSCALL_TRACE_MODE_CONTROL and LKMDBG_SYSCALL_TRACE_PHASE_ENTER
• wait for a LKMDBG_STOP_REASON_SYSCALL stop with LKMDBG_STOP_FLAG_SYSCALL_CONTROL
• issue LKMDBG_IOC_RESOLVE_SYSCALL with one of: ALLOW, SKIP, or REWRITE
• LKMDBG_IOC_CONTINUE_TARGET then releases the frozen target and lets the intercepted syscall complete with the chosen behavior

The helper tool now exposes the same flow directly:

sudo ./tools/lkmdbg_mem_test sysset <pid> control enter [tid] [syscall_nr]
sudo ./tools/lkmdbg_mem_test stop <pid>
sudo ./tools/lkmdbg_mem_test sysresolve <pid> allow
sudo ./tools/lkmdbg_mem_test cont <pid>

Remote call now supports optional arm64 register overrides through request flags:

• LKMDBG_REMOTE_CALL_FLAG_SET_SP
• LKMDBG_REMOTE_CALL_FLAG_SET_RETURN_PC
• LKMDBG_REMOTE_CALL_FLAG_SET_X8

LKMDBG_IOC_REMOTE_THREAD_CREATE is a synchronous convenience wrapper built on top of remote call:

• user space provides a target-side launcher helper address
• the module runs that helper on a parked frozen thread
• it auto-continues until the launcher returns and then re-freezes at the normal remote-call stop
• the ioctl returns the launcher result, created tid, remote-call id, and stop cookie

The helper tool also provides end-to-end wrappers that freeze, pick a parked thread, run the operation, print the result, and resume the target:

sudo ./tools/lkmdbg_mem_test rcall <pid> <target_pc_hex> [arg0 ... arg7]
sudo ./tools/lkmdbg_mem_test rcallx8 <pid> <target_pc_hex> <x8_hex> [arg0 ... arg7]
sudo ./tools/lkmdbg_mem_test rthread <pid> <launcher_pc_hex> <start_pc_hex> <arg_hex> <stack_top_hex> [tls_hex]

Memory transfer requests now use a single batched shape:

• one READ_MEM or WRITE_MEM ioctl carries an array of transfer ops
• a single-op request is just op_count=1
• the kernel processes ops against one target mm without faulting missing remote pages in
• large contiguous transfers are handled as one op; sparse transfers can be grouped into one batched request
• lkmdbg_mem_op.flags currently supports LKMDBG_MEM_OP_FLAG_FORCE_ACCESS for privileged access to already-present pages even when user permissions are removed
• forced writes are now kernel-permitted on already-present pages as well; user space is responsible for avoiding COW-sensitive or file-backed targets if that distinction matters

Current session events include:

• LKMDBG_EVENT_SESSION_OPENED
• LKMDBG_EVENT_SESSION_RESET
• LKMDBG_EVENT_INTERNAL_NOTICE
• LKMDBG_EVENT_TARGET_CLONE
• LKMDBG_EVENT_TARGET_EXEC
• LKMDBG_EVENT_TARGET_EXIT
• LKMDBG_EVENT_TARGET_SIGNAL
• LKMDBG_EVENT_TARGET_SYSCALL
• LKMDBG_EVENT_TARGET_MMAP
• LKMDBG_EVENT_TARGET_MUNMAP
• LKMDBG_EVENT_TARGET_MPROTECT
• LKMDBG_EVENT_TARGET_STOP

Event delivery is session-configurable:

• LKMDBG_IOC_GET_EVENT_CONFIG returns the current mask and supported mask
• LKMDBG_IOC_SET_EVENT_CONFIG applies a per-session event mask

Execution and watchpoints now have two backends behind the existing session fd API:

• hardware breakpoints/watchpoints via LKMDBG_HWPOINT_TYPE_*
• arm64 mmu breakpoints/watchpoints via LKMDBG_HWPOINT_FLAG_MMU

The MMU backend currently supports read, write, exec, and mixed combinations such as rx, rw, wx, and rwx, with one guarded page per target page.

The current MMU backend is intentionally conservative:

• it rewrites the target user PTEs in kernel space to remove the requested access class from the guarded page
• it reports the requested guard address in value0; execute faults also report the actual faulting PC in value1
• after a hit the backend behaves as a one-shot guard until user space explicitly rearms or removes it
• LKMDBG_CONTINUE_FLAG_REARM_HWPOINTS does not automatically rearm MMU hwpoints; user space must call LKMDBG_IOC_REARM_HWPOINT
• only one MMU hwpoint is allowed per target page today

LKMDBG_IOC_QUERY_HWPOINTS exposes live hwpoint state bits:

• ACTIVE: the backend is currently armed
• LATCHED: a stop was delivered and explicit rearm or remove is required
• LOST: the original mapping disappeared and the hwpoint can no longer be rearmed
• MUTATED: the guarded mapping or effective PTEs no longer match the originally armed baseline

For MMU hwpoints, MUTATED is not necessarily a crash condition. mprotect and similar mapping changes will intentionally surface as MUTATED, and some external kernel access paths can disturb a READ-style MMU trap enough to require remove-and-recreate instead of rearm.

Session fds are readable and pollable:

• read() now drains one or more whole struct lkmdbg_event_record instances per call when the caller provides a larger buffer
• poll()/POLLIN still signal that at least one queued event is available
• event.seq and event.reserved0 let user space detect queue drops and recover after bursts

Shared definitions live in lkmdbg_ioctl.h.

Example bootstrap flow:

sudo insmod lkmdbg.ko hook_proc_version=1
cc -O2 -Wall -Wextra -o tools/lkmdbg_open_session tools/lkmdbg_open_session.c tools/driver/bridge_c.c
sudo ./tools/lkmdbg_open_session

Example direct memory access flow:

cc -O2 -Wall -Wextra -pthread -o tools/lkmdbg_mem_test tools/lkmdbg_mem_test.c tools/driver/bridge_c.c tools/driver/bridge_events.c tools/driver/bridge_memory.c tools/driver/bridge_control.c
sudo ./tools/lkmdbg_mem_test selftest
sudo ./tools/lkmdbg_mem_test read <pid> <remote_addr_hex> <length>
sudo ./tools/lkmdbg_mem_test write <pid> <remote_addr_hex> <ascii_data>
sudo ./tools/lkmdbg_mem_test hwadd <pid> <tid> rwx <addr_hex> <len> mmu
sudo ./tools/lkmdbg_mem_test hwlist <pid>

The current selftest path exercises:

• batched target memory read/write and partial-progress accounting
• signal, syscall event/stop/control flows
• hardware and MMU break/watchpoints
• single-step stop delivery
• arm64 thread register get/set for GPR and FP/SIMD state (v0..v31, fpsr, fpcr)
• remote call, x8 override, and remote thread create
• stealth controls and event queue behavior

Android GKI note

The GitHub Actions workflow in this repository now covers:

• a host-side build against Ubuntu kernel headers
• an Android common kernel arm64 QEMU matrix for 5.10, 5.15, 6.1, 6.6, 6.12, and 6.18
• per-version smoke runs that exercise session bootstrap, memory/VMA/page/phys paths, runtime events, input injection with enable_input_tracking=1, and userspace examples
• per-version hook-soak runs that repeatedly load the module-local hook selftests

Two CI compatibility notes are worth preserving:

• the QEMU guest cache is intentionally strict; stale guest kernels can otherwise look reusable while missing CONFIG_VIRTIO_INPUT and make the keyboard smoke fail for the wrong reason
• the arm64 syscall/trace fallback order is intentionally version-sensitive; 5.15+ prefers do_el0_svc, and 6.18 needs the updated sched_process_exit(task, group_dead) tracepoint callback signature

For actual Android bring-up you should point KDIR at the exact android14-6.1 kernel build tree or exported module build headers used by your target device.

License

This project is licensed under GPL-2.0-or-later. See LICENSE.