Skip to content

socket_tracer and perf_profiler BPF programs fail to initialize on GKE 1.34 (COS 125, kernel 6.12.55) #2344

New issue
New issue
Open
Open
socket_tracer and perf_profiler BPF programs fail to initialize on GKE 1.34 (COS 125, kernel 6.12.55)#2344
Assignees
ddelnano

Description

@utkarsh-ginco
utkarsh-ginco
opened on Apr 2, 2026

Describe the bug
After upgrading a GKE cluster from 1.33 to 1.34, Pixie's socket_tracer and perf_profiler probes fail to initialize, causing all network-related data tables (conn_stats, http_events, dns_events) to be missing. Scripts like px/cluster fail with "Table 'conn_stats' not found."

GKE 1.34 uses Container-Optimized OS (COS) milestone 125, which ships with Linux kernel 6.12.55. COS does not include host kernel headers at /lib/modules/6.12.55+/build. The PEM falls back to the closest bundled packaged header (6.1.8), which is too far from kernel 6.12 for the socket_tracer BPF program to compile successfully.

As per #2275, the Pixie team runs primarily on GKE COS nodes. GKE 1.34 with COS 125 is now the current default version in GKE release channels, so this affects all GKE COS users who upgrade.

To Reproduce
Steps to reproduce the behavior:

  1. Deploy Pixie on a GKE 1.34 cluster with COS node images
  2. Check stirling_error table: socket_tracer shows "Unable to initialize BCC BPF program"
  3. Run px/cluster — fails with "Table 'conn_stats' not found"

Expected behavior
PEM should either bundle compatible headers for kernel 6.12.x, download them from Google's cos-tools bucket (https://storage.googleapis.com/cos-tools//), or use CO-RE/BTF to avoid the header dependency.

Screenshots
If applicable, add screenshots to help explain your problem. Please make sure the screenshot does not contain any sensitive information such as API keys or access tokens.

Logs
Please attach the logs by running the following command: (only logs deemed relevant to the issue have been attached. We can share more logs if necessary)

# PEM startup — version and kernel detection
I20260402 06:52:11.168628  6953 pem_main.cc:69] Pixie PEM. Version: v0.14.15+Distribution.623e988.202501242347.1.RELEASE.jenkins, id: 5661a75d-639d-44ef-9440-2d829ea172a1, kernel version: 6.12.55
I20260402 06:52:11.168690  6953 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.55+
I20260402 06:52:11.168726  6953 stirling.cc:958] Creating Stirling, registered sources: [process_stats, network_stats, jvm_stats, socket_tracer, perf_profiler, proc_exit_tracer, stirling_error]

# Header resolution — host headers not found, falls back to bundled 6.1.8
I20260402 06:52:11.169289  6953 source_connector.cc:35] Initializing source connector: socket_tracer
I20260402 06:52:11.169345  6953 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.55+
I20260402 06:52:11.171020  6953 linux_headers.cc:369] Not Found : Could not find 'source' or 'build' under /lib/modules/6.12.55+.
I20260402 06:52:11.171041  6953 linux_headers.cc:215] Looking for host Linux headers at /host/lib/modules/6.12.55+/build.
I20260402 06:52:11.171056  6953 linux_headers.cc:372] Not Found : Did not find the host headers at path: /host/lib/modules/6.12.55+/build, No such file or directory.
I20260402 06:52:11.171062  6953 linux_headers.cc:314] Attempting to install packaged headers.
I20260402 06:52:11.171190  6953 linux_headers.cc:320] Using packaged header: /px/linux-headers-x86_64-6.1.8.tar.gz
I20260402 06:52:11.498617  6953 linux_headers.cc:345] Successfully installed packaged copy of headers at /lib/modules/6.12.55+/build

# BPF compilation fails — 6.1.8 headers incompatible with 6.12.55 kernel
In file included from src/stirling/source_connectors/socket_tracer/bcc_bpf/socket_trace.c:25:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/net.h:1:
In file included from include/linux/net.h:23:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1151:20: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        void __rcu      *slots[XA_CHUNK_SIZE];
include/linux/xarray.h:1697:38: error: no member named 'marks' in 'struct xa_node'
        unsigned long *addr = xas->xa_node->marks[(__force unsigned)mark];
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

# socket_tracer fails to initialize
W20260402 06:52:14.420778  6953 stirling.cc:416] Source Connector (registry name=socket_tracer) not instantiated, error: Internal : Unable to initialize BCC BPF program: Unable to initialize BPF program

App information (please complete the following information):

  • Pixie version: 0.14.15 (operator chart 0.1.7)
  • Cloud: Cosmic Cloud (getcosmic.ai)
  • K8s cluster version: GKE 1.34.4-gke.1193000
  • Node OS: Container-Optimized OS (COS 125)
    • Kernel: 6.12.55+

Additional context

Available data tables (missing network tables):

  • present: process_stats, network_stats, jvm_stats, proc_exit_events, probe_status, stirling_error
  • missing: conn_stats, http_events, dns_events, mysql_events, pgsql_events, redis_events

Activity

  1. ddelnano
    self-assigned this
    on Apr 3, 2026

  2. ddelnano commented on Apr 3, 2026

    @ddelnano
    ddelnano
    on Apr 3, 2026
    Member

    Hey @utkarsh-ginco, you're right that we are behind on updating our bundled linux kernel headers. Pixie has dynamic tracing features that require BCC, which is why we haven't migrated to CO-RE.

    I'm working on updating the kernel headers now. It probably won't be until sometime this weekend that I'll have that updated, but I'll keep you updated on the progress of it.

  3. utkarsh-ginco commented on Apr 3, 2026

    @utkarsh-ginco
    utkarsh-ginco
    on Apr 3, 2026
    Author

    Got it, thanks!

  4. ddelnano commented on Apr 6, 2026

    @ddelnano
    ddelnano
    on Apr 6, 2026
    Member

    @utkarsh-ginco I should have a PR open by tomorrow.

  5. utkarsh-ginco commented on Apr 9, 2026

    @utkarsh-ginco
    utkarsh-ginco
    on Apr 9, 2026
    Author

    @ddelnano
    How does the timeline for merging this PR look like? #2350

  6. ddelnano commented on Apr 10, 2026

    @ddelnano
    ddelnano
    on Apr 10, 2026
    Member

    @utkarsh-ginco I'm hoping it will be reviewable by tomorrow, but it will take time for a new release to be made.

  7. utkarsh-ginco commented on Apr 10, 2026

    @utkarsh-ginco
    utkarsh-ginco
    on Apr 10, 2026
    Author

    @ddelnano Understood, thank you for the update!

    How about the timeline for the release? When do you think the fix can be released, once the fix is merged? We are at a stage where some of our production environment clusters have already been upgraded to GKE 1.34.1, and some of our monitoring stack depends on Pixie.

  8. utkarsh-ginco commented on Apr 24, 2026

    @utkarsh-ginco
    utkarsh-ginco
    on Apr 24, 2026
    Author

    Hi @ddelnano. Any updates on the fix?

  9. utkarsh-ginco commented on May 11, 2026

    @utkarsh-ginco
    utkarsh-ginco
    on May 11, 2026
    Author

    Hi @ddelnano
    It would be really helpful if you could let me know the updated timeline. And please let me know if I can help in any way.
    Thanks for working on the fix! 🙏

  10. ddelnano commented on May 12, 2026

    @ddelnano
    ddelnano
    on May 12, 2026
    Member

    Hi @utkarsh-ginco, there is a vizier pre-release build running that includes that change (source). You will be able to update your deployment to use those images once it's available. I'll post the image tags here once it's finished.

    I'm still working on getting the PR merged fully but it's been tested on all the important k8s variants.

  11. ddelnano commented on May 13, 2026

    @ddelnano
    ddelnano
    on May 13, 2026
    Member

    @utkarsh-ginco you can install this pre-release version by with the px cli: px deploy -v 0.14.16-pre-ddelnano-update-kernel-versions.38. That will pull the 0.14.16-pre-ddelnano-update-kernel-versions.38 tagged set of images under the hood.

  12. ddelnano
    added a commit that references this issue on May 15, 2026

    Trim linux headers to LTS kernel versions for 4.x and 5.x and include…

    c674687
  13. k8sstormcenter-buildbot
    added a commit that references this issue on May 17, 2026

    Trim linux headers to LTS kernel versions for 4.x and 5.x and include…

    afa4a9b

  14. tusharguptaJ commented on May 20, 2026

    @tusharguptaJ
    tusharguptaJ
    on May 20, 2026

    @ddelnano i have installed the pre-release version using cli px deploy -v 0.14.16-pre-ddelnano-update-kernel-versions.38, but the issue still persist. My orignal issue #2275
    The cli deploy commanded ended with

    Waiting for Pixie to pass healthcheck
 ✕    Wait for PEMs/Kelvin  ERR: unexpected status for PEM 'vizier-pem-wnd6d': 'Failed'
Failed Pixie healthcheck error=unexpected status for PEM 'vizier-pem-wnd6d': 'Failed'

    The query broker log shows,

    time="2026-05-20T14:04:34Z" level=info msg="Running script" query_id=049cab35-ba57-4033-85f4-efa674aaadd1 query_name=px/cluster
time="2026-05-20T14:04:34Z" level=error msg=InvalidArgument error="rpc error: code = InvalidArgument desc = Table 'conn_stats' not found." query_id=049cab35-ba57-4033-85f4-efa674aaadd1
time="2026-05-20T14:04:39Z" level=info msg="Running script" query_id=1c9443c3-8a3b-46c4-80e6-c083abbd224b query_name=healthcheck

  15. ddelnano commented on May 20, 2026

    @ddelnano
    ddelnano
    on May 20, 2026
    Member

    @tusharguptaJ please collect your vizier's logs via px collect-logs and upload them to this thread. The "table not found" errors are too coarse grained to tell if what you are experiencing is the same as the parent thread. The vizier-pem logs will confirm if that's the case.

  16. tusharguptaJ commented on May 20, 2026

    @tusharguptaJ
    tusharguptaJ
    on May 20, 2026

    Here's the log @ddelnano

    Click to view Pixie PEM logs 
    I20260520 13:46:45.374631 2907970 env.cc:47] Started: /app/src/vizier/services/agent/pem/pem
Started external stacktrace collection signal processor thread
I20260520 13:46:45.375227 2907970 kernel_version.cc:199] Found Linux kernel version using .note section.
I20260520 13:46:45.375303 2907970 pem_main.cc:69] Pixie PEM. Version: v0.14.16-pre-ddelnano-update-kernel-versions.38+Distribution.8f52d92.202605121359.2.RELEASE.jenkins, id: 82ee9848-0562-499f-a49a-eb0e2bbc9613, kernel version: 6.12.79
I20260520 13:46:45.375456 2907970 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 13:46:45.375593 2907970 stirling.cc:958] Creating Stirling, registered sources: [process_stats, network_stats, jvm_stats, socket_tracer, perf_profiler, proc_exit_tracer, stirling_error]
I20260520 13:46:45.375681 2907970 system_info.cc:42] Location of proc: /proc
I20260520 13:46:45.375712 2907970 system_info.cc:43] Location of sysfs: /sys/fs
I20260520 13:46:45.375726 2907970 system_info.cc:44] Number of CPUs: 4
I20260520 13:46:45.375847 2907970 system_info.cc:35] /proc/version:
Linux version 6.12.79 (builder@buildkitsandbox) (x86_64-bottlerocket-linux-gnu-gcc (Buildroot 2025.05.1) 13.4.0, GNU ld (GNU Binutils) 2.43.1) #1 SMP PREEMPT_DYNAMIC Tue Apr 14 00:11:24 UTC 2026
I20260520 13:46:45.376019 2907970 system_info.cc:35] /host/etc/os-release:
NAME=Bottlerocket
ID=bottlerocket
VERSION="1.59.0 (aws-k8s-1.35)"
PRETTY_NAME="Bottlerocket OS 1.59.0 (aws-k8s-1.35)"
VARIANT_ID=aws-k8s-1.35
VERSION_ID=1.59.0
BUILD_ID=73b3be44
VENDOR_NAME=Bottlerocket
HOME_URL="https://github.com/bottlerocket-os/bottlerocket"
SUPPORT_URL="https://github.com/bottlerocket-os/bottlerocket/discussions"
BUG_REPORT_URL="https://github.com/bottlerocket-os/bottlerocket/issues"
DOCUMENTATION_URL="https://bottlerocket.dev"
I20260520 13:46:45.376056 2907970 probe_cleaner.cc:102] Cleaning probes from /sys/kernel/debug/tracing/kprobe_events with the following marker: __pixie__
I20260520 13:46:45.376152 2907970 probe_cleaner.cc:117] All Stirling probes removed (count=0)
I20260520 13:46:45.376250 2907970 probe_cleaner.cc:102] Cleaning probes from /sys/kernel/debug/tracing/uprobe_events with the following marker: __pixie__
I20260520 13:46:45.376327 2907970 probe_cleaner.cc:117] All Stirling probes removed (count=0)
I20260520 13:46:45.376345 2907970 source_connector.cc:35] Initializing source connector: process_stats
I20260520 13:46:45.376358 2907970 stirling.cc:438] Adding info class: [process_stats/process_stats]
I20260520 13:46:45.376374 2907970 source_connector.cc:35] Initializing source connector: network_stats
I20260520 13:46:45.376385 2907970 stirling.cc:438] Adding info class: [network_stats/network_stats]
I20260520 13:46:45.376399 2907970 source_connector.cc:35] Initializing source connector: jvm_stats
I20260520 13:46:45.376412 2907970 stirling.cc:438] Adding info class: [jvm_stats/jvm_stats]
I20260520 13:46:45.376695 2907970 source_connector.cc:35] Initializing source connector: socket_tracer
I20260520 13:46:45.376758 2907970 socket_trace_connector.cc:495] Kernel version greater than V5.1 detected (6.12.79), raised loop limit to 882 and chunk limit to 84
I20260520 13:46:45.376785 2907970 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 13:46:45.376798 2907970 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 13:46:45.378263 2907970 linux_headers.cc:369] Not Found : Could not find 'source' or 'build' under /lib/modules/6.12.79.
I20260520 13:46:45.378301 2907970 linux_headers.cc:215] Looking for host Linux headers at /host/lib/modules/6.12.79/build.
I20260520 13:46:45.378333 2907970 linux_headers.cc:372] Not Found : Did not find the host headers at path: /host/lib/modules/6.12.79/build, No such file or directory.
I20260520 13:46:45.378350 2907970 linux_headers.cc:314] Attempting to install packaged headers.
I20260520 13:46:45.378468 2907970 linux_headers.cc:320] Using packaged header: /px/linux-headers-x86_64-6.12.80.tar.gz
I20260520 13:46:46.609959 2907970 linux_headers.cc:57] Overriding linux version code to 396367
I20260520 13:46:46.610265 2907970 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 13:46:46.610347 2907970 linux_headers.cc:99] Found kernel config at: /proc/config.gz.
I20260520 13:46:46.650128 2907970 linux_headers.cc:345] Successfully installed packaged copy of headers at /lib/modules/6.12.79/build
I20260520 13:46:46.650187 2907970 bcc_wrapper.cc:94] Resolving task_struct offsets.
I20260520 13:46:46.718883 2908049 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 13:46:48.406548 2908049 scoped_timer.h:48] Timer(init_bpf_program) : 1.69 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 13:46:48.755604 2907970 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 13:46:50.577160 2907970 scoped_timer.h:48] Timer(init_bpf_program) : 1.82 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 13:46:50.772557 2907970 bcc_wrapper.cc:97] Successfully resolved task_struct offsets: {real_start_time=3040, group_leader=2752, exit_code=2588}
I20260520 13:46:50.772682 2907970 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 13:47:56.291277 2907970 scoped_timer.h:48] Timer(init_bpf_program) : 65.52 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
bpf: Argument list too long. Program  too large (18400 insns), at most 4096 insns

W20260520 13:48:03.006151 2907970 stirling.cc:416] Source Connector (registry name=socket_tracer) not instantiated, error: Internal : Failed to load syscall__probe_ret_writev: -1
I20260520 13:48:03.006276 2907970 source_connector.cc:35] Initializing source connector: perf_profiler
I20260520 13:48:03.006331 2907970 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 13:48:03.006345 2907970 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 13:48:03.006417 2907970 linux_headers.cc:207] Using Linux headers from: /lib/modules/6.12.79/build.
I20260520 13:48:03.006429 2907970 bcc_wrapper.cc:90] Returning the previously resolved TaskStructOffsets object
I20260520 13:48:03.006490 2907970 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 13:48:06.640654 2907970 scoped_timer.h:48] Timer(init_bpf_program) : 3.63 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 13:48:06.642256 2907970 perf_profile_connector.cc:153] PerfProfiler: Stack trace profiling sampling probe successfully deployed.
I20260520 13:48:06.642297 2907970 perf_profile_connector.cc:169] PerfProfiler: Java symbolization enabled.
I20260520 13:48:06.642349 2907970 java_symbolizer.cc:233] JavaSymbolizer found agent lib /px/libpx-java-agent.so.
I20260520 13:48:06.642370 2907970 stirling.cc:438] Adding info class: [perf_profiler/stack_traces.beta]
I20260520 13:48:06.642463 2907970 source_connector.cc:35] Initializing source connector: proc_exit_tracer
I20260520 13:48:06.642501 2907970 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 13:48:06.642516 2907970 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 13:48:06.642560 2907970 linux_headers.cc:207] Using Linux headers from: /lib/modules/6.12.79/build.
I20260520 13:48:06.642577 2907970 bcc_wrapper.cc:90] Returning the previously resolved TaskStructOffsets object
I20260520 13:48:06.642758 2907970 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 13:48:08.285061 2907970 scoped_timer.h:48] Timer(init_bpf_program) : 1.64 s
I20260520 13:48:08.285257 2907970 proc_exit_connector.cc:86] Writing exit_code's offset to BPF array: offset=2588 bpf_array=control_values index=0
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 13:48:08.297154 2907970 stirling.cc:438] Adding info class: [proc_exit_tracer/proc_exit_events]
I20260520 13:48:08.297268 2907970 source_connector.cc:35] Initializing source connector: stirling_error
I20260520 13:48:08.297408 2907970 stirling.cc:438] Adding info class: [stirling_error/stirling_error]
I20260520 13:48:08.297494 2907970 stirling.cc:438] Adding info class: [stirling_error/probe_status]
I20260520 13:48:08.297521 2907970 stirling.cc:419] Stirling successfully initialized.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1779284888.300500 2907970 event_engine_client_channel_resolver.cc:558] no server name supplied in dns URI
E0000 00:00:1779284888.300562 2907970 legacy_channel.cc:128] channel stack builder failed: UNKNOWN: the target uri is not valid: dns:///
I20260520 13:48:08.311123 2907970 manager.cc:154] Hostname: ip-10-29-83-172.ec2.internal
I20260520 13:48:46.258114 2907970 cgroup_path_resolver.cc:161] Auto-discovered CGroup base path: /sys/fs/cgroup
I20260520 13:48:46.263235 2907970 cgroup_path_resolver.cc:169] Auto-discovered example path: /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02579471_b0e9_4c42_b6e6_3ad9655f9ab0.slice/cri-containerd-509222d6ec00ea7d80603e970810bf94d3d0a483e10447f0813a7dbf9fe9a8ac.scope/cgroup.procs
I20260520 13:48:46.263401 2907970 cgroup_path_resolver.cc:179] Auto-discovered template: /sys/fs/cgroup/kubepods.slice/kubepods-$2.slice/kubepods-$2-pod$0.slice/cri-containerd-$1.scope/cgroup.procs
I20260520 13:48:46.263443 2907970 cgroup_metadata_reader.cc:41] Using path_resolver with configuration: template=/sys/fs/cgroup/kubepods.slice/kubepods-$2.slice/kubepods-$2-pod$0.slice/cri-containerd-$1.scope/cgroup.procs pod_id_separators=_ qos_spearator=-
I20260520 13:48:46.263993 2912847 stirling.cc:792] Stirling is running.
I20260520 13:48:46.268647 2912847 perf_profile_connector.cc:433] PerfProfileConnector statistics: kBPFMapSwitchoverEvent=1 kCumulativeSumOfAllStackTraces=13104 kLossHistoEvent=0 
I20260520 13:48:46.268707 2912847 perf_profile_connector.cc:447] PerfProfileConnector u_symbolizer num_symbols_cached=0 hits=0 accesses=0 hit_rate=0
I20260520 13:48:46.268719 2912847 perf_profile_connector.cc:450] PerfProfileConnector k_symbolizer num_symbols_cached=0 hits=0 accesses=0 hit_rate=0
I20260520 13:48:46.351143 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 5 ms
I20260520 13:48:46.368273 2912847 run_core_stats.cc:111] |main_loop_iters,no_work_iters,useful_iters,push+transfer,transfer,push,min_push+transfer,max_push+transfer,total_0.00_ms,total_0.01_ms,total_0.02_ms,total_0.03_ms,total_0.06_ms,total_0.10_ms,total_0.18_ms,total_0.32_ms,total_0.56_ms,total_1.00_ms,total_1.78_ms,total_3.16_ms,total_5.62_ms,total_10.00_ms,total_17.78_ms,total_31.62_ms,total_56.23_ms,total_100.00_ms,total_177.83_ms,total_316.23_ms,total_562.34_ms,total_1000.00_ms,total_10000000.00_ms,no_work_0.00_ms,no_work_0.01_ms,no_work_0.02_ms,no_work_0.03_ms,no_work_0.06_ms,no_work_0.10_ms,no_work_0.18_ms,no_work_0.32_ms,no_work_0.56_ms,no_work_1.00_ms,no_work_1.78_ms,no_work_3.16_ms,no_work_5.62_ms,no_work_10.00_ms,no_work_17.78_ms,no_work_31.62_ms,no_work_56.23_ms,no_work_100.00_ms,no_work_177.83_ms,no_work_316.23_ms,no_work_562.34_ms,no_work_1000.00_ms,no_work_10000000.00_ms
W20260520 13:48:51.262717 2907970 state_manager.cc:276] Failed to read PID info for pod=02579471-b0e9-4c42-b6e6-3ad9655f9ab0, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02579471_b0e9_4c42_b6e6_3ad9655f9ab0.slice/cri-containerd-.scope/cgroup.procs]
W20260520 13:48:51.269757 2907970 state_manager.cc:276] Failed to read PID info for pod=05410694-b8b9-4cad-827b-8c1674de6bde, cid=421848d264a82ca3ee41ef746c50479f7f4fc73f93f0aa48e214f4ead58e5f45 [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05410694_b8b9_4cad_827b_8c1674de6bde.slice/cri-containerd-421848d264a82ca3ee41ef746c50479f7f4fc73f93f0aa48e214f4ead58e5f45.scope/cgroup.procs]
I20260520 13:48:51.275113 2907970 metadata_state.h:123] Service CIDR updated to 172.20.0.0/16
I20260520 13:50:23.107036 2912847 run_core_stats.cc:88] |1000,0,1000,1737,1275,462,1,12,84,0,0,0,0,0,0,0,0,1,0,0,5,1,0,1,2,906,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:51:55.650887 2912847 run_core_stats.cc:88] |2000,0,2000,3471,2548,923,1,12,175,0,0,0,0,0,0,0,0,1,0,0,5,1,0,2,4,1812,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:53:27.256839 2912847 run_core_stats.cc:88] |3000,0,3000,5191,3811,1380,1,12,269,0,0,0,0,0,0,0,0,3,0,0,5,2,1,3,7,2710,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:53:46.462909 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 4 ms
I20260520 13:54:58.850056 2912847 run_core_stats.cc:88] |4000,0,4000,6919,5079,1840,1,12,361,0,0,0,0,0,0,0,0,3,0,0,5,2,1,6,11,3611,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:56:30.287045 2912847 run_core_stats.cc:88] |5000,0,5000,8640,6344,2296,1,12,452,0,0,0,0,0,0,0,0,3,0,2,5,2,2,7,15,4512,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:58:01.545836 2912847 run_core_stats.cc:88] |6000,0,6000,10371,7614,2757,1,12,543,0,0,0,0,0,0,0,0,4,0,2,5,3,2,9,17,5415,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 13:58:46.628852 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 7 ms
I20260520 13:58:58.535928 2912847 perf_profile_connector.cc:433] PerfProfileConnector statistics: kBPFMapSwitchoverEvent=21 kCumulativeSumOfAllStackTraces=235585 kLossHistoEvent=1302 
I20260520 13:58:58.536024 2912847 perf_profile_connector.cc:447] PerfProfileConnector u_symbolizer num_symbols_cached=2058 hits=42836 accesses=71620 hit_rate=59.8101
I20260520 13:58:58.536042 2912847 perf_profile_connector.cc:450] PerfProfileConnector k_symbolizer num_symbols_cached=1007 hits=24867 accesses=27986 hit_rate=88.8551
I20260520 13:59:32.784148 2912847 run_core_stats.cc:88] |7000,0,7000,12091,8878,3213,1,12,630,0,0,0,0,0,0,0,0,5,0,3,7,6,3,10,22,6314,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:01:04.206278 2912847 run_core_stats.cc:88] |8000,0,8000,13822,10148,3674,1,12,722,0,0,0,0,0,0,0,0,5,0,3,7,7,3,12,25,7216,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:02:35.291898 2912847 run_core_stats.cc:88] |9000,0,9000,15540,11410,4130,1,12,814,0,0,0,0,0,0,0,0,5,0,6,7,8,4,13,28,8115,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
W20260520 14:02:36.397162 2907970 state_manager.cc:276] Failed to read PID info for pod=c1635d35-4503-4a9f-9f55-cfbc4ac70af5, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1635d35_4503_4a9f_9f55_cfbc4ac70af5.slice/cri-containerd-.scope/cgroup.procs]
W20260520 14:02:41.399854 2907970 state_manager.cc:276] Failed to read PID info for pod=c1635d35-4503-4a9f-9f55-cfbc4ac70af5, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1635d35_4503_4a9f_9f55_cfbc4ac70af5.slice/cri-containerd-.scope/cgroup.procs]
I20260520 14:03:46.923703 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 145 ms
I20260520 14:04:05.595376 2912847 run_core_stats.cc:88] |10000,0,10000,17250,12663,4587,1,12,902,0,0,0,0,0,0,0,0,9,0,8,12,14,8,13,31,9003,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:05:36.873612 2912847 run_core_stats.cc:88] |11000,0,11000,18971,13928,5043,1,12,995,0,0,0,0,0,0,0,0,10,0,8,13,14,8,15,33,9904,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:07:07.435375 2912847 run_core_stats.cc:88] |12000,0,12000,20687,15188,5499,1,12,1087,0,0,0,0,0,0,0,0,11,0,11,15,14,9,17,37,10799,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:08:38.322992 2912847 run_core_stats.cc:88] |13000,0,13000,22406,16451,5955,1,12,1180,0,0,0,0,0,0,0,0,11,0,12,15,16,10,18,41,11697,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:08:46.980561 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 5 ms
I20260520 14:09:03.578841 2912847 perf_profile_connector.cc:433] PerfProfileConnector statistics: kBPFMapSwitchoverEvent=41 kCumulativeSumOfAllStackTraces=455666 kLossHistoEvent=1302 
I20260520 14:09:03.578931 2912847 perf_profile_connector.cc:447] PerfProfileConnector u_symbolizer num_symbols_cached=1799 hits=98619 accesses=161306 hit_rate=61.1378
I20260520 14:09:03.578945 2912847 perf_profile_connector.cc:450] PerfProfileConnector k_symbolizer num_symbols_cached=1521 hits=51458 accesses=57572 hit_rate=89.3803
I20260520 14:10:08.649276 2912847 run_core_stats.cc:88] |14000,0,14000,24121,17710,6411,1,12,1272,0,0,0,0,0,0,0,0,12,0,13,15,16,11,23,46,12592,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
W20260520 14:10:31.449007 2907970 state_manager.cc:276] Failed to read PID info for pod=db376eb0-2372-4067-b1cc-400353fafabe, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb376eb0_2372_4067_b1cc_400353fafabe.slice/cri-containerd-.scope/cgroup.procs]
I20260520 14:11:35.873616 2912847 run_core_stats.cc:88] |15000,0,15000,25777,18924,6853,1,12,1368,0,0,0,0,0,0,0,0,17,0,16,20,22,16,30,67,13444,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:13:05.731081 2912847 run_core_stats.cc:88] |16000,0,16000,27478,20174,7304,1,12,1462,0,0,0,0,0,0,0,0,19,0,17,21,24,20,35,69,14333,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:13:47.182265 2907970 heartbeat.cc:143] Heartbeat ACK latency moving average: 5 ms
I20260520 14:14:32.764245 2912847 run_core_stats.cc:88] |17000,0,17000,29133,21389,7744,1,12,1552,0,0,0,0,0,0,0,0,20,0,17,24,28,24,54,96,15185,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

  17. ddelnano commented on May 20, 2026

    @ddelnano
    ddelnano
    on May 20, 2026
    Member

    @tusharguptaJ your issue is #2042. Adding these flags (#2042 (comment)) should allow the BPF programs to load properly. Please provide the next series of logs if that isn't the case.

  18. tusharguptaJ commented on May 20, 2026

    @tusharguptaJ
    tusharguptaJ
    on May 20, 2026

    @ddelnano Here's the further logs

    Click to view logs 
    I20260520 14:35:31.138738 569849 env.cc:47] Started: /app/src/vizier/services/agent/pem/pem
Started external stacktrace collection signal processor thread
I20260520 14:35:31.139312 569849 kernel_version.cc:199] Found Linux kernel version using .note section.
I20260520 14:35:31.139369 569849 pem_main.cc:69] Pixie PEM. Version: v0.14.15+Distribution.623e988.202501242347.1.RELEASE.jenkins, id: 593359df-dfb3-4227-8270-8ec066abb734, kernel version: 6.12.79
I20260520 14:35:31.139516 569849 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 14:35:31.139645 569849 stirling.cc:958] Creating Stirling, registered sources: [process_stats, network_stats, jvm_stats, socket_tracer, perf_profiler, proc_exit_tracer, stirling_error]
I20260520 14:35:31.139726 569849 system_info.cc:42] Location of proc: /proc
I20260520 14:35:31.139775 569849 system_info.cc:43] Location of sysfs: /sys/fs
I20260520 14:35:31.139791 569849 system_info.cc:44] Number of CPUs: 4
I20260520 14:35:31.139904 569849 system_info.cc:35] /proc/version:
Linux version 6.12.79 (builder@buildkitsandbox) (x86_64-bottlerocket-linux-gnu-gcc (Buildroot 2025.05.1) 13.4.0, GNU ld (GNU Binutils) 2.44) #1 SMP PREEMPT_DYNAMIC Fri May  1 00:25:24 UTC 2026
I20260520 14:35:31.140107 569849 system_info.cc:35] /host/etc/os-release:
NAME=Bottlerocket
ID=bottlerocket
VERSION="1.60.0 (aws-k8s-1.35)"
PRETTY_NAME="Bottlerocket OS 1.60.0 (aws-k8s-1.35)"
VARIANT_ID=aws-k8s-1.35
VERSION_ID=1.60.0
BUILD_ID=c1f9ba0c
VENDOR_NAME=Bottlerocket
HOME_URL="https://github.com/bottlerocket-os/bottlerocket"
SUPPORT_URL="https://github.com/bottlerocket-os/bottlerocket/discussions"
BUG_REPORT_URL="https://github.com/bottlerocket-os/bottlerocket/issues"
DOCUMENTATION_URL="https://bottlerocket.dev"
I20260520 14:35:31.140151 569849 probe_cleaner.cc:102] Cleaning probes from /sys/kernel/debug/tracing/kprobe_events with the following marker: __pixie__
I20260520 14:35:31.140290 569849 probe_cleaner.cc:117] All Stirling probes removed (count=0)
I20260520 14:35:31.140321 569849 probe_cleaner.cc:102] Cleaning probes from /sys/kernel/debug/tracing/uprobe_events with the following marker: __pixie__
I20260520 14:35:31.140379 569849 probe_cleaner.cc:117] All Stirling probes removed (count=0)
I20260520 14:35:31.140405 569849 source_connector.cc:35] Initializing source connector: process_stats
I20260520 14:35:31.140419 569849 stirling.cc:438] Adding info class: [process_stats/process_stats]
I20260520 14:35:31.140435 569849 source_connector.cc:35] Initializing source connector: network_stats
I20260520 14:35:31.140448 569849 stirling.cc:438] Adding info class: [network_stats/network_stats]
I20260520 14:35:31.140461 569849 source_connector.cc:35] Initializing source connector: jvm_stats
I20260520 14:35:31.140486 569849 stirling.cc:438] Adding info class: [jvm_stats/jvm_stats]
I20260520 14:35:31.142364 569849 source_connector.cc:35] Initializing source connector: socket_tracer
I20260520 14:35:31.143443 569849 socket_trace_connector.cc:475] Kernel version greater than V5.1 detected (6.12.79), raised loop limit to 882 and chunk limit to 84
I20260520 14:35:31.143501 569849 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 14:35:31.143523 569849 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 14:35:31.145102 569849 linux_headers.cc:369] Not Found : Could not find 'source' or 'build' under /lib/modules/6.12.79.
I20260520 14:35:31.145153 569849 linux_headers.cc:215] Looking for host Linux headers at /host/lib/modules/6.12.79/build.
I20260520 14:35:31.145188 569849 linux_headers.cc:372] Not Found : Did not find the host headers at path: /host/lib/modules/6.12.79/build, No such file or directory.
I20260520 14:35:31.145205 569849 linux_headers.cc:314] Attempting to install packaged headers.
W20260520 14:35:31.146703 569849 linux_headers.cc:287] Ignoring /px/linux-headers-x86_64-4.14.304.tar.gz since it does not conform to the naming convention
W20260520 14:35:31.146770 569849 linux_headers.cc:287] Ignoring /px/linux-headers-x86_64-4.19.271.tar.gz since it does not conform to the naming convention
I20260520 14:35:31.146845 569849 linux_headers.cc:320] Using packaged header: /px/linux-headers-x86_64-6.1.8.tar.gz
I20260520 14:35:32.184111 569849 linux_headers.cc:57] Overriding linux version code to 396367
I20260520 14:35:32.184432 569849 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 14:35:32.184525 569849 linux_headers.cc:99] Found kernel config at: /proc/config.gz.
I20260520 14:35:32.219754 569849 linux_headers.cc:345] Successfully installed packaged copy of headers at /lib/modules/6.12.79/build
I20260520 14:35:32.219806 569849 bcc_wrapper.cc:94] Resolving task_struct offsets.
I20260520 14:35:32.543787 569863 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 14:35:34.325178 569863 scoped_timer.h:48] Timer(init_bpf_program) : 1.78 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 14:35:34.669260 569849 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 14:35:36.123142 569849 scoped_timer.h:48] Timer(init_bpf_program) : 1.45 s
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 14:35:36.337617 569849 bcc_wrapper.cc:97] Successfully resolved task_struct offsets: {real_start_time=3040, group_leader=2752, exit_code=2588}
I20260520 14:35:36.337733 569849 bcc_wrapper.cc:166] Initializing BPF program ...
In file included from src/stirling/source_connectors/socket_tracer/bcc_bpf/socket_trace.c:25:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/net.h:1:
In file included from include/linux/net.h:23:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1151:20: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        void __rcu      *slots[XA_CHUNK_SIZE];
                               ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1153:36: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                unsigned long   tags[XA_MAX_MARKS][XA_MARK_LONGS];
                                                   ^
include/linux/xarray.h:1131:37: note: expanded from macro 'XA_MARK_LONGS'
#define XA_MARK_LONGS           DIV_ROUND_UP(XA_CHUNK_SIZE, BITS_PER_LONG)
                                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1154:37: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                unsigned long   marks[XA_MAX_MARKS][XA_MARK_LONGS];
                                                    ^
include/linux/xarray.h:1131:37: note: expanded from macro 'XA_MARK_LONGS'
#define XA_MARK_LONGS           DIV_ROUND_UP(XA_CHUNK_SIZE, BITS_PER_LONG)
                                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1198:9: error: returning 'void' from a function with incompatible result type 'void *'
        return rcu_dereference_check(node->slots[offset],
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:575:2: note: expanded from macro 'rcu_dereference_check'
        __rcu_dereference_check((p), __UNIQUE_ID(rcu), \
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:424:53: note: expanded from macro '__rcu_dereference_check'
#define __rcu_dereference_check(p, local, c, space) \
                                                    ^
In file included from src/stirling/source_connectors/socket_tracer/bcc_bpf/socket_trace.c:25:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/net.h:1:
In file included from include/linux/net.h:23:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1207:9: error: returning 'void' from a function with incompatible result type 'void *'
        return rcu_dereference_protected(node->slots[offset],
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:638:2: note: expanded from macro 'rcu_dereference_protected'
        __rcu_dereference_protected((p), __UNIQUE_ID(rcu), (c), __rcu)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:432:57: note: expanded from macro '__rcu_dereference_protected'
#define __rcu_dereference_protected(p, local, c, space) \
                                                        ^
In file included from src/stirling/source_connectors/socket_tracer/bcc_bpf/socket_trace.c:25:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/net.h:1:
In file included from include/linux/net.h:23:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1266:26: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                (entry < xa_mk_sibling(XA_CHUNK_SIZE - 1));
                                       ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1576:45: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                offset = (xas->xa_index >> node->shift) & XA_CHUNK_MASK;
                                                          ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1617:38: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_offset = (index >> shift) & XA_CHUNK_MASK;
                                            ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIinclude/linux/xarray.h:1697:38: error: no member named 'marks' in 'struct xa_node'
        unsigned long *addr = xas->xa_node->marks[(__force unsigned)mark];
                              ~~~~~~~~~~~~  ^
include/linux/xarray.h:1702:6: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        if (XA_CHUNK_SIZE == BITS_PER_LONG) {
            ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
G_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1631:35: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_shift = order - (order % XA_CHUNK_SHIFT);
                                         ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1632:32: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_sibs = (1 << (order % XA_CHUNK_SHIFT)) - 1;
                                      ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1675:39: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                        xas->xa_offset != (xas->xa_index & XA_CHUNK_MASK)))
                                                           ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1681:34: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                if (unlikely(xas->xa_offset == XA_CHUNK_MASK))
                                               ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1697:38: error: no member named 'marks' in 'struct xa_node'
        unsigned long *addr = xas->xa_node->marks[(__force unsigned)mark];
                              ~~~~~~~~~~~~  ^
include/linux/xarray.h:1702:6: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        if (XA_CHUNK_SIZE == BITS_PER_LONG) {
            ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1703:16: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                if (offset < XA_CHUNK_SIZE) {
                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1708:10: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                return XA_CHUNK_SIZE;
                       ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1711:29: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        return find_next_bit(addr, XA_CHUNK_SIZE, offset);
                                   ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1737:36: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_index = (xas->xa_index & ~XA_CHUNK_MASK) + offset;
                                          ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1740:16: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        if (offset == XA_CHUNK_SIZE)
                      ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
I20260520 14:35:40.017246 569849 scoped_timer.h:48] Timer(init_bpf_program) : 3.68 s
W20260520 14:35:40.017443 569849 stirling.cc:416] Source Connector (registry name=socket_tracer) not instantiated, error: Internal : Unable to initialize BCC BPF program: Unable to initialize BPF program
I20260520 14:35:40.017611 569849 source_connector.cc:35] Initializing source connector: perf_profiler
I20260520 14:35:40.017694 569849 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 14:35:40.017711 569849 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 14:35:40.017776 569849 linux_headers.cc:207] Using Linux headers from: /lib/modules/6.12.79/build.
I20260520 14:35:40.017803 569849 bcc_wrapper.cc:90] Returning the previously resolved TaskStructOffsets object
I20260520 14:35:40.017869 569849 bcc_wrapper.cc:166] Initializing BPF program ...
In file included from src/stirling/source_connectors/perf_profiler/bcc_bpf/profiler.c:24:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/bpf_perf_event.h:1:
In file included from include/uapi/linux/bpf_perf_event.h:11:
In file included from arch/x86/include/generated/uapi/asm/bpf_perf_event.h:1:
In file included from include/uapi/asm-generic/bpf_perf_event.h:4:
In file included from include/linux/ptrace.h:10:
In file included from include/linux/pid_namespace.h:7:
In file included from include/linux/mm.h:737:
In file included from include/linux/huge_mm.h:8:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1151:20: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        void __rcu      *slots[XA_CHUNK_SIZE];
                               ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1153:36: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                unsigned long   tags[XA_MAX_MARKS][XA_MARK_LONGS];
                                                   ^
include/linux/xarray.h:1131:37: note: expanded from macro 'XA_MARK_LONGS'
#define XA_MARK_LONGS           DIV_ROUND_UP(XA_CHUNK_SIZE, BITS_PER_LONG)
                                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1154:37: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                unsigned long   marks[XA_MAX_MARKS][XA_MARK_LONGS];
                                                    ^
include/linux/xarray.h:1131:37: note: expanded from macro 'XA_MARK_LONGS'
#define XA_MARK_LONGS           DIV_ROUND_UP(XA_CHUNK_SIZE, BITS_PER_LONG)
                                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1198:9: error: returning 'void' from a function with incompatible result type 'void *'
        return rcu_dereference_check(node->slots[offset],
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:575:2: note: expanded from macro 'rcu_dereference_check'
        __rcu_dereference_check((p), __UNIQUE_ID(rcu), \
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:424:53: note: expanded from macro '__rcu_dereference_check'
#define __rcu_dereference_check(p, local, c, space) \
                                                    ^
In file included from src/stirling/source_connectors/perf_profiler/bcc_bpf/profiler.c:24:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/bpf_perf_event.h:1:
In file included from include/uapi/linux/bpf_perf_event.h:11:
In file included from arch/x86/include/generated/uapi/asm/bpf_perf_event.h:1:
In file included from include/uapi/asm-generic/bpf_perf_event.h:4:
In file included from include/linux/ptrace.h:10:
In file included from include/linux/pid_namespace.h:7:
In file included from include/linux/mm.h:737:
In file included from include/linux/huge_mm.h:8:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1207:9: error: returning 'void' from a function with incompatible result type 'void *'
        return rcu_dereference_protected(node->slots[offset],
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:638:2: note: expanded from macro 'rcu_dereference_protected'
        __rcu_dereference_protected((p), __UNIQUE_ID(rcu), (c), __rcu)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:432:57: note: expanded from macro '__rcu_dereference_protected'
#define __rcu_dereference_protected(p, local, c, space) \
                                                        ^
In file included from src/stirling/source_connectors/perf_profiler/bcc_bpf/profiler.c:24:
In file included from src/stirling/bpf_tools/bcc_bpf/system-headers/linux/bpf_perf_event.h:1:
In file included from include/uapi/linux/bpf_perf_event.h:11:
In file included from arch/x86/include/generated/uapi/asm/bpf_perf_event.h:1:
In file included from include/uapi/asm-generic/bpf_perf_event.h:4:
In file included from include/linux/ptrace.h:10:
In file included from include/linux/pid_namespace.h:7:
In file included from include/linux/mm.h:737:
In file included from include/linux/huge_mm.h:8:
In file included from include/linux/fs.h:13:
In file included from include/linux/list_lru.h:14:
include/linux/xarray.h:1266:26: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                (entry < xa_mk_sibling(XA_CHUNK_SIZE - 1));
                                       ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1576:45: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                offset = (xas->xa_index >> node->shift) & XA_CHUNK_MASK;
                                                          ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1617:38: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_offset = (index >> shift) & XA_CHUNK_MASK;
                                            ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1631:35: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_shift = order - (order % XA_CHUNK_SHIFT);
                                         ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1632:32: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_sibs = (1 << (order % XA_CHUNK_SHIFT)) - 1;
                                      ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1675:39: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                        xas->xa_offset != (xas->xa_index & XA_CHUNK_MASK)))
                                                           ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1681:34: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                if (unlikely(xas->xa_offset == XA_CHUNK_MASK))
                                               ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1697:38: error: no member named 'marks' in 'struct xa_node'
        unsigned long *addr = xas->xa_node->marks[(__force unsigned)mark];
                              ~~~~~~~~~~~~  ^
include/linux/xarray.h:1702:6: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        if (XA_CHUNK_SIZE == BITS_PER_LONG) {
            ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1703:16: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                if (offset < XA_CHUNK_SIZE) {
                             ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1708:10: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
                return XA_CHUNK_SIZE;
                       ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1711:29: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        return find_next_bit(addr, XA_CHUNK_SIZE, offset);
                                   ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1737:36: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        xas->xa_index = (xas->xa_index & ~XA_CHUNK_MASK) + offset;
                                          ^
include/linux/xarray.h:1129:25: note: expanded from macro 'XA_CHUNK_MASK'
#define XA_CHUNK_MASK           (XA_CHUNK_SIZE - 1)
                                 ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
include/linux/xarray.h:1740:16: error: use of undeclared identifier 'CONFIG_BASE_SMALL'
        if (offset == XA_CHUNK_SIZE)
                      ^
include/linux/xarray.h:1128:32: note: expanded from macro 'XA_CHUNK_SIZE'
#define XA_CHUNK_SIZE           (1UL << XA_CHUNK_SHIFT)
                                        ^
include/linux/xarray.h:1126:26: note: expanded from macro 'XA_CHUNK_SHIFT'
#define XA_CHUNK_SHIFT          (CONFIG_BASE_SMALL ? 4 : 6)
                                 ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
I20260520 14:35:42.149693 569849 scoped_timer.h:48] Timer(init_bpf_program) : 2.13 s
W20260520 14:35:42.149819 569849 stirling.cc:416] Source Connector (registry name=perf_profiler) not instantiated, error: Internal : Unable to initialize BCC BPF program: Unable to initialize BPF program
I20260520 14:35:42.149897 569849 source_connector.cc:35] Initializing source connector: proc_exit_tracer
I20260520 14:35:42.149935 569849 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.12.79
I20260520 14:35:42.149947 569849 linux_headers.cc:353] Detected kernel release (uname -r): 6.12.79
I20260520 14:35:42.149993 569849 linux_headers.cc:207] Using Linux headers from: /lib/modules/6.12.79/build.
I20260520 14:35:42.150063 569849 bcc_wrapper.cc:90] Returning the previously resolved TaskStructOffsets object
I20260520 14:35:42.150137 569849 bcc_wrapper.cc:166] Initializing BPF program ...
I20260520 14:35:43.541858 569849 scoped_timer.h:48] Timer(init_bpf_program) : 1.39 s
I20260520 14:35:43.541916 569849 proc_exit_connector.cc:86] Writing exit_code's offset to BPF array: offset=2588 bpf_array=control_values index=0
cannot create /var/tmp/bcc
WARNING: cannot get prog tag, ignore saving source with program tag
I20260520 14:35:43.549863 569849 stirling.cc:438] Adding info class: [proc_exit_tracer/proc_exit_events]
I20260520 14:35:43.549939 569849 source_connector.cc:35] Initializing source connector: stirling_error
I20260520 14:35:43.550040 569849 stirling.cc:438] Adding info class: [stirling_error/stirling_error]
I20260520 14:35:43.550072 569849 stirling.cc:438] Adding info class: [stirling_error/probe_status]
I20260520 14:35:43.550087 569849 stirling.cc:419] Stirling successfully initialized.
E0520 14:35:43.552077832  569849 dns_resolver_ares.cc:456]             no server name supplied in dns URI
E0520 14:35:43.552123622  569849 channel.cc:120]                       channel stack builder failed: UNKNOWN: the target uri is not valid: dns:///
I20260520 14:35:43.564939 569849 manager.cc:154] Hostname: ip-10-3-99-169.ec2.internal
I20260520 14:36:17.952512 569849 cgroup_path_resolver.cc:150] Auto-discovered CGroup base path: /sys/fs/cgroup
I20260520 14:36:17.962616 569849 cgroup_path_resolver.cc:158] Auto-discovered example path: /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bfe518_8976_4871_a0c4_943428cccd67.slice/cri-containerd-462d886cb54ceff8145f74e5a8afdab83f83c987v1522d20f11857c7f1eb9124.scope/cgroup.procs
I20260520 14:36:17.962754 569849 cgroup_path_resolver.cc:168] Auto-discovered template: /sys/fs/cgroup/kubepods.slice/kubepods-$2.slice/kubepods-$2-pod$0.slice/cri-containerd-$1.scope/cgroup.procs
I20260520 14:36:17.962783 569849 cgroup_metadata_reader.cc:41] Using path_resolver with configuration: template=/sys/fs/cgroup/kubepods.slice/kubepods-$2.slice/kubepods-$2-pod$0.slice/cri-containerd-$1.scope/cgroup.procs pod_id_separators=_ qos_spearator=-
I20260520 14:36:17.963248 571130 stirling.cc:792] Stirling is running.
I20260520 14:36:18.062635 571130 run_core_stats.cc:111] |main_loop_iters,no_work_iters,useful_iters,push+transfer,transfer,push,min_push+transfer,max_push+transfer,total_0.00_ms,total_0.01_ms,total_0.02_ms,total_0.03_ms,total_0.06_ms,total_0.10_ms,total_0.18_ms,total_0.32_ms,total_0.56_ms,total_1.00_ms,total_1.78_ms,total_3.16_ms,total_5.62_ms,total_10.00_ms,total_17.78_ms,total_31.62_ms,total_56.23_ms,total_100.00_ms,total_177.83_ms,total_316.23_ms,total_562.34_ms,total_1000.00_ms,total_10000000.00_ms,no_work_0.00_ms,no_work_0.01_ms,no_work_0.02_ms,no_work_0.03_ms,no_work_0.06_ms,no_work_0.10_ms,no_work_0.18_ms,no_work_0.32_ms,no_work_0.56_ms,no_work_1.00_ms,no_work_1.78_ms,no_work_3.16_ms,no_work_5.62_ms,no_work_10.00_ms,no_work_17.78_ms,no_work_31.62_ms,no_work_56.23_ms,no_work_100.00_ms,no_work_177.83_ms,no_work_316.23_ms,no_work_562.34_ms,no_work_1000.00_ms,no_work_10000000.00_ms
I20260520 14:36:18.083422 569849 heartbeat.cc:143] Heartbeat ACK latency moving average: 4 ms
W20260520 14:36:22.957237 569849 state_manager.cc:276] Failed to read PID info for pod=80bfe518-1a7f-4871-a0c4-943428cccd67, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bfe518_1a7f_4871_a0c4_943428cccd67.slice/cri-containerd-.scope/cgroup.procs]
W20260520 14:36:22.960867 569849 state_manager.cc:276] Failed to read PID info for pod=1373b0cf-415b-42b2-88f1-2abe075ecc26, cid=87ec8e521ea6d73b7ad696cdd8e24cde378c8b08459291f17e7d8c35a8e77294 [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1373b0cf_415b_42b2_88f1_2abe075ecc26.slice/cri-containerd-87ec8e521ea6d73b7ad696cdd8e24cde378c8b08459291f17e7d8c35a8e77294.scope/cgroup.procs]
I20260520 14:36:22.963510 569849 metadata_state.h:123] Service CIDR updated to 172.20.0.0/16
I20260520 14:37:49.261189 571130 run_core_stats.cc:88] |1000,0,1000,1731,1275,456,1,10,86,0,0,0,0,0,0,0,0,0,0,1,1,2,0,0,0,910,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
W20260520 14:39:08.003165 569849 state_manager.cc:276] Failed to read PID info for pod=4c5f4b32-c144-4d06-a807-03668e08462e, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5f4b32_c144_4d06_a807_03668e08462e.slice/cri-containerd-.scope/cgroup.procs]
W20260520 14:39:18.003816 569849 state_manager.cc:276] Failed to read PID info for pod=4c5f4b32-c144-4d06-a807-03668e08462e, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5f4b32_c144_4d06_a807_03668e08462e.slice/cri-containerd-.scope/cgroup.procs]
I20260520 14:39:20.283061 571130 run_core_stats.cc:88] |2000,0,2000,3455,2545,910,1,10,177,0,0,0,0,0,0,0,0,1,0,1,2,2,0,0,0,1817,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
W20260520 14:40:03.012014 569849 state_manager.cc:276] Failed to read PID info for pod=dcba6b44-25c5-4eb8-b7f4-7fc250c28c2b, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcba6b44_25c5_4eb8_b7f4_7fc250c28c2b.slice/cri-containerd-.scope/cgroup.procs]
I20260520 14:40:50.981858 571130 run_core_stats.cc:88] |3000,0,3000,5178,3813,1365,1,10,269,0,0,0,0,0,0,0,0,2,0,2,3,3,0,0,0,2721,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20260520 14:41:18.148407 569849 heartbeat.cc:143] Heartbeat ACK latency moving average: 4 ms
W20260520 14:42:03.043623 569849 state_manager.cc:276] Failed to read PID info for pod=bf05199d-afb0-454f-8276-9309b19aa091, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf05199d_afb0_454f_8276_9309b19aa091.slice/cri-containerd-.scope/cgroup.procs]
I20260520 14:42:21.677521 571130 run_core_stats.cc:88] |4000,0,4000,6892,5077,1815,1,10,361,0,0,0,0,0,0,0,0,3,0,5,3,3,0,0,0,3625,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
W20260520 14:43:03.053742 569849 state_manager.cc:276] Failed to read PID info for pod=954c706f-bd18-4eec-9de2-8c7c94d1f420, cid= [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod954c706f_bd18_4eec_9de2_8c7c94d1f420.slice/cri-containerd-.scope/cgroup.procs]
W20260520 14:43:03.053879 569849 state_manager.cc:276] Failed to read PID info for pod=954c706f-bd18-4eec-9de2-8c7c94d1f420, cid=0ebced95e77667a4b388bcb8ffd7a1d18a1a131b817b971839091fa4ed4ff09b [msg=Failed to open file /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod954c706f_bd18_4eec_9de2_8c7c94d1f420.slice/cri-containerd-0ebced95e77667a4b388bcb8ffd7a1d18a1a131b817b971839091fa4ed4ff09b.scope/cgroup.procs]

  19. tusharguptaJ commented on May 25, 2026

    @tusharguptaJ
    tusharguptaJ
    on May 25, 2026

    @ddelnano anything on this?

  20. entlein
    added a commit that references this issue on Jun 2, 2026

    Trim linux headers to LTS kernel versions for 4.x and 5.x and include…

    192e3d9
  21. moleus
    added a commit that references this issue on Jun 20, 2026

    Trim linux headers to LTS kernel versions for 4.x and 5.x and include…

    796b7c1

  22. tusharguptaJ commented on Aug 5, 2026

    @tusharguptaJ
    tusharguptaJ
    on Aug 5, 2026

    @ddelnano Is there any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions