From 6ecb9038b4598b6b0b251dce411cbcb0247720df Mon Sep 17 00:00:00 2001 From: Komh Date: Sat, 30 May 2026 05:07:44 +0000 Subject: [PATCH] =?UTF-8?q?[security]=20Third-party=20scanner=20reports=20?= =?UTF-8?q?private=20keys=20under=20/proc=20=E2=80=94=20runtime=20artifact?= =?UTF-8?q?,=20not=20image=20content?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rerun (batch7, 2026-05-30): full 8-phase pipeline on lab-base. terminal_route=convert_adapted. --- ...ner_Flags_Transient_TLS_Keys_Under_proc.md | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 docs/en/solutions/Container_Image_Scanner_Flags_Transient_TLS_Keys_Under_proc.md diff --git a/docs/en/solutions/Container_Image_Scanner_Flags_Transient_TLS_Keys_Under_proc.md b/docs/en/solutions/Container_Image_Scanner_Flags_Transient_TLS_Keys_Under_proc.md new file mode 100644 index 00000000..41ae5483 --- /dev/null +++ b/docs/en/solutions/Container_Image_Scanner_Flags_Transient_TLS_Keys_Under_proc.md @@ -0,0 +1,71 @@ +--- +title: Third-party scanner reports private keys under /proc — runtime artifact, not image content +component: security +scenario: troubleshooting +tags: [scanner, false-positive, procfs, tls, library-go] +date_created: 2026-05-30 +date_updated: 2026-05-30 +--- + +# Third-party scanner reports private keys under /proc — runtime artifact, not image content + +## Issue + +A third-party container image scanner running in node-mode on an Alauda Container Platform worker reports findings of the form "Private keys stored in image" against many platform and add-on workloads, where the offending file paths begin with `/proc//root/...` — for example `/proc//root/tmp/tls.key` or `/proc//root/tmp/serving-certs/tls.key` [ev:c1]. The scanner attributes the finding to the container image, but the paths only exist via the kernel's per-PID procfs view of a running container's mount namespace and have no presence in the published image layers [ev:c1]. + +A single ACP worker typically surfaces several such paths simultaneously. Examples observed on `cpaas-system` controllers on kernel 5.15 / containerd 2.2.1 [ev:c1]: + +```text +PID 50397 (base-operator) -> /proc/50397/root/tmp/tls.crt +PID 50397 (base-operator) -> /proc/50397/root/tmp/tls.key +PID 63628 (apollo) -> /proc/63628/root/tmp/serving-certs +``` + +## Root Cause + +The `/proc//root` symlink is a Linux kernel feature: it gives any process able to read `/proc` a view of the addressed process's mount-namespace root [ev:c1]. So `/proc/50397/root/tmp/tls.key` is just the path `/tmp/tls.key` *inside* the container of PID 50397, surfaced through the host's procfs. The kernel exposes it whether the file lives on a tmpfs, on a projected `Secret` volume, or on the container's writable overlay layer [ev:c1]. + +The reported `tls.key` / `tls.crt` files are written by the workload process itself after the container starts; they are not present in the image manifest or in any image layer [ev:c2]. Two patterns produce them on ACP: + +- **Process-written, on the overlay**: a controller generates a fresh TLS keypair into `/tmp/.key` (or `/tmp/serving-cert-/serving-signer.key` in some libraries) at startup and rotates the key periodically. The path has no entry in `/proc//mounts`, confirming the file is on the container's writable upper layer rather than a mounted volume [ev:c3_a][ev:c3_b]. The `base-operator` controller on `cpaas-system` shows this pattern (`/tmp/tls.key` rotated within the lifetime of the running container, `mtime` later than the start of PID 50397) [ev:c2][ev:c3_a]. +- **Projected Secret on tmpfs**: a controller is started with TLS flags pointing into `/tmp/serving-certs/` (for example `--tls-cert-file=/tmp/serving-certs/tls.crt --tls-private-key-file=/tmp/serving-certs/tls.key`); the kubelet mounts a `Secret` there as a tmpfs and populates it with the atomic `..data` symlink layout. The bytes come from the API server, not from the image [ev:c2][ev:c3_b]. The platform's `apollo` service shows this pattern. + +In either case the image artifact itself contains no key material at the reported path — the scanner's pattern matcher fired on a runtime file it found by walking `/proc` on the node, not on something embedded in the image [ev:c3_a][ev:c3_b][ev:c4_a]. + +## Resolution + +Findings whose path is prefixed by `/proc//root/` are runtime-state artifacts and do not represent secret material baked into the image; they can be acknowledged as scanner false positives for the purpose of image-supply-chain risk [ev:c4_a]. The triage rule is: + +- If the scanner-reported path starts with `/proc//root/...` — the file is per-process container state surfaced through procfs and is not part of the image artifact. No image rebuild or vendor escalation is required on this basis [ev:c1][ev:c4_a]. +- If the scanner-reported path is inside the actual image filesystem (no `/proc` prefix; for example a layer-relative `/usr/share/doc/.../server_key.pem`), the file is genuinely shipped in the image and the platform support process should be engaged to confirm whether the file is a real key or a documentation sample [ev:c4_a]. + +Where the scanner allows configuration, narrow its scope so this distinction is automatic — most enterprise scanners support an "image-only" mode (registry pull, analyse layers) separate from "node-mode" (walk `/proc` on the host). Image-only mode never traverses `/proc` and so cannot produce this class of false positive [ev:c3_b]. + +## Diagnostic Steps + +Confirm a reported path is process-state rather than image content by reading the live file's metadata through procfs from a node-debug shell. The relevant facts are: (a) the mount entry for the directory in `/proc//mounts` (none = on the writable overlay; `tmpfs` = projected Secret); (b) the file's `mtime` relative to the container start (anything after start = generated by the process); and (c) what the process binary actually points at in its argument list [ev:c1][ev:c2][ev:c3_a]. + +A short read-only probe across all running containers on a worker is: + +```bash +kubectl debug node/ -it=false --profile=sysadmin \ + --image=registry.alauda.cn:60070/acp/container-debug:v4.3.2 -- \ + sh -c 'for p in $(ls /proc | grep -E "^[0-9]+$"); do + root=/proc/$p/root; [ -d "$root/tmp" ] || continue + for f in $(ls -A "$root/tmp" 2>/dev/null); do + case "$f" in *.key|*.pem|*.crt|serving-cert*|tls-*) + echo "PID $p ($(cat /proc/$p/comm)) -> /proc/$p/root/tmp/$f" ;; + esac + done + done' +``` + +For each PID returned, cross-check the mount type and the file timestamp: + +```bash +kubectl debug node/ -it=false --profile=sysadmin \ + --image=registry.alauda.cn:60070/acp/container-debug:v4.3.2 -- \ + sh -c 'cat /proc//mounts | grep " /tmp"; stat /proc//root/tmp/' +``` + +A `tmpfs` line for `/tmp/...` plus the kubelet `..data` symlink layout indicates a projected `Secret` (the key is from a Kubernetes Secret, not from the image). No mount entry plus an `mtime` after the container's start time indicates a process-written file on the writable overlay (the key was generated at runtime by the controller). In both situations the path resides outside the image layers, and a scanner finding tied to the `/proc` path is a false positive at the image-supply-chain level [ev:c3_a][ev:c3_b][ev:c4_a].