Skip to content

feat(telegraf): Add distroless container based on scratch base-image#892

Open
victor-gama wants to merge 1 commit into
influxdata:masterfrom
victor-gama:victor-gama/addTelegrafDistroless
Open

feat(telegraf): Add distroless container based on scratch base-image#892
victor-gama wants to merge 1 commit into
influxdata:masterfrom
victor-gama:victor-gama/addTelegrafDistroless

Conversation

@victor-gama

@victor-gama victor-gama commented Jun 26, 2026

Copy link
Copy Markdown

Add a distroless Telegraf variant (1.39)

This adds an opt-in distroless variant for Telegraf 1.39, producing telegraf:1.39-distroless. The image is built FROM scratch and ships only the static telegraf binary plus the handful of files it needs at runtime (no shell, no libc, no package manager, no OS userland), and runs non-root (uid 65532). The change is purely additive: the existing default and alpine images are untouched.

What to review

File What changed
telegraf/1.39/distroless/Dockerfile New. Two-stage build (alpine fetchscratch). The heart of the PR.
telegraf/manifest.json Registers distroless for 1.39 on amd64 + arm64v8.

1.39-distroless maps to Telegraf release 1.39.1 (ENV TELEGRAF_VERSION 1.39.1).

Scope note: this revision covers 1.39 only, and ships no test scripts — both per @srebhan's feedback (drop the extra versions; remove the TESTING.md/local-test.sh/circle-test.sh additions to avoid CI credit consumption). If 1.37/1.38 variants are wanted, they're a trivial follow-up (same Dockerfile, different version).

How the Dockerfile works

It's a two-stage build:

  • An alpine fetch stage downloads the release with wget, GPG-verifies the tarball against InfluxData's signing key (24C975CBA61A024EE1B631787C3D57159FC2F927 — the same key the default and alpine siblings use), unpacks it, and assembles /rootfs as the exact final image tree (binary + config + CA bundle + tzdata + nsswitch.conf + passwd/group + home/tmp).
  • The final scratch stage brings that tree over with a single plain COPY --from=fetch /rootfs/ / — no ADD --unpack, no COPY --parents, no other BuildKit-only feature — so it isn't tied to any one builder.
  • Integrity is the gpg --batch --verify check in the fetch stage; the build runs under set -eux and fails loudly if verification fails. Nothing from alpine reaches the final image except the files explicitly staged under /rootfs.
  • The runtime base is scratch, so there are no OS packages to track or bump.
What the final image contains: the complete inventory (everything a distro base would add is deliberately omitted)
Path Why it's needed
/usr/bin/telegraf the GPG-verified static (CGO_ENABLED=0) binary
/etc/telegraf/{telegraf.conf,telegraf.d} default config + drop-in dir
/etc/ssl/certs/ca-certificates.crt TLS trust roots for outbound HTTPS (e.g. influxdb_v2)
/usr/share/zoneinfo IANA tz database: time.LoadLocation, json_timezone, cron schedules
/etc/nsswitch.conf hosts: files dns so Go's resolver checks /etc/hosts before DNS
/etc/passwd, /etc/group deterministic identities: root, nobody (65534), nonroot (65532)
/home/nonroot (owned 65532) $HOME for uid 65532
/tmp (owned 65532) os.TempDir() default; scratch ships neither

Not a drop-in replacement

This is a hardening-focused variant, not a swap-in for the default/alpine images. Because there is no shell and no setcap/setpriv, it does not support the ICMP ping input, privileged ports (below 1024), or shell-out plugins like exec, snmp, and sensors. It targets network-listener and push workloads. The Dockerfile header notes the image ships no shell or OS userland — the root cause of these limitations.

Why scratch, and why a tag not a digest

Most CVEs flagged against the default/alpine images come from the parent image's userland (distro packages, the shell, coreutils) rather than from Telegraf itself (e.g. CVE-2026-48962), which is why we periodically swap between the Debian- and Alpine-based images to stay ahead of scanners. Building FROM scratch drops that whole layer, so CVE exposure comes down to the Telegraf binary plus a maintained CA bundle and tzdata. The image also runs non-root, so it satisfies Kubernetes runAsNonRoot out of the box.

Two integrity/versioning choices in this revision, per @srebhan's feedback:

  • FROM scratch, not a distroless base image. Rather than depend on gcr.io/distroless/static, we assemble the rootfs ourselves in the alpine stage. That gives a complete inventory of the image (see the table above) with nothing we don't use, and one fewer external base to track.
  • Pull by tag/version, not by pinned digest. The release is selected by TELEGRAF_VERSION and GPG-verified; the fetch base is the alpine:3.23 tag. Neither is sha256-pinned. This matches how InfluxData publishes to Docker Hub — a new release is picked up by bumping the version and rebuilding, with no per-release digest to hand-edit. (An earlier revision sha256-pinned both the tarball via ADD --checksum= and a gcr.io/distroless/static base, each of which would need a manual digest bump every release.) Integrity is still enforced, now by the GPG signature instead of a committed hash.

The tradeoff we accept: we own refreshing the CA bundle and tzdata, both sourced from the alpine:3.23 fetch base at build time. Because 3.23 is a patch-rolling tag, a plain rebuild picks up alpine's latest CA/tzdata patches within the 3.23 series — so a periodic rebuild keeps the data fresh. Once 3.23 reaches EOL, the fetch base must be bumped to the next series (an explicit part of that accepted cost). One scanning caveat: a scratch image exposes no OS-package database, so scanners report zero OS packages — a cleaner report, but the copied CA bundle and tzdata aren't scanner-visible.

Testing

Built and run locally on linux/arm64 with Docker (plain build, no --privileged). The fetch-stage GPG check gates every build.

  • Build + integrity: docker build succeeds and the fetch stage reports Good signature from "InfluxData Package Signing Key"; the build aborts if verification fails.
  • Version: telegraf --versionTelegraf 1.39.1.
  • Non-root identity: the image's USER is 65532:65532, and a procstat run (pid_finder = "native", --test) emits user="nonroot" — i.e. uid 65532 resolves to the nonroot name from the image's own /etc/passwd.
  • Genuinely shell-less: no shell or pgrep in the image — procstat's default pgrep finder errors with could not find pgrep binary, and the pure-Go native finder is required.
  • Kubernetes runAsNonRoot: in a local kind cluster, a pod with securityContext.runAsNonRoot: true referencing this image is admitted and runs (Completed), confirming the numeric USER 65532 passes the kubelet's non-root check. (A named USER would fail that check — hence numeric.)

Reproduce locally:

cd telegraf/1.39/distroless
docker build -t telegraf:1.39-distroless .   # plain `podman build` works too

docker run --rm telegraf:1.39-distroless --version           # -> Telegraf 1.39.1
docker inspect -f '{{.Config.User}}' telegraf:1.39-distroless # -> 65532:65532

@srebhan srebhan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your work @victor-gama!

I don't think we need the staging approach as we can utilize the docker commands to get, check and unpack the released Telegraf binary.
Furthermore, I would really love to learn on how the circle-test.sh approach works. Do we need to execute this manually or is this something auto-executed?

Please also test at least the following:

  • use a plugin to connect to an https endpoint for verifying TLS works
  • use a plugin to parse times with timezone information to make sure the timezone-info is available

Comment thread telegraf/1.39/distroless/Dockerfile Outdated
@victor-gama
victor-gama force-pushed the victor-gama/addTelegrafDistroless branch from 53f8cf0 to d81672b Compare July 5, 2026 03:30
Comment thread circle-test.sh Outdated
@victor-gama
victor-gama force-pushed the victor-gama/addTelegrafDistroless branch 4 times, most recently from dece662 to d587fc1 Compare July 9, 2026 21:41
@victor-gama
victor-gama requested a review from srebhan July 13, 2026 16:03
@srebhan srebhan changed the title feat: add telegraf:<version>-distroless variant feat(telegraf): Add container based on the scratch base-image Jul 14, 2026

@srebhan srebhan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @victor-gama! The PR looks mostly good, I have some small comments/questions...

Furthermore, can you please remove everything except v1.39!? I'm not planning to add this for older versions...

Another point are the testing scripts. Can we please remove them as we don't have those anywhere else and I don't want to add to credit consumption here...

Comment thread telegraf/1.39/distroless/Dockerfile
Comment thread telegraf/1.39/distroless/Dockerfile
Comment thread telegraf/1.39/distroless/Dockerfile Outdated
@srebhan srebhan changed the title feat(telegraf): Add container based on the scratch base-image feat(telegraf): Add distroless container based on scratch base-image Jul 15, 2026
@victor-gama
victor-gama force-pushed the victor-gama/addTelegrafDistroless branch from d587fc1 to caf14d7 Compare July 17, 2026 03:08
Comment thread telegraf/1.39/distroless/Dockerfile Outdated
Comment thread telegraf/1.39/distroless/Dockerfile
Comment thread telegraf/1.39/distroless/Dockerfile
@victor-gama
victor-gama force-pushed the victor-gama/addTelegrafDistroless branch from caf14d7 to 37369f2 Compare July 21, 2026 04:45
@victor-gama
victor-gama requested a review from srebhan July 21, 2026 04:58

@srebhan srebhan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @victor-gama!

# unmapped userns id -> overflow: https://man7.org/linux/man-pages/man7/user_namespaces.7.html
# distroless nonroot = 65532: https://github.com/GoogleContainerTools/distroless/blob/main/common/variables.bzl
# (nobody is kept in the map so 65534-owned files still resolve; we just never run as it.)
printf 'root:x:0:0:root:/root:/sbin/nologin\nnobody:x:65534:65534:nobody:/nonexistent:/sbin/nologin\nnonroot:x:65532:65532:nonroot:/home/nonroot:/sbin/nologin\n' > /rootfs/etc/passwd; \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would name the uid 65532 entry "telegraf" rather than "nonroot" (keeping the numeric USER and uid exactly as they are). The name is about to become a contract: procstat's user tag, anything else resolving the uid, and our docs all say "telegraf" in the other variants, and renaming after release is a visible behavior change. "nonroot" buys familiarity with the distroless ecosystem, but for a telegraf image I think parity with the siblings wins. Happy to be argued out of it, but let's decide now rather than after the tag exists. @srebhan please let me know you thoughts on this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also prefer telegraf if this doesn't collide with k8s conventions of "non-root".

# abort under set -eux. A numeric id skips the passwd lookup entirely.
# busybox chown name lookup + err string (xget_uidgid; mirror linked from busybox.net/source.html):
# https://github.com/vda-linux/busybox_mirror/blob/ec0c5cc142f1f9ea57235df5d093fbe180ad9c7d/libpwdgrp/uidgid_get.c#L77-L80
mkdir -p /rootfs/home/nonroot /rootfs/tmp; chown 65532:65532 /rootfs/home/nonroot /rootfs/tmp

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth a note in the image docs: /tmp and /home/nonroot are writable only by uid 65532, so runtimes that assign an arbitrary uid (OpenShift's default SCC, an explicit runAsUser) get no writable /tmp or HOME, unlike the siblings whose base images ship /tmp as 1777. Given the buildah mode-preservation caveat you documented, I think documenting the constraint is the right call rather than fighting it, but it should be written down where users will find it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is really: How about for buildx? If any of those does preserve the mode I would set it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildx does preserve it, mode and ownership both. I built the same shape the Dockerfile uses:

FROM alpine:3.23 AS fetch
RUN mkdir -p /rootfs/tmp /rootfs/home/nonroot; \
    chmod 1777 /rootfs/tmp; chmod 0750 /rootfs/home/nonroot; \
    chown 65532:65532 /rootfs/tmp /rootfs/home/nonroot
FROM scratch
COPY --from=fetch /rootfs/ /

Exporting the final image (buildx v0.25.0) gives:

drwxrwxrwt  65532 65532  tmp/
drwxr-x---  65532 65532  home/nonroot/

So the sticky bit survives, and since official-images publishes through BuildKit that is the path that matters here. Worth setting chmod 1777 on /rootfs/tmp: it takes effect under buildx, and under buildah it is a no-op that costs nothing.

That also closes something I was going to raise separately. As it stands /tmp is writable only by uid 65532, so a runtime that assigns an arbitrary uid (OpenShift's default SCC, an explicit runAsUser) gets no writable /tmp, unlike the alpine and debian variants whose bases ship /tmp as 1777. With the mode set, that difference goes away rather than needing a caveat in the docs.

I only verified the buildx half, so I can't speak to whether the buildah behavior in the comment is accurate. Either way the comment needs a reword, since as written it reads as "mode bits do not survive COPY" in general when it is specific to the builder we do not publish with.

Comment thread telegraf/1.39/distroless/Dockerfile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants