[dmt] Harden chart loading against symlink loops and oversized files#421
Open
ldmonster wants to merge 5 commits into
Open
[dmt] Harden chart loading against symlink loops and oversized files#421ldmonster wants to merge 5 commits into
ldmonster wants to merge 5 commits into
Conversation
Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
85379c7 to
ee8fdca
Compare
…ink-occures-scan-loop Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dmt lints modules by handing each chart directory to nelm's chart loader. That loader walks the tree following symbolic links with no cycle detection and reads every file fully into memory. Two classes of pathological input therefore take down the whole process:
found symbolic link in path: …line per step, so the process also floods stdout/journald before it dies.This branch makes dmt survive both: it pre-flights every chart directory, lints a cleaned copy when a symlink loop is present (surfacing the loop as a finding instead of crashing), caps the size of any single file it will read, and lets the large-file warning be excluded per module. nelm is bumped to a release whose own loader no longer follows symlink cycles unbounded.
Motivation
A module in the wild shipped a chart with a
templates/loop -> ..symlink. Running dmt against it drove nelm's loader into an unbounded walk that consumed all memory and produced an effectively unbounded log stream before the process was killed. Linting should degrade gracefully on hostile input, not crash and flood the logs.What changed
1. Symlink-loop protection (
add symlink resolver)internal/helm/guard.go(new) — a pre-flight pass over the chart dir that mirrors how nelm's loader classifies files (follows symlinks, honours.helmignore) but tracks the resolved real path of every directory on the descent stack, so it can:scanChartDir→hasLoop), andmaterializeCleanChartDircopies the chart into a temp dir with symlinks dereferenced and loops skipped, so nelm's loader can walk it safely.internal/helm/render.go—RenderChartFromDirnow pre-flights the directory: on a symlink loop it renders a cleaned copy (loops skipped, symlinks dereferenced) so linting still proceeds; an oversized tree is a hard error. A newOnSymlinkLoopcallback lets the caller surface each skipped loop as a lint finding rather than dropping it silently.internal/module/controller.go— wiresOnSymlinkLoopto emit asymlink-loopwarn-level finding, so the module is still linted but the loop is reported.internal/module/sympath.go— dmt's own forkedWalkgains the same ancestor-based cycle detection (skips an already-visited real path with a warning instead of recursing forever).go.mod/go.sum— bumpgithub.com/werf/nelmv1.24.3 → v1.26.1.2. Cap the size of any single file dmt reads (
add truncate big files)internal/fsutils/fsutils.go— newfsutils.ReadFile, which behaves likeos.ReadFilebut refuses files larger thanMaxLintableFileSize(10 MiB), returningErrFileTooLarge. HelperIsFileTooLarge(err)lets scanners treat it as "skip this file" rather than a hard failure.pkg/linters/no-cyrillic/rules/{files,library}.gopkg/linters/docs/rules/{cyrillic_in_english,no_lang_key}.gopkg/linters/openapi/rules/crds.gopkg/linters/templates/rules/{grafana_dashboards,prometheus_rules,mount_points}.gopkg/linters/container/rules/mount_points.go3. Exclude the large-file warning per module (
add exclude for big files)pkg/config.go/pkg/config/linters_settings.go— newdocumentation.exclude-rules.file-sizeconfig (files/directories). These excludes affect the large-file size warning only; the documentation content checks themselves are unchanged.internal/module/module.go—mapDocumentationExclusionswires the new config through (previously documentation had no exclusion mapping).Behavior changes
symlink-loopwarning.exclude-rules.file-size.