-
Notifications
You must be signed in to change notification settings - Fork 48
fix: Remove gulp-decompress and align with upstream untar #788
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
750dbed
062b3fe
a7f3e21
a87de45
e8f7c53
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,8 +3,7 @@ | |
| * Licensed under the MIT License. See License.txt in the project root for license information. | ||
| *--------------------------------------------------------------------------------------------*/ | ||
|
|
||
| import { gulp, rename, replace, filter, jsonEditor } from './lib/gulp/facade.ts'; | ||
| import decompress from 'gulp-decompress'; | ||
| import { gulp, rename, replace, filter, flatmap, gunzip, jsonEditor } from './lib/gulp/facade.ts'; | ||
| import * as path from 'path'; | ||
| import es from 'event-stream'; | ||
| import * as util from './lib/util.ts'; | ||
|
|
@@ -18,6 +17,7 @@ import { getProductionDependencies } from './lib/dependencies.ts'; | |
| import { readISODate } from './lib/date.ts'; | ||
| import vfs from 'vinyl-fs'; | ||
| import packageJson from '../package.json' with { type: 'json' }; | ||
| import { untar } from './lib/util.ts'; | ||
| import File from 'vinyl'; | ||
| import * as fs from 'fs'; | ||
| import glob from 'glob'; | ||
|
|
@@ -319,7 +319,7 @@ function nodejs(platform: string, arch: string): NodeJS.ReadWriteStream | undefi | |
| const downloaded = (nodejsArtifactFeed ? | ||
| fetchNodejs(expectedName!, checksumSha256) : | ||
| fetchUrls(`/dist/v${nodeVersion}/node-v${nodeVersion}-${platform}-${arch}.tar.gz`, { base: 'https://nodejs.org', checksumSha256 }) | ||
| ).pipe(decompress()) | ||
| ).pipe(flatmap(stream => stream.pipe(gunzip()).pipe(untar()))) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- target context ---'
sed -n '1,45p;300,345p' code/build/gulpfile.reh.ts
printf '%s\n' '--- flatmap declarations and usages ---'
rg -n -S 'flatmap|from[[:space:]]+.*flatmap|require\(.*flatmap' --glob '!node_modules' .
printf '%s\n' '--- dependency declarations ---'
rg -n -S '"(flatmap|gulp-flatmap|through2|gunzip-maybe|untar|tar-stream|decompress)"|flatmap|gulp-flatmap' package.json code package.json yarn.lock package-lock.json pnpm-lock.yaml 2>/dev/null || true
printf '%s\n' '--- relevant tracked files ---'
git ls-files | rg '(^|/)(package(-lock)?\.json|yarn\.lock|pnpm-lock\.yaml|gulpfile\.reh\.ts|.*flatmap.*)$'Repository: che-incubator/che-code Length of output: 50379 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- package metadata ---'
find . -maxdepth 3 -type f \( -name 'package.json' -o -name 'yarn.lock' -o -name 'package-lock.json' -o -name 'pnpm-lock.yaml' \) -print
printf '%s\n' '--- exact imports and calls ---'
rg -n -C 8 'flatmap|gunzip|untar|decompress' code/build/gulpfile.reh.ts
printf '%s\n' '--- local flatmap source or documentation ---'
rg -n -S -g '*.js' -g '*.ts' -g '*.md' -g '*.json' 'flatmap|gulp-flatmap' . --glob '!node_modules' | head -200Repository: che-incubator/che-code Length of output: 50379 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- facade implementation ---'
sed -n '1,70p' code/build/lib/gulp/facade.ts
printf '%s\n' '--- local untar and gunzip implementations ---'
rg -n -C 12 'function untar|const untar|export .*untar|function gunzip|const gunzip|export .*gunzip' code/build/lib code/build
printf '%s\n' '--- gulp-flatmap package metadata ---'
python3 - <<'PY'
import json
from pathlib import Path
p = json.loads(Path('code/package-lock.json').read_text())
for key in ('node_modules/gulp-flatmap', 'node_modules/gulp-flatmap/node_modules/through2'):
print(key, json.dumps(p['packages'].get(key), indent=2))
PY
printf '%s\n' '--- gulp-flatmap published source ---'
curl --fail --silent --show-error https://registry.npmjs.org/gulp-flatmap/1.0.2 \
| python3 -c 'import json,sys; print(json.load(sys.stdin)["dist"]["tarball"])' \
| xargs -r curl --fail --silent --show-error \
| tar -xzO --wildcards 'package/*.js' 'package/*.ts' 2>/dev/nullRepository: che-incubator/che-code Length of output: 13616 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- untar parser error paths ---'
sed -n '440,515p' code/build/lib/util.ts
printf '%s\n' '--- gulp-flatmap error-handler behavior ---'
curl --fail --silent --show-error https://registry.npmjs.org/gulp-flatmap/1.0.2 \
| python3 -c 'import json,sys; print(json.load(sys.stdin)["dist"]["tarball"])' \
| xargs -r curl --fail --silent --show-error \
| tar -xzO package/index.js \
| nl -ba | sed -n '35,85p'
printf '%s\n' '--- standalone Node pipe error propagation probe ---'
node - <<'JS'
const { Readable, Transform } = require('node:stream');
const source = new Readable({
read() {
this.push(Buffer.from('input'));
this.push(null);
process.nextTick(() => this.emit('error', new Error('source-error')));
}
});
const destination = new Transform({
transform(chunk, encoding, callback) {
callback(null, chunk);
}
});
let destinationErrors = 0;
let sourceErrors = 0;
source.on('error', () => sourceErrors++);
destination.on('error', () => destinationErrors++);
source.pipe(destination);
setTimeout(() => {
console.log(JSON.stringify({ sourceErrors, destinationErrors }));
}, 25);
JSRepository: che-incubator/che-code Length of output: 1290 Use an error-propagating pipeline for
🤖 Prompt for AI Agents |
||
| .pipe(filter('**/node')) | ||
| .pipe(util.setExecutableBit('**')) | ||
| .pipe(rename('node')); | ||
|
|
@@ -328,7 +328,7 @@ function nodejs(platform: string, arch: string): NodeJS.ReadWriteStream | undefi | |
| case 'alpine': | ||
| return nodejsArtifactFeed ? | ||
| fetchNodejs(expectedName!, checksumSha256) | ||
| .pipe(decompress()) | ||
| .pipe(flatmap(stream => stream.pipe(gunzip()).pipe(untar()))) | ||
| .pipe(filter('**/node')) | ||
| .pipe(util.setExecutableBit('**')) | ||
| .pipe(rename('node')) | ||
|
|
||
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 20104
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 44774
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 367
Propagate extraction errors to the task callback.
gunzip()anduntar()can emit errors that the listener aftergulp.dest()does not receive. Usestream.pipeline()or forward both errors to a single callback and clean up the temporary directory on failure.🤖 Prompt for AI Agents
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 6070
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 50378
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 211
🌐 Web query:
node-tar v7.5.22 Parser entry.path path traversal validation versus Unpack source💡 Result:
In node-tar, the path traversal protections rely on validating entry paths and link targets against the extraction root [1][2][3]. However, vulnerabilities have historically stemmed from mismatches between how these paths are validated and how they are eventually resolved or used during file system operations [1][4][5]. Key differences between entry.path validation and Unpack source behavior include: 1. Path Resolution Semantics: Vulnerabilities often arise because security checks perform path resolution (e.g., using path.resolve or normalize) based on different baselines or context than the actual file system operation [1][5]. For example, a validation check might treat a path as safe because it appears relative to an entry's parent directory, while the subsequent extraction operation resolves it against the entire extraction root (this.cwd), allowing it to escape the intended directory [1]. 2. Hardlink and Symlink Target Handling: Even when entry.path itself is validated, the linkpath (for hardlinks and symlinks) may not be correctly constrained [2][5]. Attackers have historically exploited this by providing linkpaths that resolve to absolute paths outside the extraction directory [2]. Because the library's security checks were sometimes string-based or failed to account for symlink chains on disk, the validation could be bypassed, leading to arbitrary file read or overwrite [5]. 3. Normalization and Stripping Inconsistencies: Issues have been identified where paths are validated before they are "stripped" (e.g., removing absolute roots or drive letters) [4][6]. In such cases, the security check runs on the original, un-sanitized input, while the extraction operation proceeds with a modified, unsafe path that was incorrectly permitted by the initial check [4]. In versions like 7.5.22, the library has undergone numerous fixes to align these validation and resolution logics, but historically, the complexity of correctly handling absolute paths, drive-relative paths (on Windows), and symlink-based escapes across varying Node.js environments has made achieving perfect isolation difficult [7][4][8][5]. Users are strongly advised to keep node-tar updated to the latest version and, when processing untrusted archives, to use a filter function to explicitly reject or validate suspicious entries [6][3].
Citations:
🌐 Web query:
gulp.dest vinyl file path relative path traversal behavior vinyl-fs dest💡 Result:
In gulp and vinyl-fs, the gulp.dest method determines the output file path by joining the destination directory with the file's relative path [1][2]. The relative path is calculated as the portion of the file's path that follows its base directory (file.relative = path.relative(file.base, file.path)) [3][2]. Regarding path traversal behavior, vinyl-fs does not explicitly sanitize the relative path to prevent directory traversal attacks (e.g., sequences like../) [1][4]. If a Vinyl object enters the stream with a relative path containing such segments, gulp.dest will resolve that path relative to the destination directory [2][5]. This can result in files being written outside the intended destination directory if the relative path resolves to a location above it [1][2]. Because gulp.dest relies on the integrity of the Vinyl object's properties (specifically path, base, and relative), developers should ensure that any user-controlled input or untrusted file paths are validated before they are processed as Vinyl objects [6]. If a source provides malicious path information, the destination stream will treat it as legitimate and perform the write operation at the resolved location on the file system [4][7]. In summary, the design of vinyl-fs assumes that the file metadata (base and path) is trustworthy [6]. To mitigate potential path traversal, you should sanitize or validate the file paths in the pipeline before they reach gulp.dest if the source of those files is not fully trusted [6][5].
Citations:
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 5057
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 26830
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 868
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 12656
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 953
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 2173
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 43903
🏁 Script executed:
Repository: che-incubator/che-code
Length of output: 11599
Confine archive paths and forward all extraction errors.
tar.Parserdoes not reject../or absolute entry paths.VinylFilepreserves these paths, andgulp.dest()/vfs.dest()resolves them against the destination. A crafted archive can therefore write outside the target directory.untar()before emitting files forcode/build/gulpfile.cli.ts#L88-L89,code/build/gulpfile.reh.ts#L322, andcode/build/gulpfile.reh.ts#L331.gunzip()anduntar()errors tocallback; the listener ongulp.dest()does not receive intermediate stream errors.gunzip()errors.gulp-flatmapforwardsuntar()errors, but the innergunzip()error remains unhandled.📍 Affects 2 files
code/build/gulpfile.cli.ts#L88-L89(this comment)code/build/gulpfile.reh.ts#L322-L322code/build/gulpfile.reh.ts#L331-L331🤖 Prompt for AI Agents