Skip to content

Report what a scan actually examined, and stop the manifest under scan from steering the fetch - #5

Merged
thebenignhacker merged 27 commits into
mainfrom
fix/deep-analysis-verdict-and-cache
Jul 30, 2026
Merged

Report what a scan actually examined, and stop the manifest under scan from steering the fetch#5
thebenignhacker merged 27 commits into
mainfrom
fix/deep-analysis-verdict-and-cache

Conversation

@thebenignhacker

Copy link
Copy Markdown
Member

Everything for the v1.3.0 release. 61 commits since the last released version, v1.2.2.

What this fixes

A scanned manifest could execute code on the host running --deep. The dependency name and version were taken from the manifest and handed to a package manager and the filesystem without being held to any grammar, so a repository chosen by an attacker could read a directory, write a file, or run code on the scanning machine. Names are now held to their own registry's grammar, npm pack runs with --ignore-scripts and after a -- separator, and pip download runs with --only-binary=:all:. A security advisory with a per-vector table follows this release.

A scan reported coverage it never had. In several distinct ways: a package whose archive held nothing readable was counted as examined; a source file the analyzer refused was dropped silently while the package still claimed examination; a failed extraction was accepted as a cache hit on the next run; a file with one line over 64 KiB contributed nothing; and a name grammar added during this work refused 1,054 real published npm packages, which is the same false clean reached from the other direction.

A CI gate that loosened silently. --fail-on was validated one way and read another, accepted any value, and accepted an empty one. And, found in this release's own final review, making --risk and --min-severity actually filter let them decide the exit code: --fail-on vulnerable --risk safe exited 0 on a project that exits 1 without the filter, while the same run printed "This is not a clean result". Released 1.2.2 exits 1 for both, so that was a regression, and it is fixed rather than disclosed.

Verification

Six adversarial rounds and six fresh-user passes across three sessions. Rounds one to four each found defects inside the previous round's fix; rounds five to seven found defects only inside fixes made during this release, including two of my own corrections that were measured against the wrong artifact.

  • CBOM validates against the official CycloneDX 1.6 schema and SARIF against SARIF 2.1.0, both fetched upstream, zero errors.
  • Formats reconcile exactly, and three runs of each are byte-identical once scanDate and the CBOM serial are normalised.
  • Exit-code behaviour A/B'd against the released 1.2.2 binary with the database held constant, so the two runs differ only in the binary.
  • Nine of nine mutations of the gate are killed, on both the single-project and workspace paths.
  • Zero emoji, em-dashes or ANSI across all five formats.

Known limitations

The changelog's known-limitations section is long and deliberately so. Every entry was reproduced by hand against both the 1.2.2 and the 1.3.0 binary, and each states which database it was measured against, because two different databases report the same version label. The ones that matter most: a database record can answer for a different version than the one declared; a file skipped by name is neither analyzed nor counted; and 780 of the 849 published records are name-inferred, which fabricates primitives that packages do not contain.

No behaviour change. The binary, its output and its exit codes are untouched;
this is the changelog, the README, the GoReleaser config and the smoke doc.

CHANGELOG: the file carried both an [Unreleased] section and a [1.3.0] section
dated 2026-07-27 for a release that was never tagged, so cutting [Unreleased]
would have produced two 1.3.0 headings. Both are merged into one 1.3.0 dated
today. The 1.2.2 entry that was never written is added, and the compare links
now use the canonical repository name.

The changelog claimed "status now reports 901". That was true of the database
the fix was measured against and is already false: the shipped file holds 849
after the scheduled refresh of 2026-07-28. The claim now describes the
behaviour rather than pinning a number a weekly job can invalidate.

Added a known-limitations section. Every item was reproduced by hand against
both the 1.3.0 and the 1.2.2 binary during the release test, and all of them
are present in 1.2.2: hybrid post-quantum components reported as
quantum-vulnerable, unstable Maven coverage in the downloadable database,
findings with no line number, GITHUB_TOKEN breaking analyze <url> against
public repositories, an unverified database download, --deep needing pip rather
than pip3, and an unsupported-only tree exiting 2.

README: "Crypto database of 901 packages (69 verified, 832 inferred)" was
stated twice and is now wrong on both counts for the same reason. Replaced with
a statement that does not drift, pointing at `cryptodeps status` for the
numbers a given install actually holds. The sample output block reproduces
exactly against a current build except for the scan-root header line and
`Scanning go.mod`, where the tool prints `./go.mod`; both corrected. The
exit-code table now says what exit 2 covers, which is this release's headline
change. The subtitle promised to "find every cryptographic vulnerability in
your dependencies", which the known limitations contradict. Removed the eleven
em-dashes, which the CSNP style rule does not allow.

.goreleaser.yml: archives.format and format_overrides.format are deprecated and
failed `goreleaser check`. Migrated to the formats list. A snapshot release
produces the same five archive names as v1.2.2, version token aside, so no
download URL changes.

docs/testing/release-smoke.md was the generic web-app stub, asking whether the
site renders at 375px. Rewritten as the walkthrough this tool actually needs,
including building the previous released binary to compare against, which is
what has separated pre-existing from regression on every release here.
The verdict that says whether a scan established anything asked whether the
crypto database carried each dependency. That was the same question as "was
this dependency examined" only while the database was the only way to examine
one. --deep is a second way, so a scan that read the source of every dependency
and found no cryptography reported that it had examined nothing and told the
user to run --deep, which is what they had just run. The JSON for the same
invocation reported deepAnalyzed on every dependency, so one command produced
two contradictory answers, and the previous release printed the correct verdict
for this case.

Coverage is now counted where examination is decided, as summary.notExamined,
with DependencyResult.Examined() naming the distinction so that a third means of
examination has one place to declare itself. Every format reads that field.
summary.deepAttempted separates "not examined because you did not ask for source
analysis", where the next step is --deep, from "not examined because the package
could not be fetched", where suggesting --deep is a dead end.

That fix makes the verdict trust DeepAnalyzed, which was set for packages whose
source was never read, so the fetcher is fixed first. The npm and PyPI fetchers
returned one directory after extracting an archive and a different one on a
cache hit, and the cache-hit test asked only whether the version directory
existed; a directory holding nothing but the downloaded tarball satisfies that,
so its contents were walked, nothing was found, and the package was recorded as
analyzed. Both paths now resolve the extracted source through one helper, and an
entry that holds none is refetched rather than accepted. An npm tarball that
does not unpack to package/, as ejs does not, is now analyzed on the first run
instead of failing there and being read from the wrong root afterwards.

Fixing the cache path also closes a way for a scanned manifest to steer it. The
path was built from the dependency name and version exactly as declared, so
{"ejs": "../../../somewhere"} resolved the cache entry outside the cache
directory, and --deep walked that directory and published its absolute file
paths and line numbers as the dependency's source. Manifest-supplied segments
are now sanitized, and a version naming a local path is refused rather than
handed to npm pack or pip download, which would resolve it.

Also folded in: pyproject.toml and Pipfile left the comparison operator in the
version string, so the same package read as pycryptodome@==3.20.0 there and
pycryptodome@3.20.0 through requirements.txt, and the CBOM purl disagreed with
the JSON and SARIF of its own scan.

Every regression test here was confirmed to fail against the code it guards, and
each was mutation-tested. Four test fixtures encoded the behaviour being fixed
and were rewritten: three cache layouts that the fetcher never produces, and the
expectation that a version constraint keeps its operator.
The release-test walkthrough showed the known-limitations entry was misleading
about when the download happens. It is not something a user opts into by running
update: the first analyze on a machine with no ~/.cryptodeps fetches the
database and writes it with no output at all, so a fresh user is scanning
against downloaded, unverified data on their first command. Documentation only.
Neither blocks the release and neither is a regression, but both are things a
user can hit and neither was written down.

--fail-on any fails a project whose cryptography is entirely quantum safe and
reports it with the partial-risk exit code, while the flag help describes it as
exiting on risk. The behaviour is deliberate, the help text and the exit code
are both wrong for it, and it is byte-identical on 1.2.2.

--offline silently disables --deep, so a scan whose dependencies are all absent
from the database ends by suggesting the flag that was just passed. The verdict
itself is fixed in this release; the suggestion that follows it is not.
The traversal fix closed one field of a class that spans three. Two were still
open, and one of them is a write.

A dependency NAME reaches the package manager as a spec. cacheSegment kept it
from steering the cache path, which is what made it look handled, but npm pack
resolves a name that names a directory, so {"../../../../victim": ""} packed a
tree outside the project, walked it, and published its file names and line
numbers. A path and a package spec are different guarantees. Both fields are now
checked at the point the fetch is built, for every ecosystem.

fetchMavenArtifact joined the artifactId from the scanned pom.xml onto the path
given to curl -o, and filepath.Join resolves .. lexically, so the download target
escaped the cache. A ? in the artifactId splits the Maven URL so its path portion
still names a real artifact while the file portion traverses, supplying the 200
that curl -f needs before it will write. A 26-byte file outside the cache became
234540 bytes of an unrelated archive on one analyze --deep. Coordinates are now
validated as coordinates, which holds for the URL and the path together, and the
written file name is derived from sanitized segments regardless of that.

Both regression tests were confirmed red at runtime against the previous commit,
and both failed there for the right reason: the pre-fix code refused only
incidentally, because no downloader was on PATH. Each has a companion guard
asserting that real names and real Maven coordinates are still accepted, and the
Maven test asserts that its fixture genuinely escapes the cache before trusting
anything else it measures.

The changelog said no write or execute primitive was involved. That was written
from the shape of the first defect rather than from the subsystem, and it was
false. Corrected.
The one defect from the adversarial pass that is not fixed yet. A Maven artifact
with no sources JAR falls back to the main JAR, which holds compiled classes, so
the walker reads nothing and returns nothing, and the absence of an error is
taken as an examination. Not a regression, since 1.2.2 answers the same scan
with a clean verdict too, but the new wording claims something the old wording
did not.
A deep scan of a Maven artifact with no sources JAR falls back to the main
JAR, which holds compiled classes only. The Java walker accepts .java, .kt
and .kts, found none, and returned no usages and no error, so DeepAnalyzed
was set from the absence of that error and the report said "no cryptographic
usage detected in the 1 of 1 dependencies that were examined" for a scan that
parsed zero files.

The walkers now return what they parsed alongside what they found, the count
travels with the analysis as filesAnalyzed, and the decision to claim an
examination is made from it. A package whose archive carries nothing readable
is reported as not examined, with the reason on stderr and in the document, so
that a consumer of JSON or SARIF can tell it from a package that was skipped.
…tify

The reset is the only destructive operation in the fetcher and its argument
is built from manifest-supplied text, yet nothing stated how much it was
allowed to remove: four mutations of it, up to RemoveAll of the entire cache
root, left the suite green. It now refuses anything that is not exactly one
package entry, three levels below the cache root, and the failed-download
paths route through the same guard instead of calling RemoveAll directly.

An entry holding more than one candidate directory is now kept and reported
rather than deleted. Deleting it destroyed a cache entry that may have held
the findings, and because all three cache-hit sites discarded the error, the
user was then shown whatever the refetch failed with instead of the
diagnostic written for the case.
The guard read a colon as evidence that a reference was remote, so a drive
letter passed it: a manifest declaring C:\victim, \victim or \\host\share
reached npm as a folder spec on a target this tool ships binaries for. The
Yarn and pnpm link: and portal: spellings are refused with them, which npm
pack rejects today and which should not depend on another tool's parser
staying strict.

The Maven coordinates and aliases that legitimately carry a colon are held by
their own cases, so narrowing this does not stop packages that fetch today.
…arse

The first attempt at this class screened the dependency name for the spellings
of a local path and treated it as one opaque string. A package manager does
not: npm splits name@spec on the first @ after index 0, so a dependency named
x@/tmp/victim presented a guard-passing string to the guard and a directory to
npm, which packed it, ran its prepare and prepack scripts on the scanning host,
and published its contents as that dependency's findings. The same shape
reaches pip as a PEP 508 direct reference through a Poetry or Pipfile table
key, and git+file:// reached npm through the version, since it neither starts
with file: nor trips the traversal walk.

Names are now held to their own registry's grammar, which is the question
Maven coordinates already answered, and npm pack runs with --ignore-scripts so
that the remote git references that stay allowed cannot execute anything
either.

The examination evidence was counting files OPENED. Three of the four analyzers
are line scanners whose only failure mode was os.Open, so a zero-byte
Empty.java or compiled class bytes under a .java name counted as parsed and
produced the exact verdict this release exists to remove. A file now counts
only if it holds text the analyzer can read, and a symlink is not read at all:
tar and unzip both restore absolute symlink targets, so an archive could ship
Leak.java pointing at any file the process could read and have it published as
a dependency's source.
The rune-boundary allowance in the new text check shrank the inspected head
one byte at a time while it was invalid UTF-8, and utf8.Valid on an empty
slice is true, so any file over the head size with no NUL byte was accepted as
source: 2000 bytes of 0xff counted as a parsed file and would have restored
the examined-but-unread verdict for an archive carrying it.

The allowance is now bounded to the three continuation bytes a truncated rune
can occupy, and an empty head is not text.
The text check that made examination mean a parse required the head of a file
to be valid UTF-8. These analyzers are line scanners matching ASCII
identifiers, so they never needed valid UTF-8, and the requirement refused
legitimate source in any single-byte encoding: one Latin-1 accent in a comment
made a .java or .js file unreadable, its cryptography was never looked for, and
the released 1.2.2 had found it.

The refusal was also silent. FilesFailed was counted inside the walk and went
no further, so a package holding one file the analyzer read and one it refused
was reported as examined and clean, on every stream and in every format, with
zero bytes on stderr. Together those two made an evasion: a dependency could
hide its cryptography from this scanner by encoding the file that calls MD5 in
Latin-1, and the report said it had been examined and was clean.

A file is now text if it holds no NUL byte and its head is either valid UTF-8 or
predominantly printable ASCII, which is the property a line scanner actually
needs. The threshold is measured rather than chosen: across 140 real npm source
files carrying non-ASCII characters, re-encoded to Latin-1, the lowest
printable-ASCII fraction was 0.904 and the median 0.999, while random NUL-free
bytes reached at most 0.453 and real binaries with their NUL bytes stripped
reached 0.286. The NUL-free binary that prompted the previous commit is still
refused at 0 percent.

The count of refused files now travels with the analysis as filesUnreadable,
sums into the summary, and reaches every format through the one classifier they
all share, because the earlier fix asked the disclosure question of a package
that was not examined at all and never of one examined in part.
Five defects, four of them inside changes this release made, all reproduced by
hand against the candidate binary before being accepted.

A version traversed out of the cache behind any scheme. localPathReference
skipped its traversal walk whenever the value contained a colon, reasoning that
a colon meant a remote reference such as github:owner/repo. An invented scheme
defeats that: "a1:../../../../../../../victim" carried a colon, so the walk never
ran, npm pack resolved a directory outside the cache, and the analyzer read and
published it as that dependency's source. Reproduced by sentinel at depth 7. The
walk now runs on every value, because no legitimate version carries a ".." path
element, and whether a value is remote is decided by the scheme test rather than
by the presence of a punctuation mark. Maven coordinate validation moved into
validPackageName so that every ecosystem's grammar is the first thing a name
meets: reached later, it was masked by the traversal screen, which refused a
hostile coordinate for the wrong reason and left the coordinate validator
unexercised by the test written to pin it.

An extraction failure failed open on the next run. The three download sites
discard a partial fetch and the four extraction sites returned a bare error, but
tar and unzip both extract partially before failing, so the entry was left
holding a package directory that the cache-hit check accepts. The first scan
reported the package as not examined and the second reported it as examined,
with findings, from a partially extracted archive and nothing on any stream.

--fail-on was validated one way and read another. ValidateFailOn trimmed and
lowercased, determineExitCode only lowercased, so " partial " was accepted as
valid and then matched no policy: a project with partial-risk findings exited 3
for "partial" and 0 for " partial ", silently. Whitespace around a value is the
ordinary result of a YAML block scalar, which is where this flag is used. Both
now read it through CanonicalFailOn.

A line longer than 64 KiB cost a whole file. bufio's default token limit stops
Scan and reports an error, the three line-scanning analyzers return it, and the
walk discards every usage already found. A bundled dist file, which is normally
one very long line, therefore contributed nothing: 70 KB on one line calling
crypto.createHash('md5') produced no finding on this candidate or on 1.2.2. The
scanner is given the same bound the file already has.

A Poetry or Pipfile dependency declared by location lost its locator, so a
path = "../internal-lib" entry became a name with no version and the fetcher
downloaded whatever PyPI serves under that name. That is dependency confusion
performed by the scanner: registering the name of a company's local package is
enough to be handed the attribution, and pip builds sdists. The locator is kept
so the fetcher's guards see what was declared.

Also: the unreadable-file count is now accompanied by the names and reasons,
bounded at ten, because the coverage note promised the scan named them and it
did not, which is a dead end inside the disclosure written to prevent one. The
source-file cap moves to 32 MiB, measured against aws-sdk-go's 7,771,273-byte
service/ec2/api.go, which the 8 MiB cap was within 8 percent of. A dead
scanErr helper is deleted: golangci-lint enables unused, and nothing had been
pushed, so it would have failed the release CI on the first run.

Every regression test confirmed red at runtime against 5b00b61 in a worktree.
Two of them were vacuous first: the long-line fixture used a crypto spelling
this analyzer does not detect at all, so it failed for the wrong reason and now
carries a short-line control; and the extraction fixture emptied PATH, which
left its own stubs unable to run mkdir, so nothing was ever half-extracted and
the test passed against the defect.
…lose

Two claims in the 1.3.0 entry were false as written. It said every real package
name in the suite still fetches, across all four ecosystems; the table holds
three, it asserts that a grammar accepts a name rather than that a fetch
completed, and a dotted PyPI name only reaches the grammar intact through
pyproject.toml. It said each guard was mutation-tested, while several mutations
survived: a threshold no fixture could pin from below, a bound whose removal no
fixture could detect, three of four extraction sites with no test at all, and a
disclosure asserted in one of five formats. Both now state their limits, and the
mutation work is reported with its survivors.

Three disclosures added, none of them regressions, all reproduced. pip download
builds an sdist, so --ignore-scripts closes the npm half of the execution class
and not the pip half: resolving a PyPI dependency with no matching wheel runs the
target's build backend on the scanning host. A version may name an arbitrary
tarball URL or an npm alias, so a scanned manifest can direct the fetch at a host
of its choosing and mis-key the findings. go mod download inherits the
invocation directory and can write go.sum into the tree under scan.

The 1.3.0 fixes for this round are recorded under Fixed.
Three defects the fourth round's regression sweep found, two of them refusals
this release introduced. A guard on a security tool that refuses real input
produces a clean result it never established, which is the same false clean this
release exists to remove, reached from the other side.

The npm name grammar encoded the rules npm applies to a name from a NEW
publisher: a scope and a name each beginning with a letter or a digit. npm
grandfathered the names that predate those rules. Swept against the complete
registry, 4,240,864 names, the pattern refused 1,054 that are published and
installable now, nine of them above 100,000 downloads a month, including
@lingo.dev/_spec at 212,000, @-xun/fs, @_sh/strapi-plugin-ckeditor and
@~39/empty. Confirmed here against the registry and against the candidate
binary, which refuses all four with the dependency never analyzed. The grammar
now excludes what lets a name be read as something other than a name, which is
what a fetch guard is for: an at sign separates a name from a spec, a slash and
a backslash make it a path, a colon makes it a scheme, whitespace separates
arguments. A leading dot is refused separately so that loosening the grammar
cannot quietly admit it. PyPI, Go and Maven were swept the same way and refuse
nothing real: 0 of 860,284 PyPI names, 0 of 16,279 Go module paths, 0 of 1,800
Maven coordinates.

Maven properties were resolved in the version and not in the groupId or
artifactId, so the project.groupId placeholder, which is how a multi-module
build names a sibling, survived into the coordinate and was refused as a name
that could steer a fetch. 81 coordinates across 16 published artifacts in 2,099
real poms from Maven Central. On one real pom the tool analyzed 6 dependencies
where the same pom with the property expanded analyzed 11.

pip now runs with --only-binary=:all:, which is the founder's decision on the
half of the execution class that --ignore-scripts does not cover. Resolving a
source distribution makes pip install build dependencies and run the project's
build backend, so fetching source in order to read it was a way to execute code
named by the manifest under scan. Packages that publish no wheel are now
reported as not examined, with the reason, rather than analyzed by running them.
The npm grammar refusing 1,054 real names, the unresolved Maven property in a
coordinate, and the pip source-build execution path are recorded under Fixed.
The pip entry moves out of known limitations, where it was disclosed an hour
earlier as something this release does not close, because the founder's decision
is to close it: fetching source in order to read it must not be a way to run it.

What that costs is stated rather than left implicit. A PyPI package distributed
only as an sdist is now reported as not examined instead of being built, with the
reason named on stderr and carried in every format. Preferring an unexamined
dependency to an executed one is the trade, and a source-build mode behind an
explicit opt-in is the shape of the fix if the coverage turns out to matter.

The --ignore-scripts sentence said the release closed execution during a fetch
while only the npm half was closed. It now names both halves.
Found in my own fix from the commit before this one, which is where the defects
in this unit have been every round.

Loosening the name grammar to admit the names npm's registry actually carries
admits names that begin with a hyphen, and some are real: "-" has 153,000
downloads a month. npm reads a spec beginning with a hyphen as a FLAG, so a
dependency named "--help" made npm print its help text and pack nothing, and one
named "--ignore-scripts" would be swallowed with no trace at all. Untrusted input
parsed as a flag is the argument-injection half of the same class as untrusted
input parsed as a path, and widening an allowlist is how a class reopens.

A "--" separator ends flag parsing. Verified that ordinary and scoped specs are
unaffected: ejs@3.1.10 and @babel/core@7.25.2 both pack with it. The test asserts
the argv, because what matters is what the tool asks npm to do, and it pins the
separator's position as well as its presence.

The other three fetchers are unaffected: the PyPI and Go grammars require a name
to begin with a letter or a digit, and the Maven coordinate is interpolated into
a URL rather than passed as an argument.
The fifth adversarial round measured what raising the per-file cap from 8 MiB to
32 MiB costs: a 30 MiB single-line file dense with cryptographic calls peaks at
about 309 MB resident and takes 2.4 seconds. Files are read one at a time, so
that is the bound for a scan rather than a per-archive total. Stated in the
changelog beside the change rather than left for a user to discover.
The fifth fresh-user pass found no release-blocking regression: every P1 it rated
reproduces identically on released 1.2.2. What it did find is three claims in the
1.3.0 entry that were false or dangling, and two serious pre-existing defects
that entry did not disclose.

--fail-on "" was the last spelling of the gate loosening. The padded value was
fixed by canonicalising; the empty string still fell through, because "" was
listed as legal and then matched no policy. Verified: on a project whose only
findings are partial risk, "partial" exits 3 and "" exits 0, with nothing on
either stream. An unset workflow input is exactly how a CI gate arrives here
empty. This flag carries a default, so a blank value expresses nothing, unlike
--risk and --min-severity where empty means do not filter.

Three claims corrected. The Added section promised these fields "in JSON and
YAML output" in three places; there is no YAML emitter and --format yaml exits 2,
so the claim now says JSON. The npm-grammar entry said the dotted PyPI split was
recorded under known limitations when it was not; it is now, and the sentence
says the earlier draft was wrong. The long-line entry justified itself with
bundled and minified output while a file named *.min.js is skipped by name before
the analyzer sees it, so it now says what it does not reach.

Two disclosures added, both pre-existing and both verified by the pass against
the real published artifacts. A database record can answer for a different
version than the one declared: circl v1.3.9 is reported with ML-KEM and ML-DSA
SAFE from a v1.6.4 record, while the real v1.3.9 ships round-3 Kyber and
Dilithium and no ML-KEM at all, and bouncycastle 1.78.1 is the same. That is the
worst direction for this tool to be wrong in, and it reaches the README's own
example. And a file skipped by name is neither analyzed nor counted: a
162,025-byte bundle.min.js calling createHash('md5') produces no finding and no
trace, while byte-identical content named dist/app.js is found and reported.
…angelog

The README claimed "Scans all transitive dependencies, not just direct ones"
and the root help listed "Full dependency tree analysis". Neither is true.
Lock files are rejected as unsupported, so summary.totalDependencies equals
summary.directDependencies on every tree tested, and adding a package-lock.json
that declares a transitive dependency changes neither number.

Overstating coverage on the front page is the same defect this release exists
to close inside the scanner, reached from the documentation instead. The
behaviour is unchanged from 1.2.2 and is not a regression; the claim is what
changed.

Also disclosed in known limitations, each verified against this binary rather
than carried over from a report:

- The bundled Action passes --offline at both scan steps, so it sees the 72
  built-in packages rather than 849, and installs with go install @latest,
  which injects no ldflags, so it reports its version as dev. That is this
  release's own provenance fix undone by the install method.
- The CBOM component list carries only libraries with findings, so it is not a
  complete bill of materials.
- ML-KEM maps to key-agree where the CycloneDX enum offers kem, and three other
  mappings are legal but not the closest term.
- The per-analysis provenance block still emits zero values, which is separate
  from the tool version this release fixed.

Release date corrected to 2026-07-30.
The known limitation explained @noble/post-quantum's classical findings as the
hybrid halves of constructions such as X-Wing. Checked against the artifact the
database records, that is wrong. In @noble/post-quantum@0.2.0, ML-KEM, ML-DSA,
SLH-DSA and X25519 are present; RSA, ECDSA, Ed25519, AES and ChaCha20-Poly1305
do not occur in its code at all. It exports ml-kem, ml-dsa, slh-dsa and utils
and nothing else. Only X25519 is a real hybrid component.

The four others are fabricated by name inference, which supplies 780 of the
849 records. The inference appears to substring-match without word boundaries,
the same defect the deep analyzers have: the only occurrence of "rsa" in the
package is inside the identifier bitReversal, and ed25519, RSA and ChaCha occur
only in README prose. @noble/hashes@1.4.0 and @noble/ciphers@1.0.0 are credited
with the same four absent primitives, verified against 96 and 70 source files.

This is the third explanation in this release written from a plausible model
rather than from the artifact, so it is stated with the measurement attached.

Also disclosed, both verified to reproduce identically on released 1.2.2 and so
neither a regression nor a tag blocker:

- Following the tool's own remediation advice does not change the verdict. A
  project that swaps node-forge and elliptic for the @noble libraries the report
  names, removing DES, 3DES, MD5, SHA-1, secp256k1 and ECDH, gets a
  byte-identical summary and the same exit code. The migration target carries
  the same inferred core as the thing being migrated away from.
- --offline does not gate the GitHub fetch path, so analyze owner/repo --offline
  clones and scans over the network, and a mistyped one-slash local path becomes
  an outbound request.
Making --risk and --min-severity actually filter, earlier in this release, had
a side effect nobody asked for: the exit code was computed from the same
filtered summary as the report, so a display flag decided a CI outcome.

  analyze . --fail-on vulnerable              -> exit 1
  analyze . --fail-on vulnerable --risk safe  -> exit 0

Same project, thirteen findings withheld, and the same run printed "This is not
a clean result. Re-run without the filter to see them." to stdout. The tool told
the operator it had hidden findings and told CI the project was clean.
--min-severity reaches it identically, which matters more in practice:
"--min-severity high --fail-on vulnerable" is a plausible CI line that quietly
stops failing on anything below HIGH.

Released 1.2.2 exits 1 for all of these because its filters did nothing at all,
so this is a REGRESSION, in the one flag that decides CI outcomes, in the
release that hardened that flag three times against silent loosening.

A view flag does not answer a gate. The scan records what the filters withheld,
broken down by the risk levels --fail-on asks about, and the gate reads
found-not-shown alongside shown. The counters are not serialized, so all five
output formats stay byte-identical, filtered and unfiltered. --fail-on none
still means none and a clean project still exits 0; both are asserted.

determineExitCode and determineExitCodeMulti carried two copies of one switch
differing only in which summary they read, which is how they could drift. They
are now one function with two callers.

On the tests, because the first version of them was the actual defect here:

- The invariant is asserted as "a reporting filter never changes the exit code",
  over 4 fixtures x 6 filters x 4 thresholds, on BOTH the single-project and the
  workspace path. The first version covered the workspace path with a struct
  literal, so all three aggregation lines could be deleted with a green suite,
  restoring the bug on the path the CLI uses by default. Nine mutations, one per
  load-bearing line, are now all killed; six of them survived before.
- gate_filter_test.go names none of the new fields, so it compiles against the
  pre-fix commit and fails there at RUNTIME. The counter-level assertions live
  in their own file precisely so they cannot break that property. A test that
  only fails to compile has demonstrated nothing.
- The fixtures use database.NewEmbedded(), not NewWithCachedData(). The latter
  reads the developer's real ~/.cryptodeps, which would let a differently
  classified cache turn the only runtime protection for a CI gate into a skip.
Each of these was written from a plausible model rather than from the thing
itself, which is the same failure the release documents elsewhere. Kept separate
from the gate fix so that reverting code does not revert documentation.

- Dependency coverage, stated per ecosystem. A first correction in this release
  replaced an overstatement ("Scans all transitive dependencies") with a false
  understatement, by generalising one npm measurement to all four ecosystems.
  Go reads // indirect requirements: a go.mod with one direct require and two
  indirect reports totalDependencies 3, directDependencies 1, and the indirect
  golang.org/x/crypto contributes five findings. For Go 1.17 and later that
  block is the transitive closure. npm, Python and Maven are declared-only.

- Lock files are invisible, not refused. package-lock.json and poetry.lock leave
  the document's "skipped" field null, while a Cargo.toml, which genuinely has
  no parser, is reported with a reason. Describing a silent skip as a reported
  refusal is the wrong direction for this release in particular.

- The hybrid-PQC entry measured the wrong artifact. The database record for
  @noble/post-quantum answers for 0.6.1, not the declared 0.2.0, which is the
  version-mismatch limitation disclosed two entries above. In 0.6.1 ECDSA,
  Ed25519 and X25519 are all present, in hybrid.js, whose own comment describes
  a preset combining ML-KEM-768 with X25519, and AES and ChaCha are in
  falcon.js. So those findings really are hybrid halves and the earlier
  explanation was substantially right; only RSA is fabricated. The
  @noble/hashes, @noble/ciphers and @noble/curves fabrications are real, at
  2.2.0, which is the version their records answer for.

- Two databases report one version label. The published asset holds 849 records,
  780 inferred; a database cached on 2026-07-27 holds 901, 832 inferred. Both
  call themselves 1.1.0 and no scan output says which answered. Every count in
  this changelog now names the database it describes.

Also documented: the --fail-on "" refusal, which shipped as a behaviour change
with no entry, and whose neighbouring entry claimed parity with --risk and
--min-severity that does not exist (those two accept empty deliberately, since
blank means "do not filter"; --fail-on has a default and cannot express one by
being blank).
@thebenignhacker
thebenignhacker merged commit e5234a6 into main Jul 30, 2026
3 checks passed
@thebenignhacker
thebenignhacker deleted the fix/deep-analysis-verdict-and-cache branch July 30, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant