intel-db

Public mirror of the Codeward Intel database: signed, offline SQLite snapshots of vulnerability, package, licence and CWE data, published as GitHub Release assets.

There is no data in this git repository. Everything lives in the releases — the git tree holds only the publisher that puts it there.

Using it

Codeward scanners use this mirror automatically. With no CODEWARD_INTEL_TOKEN set, a scan downloads the segments it needs from the latest release, verifies the manifest signature and every segment checksum, and runs entirely offline afterwards. Nothing to configure.

Use latest, not a dated tag. Only the newest few snapshots are kept — older releases are pruned along with their tags, so a URL pointing at a specific snapshot-… tag will start returning 404 within days, sometimes within a day. Dated tags exist so a release has a stable identity and so an operator can roll back; they are not a pinning mechanism and this mirror makes no availability promise about them. If you need a snapshot to stay fetchable, download it and host the copy yourself.

Fetch by hand:

base=https://github.com/codeward-io/intel-db/releases/latest/download
curl -fLO $base/manifest.json
curl -fLO $base/core.db.zst      # or core.db for the uncompressed file

What a release contains

Asset	What it is
manifest.json	Segment list with SHA-256 checksums, record counts, and an Ed25519 signature over the checksum list
core.db	Licence data
cwes.db	CWE definitions
code_rules.db	Validated scanner code rules
vuln-<ecosystem>.db	Vulnerabilities for one ecosystem (npm, pypi, go, maven, cargo, rubygems, nuget, composer)
pkg-<ecosystem>.db	Package metadata for one ecosystem
*-enrichments.db	Optional enrichment segments
<name>.db.zst	zstd archive of each of the above (~16× smaller — this is what scanners actually download)

Both forms are published on purpose: consumers prefer the compressed file and fall back to the raw one if an archive fails to expand.

Verifying a release yourself

The manifest signature covers the sorted list of segment checksums (each compressed variant contributing a <segment>.zst entry), signed with Ed25519. Scanners verify it against a compiled-in public key, overridable with CODEWARD_INTEL_PUBKEY. Checksums are plain SHA-256 of each asset:

sha256sum -c <(jq -r '.segments[] | "\(.checksum)  \(.filename)"' manifest.json)

Attribution and licensing

The snapshots redistribute data from several upstream sources, some under CC-BY-4.0 or MIT. Each release body reproduces the required notices, and manifest.json carries them in its attribution array. The Apache-2.0 LICENSE in this repository covers the publisher code here, not the data in the releases.

How releases are produced

.github/workflows/publish-snapshot.yaml runs every six hours (30 minutes after Intel's own snapshot generation) and on demand:

1. Fetch manifest.json from the Intel API.
2. Fail if it is older than 24 hours — a wedged generator upstream surfaces here rather than becoming a silently stale global mirror.
3. Verify the Ed25519 signature against CODEWARD_INTEL_PUBKEY. An unsigned or unverifiable manifest is never published: deployed scanners would reject it while a dev build would look fine.
4. Compare the content digest with the currently published release and stop if nothing changed — most six-hourly runs are no-ops.
5. Download every asset, verifying each checksum before it is kept.
6. Create a draft release, upload everything, then flip it to published + latest in one step. Assets are never mutated on a published release.
7. Verify releases/latest/download/manifest.json serves what was just published, then prune releases beyond the retention window.

Run the same preparation locally against a scratch directory:

go run ./cmd/snapshotctl --api https://intel.codeward.io --out ./stage --pubkey <hex>

Operator setup, credentials and recovery procedures: docs/PUBLISHING.md.