Document thrift CPE-gap accurately + add weekly NVD-CPE thrift watch

Corrects the earlier "thrift ~=0.22.0 (no known advisory)" wording, which
was wrong: thrift 0.22.0 IS affected by open Apache Thrift advisories
(CVE-2025-48431 + the CVE-2026-41602..41636 set, all fixed in 0.23.0).

Why we still hold at ~=0.22.0 and accept them:
- Apache Thrift is one monorepo shipping ~20 language libraries; the PyPI
  `thrift` package is built only from lib/py. Each of these CVEs is in a
  NON-Python binding -- verified against the upstream oss-security
  advisories: Node.js (41636), Go (41602), c_glib/C (48431), Java (41603),
  Swift (41604, 41605). None touches the Python code paths we ship.
- The only fix (0.23.0) is the version that caused SEV0 ES-1960554 on
  DBR-LTS old setuptools, so we cannot take it until a build-safe thrift
  ships (THRIFT-6067).

Why the OSV gate doesn't flag it (and why that is NOT proof Python is safe):
- These CVEs are in OSV with `affected[].package = null` -- only a GIT/CPE
  coordinate (cpe:2.3:a:apache:thrift), no PyPI/npm/Go package entry. OSV
  and Dependabot both match by package purl, so they return nothing for
  PyPI thrift. This is a coordinate blind spot, independent of whether
  Python is affected -- a FUTURE Python-affecting thrift CVE filed the same
  way would also be missed.

Mitigation: a supplementary NVD-CPE thrift watch in securityScan.yml,
scheduled/manual only (never PR; NVD rate limits). It lists all
apache:thrift CVEs affecting the locked version in the weekly summary and
hard-fails if any description names Python. Scoped to thrift alone because
an audit of all three drivers' full dependency sets found thrift is the
only dep with this purl-vs-CPE gap (Go/Node already ship the fixed 0.23.0).
The Python-detection is a heuristic (description must say python/lib/py);
the full list is always surfaced for human review.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Author: vikrantpuppala

.github/workflows/securityScan.yml

@@ -264,6 +264,88 @@ jobs:
           echo "Full step summary: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
           exit 1
 
+      # --- Supplementary: thrift NVD-CPE watch (scheduled/manual only) ---
+      # OSV-Scanner (and Dependabot) match by PyPI purl. The Apache Thrift CVEs
+      # are filed ONLY against the upstream CPE (cpe:2.3:a:apache:thrift) with
+      # no PyPI-package coordinate, so the purl gate above is structurally BLIND
+      # to them (see the thrift note in pyproject.toml). We hold thrift at
+      # ~=0.22.0 on purpose (0.23.0 fixes the CVEs but reintroduces the
+      # ES-1960554 DBR-LTS install break), and every known thrift CVE to date
+      # is in a non-Python binding -- so we are not exposed today. But that is a
+      # point-in-time fact: a FUTURE thrift CVE could land in the Python binding
+      # and the purl gate would never see it.
+      #
+      # This step closes that one blind spot cheaply: query NVD by CPE for
+      # apache:thrift advisories affecting our locked version, and LOUDLY flag
+      # any whose description mentions Python (the only case we're exposed to).
+      # It is scoped to thrift alone (not a general CPE sweep) because thrift is
+      # the sole dependency in the lock with this purl-vs-CPE gap (audited).
+      # Runs on the weekly/manual event only -- never on PRs -- because NVD
+      # rate-limits unauthenticated callers and this is a safety net, not a gate.
+      #
+      # HEURISTIC, not a guarantee: the Python-detection relies on the NVD
+      # description containing "python" / "lib/py" / "PyPI". Apache files these
+      # per-binding, but some descriptions read only "Apache Thrift" with no
+      # language named (e.g. CVE-2026-41606). Such a finding is listed in the
+      # weekly summary (TOTAL count) but would NOT auto-escalate. So: the FULL
+      # list is always surfaced for human review; only the clearly-Python ones
+      # hard-fail. Treat the weekly summary as a prompt to eyeball new thrift
+      # CVEs, not as proof none affect Python.
+      - name: Thrift NVD-CPE watch (CPE-only advisories the purl gate can't see)
+        if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
+        run: |
+          set -uo pipefail
+
+          # Locked thrift version from poetry.lock (authoritative shipped version).
+          THRIFT_VER=$(awk '/^\[\[package\]\]/{n=""} /^name = /{gsub(/"/,"");n=$3} n=="thrift" && /^version = /{gsub(/"/,"");print $3; exit}' poetry.lock)
+          if [ -z "${THRIFT_VER:-}" ]; then
+            echo "::warning::Could not determine locked thrift version; skipping NVD-CPE thrift watch."
+            exit 0
+          fi
+          echo "Checking NVD for apache:thrift advisories affecting thrift $THRIFT_VER ..."
+
+          # Exact-version CPE match against NVD. Non-fatal on transport errors
+          # (this is a supplementary watch, not the gate) -- but a Python-
+          # affecting hit escalates to an error below.
+          NVD_URL="https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:a:apache:thrift:${THRIFT_VER}:*:*:*:*:*:*:*"
+          nvd_rc=0
+          curl -fsSL --max-time 60 "$NVD_URL" -o /tmp/thrift-nvd.json || nvd_rc=$?
+          if [ "$nvd_rc" -ne 0 ] || [ ! -s /tmp/thrift-nvd.json ]; then
+            echo "::warning::NVD query failed (rc=$nvd_rc, e.g. rate limit). Thrift CPE watch inconclusive this run."
+            exit 0
+          fi
+
+          TOTAL=$(jq '.totalResults // 0' /tmp/thrift-nvd.json)
+          echo "NVD apache:thrift CVEs affecting $THRIFT_VER: $TOTAL"
+
+          # Findings whose description mentions Python -> the ONLY class we are
+          # actually exposed to (all others are non-Python bindings we accept).
+          PY_HITS=$(jq -r '
+            [ .vulnerabilities[]?.cve
+              | select( ([.descriptions[]? | select(.lang=="en") | .value] | join(" ")) | test("python|lib/py|PyPI"; "i") )
+              | .id ] | unique | join(", ")
+          ' /tmp/thrift-nvd.json)
+
+          {
+            echo "## Thrift NVD-CPE watch"
+            echo ""
+            echo "- Locked thrift version: \`$THRIFT_VER\`"
+            echo "- NVD \`apache:thrift\` CVEs affecting it: \`$TOTAL\` (purl gate cannot see these)"
+            if [ -n "$PY_HITS" ]; then
+              echo "- **Python-affecting thrift CVEs: \`$PY_HITS\` — ACTION REQUIRED**"
+            else
+              echo "- Python-affecting thrift CVEs: none (all findings are non-Python bindings — accepted; see pyproject.toml)"
+            fi
+          } >> "$GITHUB_STEP_SUMMARY"
+
+          jq -r '.vulnerabilities[]?.cve | "  \(.id): \([.descriptions[]?|select(.lang=="en")|.value][0][0:100])"' /tmp/thrift-nvd.json || true
+
+          if [ -n "$PY_HITS" ]; then
+            echo "::error::Thrift CVE(s) mentioning Python detected ($PY_HITS) against locked thrift $THRIFT_VER. These are INVISIBLE to the OSV/purl gate. Re-evaluate the thrift pin (see the thrift note in pyproject.toml)."
+            exit 1
+          fi
+          echo "No Python-affecting thrift CVE. Non-Python binding advisories remain accepted (0.23.0 fix blocked by ES-1960554)."
+
       # --- Terminal: scheduled/manual event ---
       # Weekly reports ALL findings (not just CVSS >= 7) so emerging risk is
       # visible before it crosses the PR gate. PR-time is narrower to avoid

pyproject.toml

@@ -19,6 +19,24 @@ python = "^3.10"
 # THRIFT-6067 / apache/thrift#3584) ships. The `DBR LTS Install` CI check
 # (.github/workflows/dbr-lts-install.yml) installs the built wheel on real DBR
 # LTS clusters and catches this regression when dependency pins change.
+#
+# SECURITY NOTE: thrift 0.22.0 IS affected by open Apache Thrift advisories
+# (CVE-2025-48431 and the CVE-2026-41602..41636 set), all fixed in 0.23.0.
+# We accept them here for two reasons:
+#   (1) Apache Thrift is one monorepo shipping ~20 language libraries; the PyPI
+#       `thrift` package is built only from `lib/py`. Each of these CVEs is in
+#       a NON-Python binding -- verified against the upstream advisories:
+#       Node.js (41636), Go (41602), c_glib/C (48431), Java (41603), Swift
+#       (41604, 41605) -- so none touches the Python code paths we ship.
+#   (2) The only fix (0.23.0) is the version that caused ES-1960554, so we
+#       cannot take it until a build-safe thrift ships (see above).
+# NOTE these advisories are INVISIBLE to the OSV-Scanner gate: OSV and
+# Dependabot match by PyPI purl, but these CVEs are filed only against the
+# upstream CPE (cpe:2.3:a:apache:thrift) with no PyPI-package coordinate, so
+# neither reports them. This is a point-in-time, per-CVE determination -- a
+# FUTURE thrift CVE could land in `lib/py`. The weekly Security Scan therefore
+# runs a supplementary NVD-CPE check for thrift (see securityScan.yml) so a
+# Python-affecting thrift CVE surfaces instead of being silently missed.
 thrift = "~=0.22.0"
 pandas = [
     { version = ">=1.2.5,<4.0.0", python = ">=3.10,<3.13" },