Skip to content

Enhancement: add a fail-on-epss input to the GitHub Action (maps to --fail-on-epss) #487

Closed

Description

@vibgrate-team

Context

vg scan --fail-on-epss <threshold> landed in #474 (closing #420). It fails the run with exit 2 when any advisory's EPSS score is at or above a 0–1 threshold, and it never treats a missing score as 0. The CLI docs (DOCS.md, docs/ci/github-actions.md, examples/github-actions/README.md) all describe it.

The composite GitHub Action in action.yml doesn't expose it yet. It has first-class inputs for the other gates (fail-on, max-score for --drift-budget, max-worsening for --drift-worsening) with input validation and clear ::error:: messages, but EPSS can only be reached today through the free-form args string. That skips validation and is easy to miss.

What to do

  • Add a fail-on-epss input to action.yml, described in the same style as max-score (a probability from 0 to 1, empty means never fail on EPSS, maps to --fail-on-epss).
  • In the scan step, trim the value and reject anything that isn't a number from 0 to 1 with an ::error:: line before the scan runs, matching how max-score is validated.
  • Because the gate reads advisories from the vulnerability pass, either require vulns: true (with an actionable ::error:: when it's missing) or document clearly that the gate is skipped without it. Pick one and say which in the input description.
  • Older scanner images won't know the flag. Probe scan --help the same way the step already does for --summary-out, and emit a ::notice:: instead of failing on an unknown flag.
  • Add a with: fail-on-epss: '0.1' example to docs/ci/github-actions.md and examples/github-actions/README.md.

Acceptance

  • fail-on-epss: '0.1' with vulns: true passes --fail-on-epss 0.1 to the scan, and an advisory at or above 0.1 fails the job.
  • An empty input adds no flag and behaves exactly as today.
  • Values such as abc, -0.1, 1.5 or whitespace-only fail the step before the scan with a clear message.
  • An image that predates --fail-on-epss gets a notice, not a broken run.
  • The README/docs examples show the new input alongside fail-on and max-score.

Activity

  1. vibgrate-team commented on Oct 20, 2026

    @vibgrate-team
    ContributorAuthor

    Thanks for writing this up so clearly. A few pointers for whoever picks it up:

    • Where it goes: everything lives in the Run Vibgrate scan step of action.yml. Add INPUT_FAIL_ON_EPSS: ${{ inputs.fail-on-epss }} to that step's env: block, and keep reading it from env rather than interpolating it into the script, the same way the other inputs are passed.
    • Validation: the max-score block is the model to copy. It trims whitespace with parameter expansion, checks the shape with ^[0-9]+(\.[0-9]+)?$, then range-checks with awk. For EPSS the range is 0 to 1, so the awk test becomes score + 0 <= 1. A whitespace-only value trims to an empty string and fails the regex, which covers that acceptance case without extra code. Keep the CLI's own rule in mind too: parseEpssThreshold in src/reporting/epss-gate.ts rejects a blank value, non-finite numbers, and anything outside 0 to 1, so the Action should never accept something the CLI would reject.
    • The vulns dependency: the gate reads advisories from the vulnerability pass. Without --vulns, evaluateEpssGate sees source: null and reports a skip rather than a pass or a failure. Failing fast with ::error::fail-on-epss needs vulns: true ... when INPUT_VULNS isn't true is probably the least surprising choice, because otherwise a gate someone set on purpose would quietly do nothing. Whichever way you go, say it in the input description.
    • Older images: help_text is already captured once for the --summary-out probe. Reuse it with grep -q -- '--fail-on-epss' rather than running docker run ... scan --help a second time, and emit a ::notice:: when the flag is missing.
    • Ordering: add the flag to extra before the INPUT_ARGS word-split, so anything a caller passes in args still comes last and wins.
    • Tests: if there's an existing shell or YAML test over action.yml inputs, add cases for 0.1, empty, abc, -0.1, 1.5 and ' '. Then add the with: fail-on-epss: '0.1' example next to fail-on and max-score in docs/ci/github-actions.md and examples/github-actions/README.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions