Skip to content

Generalize the smoke harness to per-snippet coverage + emit a coverage table #30

Description

@TMHSDigital

Context / Problem Statement

tests/smoke/run_smoke.py is excellent but exercises only ~6 hand-picked headline behaviors and blender-smoke.yml runs 4 examples. The 17 snippets and 12 skills are largely un-run in real Blender, and there is no record of what is covered. A new snippet can ship having only passed py_compile, and coverage can erode invisibly (silent-truncation risk).

Proposed Solution / Implementation Steps

  1. Add tests/smoke/run_snippets.py: iterate snippets/*.py, exec each inside a reset() factory-empty scene in headless Blender, treating any raised exception as failure (mirror the try/except wrapper already in run_smoke.py).
  2. Support a # smoke: skip <reason> first-line pragma for snippets that need scene state they can't self-provide; print the skipped list so skips are visible, never silent.
  3. Emit a coverage table at the end: N snippets run, M skipped (reasons), K skills with a smoke check. Print it to the CI log.
  4. Wire a new step into blender-smoke.yml after the existing smoke driver, on the same 5.1/4.5 matrix, xvfb-run-wrapped.

Definition of Done

  • Every snippet is either executed in headless Blender or explicitly # smoke: skip-ged with a reason.
  • CI log prints a run/skip/coverage summary table.
  • A snippet that raises in Blender fails the job (proven once).
  • Runs green on both 5.1 and 4.5 matrix legs.

Filed from the repository technical audit — see docs/technical-audit.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttestingTest harness and CI test coverage

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions