Skip to content

Add minimal smoke validation script and documentation - #27

Draft
binrogithub wants to merge 1 commit into
masterfrom
codex/create-smoke-validation-script
Draft

Add minimal smoke validation script and documentation#27
binrogithub wants to merge 1 commit into
masterfrom
codex/create-smoke-validation-script

Conversation

@binrogithub

Copy link
Copy Markdown
Owner

Motivation

  • Provide a small, dependency-light smoke check for key CLI flows (preview -> run_id/report.md, MCP tool listing, hc_tools_search and policy/apply behavior).
  • Enable quick offline validation with a mock mode so the wiring and output-matching can be validated without external services.
  • Make expected-output patterns configurable so the script can be adapted to different CLI output formats.

Description

  • Add an executable scripts/smoke_validation.sh that: runs a preview command and parses run_id, verifies the generated report.md, calls mcp list_tools and hc_tools_search, asserts that a demo tenant apply is blocked (expected failure), and asserts that an ops tenant apply succeeds when approvals are supplied; it supports a mock mode (SMOKE_USE_MOCK=1) and configurable commands/patterns via environment variables.
  • Add documentation/smoke_validation.md with exact example invocations for mocked and real runs, expected (abridged) outputs, and environment variables to customize matching patterns and command overrides.
  • The script uses sed to extract run_id and rg (ripgrep) for pattern assertions, prints clear sectioned logs, and is installed as an executable file in scripts/.

Testing

  • No automated tests were executed as part of this change; the script is intended to be run (manually or in CI) using the documented mock example or real CLI commands to validate behavior.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant