Skip to content

[HFH-4795] Agentic acceptance criteria - #45

Merged
garettarrowood merged 3 commits into
mainfrom
hfh/acceptance_criteria
Jul 27, 2026
Merged

[HFH-4795] Agentic acceptance criteria#45
garettarrowood merged 3 commits into
mainfrom
hfh/acceptance_criteria

Conversation

@garettarrowood

@garettarrowood garettarrowood commented Jul 27, 2026

Copy link
Copy Markdown
Member

Summary

Adds a reusable agentic-acceptance-criteria action that generates a non-technical manual QA plan from a pull request’s merge-base diff and posts it as an updatable PR comment.

What changed

  • Added a composite action that:
    • Reuses the existing agentic-review GitHub App credentials and Cursor provider pool.
    • Checks out the PR head and computes the merge-base diff.
    • Runs Cursor with read-only repository permissions.
    • Uploads the raw generated JSON for debugging.
    • Upserts one authoritative acceptance-criteria comment instead of creating duplicates.
    • Preserves the last successful QA plan when a later run fails.
  • Added a dedicated prompt that produces application-focused test coverage without relying on the PR description.
  • Added structured parsing and deterministic Markdown formatting for:
    • Permissions and roles.
    • Functional test scenarios.
    • Regression testing.
  • Added explicit fallback output when no manual application QA is identified.
  • Documented the future nitro-web workflow using:
    • The agentic-acceptance-criteria label.
    • The /agentic-acceptance-criteria comment command.
    • No automatic PR-open, ready-for-review, or commit-push triggers.
  • Documented the two-repository rollout and validation plan.

Example output

## ✅ Test Plan: PR #123 — Example change

---

### Permissions / Roles

- **Required Permissions:** Yes
- **Roles to Test:**
  - User with access to the affected feature

---

### Functional / Features to Test

#### Affected Feature

- **AF-1 — Default behavior**
  - Open the affected feature.
  - Verify the expected information appears.
  - Verify the updated behavior works as expected.

---

### Regression Testing

- Verify the neighboring workflow remains unchanged.
- Verify users cannot access data outside their permissions.

Validation

  • 18 new examples passing across the parser, formatter, and command-line renderer.
  • Ruby and shell syntax checks passing.
  • Composite-action YAML validation passing.
  • RuboCop lint checks passing.

Next steps

  1. Merge this PR into github-actions-workflows.
  2. Copy the resulting immutable commit SHA from main.
  3. Create the agentic-acceptance-criteria label in nitro-web
  4. Open a separate nitro-web PR adding the documented acceptance-criteria workflow, pinned to that SHA.
  5. After the workflow reaches nitro-web’s default branch, validate both trigger paths on an open PR:
    • Apply the agentic-acceptance-criteria label.
    • Comment /agentic-acceptance-criteria.
  6. Confirm both paths update the same QA comment and do not trigger the existing code-review workflow.

Parse Cursor output into a normalized QA-plan schema and render the agreed permissions, feature scenarios, and regression sections as deterministic Markdown.

Handle fenced or prefixed JSON, deduplicate repeated content, provide explicit fallbacks, and cover the parser, formatter, and command-line renderer with standalone specs.
Generate a merge-base diff with the existing GitHub App credentials, invoke Cursor through a read-only provider, and upsert a persistent non-technical QA plan on the pull request.

Preserve the last successful plan on failures and document the SHA-pinned nitro-web label and slash-command workflow that reuses the existing Cursor pool without triggering code review.
Record the generation schema, QA comment format, trigger isolation, failure behavior, and validation expectations for the new action.

Define the two-repository rollout order so nitro-web adopts the action only after it can pin an immutable commit from this repository.
@garettarrowood garettarrowood self-assigned this Jul 27, 2026
@garettarrowood
garettarrowood requested a review from a team as a code owner July 27, 2026 14:44
@powerhome-portal

Copy link
Copy Markdown

A change to documentation files was detected in your PR. Please visit this link to preview changes: https://portal-staging.powerapp.cloud/docs?filters[kind]=all&filters[user]=all&filters[namespaceFilter]=hfh-acceptancecriteria

@garettarrowood
garettarrowood merged commit e13b5a0 into main Jul 27, 2026
4 checks passed
@garettarrowood
garettarrowood deleted the hfh/acceptance_criteria branch July 27, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants