Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Trajectory: Add PR-specific Cloud RelayFlow red-green proof infrastructure

> **Status:** ✅ Completed
> **Confidence:** 92%
> **Started:** August 25, 2026 at 10:45 AM
> **Completed:** August 25, 2026 at 11:14 AM

---

## Summary

Implemented trusted PR classification and single-case dispatch, a fail-closed Cloud RelayFlow that proves base then head in distinct per-step sandboxes, provenance/evidence validation, GitHub Action wiring, tests, template guidance, and credential rollout documentation.

**Approach:** Kept pull_request_target as a data-only trusted dispatcher, ran PR code only in Cloud, represented expected-red as a structured successful observation, disabled repair retries, validated exact base/head SHAs and sandbox IDs, and tested both the contracts and the complete repository.

---

## Key Decisions

### Require exactly one PR proof case and a structured observation result
- **Chose:** Require exactly one PR proof case and a structured observation result
- **Reasoning:** One feature or fix should trigger only its own case. The case runner must exit successfully after observing behavior and write bug/absent/fixed plus an exact signature; test crashes, missing tests, skips, and build failures remain infrastructure failures and cannot count as expected red.

### Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes
- **Chose:** Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes
- **Reasoning:** The GitHub runner must never checkout or execute untrusted head code while holding Cloud credentials. It will fetch and validate one declarative case manifest, stage exact base/head SHAs, and submit the trusted base-branch RelayFlow.

### Disable automatic repair and retries for PR proof workflows
- **Chose:** Disable automatic repair and retries for PR proof workflows
- **Rejected:** Default retry policy, Continue on failure
- **Reasoning:** A red/green gate must preserve the first observation and fail closed. RelayFlow defaults to agent-assisted repair retries when agents exist, so the proof workflow explicitly uses fail-fast with zero step retries.

### Stage the validated generated input in the disposable CI git index
- **Chose:** Stage the validated generated input in the disposable CI git index
- **Rejected:** Change global code-sync semantics, Commit a placeholder input file
- **Reasoning:** Cloud code sync uploads git-known files. The per-PR input is generated after checkout, so the action must git-add it without committing or pushing or the Cloud workflow cannot see the exact case and SHAs.

### Require explicit proof classification on every PR
- **Chose:** Require explicit proof classification on every PR
- **Rejected:** Infer only from conventional titles, Run Cloud for every PR
- **Reasoning:** Silently treating missing metadata as non-functional would let non-conventional feature and fix titles bypass the gate. Every PR must declare feature, bugfix, or non-functional; only the first two launch Cloud.

---

## Chapters

### 1. Work
*Agent: default*

- Require exactly one PR proof case and a structured observation result: Require exactly one PR proof case and a structured observation result
- Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes: Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes
- Disable automatic repair and retries for PR proof workflows: Disable automatic repair and retries for PR proof workflows
- Stage the validated generated input in the disposable CI git index: Stage the validated generated input in the disposable CI git index
- Require explicit proof classification on every PR: Require explicit proof classification on every PR
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
{
"id": "traj_yr3f5u3r6zdz",
"version": 1,
"task": {
"title": "Add PR-specific Cloud RelayFlow red-green proof infrastructure"
},
"status": "completed",
"startedAt": "2026-08-25T08:45:32.835Z",
"completedAt": "2026-08-25T09:14:31.949Z",
"agents": [
{
"name": "default",
"role": "lead",
"joinedAt": "2026-08-25T08:50:30.315Z"
}
],
"chapters": [
{
"id": "chap_cco5mxx3i8f5",
"title": "Work",
"agentName": "default",
"startedAt": "2026-08-25T08:50:30.315Z",
"endedAt": "2026-08-25T09:14:31.949Z",
"events": [
{
"ts": 1787647830316,
"type": "decision",
"content": "Require exactly one PR proof case and a structured observation result: Require exactly one PR proof case and a structured observation result",
"raw": {
"question": "Require exactly one PR proof case and a structured observation result",
"chosen": "Require exactly one PR proof case and a structured observation result",
"alternatives": [],
"reasoning": "One feature or fix should trigger only its own case. The case runner must exit successfully after observing behavior and write bug/absent/fixed plus an exact signature; test crashes, missing tests, skips, and build failures remain infrastructure failures and cannot count as expected red."
},
"significance": "high"
},
{
"ts": 1787647830374,
"type": "decision",
"content": "Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes: Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes",
"raw": {
"question": "Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes",
"chosen": "Use pull_request_target only as a trusted dispatcher; execute PR case code only inside Cloud sandboxes",
"alternatives": [],
"reasoning": "The GitHub runner must never checkout or execute untrusted head code while holding Cloud credentials. It will fetch and validate one declarative case manifest, stage exact base/head SHAs, and submit the trusted base-branch RelayFlow."
},
"significance": "high"
},
{
"ts": 1787649023999,
"type": "decision",
"content": "Disable automatic repair and retries for PR proof workflows: Disable automatic repair and retries for PR proof workflows",
"raw": {
"question": "Disable automatic repair and retries for PR proof workflows",
"chosen": "Disable automatic repair and retries for PR proof workflows",
"alternatives": [
{
"option": "Default retry policy",
"reason": ""
},
{
"option": "Continue on failure",
"reason": ""
}
],
"reasoning": "A red/green gate must preserve the first observation and fail closed. RelayFlow defaults to agent-assisted repair retries when agents exist, so the proof workflow explicitly uses fail-fast with zero step retries."
},
"significance": "high"
},
{
"ts": 1787649024705,
"type": "decision",
"content": "Stage the validated generated input in the disposable CI git index: Stage the validated generated input in the disposable CI git index",
"raw": {
"question": "Stage the validated generated input in the disposable CI git index",
"chosen": "Stage the validated generated input in the disposable CI git index",
"alternatives": [
{
"option": "Change global code-sync semantics",
"reason": ""
},
{
"option": "Commit a placeholder input file",
"reason": ""
}
],
"reasoning": "Cloud code sync uploads git-known files. The per-PR input is generated after checkout, so the action must git-add it without committing or pushing or the Cloud workflow cannot see the exact case and SHAs."
},
"significance": "high"
},
{
"ts": 1787649025196,
"type": "decision",
"content": "Require explicit proof classification on every PR: Require explicit proof classification on every PR",
"raw": {
"question": "Require explicit proof classification on every PR",
"chosen": "Require explicit proof classification on every PR",
"alternatives": [
{
"option": "Infer only from conventional titles",
"reason": ""
},
{
"option": "Run Cloud for every PR",
"reason": ""
}
],
"reasoning": "Silently treating missing metadata as non-functional would let non-conventional feature and fix titles bypass the gate. Every PR must declare feature, bugfix, or non-functional; only the first two launch Cloud."
},
"significance": "high"
}
]
}
],
"retrospective": {
"summary": "Implemented trusted PR classification and single-case dispatch, a fail-closed Cloud RelayFlow that proves base then head in distinct per-step sandboxes, provenance/evidence validation, GitHub Action wiring, tests, template guidance, and credential rollout documentation.",
"approach": "Kept pull_request_target as a data-only trusted dispatcher, ran PR code only in Cloud, represented expected-red as a structured successful observation, disabled repair retries, validated exact base/head SHAs and sandbox IDs, and tested both the contracts and the complete repository.",
"confidence": 0.92
},
"commits": [],
"filesChanged": [],
"projectId": "AgentWorkforce/relay",
"tags": [],
"_trace": {
"startRef": "191e5f14a343431ec42282c1103c9f014854ac0b",
"endRef": "191e5f14a343431ec42282c1103c9f014854ac0b"
}
}
9 changes: 9 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@
- [ ] Tests added/updated
- [ ] Manual testing completed

## RelayFlow Proof

Replace both values below. Use `feature` or `bugfix` for user-visible behavior
changes and add exactly one case under `tests/relayflows/cases/<case-id>/`.
Use `non-functional` and `n/a` only when runtime behavior is unchanged.

- Change type: `replace-me` <!-- relay-pr-proof:type -->
- RelayFlow case: `replace-me` <!-- relay-pr-proof:case -->

## Screenshots

<!-- If applicable -->
113 changes: 113 additions & 0 deletions .github/workflows/relayflow-pr-proof.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: RelayFlow PR Proof

# Security boundary: pull_request_target gives this trusted base-branch
# dispatcher access to Cloud credentials. It MUST NOT checkout, import, or run
# PR-head code on the GitHub runner. The exact head is cloned only inside the
# isolated Cloud proof sandboxes by scripts/pr-proof/run-arm.mjs.
on:
pull_request_target:
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
types: [opened, synchronize, reopened, edited, ready_for_review]
Comment thread
khaliqgant marked this conversation as resolved.
workflow_dispatch:
inputs:
pr_number:
description: Pull request number to prove
required: true
type: number

permissions:
contents: read
pull-requests: read
statuses: write

concurrency:
group: relayflow-pr-proof-${{ github.event.pull_request.number || inputs.pr_number }}
cancel-in-progress: true
Comment thread
khaliqgant marked this conversation as resolved.
Comment thread
khaliqgant marked this conversation as resolved.

jobs:
proof:
name: RelayFlow PR proof dispatcher
runs-on: ubuntu-latest
timeout-minutes: 70
steps:
- name: Checkout trusted base
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
ref: ${{ github.event.pull_request.base.sha || github.sha }}
persist-credentials: false

- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22.14.0

# pull_request_target runs are attached to the base SHA. Publish the
# stable required context explicitly on the exact validated PR head.
- name: Start required status on PR head
id: status
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
node scripts/pr-proof/report-status.mjs start
--event "$GITHUB_EVENT_PATH"
--github-output "$GITHUB_OUTPUT"

- name: Classify PR and validate declared case
id: prepare
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
node scripts/pr-proof/prepare.mjs
--event "$GITHUB_EVENT_PATH"
--output .relayflow/pr-proof-input.json
--expected-head-sha "${{ steps.status.outputs.head_sha }}"
--github-output "$GITHUB_OUTPUT"
--summary "$GITHUB_STEP_SUMMARY"

- name: Confirm Cloud proof credential
if: steps.prepare.outputs.required == 'true'
env:
CLOUD_API_URL: ${{ secrets.CLOUD_API_URL }}
CLOUD_API_KEY: ${{ secrets.RELAYFLOW_PR_PROOF_CLOUD_API_KEY }}
run: |
test -n "$CLOUD_API_URL"
test -n "$CLOUD_API_KEY"

# Cloud code sync intentionally uploads only paths known to git. Add the
# generated, validated input to this disposable runner's index so it is
# present in the Cloud workspace; this does not create a commit or push.
- name: Include proof input in Cloud code sync
if: steps.prepare.outputs.required == 'true'
run: git add -f -- .relayflow/pr-proof-input.json

- name: Install the released Agent Relay CLI
if: steps.prepare.outputs.required == 'true'
run: npm install --global "agent-relay@$(node -p "require('./package.json').version")"

- name: Run isolated Cloud red-green proof
if: steps.prepare.outputs.required == 'true'
id: cloud
env:
CLOUD_API_URL: ${{ secrets.CLOUD_API_URL }}
CLOUD_API_KEY: ${{ secrets.RELAYFLOW_PR_PROOF_CLOUD_API_KEY }}
PR_PROOF_CLOUD_TIMEOUT_MS: '3600000'
run: node scripts/pr-proof/run-cloud.mjs workflows/pr-proof.ts

- name: Upload Cloud proof logs
if: always() && steps.prepare.outputs.required == 'true'
Comment thread
khaliqgant marked this conversation as resolved.
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: relayflow-pr-proof-${{ github.event.pull_request.number || inputs.pr_number }}
path: .workflow-artifacts/pr-proof/cloud.log
if-no-files-found: warn

- name: Finish required status on PR head
# report-status verifies that this run still owns the latest pending
# context before publishing. That makes cancellation terminal without
# allowing a cancelled predecessor to overwrite a replacement run.
if: always() && steps.status.outputs.head_sha != ''
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
node scripts/pr-proof/report-status.mjs finish
--sha "${{ steps.status.outputs.head_sha }}"
--job-status "${{ job.status }}"
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ __pycache__/
.agentworkforce/relay/
.relay/
.agent-relay/
.relayflow/pr-proof-input.json

.msd/
/workflows/*
Expand All @@ -91,6 +92,7 @@ __pycache__/
!/workflows/cloud-connect/
!/workflows/verify-features.ts
!/workflows/audit-feature-manifest.ts
!/workflows/pr-proof.ts

# Eval harness JSON reports (generated per run)
tests/integration/broker/evals-reports/
Expand Down
37 changes: 35 additions & 2 deletions packages/cloud/src/api-client.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,41 @@
import { describe, expect, it, vi } from 'vitest';
import { afterEach, describe, expect, it, vi } from 'vitest';

import { CloudApiClient } from './api-client.js';
import { CloudApiClient, WorkflowApiKeyClient } from './api-client.js';
import { CloudAuthError } from './types.js';

afterEach(() => {
vi.unstubAllGlobals();
});

describe('WorkflowApiKeyClient', () => {
it('uses one API key at the explicit URL and returns a 401 without retrying', async () => {
const fetchSpy = vi.fn(async () => new Response('{}', { status: 401 }));
vi.stubGlobal('fetch', fetchSpy);

const client = WorkflowApiKeyClient.fromEnv('https://explicit.example/cloud', {
CLOUD_API_URL: 'https://ignored.example/cloud',
CLOUD_API_KEY: 'ci-api-key',
});
expect(client).not.toBeNull();

const response = await client!.fetch('/api/v1/workflows/run', { method: 'POST' });

expect(response.status).toBe(401);
expect(fetchSpy).toHaveBeenCalledOnce();
expect(String(fetchSpy.mock.calls[0][0])).toBe('https://explicit.example/cloud/api/v1/workflows/run');
const headers = new Headers(fetchSpy.mock.calls[0][1]?.headers);
expect(headers.get('authorization')).toBe('Bearer ci-api-key');
});

it('fails closed when the API key has a malformed Cloud URL', () => {
expect(() =>
WorkflowApiKeyClient.fromEnv('not-a-url', {
CLOUD_API_KEY: 'ci-api-key',
})
).toThrowError(expect.objectContaining({ code: 'AUTH_ENV_REPROVISION_REQUIRED' }));
});
});

describe('CloudApiClient', () => {
it('refreshes before an otherwise-valid session reaches refresh-token expiry', async () => {
const fetchSpy = vi.fn(async (input: string | URL) => {
Expand Down
Loading
Loading