USHIFT-7116: add microshift-release-evaluation periodic job#79994
USHIFT-7116: add microshift-release-evaluation periodic job#79994agullon wants to merge 1 commit into
Conversation
Adds a new Prow periodic job that runs the microshift-release:pre-check AI skill every Thursday and Friday at 16:00 UTC. The job evaluates upcoming OCP z-stream releases for MicroShift participation and posts results to #team-ocp-edge-notifications via Prow reporter_config. New step registry files: - workflow, ref, and commands script under openshift/edge-tooling/microshift-release/evaluation/ Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
|
Skipping CI for Draft Pull Request. |
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
WalkthroughThis PR introduces a complete scheduled CI workflow for evaluating OpenShift z-stream releases against MicroShift compatibility. A new test job runs Thursdays and Fridays at 16:00 UTC, invoking a Claude-based evaluation workflow that manages credentials, configures AI connectivity via MCP servers, executes a pre-check skill, and reports results to Slack via the Prow reporter. ChangesMicroShift Release Evaluation Workflow
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
Suggested reviewers
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error)
✅ Passed checks (14 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
agullon
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@agullon: This pull request references USHIFT-7116 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: agullon The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
[REHEARSALNOTIFIER] Note: If this PR includes changes to step registry files (
|
openshift-merge-bot
Bot
added
the
rehearsals-ack
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-commands.sh`:
- Around line 2-3: The script currently enables bash xtrace globally via the
"set -x" invocation; remove that global "set -x" so the script starts with only
"set -euo pipefail" and never leaves tracing on for the whole step, and if debug
output is needed wrap the minimal sensitive sections with scoped tracing blocks
using "set -x" immediately before the debug lines and "set +x" right after
(refer to the existing "set -euo pipefail" and "set -x" entries to locate and
change this).
- Around line 37-45: The fallback that extracts Claude output writes results
only to ${SHARED_DIR} (variables result_text, CLAUDE_LOG, and files
precheck-results.txt / precheck-completed) but not to the artifact location,
breaking the advertised artifacts link; update the branch that handles
result_text to also persist the same files into ${ARTIFACT_DIR} (write
${ARTIFACT_DIR}/precheck-results.txt and touch
${ARTIFACT_DIR}/precheck-completed) so both SHARED_DIR and ARTIFACT_DIR contain
the fallback outputs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 29c69d3e-f50f-40d0-9554-5b0b41f19781
📒 Files selected for processing (7)
ci-operator/config/openshift-eng/edge-tooling/openshift-eng-edge-tooling-main.yamlci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/OWNERSci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-commands.shci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-ref.metadata.jsonci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-ref.yamlci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-workflow.metadata.jsonci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-workflow.yaml
| set -euo pipefail | ||
| set -x |
There was a problem hiding this comment.
Do not enable xtrace by default here.
Line 3 makes tracing opt-out for the whole script. This step loads GitHub and Jira credentials later, so future edits outside the two set +x blocks can easily spill sensitive operations into CI logs. Start without -x and only scope it to temporary debug sections when needed.
Suggested fix
#!/usr/bin/bash
set -euo pipefail
-set -xAs per coding guidelines, step registry command scripts should use set -euo pipefail by default (without -x tracing) and only enable -x for active debugging, keeping the scope of disabled tracing minimal.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-commands.sh`
around lines 2 - 3, The script currently enables bash xtrace globally via the
"set -x" invocation; remove that global "set -x" so the script starts with only
"set -euo pipefail" and never leaves tracing on for the whole step, and if debug
output is needed wrap the minimal sensitive sections with scoped tracing blocks
using "set -x" immediately before the debug lines and "set +x" right after
(refer to the existing "set -euo pipefail" and "set -x" entries to locate and
change this).
| # Fallback: extract text from Claude stream-json result event | ||
| if [[ -f "${CLAUDE_LOG}" ]]; then | ||
| local result_text | ||
| result_text="$(grep '"type":"result"' "${CLAUDE_LOG}" | tail -1 | jq -r '.result // empty' 2>/dev/null || true)" | ||
| if [[ -n "${result_text}" ]]; then | ||
| echo "${result_text}" > "${SHARED_DIR}/precheck-results.txt" | ||
| touch "${SHARED_DIR}/precheck-completed" | ||
| echo "Pre-check results extracted from Claude log." | ||
| return 0 |
There was a problem hiding this comment.
Persist fallback results into ARTIFACT_DIR as well.
When RESULTS_TEXT is missing, this branch reconstructs the output only into ${SHARED_DIR}. The periodic job advertises artifacts/precheck-results.txt in Slack, so the fallback path still leaves users with a broken “View results” link.
Suggested fix
result_text="$(grep '"type":"result"' "${CLAUDE_LOG}" | tail -1 | jq -r '.result // empty' 2>/dev/null || true)"
if [[ -n "${result_text}" ]]; then
- echo "${result_text}" > "${SHARED_DIR}/precheck-results.txt"
+ echo "${result_text}" > "${SHARED_DIR}/precheck-results.txt"
+ echo "${result_text}" > "${ARTIFACT_DIR}/precheck-results.txt"
touch "${SHARED_DIR}/precheck-completed"
echo "Pre-check results extracted from Claude log."
return 0
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Fallback: extract text from Claude stream-json result event | |
| if [[ -f "${CLAUDE_LOG}" ]]; then | |
| local result_text | |
| result_text="$(grep '"type":"result"' "${CLAUDE_LOG}" | tail -1 | jq -r '.result // empty' 2>/dev/null || true)" | |
| if [[ -n "${result_text}" ]]; then | |
| echo "${result_text}" > "${SHARED_DIR}/precheck-results.txt" | |
| touch "${SHARED_DIR}/precheck-completed" | |
| echo "Pre-check results extracted from Claude log." | |
| return 0 | |
| # Fallback: extract text from Claude stream-json result event | |
| if [[ -f "${CLAUDE_LOG}" ]]; then | |
| local result_text | |
| result_text="$(grep '"type":"result"' "${CLAUDE_LOG}" | tail -1 | jq -r '.result // empty' 2>/dev/null || true)" | |
| if [[ -n "${result_text}" ]]; then | |
| echo "${result_text}" > "${SHARED_DIR}/precheck-results.txt" | |
| echo "${result_text}" > "${ARTIFACT_DIR}/precheck-results.txt" | |
| touch "${SHARED_DIR}/precheck-completed" | |
| echo "Pre-check results extracted from Claude log." | |
| return 0 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/openshift/edge-tooling/microshift-release/evaluation/openshift-edge-tooling-microshift-release-evaluation-commands.sh`
around lines 37 - 45, The fallback that extracts Claude output writes results
only to ${SHARED_DIR} (variables result_text, CLAUDE_LOG, and files
precheck-results.txt / precheck-completed) but not to the artifact location,
breaking the advertised artifacts link; update the branch that handles
result_text to also persist the same files into ${ARTIFACT_DIR} (write
${ARTIFACT_DIR}/precheck-results.txt and touch
${ARTIFACT_DIR}/precheck-completed) so both SHARED_DIR and ARTIFACT_DIR contain
the fallback outputs.
2 participants
Summary
Adds a new Prow periodic job (
microshift-release-evaluation) that automates MicroShift release evaluation for upcoming OCP z-stream releases.microshift-release:pre-checkAI skill every Thursday and Friday at 16:00 UTC#team-ocp-edge-notificationsvia Prowreporter_configmicroshift-ci-doctorrestrict_network_access: false) for Brew/advisory checksNew files
openshift/edge-tooling/microshift-release/evaluation/openshift-eng-edge-tooling-main.yamlBlocked by
This job depends on the edge-tooling AI helpers image containing the latest pre-check skill changes:
Jira
USHIFT-7116 (parent: USHIFT-6766)
Test plan
Summary by CodeRabbit
This PR adds a new periodic Prow CI job to automate MicroShift release evaluation against upcoming OCP z-stream releases. The job is configured to run every Thursday and Friday at 16:00 UTC and will post color-coded evaluation results to the
#team-ocp-edge-notificationsSlack channel.What's being added:
The implementation consists of:
microshift-release-evaluation) in the OpenShift edge-tooling CI configurationopenshift/edge-tooling/microshift-release/evaluation/that includes:Technical details:
The job reuses existing patterns from the
microshift-ci-doctorjob, including the same edge-tooling AI helpers container image and credential management approach. The evaluation script:microshift-release:pre-checkAI skill with a time-bounded promptThe job requires intranet access (
restrict_network_access: false) to perform Brew and advisory checks needed for the evaluation.Status and dependencies:
This PR is currently blocked on upstream edge-tooling PRs that need to update the pre-check skill to migrate Jira queries to MCP OAuth and replace the Product Pages MCP integration with ART Jira support.