Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .agents/WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ specific one.
| `new-resource` | New Resource Creation | `.agents/skills/workflows/new_resource/SKILL.md` |
| `list-resource` | List Resource Addition | `.agents/skills/workflows/add_list_resource/SKILL.md` |
| `enhancement` (field additions) | New Field / Property Addition | `.agents/skills/workflows/add_fields/SKILL.md` |
| `deprecation` | Deprecation Notice Addition | `.agents/skills/workflows/deprecate_resource_or_field/SKILL.md` |
| `breaking-change` | Major Release Removal / Breaking Change | `.agents/skills/workflows/major_release_removal/SKILL.md` |
| *No matching label / un-labeled* | General Modification / Bug Fix | Fallback to `.agents/skills/workflows/default/SKILL.md` (or inspect issue body) |

## Available Workflows:
Expand All @@ -34,8 +36,11 @@ specific one.
* **Test Monitor Workflow** (`.agents/skills/workflows/test_monitor/SKILL.md`): For fetching, triaging, analyzing, and reporting on nightly acceptance test results across Beta and GA providers.
* **Add IAM Support Workflow** (`.agents/skills/workflows/add_iam_resources/SKILL.md`): Adds IAM support to an existing MMv1 resource.
* **Bug Fix Workflow** (`.agents/skills/workflows/bug_fix/SKILL.md`): For triaging, planning, fixing, and verifying reported provider bugs.
* **Deprecate Resource or Field Workflow** (`.agents/skills/workflows/deprecate_resource_or_field/SKILL.md`): Specifically for deprecating existing resources or fields on `main` prior to a major release.
* **Major Release Removal Workflow** (`.agents/skills/workflows/major_release_removal/SKILL.md`): Specifically for executing resource and field removals on a major release feature branch.
* *(Future workflows can be added here)*

## Subagents:

* **`test-fixer`** (`.agents/agents/test-fixer/`): Automates diagnosis, remediation in Magic Modules, provider generation, and re-testing for failing acceptance tests.
* **`removal-auditor`** (`.agents/agents/removal-auditor/`): Audits deprecation status on `main`, sync status on the major release branch, and scans repository dependencies for resource and field removals.
6 changes: 6 additions & 0 deletions .agents/agents/removal-auditor/agent.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "removal-auditor",
"configPath": {
"relativePathToConfig": "config.yaml"
}
}
68 changes: 68 additions & 0 deletions .agents/agents/removal-auditor/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
custom_agent:
system_prompt_sections:
- title: "Core Instructions"
content: |
Your task is to perform an end-to-end pre-flight audit for a planned resource or field removal for a major release.

Input Target:
- `product`: Product directory name (e.g. `compute`, `dlp`, `notebooks`, `vertexai`).
- `resource`: Resource name (e.g. `JobTrigger`, `google_notebooks_instance`, `Schedule`).
- `field` (Optional): Specific field name (e.g. `publish_findings_to_cloud_data_catalog`). If omitted or set to `all`, treat as a full resource removal.
- `major_release_branch`: Major release branch (e.g. `FEATURE-BRANCH-major-release-8.0.0`).

Follow these audit steps:

1. **Pre-Flight `main` Branch Audit**:
- Check the state of the target resource or field on `upstream/main`.
- **Deprecation Check**: Verify whether `deprecation_message` (for MMv1 YAML) or `Deprecated` / `DeprecationMessage` (for handwritten Go) is set on `upstream/main`.
- **Replacement Availability**: If the field/resource is being replaced (e.g., in favor of another field or resource), verify that the replacement is fully implemented and available on `upstream/main`.
- **Forward Compatibility**: If a required field is slated for removal, verify whether it was already changed to optional on `upstream/main`.
- If `main` is missing the deprecation or replacement, flag this immediately as a **BLOCKER: Requires Deprecation on `main` First**.

2. **Major Release Branch Sync Audit**:
- Check `upstream/<major_release_branch>` to verify whether the deprecation and replacement commits from `main` have been merged into the major release branch.
- If the major release branch is missing the deprecation commits from `main`, flag this as a **BLOCKER: Needs Main Sync**.

3. **Comprehensive Dependency & Blast Radius Scan**:
Search across the codebase for all references to the resource or field:
- **MMv1 Product & Resource YAMLs**: `mmv1/products/<product>/...`
- **Custom Code Templates**: `mmv1/templates/terraform/` (e.g., `constants/`, `custom_check_destroy/`, `custom_expand/`, `custom_flatten/`, `encoder/`, `update_encoder/`, `pre_create/`, `post_create/`, `pre_update/`, `post_update/`, `state_migrations/`).
- **Samples & Examples**: `mmv1/templates/terraform/samples/services/<product>/` and `mmv1/templates/terraform/examples/`.
- **Handwritten Code & Tests**: `mmv1/third_party/terraform/services/<product>/...` (including `ImportStateVerifyIgnore` entries).
- **Issue Labeler**: `tools/issue-labeler/labeler/enrolled_teams.yml`.
- **TGC Converters**: `mmv1/third_party/tgc/resource_converters.go.tmpl` (converter mappings and IAM registrations).
- **Documentation**: `mmv1/third_party/terraform/website/docs/...`
- **Cross-Resource Links**: Any other product YAMLs or test fixtures referencing the target.

4. **Compile Removal Audit Report Artifact**:
Generate a structured Markdown report detailing:
- Target summary (Product, Resource, Field, Scope: MMv1 vs. Handwritten, Resource vs. Field).
- Pre-flight status on `main` (Deprecation message present? Replacement ready?).
- Sync status on the major release branch.
- Complete file modification list (files to delete, files to edit, exact sections).
- Acceptance tests requiring updates or execution.
- Proposed Upgrade Guide entry text for `version_<X>_upgrade.html.markdown`.

- title: "Report Format"
content: |
You must return a clear Markdown audit report with the following structure:
- **Target Summary**: Target name, type (MMv1 vs Handwritten), removal scope (Field vs Resource vs Entire Product).
- **Pre-Flight Readiness**:
- `main` branch deprecation status: `READY` or `BLOCKED` (with explanation).
- Major release branch sync status: `SYNCED` or `NEEDS_SYNC`.
- **Actionable Removal Plan**:
- Files to delete.
- Files to edit (with line numbers / property names).
- Test adjustments (`ImportStateVerifyIgnore`, sample deletions, test fixture updates).
- Issue labeler adjustments (`tools/issue-labeler/labeler/enrolled_teams.yml`).
- TGC converter cleanup requirements.
- **Proposed Upgrade Guide Entry**: Draft markdown text ready to insert into `version_<X>_upgrade.html.markdown`.

tool_names:
- view_file
- list_dir
- run_command
- grep_search
- write_to_file

command_execution_policy: "eager"
2 changes: 2 additions & 0 deletions .agents/knowledge/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ Read this index at decision points; open only the source the task needs. Format
- Create a PR — `docs/content/code-review/create-pr.md`
- Write release notes — `docs/content/code-review/release-notes.md`
- Review a PR — `docs/content/code-review/review-pr.md`
- Make a breaking change / deprecations / removals — `docs/content/breaking-changes/make-a-breaking-change.md`
- Types of breaking changes — `docs/content/breaking-changes/breaking-changes.md`

**Documentation:**

Expand Down
113 changes: 49 additions & 64 deletions .agents/skills/operations/create-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,13 @@ description: "Create a Pull Request (PR) against GoogleCloudPlatform/magic-modul

> **Note to AI Agents:** You MUST read the YAML frontmatter above first. Only read the rest of this file if the `description` matches your current roadblock or required task.

This skill provides step-by-step instructions for preparing, formatting, and opening a Pull Request (PR) for `magic-modules` following the official contribution guidelines.
This skill provides step-by-step instructions for preparing, formatting, and opening a Pull Request (PR) for `magic-modules` following official contribution guidelines.

## Prerequisites

* You are in the `magic-modules` root directory.
* Your git working directory is clean except for the files intended for the PR.
* Downstream provider changes are NOT staged or committed to `magic-modules`.
* `gh` CLI is installed and authenticated (`gh auth status`).
* Remote repositories are configured (e.g., `upstream` pointing to `GoogleCloudPlatform/magic-modules` and a personal fork remote such as `origin`).

---
Expand All @@ -26,49 +25,38 @@ Before creating a branch or opening a PR, verify all of the following rules:
1. **Single Self-Contained Change:** Each PR must contain only **one** logical change.
- Adding multiple resources? Put **one resource per PR**.
- Fixing a bug and adding new fields? Split into **two separate PRs**.
2. **No Downstream Artifacts in Magic Modules:**
- Do NOT commit generated downstream provider code (e.g. `$GOPATH/src/github.com/hashicorp/terraform-provider-google`) into `magic-modules`.
3. **Plan Completeness:** Verify that every file listed in the implementation plan (including any necessary documentation) has been generated and staged.
4. **Pre-Gen Static Checks:** Execute the `run-pre-gen-checks` skill (`.agents/skills/utils/run-pre-gen-checks/`) to run fast static checks directly against Magic Modules (Go formatting, YAML linting, template validation, and MMv1 unit tests).
5. **Workspace Cleanup:** Run `git status --porcelain` and remove any untracked `.log`, `.test`, or temporary test artifacts across both `magic-modules` and downstream repositories before opening the PR.
2. **Strict PR Title Length Limit:**
- PR title must be concise and strictly **under 70 characters**.
- Format: `<product>: <action> <target>` (e.g. `beyondcorp: deprecate google_beyondcorp_app_*` or `compute: add foo field to google_compute_instance`).
3. **No Downstream Artifacts in Magic Modules:**
- Do NOT commit generated downstream provider code into `magic-modules`.
4. **Workspace Cleanup:** Run `git status --porcelain` and ensure no untracked temporary test files exist before opening the PR.

---

## Execution Steps

### 1. Sync and Create Feature Branch

Identify the upstream remote (pointing to `GoogleCloudPlatform/magic-modules`), fetch `main`, and create a clean topic branch:

```bash
# Discover the remote for GoogleCloudPlatform/magic-modules (defaults to 'upstream' if unmatched)
UPSTREAM_REMOTE=$(git remote -v | grep -i "GoogleCloudPlatform/magic-modules" | head -n 1 | awk '{print $1}')
UPSTREAM_REMOTE="${UPSTREAM_REMOTE:-upstream}"

git fetch "$UPSTREAM_REMOTE" main
BRANCH="<short-descriptive-branch-name>" # e.g. add-compute-foo-field
BRANCH="<short-descriptive-branch-name>" # e.g. deprecate-beyondcorp-app
git checkout -b "$BRANCH" "$UPSTREAM_REMOTE/main"
```

### 2. Stage and Commit Changes

Stage ONLY the files in `magic-modules` (such as YAML definitions under `mmv1/products/`, templates, or docs):

```bash
git add mmv1/products/<product>/
git commit -m "<product>: <concise description of change>"
git add mmv1/products/<product>/ # or other relevant files
git commit -m "<product>: <concise description under 70 chars>"
```

### 3. Push to Fork

Identify your personal fork remote. If multiple non-upstream remotes exist or the target remote is ambiguous, stop and ask the user which remote to push to:
### 3. Push to Personal Fork

```bash
# Check available remotes
git remote -v

# Automatically select personal fork remote (remote not pointing to GoogleCloudPlatform).
# If multiple non-upstream remotes exist, ask the user to confirm the remote name.
FORK_REMOTE=$(git remote -v | grep -v -i "GoogleCloudPlatform/magic-modules" | head -n 1 | awk '{print $1}')
FORK_REMOTE="${FORK_REMOTE:-origin}"

Expand All @@ -77,78 +65,75 @@ git push -u "$FORK_REMOTE" "$BRANCH"

---

### 4. Format Release Notes & PR Body

Every PR must contain at least one release note block in the PR body.
### 4. Format PR Body & Release Notes

Refer to the official guide for detailed release note rules, categories, and examples:
* [docs/content/code-review/release-notes.md](../../../../docs/content/code-review/release-notes.md)
Every PR must contain a clear summary and a release note block in the PR body.

> [!IMPORTANT]
> **Note to AI Agents:** View `docs/content/code-review/release-notes.md` from the repository root to determine the correct release note type (e.g., `enhancement`, `bug`, `none`, `new-resource`, `deprecation`, etc.) and follow the end-user impact guidelines.
Refer to [docs/content/code-review/release-notes.md](../../../../docs/content/code-review/release-notes.md) for details on categories (`enhancement`, `bug`, `none`, `new-resource`, `deprecation`, `breaking-change`).

#### Release Note Block Format
```markdown
```release-note:TYPE
CONTENT
```
```

#### Sample PR Body Content
#### Sample PR Body
```markdown
Summary of what changed and why in a few concise sentences.

Modeled after: https://github.com/GoogleCloudPlatform/magic-modules/pull/12344
Fixes https://github.com/hashicorp/terraform-provider-google/issues/12345

```release-note:enhancement
compute: added `foo` field to `google_compute_instance` resource
<product>: added `foo` field to `google_compute_instance`
```
```

---

### 5. Create Pull Request with `gh` CLI Using `--body-file`

> [!CAUTION]
> **DO NOT pass inline double-quoted `--body "..."` strings containing backticks.**
> Enclosing triple backticks (` ```release-note:type ``` `) in double quotes causes `zsh`/`bash` to execute `` `release-note:type` `` as a live shell command substitution. The command fails, silently stripping the release note block from the published PR body!

* **Upstream Target:** Most likely target `--repo GoogleCloudPlatform/magic-modules` (upstream magic modules), and when in doubt ask the user.
* **PR Title Length:** Keep the PR title strictly under 60 characters, ideally < 50 characters (format: `<product>: <concise description>`).
### 5. Create PR or Provide Pre-Filled Web Link

Always write the body to a temporary file via a single-quoted HEREDOC (`cat <<'EOF'`) and invoke `gh pr create` with `--body-file`:
Write the PR body to a temporary file (`/tmp/pr_body.txt`) using a single-quoted HEREDOC:

```bash
PR_TITLE="<product>: <short description>" # e.g. compute: add foo field to google_compute_instance (under 60 chars)
PR_TITLE="<product>: <short description under 70 chars>"
BASE_BRANCH="main" # or FEATURE-BRANCH-major-release-8.0.0

cat <<'EOF' > /tmp/pr_body.txt
<summary of what changed and why>

Fixes <issue link if applicable>

```release-note:<type>
<release note content>
<product>: <release note description>
```
EOF
```

#### Attempt `gh pr create`:
```bash
gh pr create \
--repo GoogleCloudPlatform/magic-modules \
--base main \
--base "$BASE_BRANCH" \
--head "$(gh api user -q .login):$BRANCH" \
--title "$PR_TITLE" \
--body-file /tmp/pr_body.txt
```

#### Generate Pre-Filled Web URL (Always Provide as Hyperlink):
Always generate and present a clickable markdown link with the PR title and description pre-filled in query parameters for easy user review and submission:

```python
import urllib.parse

base_branch = "main" # or "FEATURE-BRANCH-major-release-8.0.0"
head_ref = f"{username}:{branch}" # e.g. "c2thorn:deprecate-beyondcorp-app"
title = "..."
body = "..."

url = f"https://github.com/GoogleCloudPlatform/magic-modules/compare/{base_branch}...{head_ref}?expand=1&title={urllib.parse.quote(title)}&body={urllib.parse.quote(body)}"
print(url)
```

Present the result in chat as:
👉 **[Create Pull Request on GitHub](<URL>)**

---

## Verification & Auto-Repair Handoff

1. Verify that `gh pr create` completed successfully and returned a valid Pull Request URL.
2. **Verify PR Body Integrity:** Run `gh pr view` to verify the published body rendered the release note block:
```bash
gh pr view --repo GoogleCloudPlatform/magic-modules
```
3. **Auto-Repair Missing Release Note:** If the output body does NOT contain `release-note:`, repair it immediately:
```bash
gh pr edit --repo GoogleCloudPlatform/magic-modules --body-file /tmp/pr_body.txt
```
4. Share the confirmed PR URL with the user and complete the execution.
## Verification & Summary

1. If `gh pr create` succeeds, view the published PR: `gh pr view --repo GoogleCloudPlatform/magic-modules`.
2. Share the confirmed PR URL (or the pre-filled direct compare URL) with the user.
64 changes: 64 additions & 0 deletions .agents/skills/workflows/deprecate_resource_or_field/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
name: deprecate-resource-or-field-workflow
description: "Workflow for deprecating existing resources, data sources, or fields on the main branch prior to a major release."
---

# `deprecate-resource-or-field-workflow`

> **Note to AI Agents:** You MUST read the YAML frontmatter above first. Only read the rest of this file if the `description` matches your required task.

This workflow governs adding deprecation notices and establishing forwards-compatibility on the `main` branch prior to a major release.

## Prerequisites

- You must be operating on the `main` branch in the `magic-modules` root directory.
- You must know the target resource/data source (e.g., `google_beyondcorp_app_connector`) and the field (or entire resource/data source) to deprecate.
- You must know the replacement path (if applicable) or the reason for deprecation.

---

## Execution Steps

### 1. Context & Guidance

Before beginning implementation, consult:
- `docs/content/breaking-changes/make-a-breaking-change.md` — Section *Add deprecations and warnings to the main branch*.
- `docs/content/reference/field.md` and `docs/content/reference/resource.md` — Configuration references.
- `.agents/knowledge/index.md` — General knowledge index.

### 2. Forwards-Compatibility & Replacement

- **Required Fields**: If a required field is being deprecated for future removal, make it optional (`required: true` $\rightarrow$ `optional: true` in MMv1 YAML or `Required: false, Optional: true` in Go) so configurations can begin omitting it prior to the major release.
- **Renames / Replacements**: Ensure the replacement field or resource is implemented, tested, and available on `main` before or alongside the deprecation.

### 3. Apply Deprecation Notice

Follow the conventions in `docs/content/breaking-changes/make-a-breaking-change.md`:
- **MMv1 Resources/Fields**: Set `deprecation_message` on the target property or resource in `mmv1/products/<product>/<Resource>.yaml`.
- **Handwritten Resources/Data Sources**: Set `Deprecated` (fields) or `DeprecationMessage` (resources / data sources) in Go schema (`mmv1/third_party/terraform/services/<product>/...`) and add deprecation warnings to markdown documentation in `website/docs/r/` or `website/docs/d/`.

### 4. Run Pre-Gen Checks

- Execute [run-pre-gen-checks](.agents/skills/utils/run-pre-gen-checks/SKILL.md) to verify Go formatting, YAML linting, template validation, and unit tests.

### 5. Generate Provider & Verify Build

- Execute [generate-provider](.agents/skills/operations/generate-provider/SKILL.md).
- Run `make build` in the downstream provider repository to verify compilation.

### 6. Verification Testing

- Invoke [qa-test-runner](.agents/skills/operations/qa-test-runner/SKILL.md) to verify acceptance tests pass (`PASS`).

### 7. PR Creation & Release Note

Execute [create-pr](.agents/skills/operations/create-pr/SKILL.md) targeting `main`:
- **Title Length Limit**: Must be strictly **under 70 characters** (e.g. `<product>: deprecate <target>`).
- **Body**:
```markdown
```release-note:deprecation
<product>: deprecated `<field_name>` on `google_<resource_name>`. Use `<replacement>` instead.
```
*(or `<product>: deprecated `google_<resource_name>` resource/data source`)*
```
- **Pre-Filled Hyperlink**: Always generate and provide a pre-filled markdown compare URL in chat for easy user submission.
Loading
Loading