Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions repo_policy_sync/docs/explanation/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,15 @@ references; operation handlers do not know where those values came from.
`operations/`. An operation owns its YAML validation, compliance check,
remediation description, and application.

GitHub target ref operations scan workflow text directly so YAML comments and
formatting survive updates. `GitHubResolver` is created once for a run, uses
the authenticated `gh api` boundary for repository tag and commit metadata,
and shares a synchronized in-memory cache across policy and repository
workers. Exact refs do not require remote resolution; minimal refs resolve
tags and use GitHub's commit ancestry comparison for full SHA pins. The
workflow target can be an action path or an external reusable-workflow path;
the API repository is derived from its first two path components.

The registry is intentionally built in. It makes supported operations visible
and testable without runtime discovery, third-party code loading, or an
extension ABI. Add a new operation by implementing it in `operations/` and
Expand Down
36 changes: 36 additions & 0 deletions repo_policy_sync/docs/reference/policy-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,42 @@ existing match; without one, it appends the line. A missing file is created.
Globs use `*`, `?`, and `[...]` and match complete raw lines; comments and
whitespace have no special meaning.

### `ensure_exact`

```yaml
- type: ensure_exact
target: actions/checkout
ref: 11bd719055c2f7a3f9e0f8c1e4f2a5b6c7d8e9f0
```

Updates every matching external action or reusable-workflow `uses:` reference
in workflow files under `.github/workflows/`. `target` uses the complete
external target before `@`, such as `actions/checkout`,
`eclipse-score/cicd-actions/setup-bazel-cache`, or
`eclipse-score/cicd-actions/.github/workflows/build.yml`. `ref` may be a
branch, tag, or complete 40-character commit SHA. The operation preserves
surrounding YAML formatting, quotes, and comments. Local actions and local
reusable workflows are outside this operation.

### `ensure_minimal`

```yaml
- type: ensure_minimal
target: actions/setup-python
minimum_version: v5.1
```

The operation obtains the target repository's tags through the authenticated
`gh api` client and selects the lowest semantic tag that satisfies
`minimum_version`. Older semantic tags are replaced with that tag. A complete
SHA is compared with the resolved tag commit through GitHub's commit comparison
API; an ancestor is updated to the resolved commit SHA, while an identical or
newer commit is left alone. Diverged commit histories are rejected because
their order is not unambiguous. Branches and other non-semantic refs are
intentionally unchanged. Repository tags and commit comparisons are cached for
the duration of one policy run. A policy can contain any number of
`ensure_exact` and `ensure_minimal` entries for actions or reusable workflows.

### `remove_file`

```yaml
Expand Down
1 change: 1 addition & 0 deletions repo_policy_sync/policies/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ use the [documentation index](../docs/README.md).
| `docs-as-code-gitignore` | Update `score_docs_as_code` Git ignore entries and remove legacy configuration files. | One-time cleanup |
| `minimal-bazel-module-declaration` | Keep `MODULE.bazel` limited to the repository-owned module name by removing version metadata. | One-time cleanup |
| `minimum-bazel-version` | Upgrade repositories to at least Bazel `8.6.0` and regenerate the lockfile when required. | Baseline maintenance |
| `minimum-github-action-versions` | Keep `actions/checkout` at semantic version `v6` or newer in GitHub workflows. | Baseline maintenance |
| `score-devcontainer-dependency-alignment` | Declare the direct SCORE devcontainer Bazel dependency using the Dockerfile image version. | One-time integration |

The policy definitions and their executable before/after cases are the
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# *******************************************************************************
# Copyright (c) 2026 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
title: "chore(actions): keep actions/checkout at v6 or newer"
description: Ensure workflows use a supported release of the standard checkout action.

ensure:
- type: ensure_minimal
target: actions/checkout
minimum_version: v6
rationale: Keep the checkout action on v6 or a newer semantic release.
42 changes: 36 additions & 6 deletions repo_policy_sync/src/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
starlark_call_ranges,
)
from .errors import CommandError, RepoPolicySyncError, redact_sensitive_text
from .github import GitHubResolver
from .models import Change, EnsureOperation, Evaluation, Policy
from .operations import apply as apply_operation
from .operations import describe_changes, resolve_operation
Expand All @@ -52,18 +53,29 @@


def evaluate_policy(
root: Path, policy: Policy, *, organization: str | None = None
root: Path,
policy: Policy,
*,
organization: str | None = None,
github_resolver: GitHubResolver | None = None,
) -> Evaluation:
"""Evaluate a policy against a checked-out repository without changing it."""

evaluation, _ = _evaluate_policy_with_operations(
root, policy, organization=organization
root,
policy,
organization=organization,
github_resolver=github_resolver,
)
return evaluation


def _evaluate_policy_with_operations(
root: Path, policy: Policy, *, organization: str | None = None
root: Path,
policy: Policy,
*,
organization: str | None = None,
github_resolver: GitHubResolver | None = None,
) -> tuple[Evaluation, tuple[EnsureOperation, ...]]:
"""Evaluate a policy and retain the materialized operations for applying it."""

Expand All @@ -75,9 +87,17 @@ def _evaluate_policy_with_operations(
operations = tuple(
resolve_operation(operation, values) for operation in policy.ensure
)
github_resolver = github_resolver or GitHubResolver()
changes: list[Change] = []
for operation in operations:
changes.extend(describe_changes(root, operation, organization=organization))
changes.extend(
describe_changes(
root,
operation,
organization=organization,
github_resolver=github_resolver,
)
)
if changes:
changes.extend(
Change(command.when_file_exists, command.description)
Expand All @@ -95,16 +115,26 @@ def apply_policy(
*,
force_after_apply: bool = False,
organization: str | None = None,
github_resolver: GitHubResolver | None = None,
) -> Evaluation:
"""Apply a matching policy and return the changes that were made."""

github_resolver = github_resolver or GitHubResolver()
evaluation, operations = _evaluate_policy_with_operations(
root, policy, organization=organization
root,
policy,
organization=organization,
github_resolver=github_resolver,
)
if not evaluation.applies:
return evaluation
for operation in operations:
apply_operation(root, operation, organization=organization)
apply_operation(
root,
operation,
organization=organization,
github_resolver=github_resolver,
)
if evaluation.changes or force_after_apply:
changed_paths = {change.path for change in evaluation.changes}
for command in policy.after_apply:
Expand Down
163 changes: 163 additions & 0 deletions repo_policy_sync/src/github.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from dataclasses import dataclass
from importlib.resources import files
from pathlib import Path
from threading import RLock

from .errors import CommandError, RepoPolicySyncError, redact_sensitive_text
from .models import Change, Policy, policy_branch_slug
Expand Down Expand Up @@ -847,6 +848,168 @@ def _run(
return result.stdout


@dataclass(frozen=True)
class GitHubTag:
"""A release tag and the commit GitHub resolves it to.

Repository identity is intentionally kept by the resolver that returned
the tag. A tag result is always scoped to one repository, so this value
object can describe the release without duplicating that context.
"""

name: str
sha: str
Comment thread
MaximilianSoerenPollak marked this conversation as resolved.


class GitHubResolver:
"""Resolve GitHub repository releases through authenticated ``gh api`` calls.

Repository metadata is immutable for the duration of a policy run. Keeping
the cache on this object avoids repeated requests when a policy has several
matching workflow entries or several targets from the same repository. The
lock also prevents concurrent repository workers from fetching the same
metadata twice.
"""

def __init__(self, client: GitHubCli | None = None) -> None:
self._client = client or GitHubCli()
self._tags: dict[str, tuple[GitHubTag, ...]] = {}
self._comparisons: dict[tuple[str, str, str], str] = {}
self._lock = RLock()

def tags(self, repository: str) -> tuple[GitHubTag, ...]:
"""Return all published tags for one ``owner/repository``.

The result is scoped to the requested repository. Keeping that scope at
the resolver boundary ensures callers cannot accidentally select a tag
from a different repository while keeping ``GitHubTag`` repository
agnostic.
"""

with self._lock:
cached = self._tags.get(repository)
if cached is not None:
return cached
output = self._client._run(
[
"gh",
"api",
"--paginate",
"--slurp",
f"/repos/{repository}/tags?per_page=100",
]
)
tags = _parse_repository_tags(output, repository)
self._tags[repository] = tags
return tags

def compare_commits(self, repository: str, base: str, head: str) -> str:
"""Return GitHub's ancestry relationship for ``base`` and ``head``.

GitHub reports ``behind`` when ``head`` is an ancestor of ``base``,
which is exactly the state in which a target pin needs updating.
``diverged`` is intentionally retained as an error by the operation:
commit timestamps cannot safely turn unrelated histories into a
minimum-version decision.
"""

key = (repository, base, head)
with self._lock:
cached = self._comparisons.get(key)
if cached is not None:
return cached
output = self._client._run(
[
"gh",
"api",
f"/repos/{repository}/compare/{base}...{head}",
]
)
try:
response = json.loads(output)
except json.JSONDecodeError as exc:
raise CommandError(
f"gh returned invalid commit comparison JSON for {repository}"
) from exc
if not isinstance(response, dict) or not isinstance(
response.get("status"), str
):
raise CommandError(
f"gh returned invalid commit comparison JSON for {repository}"
)
status = response["status"]
if status not in {"ahead", "behind", "identical", "diverged"}:
raise CommandError(
f"gh returned an unknown commit comparison status for {repository}: {status!r}"
)
self._comparisons[key] = status
return status


def _parse_repository_tags(output: str, repository: str) -> tuple[GitHubTag, ...]:
"""Validate the paginated shape returned by GitHub's tags endpoint."""

try:
pages = json.loads(output)
except json.JSONDecodeError as exc:
raise CommandError(
f"gh returned invalid repository tag JSON for {repository}"
) from exc
# `gh api --paginate --slurp` must produce a list. Treating any other
# shape as an error avoids silently interpreting an API error object as a
# repository with no usable releases.
if not isinstance(pages, list):
raise CommandError(f"gh returned invalid repository tag JSON for {repository}")
# The normal `--slurp` shape is a list of page lists, while lightweight API
# doubles and non-paginated callers commonly provide one flat entry list.
# Normalize both forms here so release selection never depends on how the
# response was paginated.
entries = (
[entry for page in pages for entry in page]
if all(isinstance(page, list) for page in pages)
else pages
)
tags: list[GitHubTag] = []
for entry in entries:
# A malformed item must fail the policy rather than being skipped and
# potentially making an incomplete release list look authoritative.
if not isinstance(entry, dict):
raise CommandError(
f"gh returned invalid repository tag JSON for {repository}"
)
name = entry.get("name")
commit = entry.get("commit")
# The tags endpoint normally exposes `name` and `commit`, but GitHub's
# ref-shaped representation uses `ref` and `object`. Accept both so
# endpoint representation details cannot change the resolved release.
if name is None:
reference = entry.get("ref")
object_value = entry.get("object")
if (
isinstance(reference, str)
and reference.startswith("refs/tags/")
and isinstance(object_value, dict)
):
name = reference.removeprefix("refs/tags/")
commit = object_value
sha = commit.get("sha") if isinstance(commit, dict) else None
# The policy compares semantic tag names and pins to commit SHAs. Both
# values therefore need to be validated before they influence either
# decision; malformed metadata is safer as an error than as a partial
# or incorrect update.
if (
not isinstance(name, str)
or not name
or not isinstance(sha, str)
or re.fullmatch(r"[0-9a-fA-F]{40}", sha) is None
):
raise CommandError(
f"gh returned invalid repository tag JSON for {repository}"
)
tags.append(GitHubTag(name, sha))
return tuple(tags)


def policy_branch(policy_id: str) -> str:
"""Map a stable policy identifier to a safe, deterministic branch name."""

Expand Down
20 changes: 20 additions & 0 deletions repo_policy_sync/src/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,24 @@ class EnsureMinimumVersion:
rationale: str | None = None


@dataclass(frozen=True)
class EnsureExactGitHubRef:
"""Ensure every use of one external GitHub target points at one ref."""

target: str
ref: str
rationale: str | None = None


@dataclass(frozen=True)
class EnsureMinimalGitHubRef:
"""Ensure GitHub target tags and commit pins meet a minimum release."""

target: str
minimum_version: str
rationale: str | None = None


@dataclass(frozen=True)
class ValueReference:
"""A reference to a named value derived by the containing policy."""
Expand Down Expand Up @@ -155,6 +173,8 @@ class EnsureBazelDependencyDevDependency:
| RemoveFile
| ReplaceRegex
| EnsureMinimumVersion
| EnsureExactGitHubRef
| EnsureMinimalGitHubRef
| EnsureBazelDependency
| EnsureBazelDependencyDevDependency
)
Expand Down
2 changes: 2 additions & 0 deletions repo_policy_sync/src/operations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ authoritative source for the complete schema, validation rules, and examples.
| `ensure_bazel_dependency` | Declaring a direct bzlmod dependency | Adds a `bazel_dep` with the configured module name and version when it is missing. | [`test_ensure_bazel_dependency.py`](../../tests/operations/test_ensure_bazel_dependency.py) |
| `ensure_bazel_dependency_dev_dependency` | Controlling whether a direct bzlmod dependency is development-only | Adds or changes `dev_dependency = True`, or removes the attribute when configured as false, in the repository-root `MODULE.bazel`. | [`test_ensure_bazel_dependency_dev_dependency.py`](../../tests/operations/test_ensure_bazel_dependency_dev_dependency.py) |
| `ensure_line` | Keeping one exact line in a text file | Inserts the desired line, removes configured replacements and duplicates, and creates a missing file. | [`test_ensure_line.py`](../../tests/operations/test_ensure_line.py) |
| `ensure_exact` | Pinning one external GitHub target to one exact ref | Updates every matching action or reusable-workflow `uses: target@ref` in `.github/workflows/**/*.yml` and `.yaml` to the configured branch, tag, or full commit SHA. | [`test_ensure_github_ref.py`](../../tests/operations/test_ensure_github_ref.py) |
| `ensure_minimal` | Maintaining a minimum release for one external GitHub target | Resolves the lowest available semantic tag meeting the configured minimum, upgrades older tags and stale commit pins, and leaves branches unchanged. | [`test_ensure_github_ref.py`](../../tests/operations/test_ensure_github_ref.py) |
| `ensure_minimum_version` | Maintaining a simple version file such as `.bazelversion` | Replaces a lower `major.minor.patch` value; equal or higher versions and missing files are compliant. | [`test_ensure_minimum_version.py`](../../tests/operations/test_ensure_minimum_version.py) |
| `remove_file` | Removing an obsolete file | Deletes an existing file; a missing file is compliant and directories are rejected. | [`test_remove_file.py`](../../tests/operations/test_remove_file.py) |
| `replace_regex` | Applying a narrow text substitution | Applies Python `re.sub` to a complete UTF-8 file; missing files and non-matching patterns are compliant. | [`test_replace_regex.py`](../../tests/operations/test_replace_regex.py) |
Expand Down
Loading