Skip to content

Pin branch-tracked mod versions to their real commit, not the moving branch tip - #55

Open
12problems wants to merge 2 commits into
mqttfrom
fix/branch-tracked-mod-version-pinning
Open

Pin branch-tracked mod versions to their real commit, not the moving branch tip#55
12problems wants to merge 2 commits into
mqttfrom
fix/branch-tracked-mod-version-pinning

Conversation

@12problems

Copy link
Copy Markdown

What

mod_registry_versions rows for branch-tracked mods (no real GitHub releases) all carry the exact same downloadUrl and (whenever the branch happened not to move between two sync cycles) the exact same sha256, no matter how many distinct commit-hash "version" labels get recorded over time. Confirmed live against Aikoyori@Aikoyoris-Shenanigans from the real skyline69/balatro-mod-index data.

Root cause

Upstream's own update_mod_versions.py bumps a branch-tracked mod's version field to the latest commit SHA of the whole repo (an unscoped /repos/{owner}/{repo}/commits call — fires on any commit anywhere, not just ones touching the mod itself), but only ever rewrites downloadURL for its tag/release cases — never for the VersionSource.HEAD case. So every version label ever recorded for such a mod points at the exact same branch-archive URL, and downloading it always fetches "whatever's on the branch right now" — never the specific commit the label names.

This isn't just cosmetic duplication: it means an older label becomes permanently unfetchable once the branch advances past it. Nothing in this pipeline can ever again produce that label's original bytes, silently breaking any profile pinned to it — a Ranked rankedVersion pin, or a user manually selecting an older entry from the version dropdown.

Fix

  • resolveCommitPinnedDownloadUrl() (new, in custom-mod-version-check.service.ts alongside its existing GitHub-resolution helpers): when a version string looks like a git SHA and its downloadUrl classifies as 'branch', resolves the SHA to its full 40-char form via one GitHub API call and rebuilds a codeload.github.com/.../zip/<full-sha> URL — permanently fetchable regardless of where the branch moves next.
  • pinBranchVersionIfNew() (new, in mods-sync.service.ts): calls the above right before a (modId, version) pair is hashed and stored for the first time, for both upstream-index entries and automaticVersionCheck custom mods tracking a branch. A version that's already been hashed is left alone unconditionally — this costs one extra GitHub API call per brand-new branch-tracked version, never per sync.
  • Falls back to the literal branch URL whenever pinning isn't applicable (release/custom sources, a non-SHA-like version string) or the GitHub lookup fails/rate-limits — never a hard failure that aborts the sync over one mod.

Known follow-up (not done here)

Existing mod_registry_versions rows synced before this fix keep their stale branch URL — only new versions get pinned going forward. A one-off backfill (mirroring recomputeAllModHashes's shape) would be needed to retroactively pin already-stored branch-tracked versions.

Testing

Added unit tests for resolveCommitPinnedDownloadUrl() covering: successful pin, non-branch URL (no-op), non-SHA-like version (no-op), 404, and rate-limit — all best-effort/non-throwing. Full src/tests/mods suite passes (40/40).

🤖 Generated with Claude Code

12problems and others added 2 commits September 5, 2026 22:52
…branch tip

skyline69/balatro-mod-index's own update_mod_versions.py bumps a
branch-tracked mod's `version` field to the latest commit SHA of the
*whole repo* on any commit anywhere in it, but only ever rewrites
`downloadURL` for its tag/release cases -- never for the HEAD case. So
every version ever recorded for such a mod carries the exact same
branch-archive URL, and downloading it always fetches "whatever's on
the branch right now", never the specific commit the version label
names. Confirmed live: Aikoyori@Aikoyoris-Shenanigans' version history
has a dozen distinct commit-hash labels all sharing one identical
downloadUrl and (whenever the branch hadn't moved between two label
bumps) identical sha256.

The real cost isn't the duplication -- it's that an older label
becomes permanently unfetchable once the branch advances past it,
silently breaking any profile (a Ranked rankedVersion pin, or a user
manually pinning an older dropdown entry) sitting on it.

Add resolveCommitPinnedDownloadUrl() (custom-mod-version-check.service.ts,
alongside its existing GitHub-resolution helpers): when a version looks
like a git SHA and its downloadUrl classifies as 'branch', resolve the
SHA to its full 40-char form via one GitHub API call and rebuild a
codeload.github.com/.../zip/<full-sha> URL that stays fetchable forever,
regardless of where the branch moves next.

Wire it into mods-sync.service.ts via pinBranchVersionIfNew(), called
right before a (modId, version) pair is hashed and stored for the first
time -- both for upstream-index entries and for automaticVersionCheck
custom mods tracking a branch. A version that's already been hashed is
left alone unconditionally, so this costs one extra GitHub API call per
brand-new branch-tracked version, never per sync.

Known follow-up (not done here): existing mod_registry_versions rows
synced before this fix keep their stale branch URL -- only new versions
get pinned going forward. A one-off backfill (mirroring
recomputeAllModHashes's shape) would be needed to retroactively pin
already-stored branch-tracked versions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pinBranchVersionIfNew() (previous commit) only pins a version's downloadUrl
the first time it's synced -- every mod_registry_versions row written
before that fix exists is still sitting on its original moving-branch-tip
URL, with a sha256 computed against whatever that tip happened to be at
hash-time (not necessarily the exact commit its own version label names).

Add backfill-branch-pins.ts (pnpm backfill-branch-pins), a one-off script
in the same shape as the existing backfill-mod-hashes.ts: finds every
mod_registry_versions row whose downloadUrl still classifies as 'branch',
resolves a commit-pinned URL for it, and *re-downloads and re-hashes* it
against that pinned URL rather than trusting the existing stored hash --
the old hash could already be wrong for the commit it's about to be
pinned to, so only a fresh hash guarantees correctness.

Each row gets up to 3 resolve+hash attempts with a short backoff before
being marked via a new mod_registry_versions.pin_failed_at column
(migration 0036) and skipped on future runs -- so a genuinely dead
repo/branch/commit doesn't keep burning GitHub API calls on every
re-run. Pass --retry-failed to re-attempt previously-marked rows (e.g.
after a renamed repo or an expired rate limit).

New gateway functions in mods.gateway.ts: listVersionsWithDownloadUrl
(broad fetch, filtered/classified in TS same as listAllVersionsWithDownloadUrl),
applyBranchPin (writes the pinned URL + hash, clears pinFailedAt),
markVersionPinFailed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@12problems

Copy link
Copy Markdown
Author

Added the backfill for pre-existing stale rows (per your call in chat):

  • pnpm backfill-branch-pins (backfill-branch-pins.ts) — a one-off script, same shape as the existing backfill-mod-hashes.ts. Finds every mod_registry_versions row still on a live branch URL, resolves it to a commit-pinned one, and re-downloads + re-hashes it against that pinned commit (not just repoints the URL) — a stale row's stored hash could already be wrong for the commit it's about to be pinned to.
  • Up to 3 resolve+hash attempts per row with backoff; a row that still fails gets marked via a new pin_failed_at column (migration 0036) and is skipped on future runs, so a genuinely dead repo/commit doesn't burn API calls forever. --retry-failed re-attempts marked rows on demand.
  • Deliberately not wired into the recurring hourly sync — this stays a manual, on-demand script.

Run once after this PR deploys:

docker compose exec api pnpm --filter balatro-multiplayer-api-server backfill-branch-pins

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant