feat(prices): refresh prices once a day and record each change once (#365)

.claude/memory/MEMORY.md

@@ -90,3 +90,4 @@
 - [No git stash or sqlite3 in briefs](no-git-stash-in-briefs.md) — forbid git stash/reset/checkout and the sqlite3 command (it prompts the maintainer) in every subagent brief (except restoring a mutation-tested file); a stash/pop on a clean tree applied an unrelated old stash in spec 080
 - [Unset `config.x` answers nil](rails-config-x-unset.md) — an unset `Rails.configuration.x` key is an empty OrderedOptions; assert a boot reading's class, not only nil attributes
 - [Mutation proof in place of failing-first](mutation-proof-when-behaviour-already-built.md) — only where an earlier task already built the behaviour; each mutation and its failure go in the commit body (coordinator, spec 085 Gate B)
+- [Harness changes the model mid-run](harness-changes-the-model-mid-run.md) — a worker's main session can change model without being asked; state the model per commit at gates, answer from what the harness reports

.claude/memory/harness-changes-the-model-mid-run.md

@@ -0,0 +1,12 @@
+---
+name: harness-changes-the-model-mid-run
+description: a worker's main session can change model mid-run without being asked; state the model per commit at gates
+metadata:
+  type: feedback
+---
+
+During spec 089's run the harness moved a worker's main session from Sonnet 5.5 to Opus 5.5 and back (the commit attribution reminder changed with it), without the worker or the coordinator asking; a sibling worker saw the same.
+
+**Why:** the maintainer chose the model split for the run and wants it visible while it goes, not at the end (coordinator's brief, 2026-11-13).
+
+**How to apply:** at every gate and in `worker_done`, say which model wrote what, by commit range; when asked which model the session is on, answer from what the harness reports now and whether it changed, and change nothing.

CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Status
 
-Collector is a self-hostable, multi-tenant web app for tracking collectibles, starting with Magic: The Gathering cards. The Rails app skeleton exists (stack, RSpec, `bin/ci`, Compose and Kamal deployment). Feature 002 added a collectible-agnostic catalog core (`app/models/catalog/`) with an MTG extension (`app/models/mtg/`), Scryfall bulk-data ingestion via a background refresh, and a card search at `/catalog/entries`, which has needed sign-in since accounts (spec 004) unless the operator opens it with `COLLECTOR_PUBLIC_CATALOG` (spec 023). Accounts (row-level tenancy through `Current.account`) and collections of lots (spec 004, list view spec 006) exist, and the card scanner (specs 005, 007–009) adds scanned cards to the collection through its confirm flow. Admins start and watch catalog refreshes and art index builds at `/admin/catalog`, and see, retry and discard background jobs at `/admin/jobs` (spec 015). A collector's data has a documented export format, and a collector downloads, previews and applies an export file from the account menu or More (`CollectionExport`, `CollectionImport`, `CollectionImport::Apply`; [`docs/architecture/data-portability.md`](docs/architecture/data-portability.md)). Every page's footer ends with Collector's licence line, AGPL-3.0 with a link to the source of the running build, and shows the running version and short SHA, which the image's build passes as `COLLECTOR_VERSION`, `COLLECTOR_REVISION` and `COLLECTOR_SOURCE_URL` and `Collector::BuildFacts` reads once at boot; `bin/image-smoke` fails an image without them (spec 084). The line ends with a link to `/licences`, which anyone can open and which lists the licences of the third-party software and assets in the image: the gems from the running bundle, the OCR engine, the importmap JavaScript, the fonts and the icons from `config/third_party_licences.yml`, kept honest by a guard spec (`Collector::ThirdPartyLicences`, `LicencesController`, spec 087), and `bin/ci checks` fails a gem or a manifest entry under a licence that `config/licence_policy.yml` doesn't allow or name as an exception (`Collector::ThirdPartyLicences::Policy` and `Check`, `bin/rails licences:check`, spec 091, [ADR 0031](docs/adr/0031-a-dependency-licence-is-a-decision.md)). A collector adds cards from a set by collector number at `/collection/number_entry` ("Add by number", spec 026): one line per number, quantity and finish, each added at once to a per-account list with Undo (`NumberEntry`, `NumberEntry::Line`), which Done clears; the list is working state, left out of the export. A collector manages the places their collection is kept in at `/locations` (`Location`, spec 029) and records which location a lot is in when adding or editing it, or moves some or all of a lot's copies with Move copies (`Lot::Move`, spec 032; a location that holds copies can't be removed; export format version 3) and moves every copy of many lots at once from bulk mode with Move to… (`Lot::BulkMove`, spec 041), opens a location's lots in bulk mode from its removal refusal page with one button (`Locations::SelectionsController`, spec 059), and the collection's grid tiles and table rows show it (spec 033) and filter by it (spec 037). A collector creates, renames and removes decks at `/decks` (`Deck`, `Deck::Creation`, `Catalog::Formats`, spec 062, [ADR 0028](docs/adr/0028-decks-are-locations-with-zones.md)): a deck is a location of kind `deck` whose zones are child locations that hold its copies, the collectible extension registers the formats, a lot in a zone is shown as "Deck › Zone", the export format (version 4) carries decks, a collector changes a deck's format at `/decks/:id/format/edit` (`Deck::FormatChange`, spec 064), opens a deck's lots in bulk mode from its removal refusal page (`Decks::SelectionsController`, spec 067), and opens a deck at `/decks/:id` to read the cards they own in it, zone by zone, with their cost and legality (`Deck::Page`, the seam `Catalog::DeckCards`, spec 066), and the deck's shape, its mana curve, card types and coloured mana symbols, in a panel beside the zones (`Catalog::DeckShape`, `MTG::DeckShape`, spec 069), and records the cards a deck's zone needs and they don't own as proxies, shown beside the owned copies and counted in a deck's cards but never as owned (`Deck::Proxy`, spec 068): a change of format moves them into Considering and the export carries them (format version 4 has `proxies`). A collector swaps a proxy for a free copy they own (Use my copy) or a new copy (Add a new copy) from the row's menu, and the proxy shrinks or goes in the same write (`Deck::Proxy::UseCopies`, `Deck::Proxy::AddCopy`, `Lot.free`, `Lot::MoveIntoZone`, spec 070). A collector adds a card to a deck from its card page, their free copies or a proxy (`Deck::Addition`, `Lot::FreeCopies`, spec 071). The card page shows how many copies are free to use and which decks hold the rest, in its stats and an "In decks" table (`Catalog::CardOverview`, `Deck::CardUses`, spec 073). A collector checks a deck on demand from its page, its size, copy limits and the cards with owned copies and proxies in one zone, and nothing about the check is stored (`Catalog::DeckChecks`, `MTG::Collecting.deck_rules`, `Decks::ChecksController`, spec 079). An operator backs up a running instance and restores it onto a fresh volume with two rake tasks, `backup:create[DIRECTORY]` and `backup:restore[FILE]` (`Collector::Backup`, `lib/collector/backup/`, spec 030, ADR 0024). Market prices will live in a fifth SQLite database, `prices` (`PricesRecord`, `db/prices_migrate/`, `db/prices_structure.sql`; spec 081, ADR 0029; [prices](docs/architecture/prices.md)), whose five tables exist, with a price source contract and Scryfall's prices through it (`Prices::Sources`, `MTG::Scryfall::PriceQuotes`, `PriceSource.sync!`; spec 085, #364) and no refresh or reader yet, and the backup holds it beside the primary (backup format version 2); a server start refuses a volume that holds the prices database and no primary (`Collector::StartCheck`, `start:check`, spec 086). CI proves an upgrade from the previous release's published image (`bin/upgrade-smoke`, `script/upgrade/`, spec 031, #42; [release and deploy](docs/architecture/release-and-deploy.md)). License: AGPL-3.0.
+Collector is a self-hostable, multi-tenant web app for tracking collectibles, starting with Magic: The Gathering cards. The Rails app skeleton exists (stack, RSpec, `bin/ci`, Compose and Kamal deployment). Feature 002 added a collectible-agnostic catalog core (`app/models/catalog/`) with an MTG extension (`app/models/mtg/`), Scryfall bulk-data ingestion via a background refresh, and a card search at `/catalog/entries`, which has needed sign-in since accounts (spec 004) unless the operator opens it with `COLLECTOR_PUBLIC_CATALOG` (spec 023). Accounts (row-level tenancy through `Current.account`) and collections of lots (spec 004, list view spec 006) exist, and the card scanner (specs 005, 007–009) adds scanned cards to the collection through its confirm flow. Admins start and watch catalog refreshes and art index builds at `/admin/catalog`, and see, retry and discard background jobs at `/admin/jobs` (spec 015). A collector's data has a documented export format, and a collector downloads, previews and applies an export file from the account menu or More (`CollectionExport`, `CollectionImport`, `CollectionImport::Apply`; [`docs/architecture/data-portability.md`](docs/architecture/data-portability.md)). Every page's footer ends with Collector's licence line, AGPL-3.0 with a link to the source of the running build, and shows the running version and short SHA, which the image's build passes as `COLLECTOR_VERSION`, `COLLECTOR_REVISION` and `COLLECTOR_SOURCE_URL` and `Collector::BuildFacts` reads once at boot; `bin/image-smoke` fails an image without them (spec 084). The line ends with a link to `/licences`, which anyone can open and which lists the licences of the third-party software and assets in the image: the gems from the running bundle, the OCR engine, the importmap JavaScript, the fonts and the icons from `config/third_party_licences.yml`, kept honest by a guard spec (`Collector::ThirdPartyLicences`, `LicencesController`, spec 087), and `bin/ci checks` fails a gem or a manifest entry under a licence that `config/licence_policy.yml` doesn't allow or name as an exception (`Collector::ThirdPartyLicences::Policy` and `Check`, `bin/rails licences:check`, spec 091, [ADR 0031](docs/adr/0031-a-dependency-licence-is-a-decision.md)). A collector adds cards from a set by collector number at `/collection/number_entry` ("Add by number", spec 026): one line per number, quantity and finish, each added at once to a per-account list with Undo (`NumberEntry`, `NumberEntry::Line`), which Done clears; the list is working state, left out of the export. A collector manages the places their collection is kept in at `/locations` (`Location`, spec 029) and records which location a lot is in when adding or editing it, or moves some or all of a lot's copies with Move copies (`Lot::Move`, spec 032; a location that holds copies can't be removed; export format version 3) and moves every copy of many lots at once from bulk mode with Move to… (`Lot::BulkMove`, spec 041), opens a location's lots in bulk mode from its removal refusal page with one button (`Locations::SelectionsController`, spec 059), and the collection's grid tiles and table rows show it (spec 033) and filter by it (spec 037). A collector creates, renames and removes decks at `/decks` (`Deck`, `Deck::Creation`, `Catalog::Formats`, spec 062, [ADR 0028](docs/adr/0028-decks-are-locations-with-zones.md)): a deck is a location of kind `deck` whose zones are child locations that hold its copies, the collectible extension registers the formats, a lot in a zone is shown as "Deck › Zone", the export format (version 4) carries decks, a collector changes a deck's format at `/decks/:id/format/edit` (`Deck::FormatChange`, spec 064), opens a deck's lots in bulk mode from its removal refusal page (`Decks::SelectionsController`, spec 067), and opens a deck at `/decks/:id` to read the cards they own in it, zone by zone, with their cost and legality (`Deck::Page`, the seam `Catalog::DeckCards`, spec 066), and the deck's shape, its mana curve, card types and coloured mana symbols, in a panel beside the zones (`Catalog::DeckShape`, `MTG::DeckShape`, spec 069), and records the cards a deck's zone needs and they don't own as proxies, shown beside the owned copies and counted in a deck's cards but never as owned (`Deck::Proxy`, spec 068): a change of format moves them into Considering and the export carries them (format version 4 has `proxies`). A collector swaps a proxy for a free copy they own (Use my copy) or a new copy (Add a new copy) from the row's menu, and the proxy shrinks or goes in the same write (`Deck::Proxy::UseCopies`, `Deck::Proxy::AddCopy`, `Lot.free`, `Lot::MoveIntoZone`, spec 070). A collector adds a card to a deck from its card page, their free copies or a proxy (`Deck::Addition`, `Lot::FreeCopies`, spec 071). The card page shows how many copies are free to use and which decks hold the rest, in its stats and an "In decks" table (`Catalog::CardOverview`, `Deck::CardUses`, spec 073). A collector checks a deck on demand from its page, its size, copy limits and the cards with owned copies and proxies in one zone, and nothing about the check is stored (`Catalog::DeckChecks`, `MTG::Collecting.deck_rules`, `Decks::ChecksController`, spec 079). An operator backs up a running instance and restores it onto a fresh volume with two rake tasks, `backup:create[DIRECTORY]` and `backup:restore[FILE]` (`Collector::Backup`, `lib/collector/backup/`, spec 030, ADR 0024). Market prices will live in a fifth SQLite database, `prices` (`PricesRecord`, `db/prices_migrate/`, `db/prices_structure.sql`; spec 081, ADR 0029; [prices](docs/architecture/prices.md)), whose five tables exist, with a price source contract and Scryfall's prices through it (`Prices::Sources`, `MTG::Scryfall::PriceQuotes`, `PriceSource.sync!`; spec 085, #364) a daily refresh that records each price change once (`Prices::Refresh`, `Prices::RefreshJob`, `PriceRefreshRun`, `CurrentPrice`, `PriceChange`; spec 089, #365) and no reader yet (#366 to #369 are still to come), and the backup holds it beside the primary (backup format version 2); a server start refuses a volume that holds the prices database and no primary (`Collector::StartCheck`, `start:check`, spec 086). CI proves an upgrade from the previous release's published image (`bin/upgrade-smoke`, `script/upgrade/`, spec 031, #42; [release and deploy](docs/architecture/release-and-deploy.md)). License: AGPL-3.0.
 
 Mission and principles: `.claude/memory/foundation.md`. Goal and spec rules: `docs/north-star.md`. Order of work: `docs/roadmap.md`. Decisions: `docs/adr/` (index in its README). Current rules per subsystem: `docs/architecture/` (area docs). Detailed coding rules: `.claude/rules/`. Feature specs: `docs/specs/` (index: `docs/specs/README.md`).
 

README.md

@@ -454,7 +454,7 @@ scp root@192.168.0.1:/srv/collector-backups/collector-backup-20261021T142100Z.ta
 
 The file is mode 0600 and owned by uid 1000 on the server; Kamal logs in as root by default (`ssh:` in `config/deploy.yml`), which can read it. Any other login needs `sudo`.
 
-**When to back up:** before every upgrade, and at least daily for a collection that changes daily. From a host's cron, for example (02:17 every day, clear of Monday 03:00 to 05:00 in the container's time zone, UTC unless you set one, when the weekly catalog refresh writes the most and a backup would hold the database's write-ahead log back):
+**When to back up:** before every upgrade, and at least daily for a collection that changes daily. From a host's cron, for example (02:17 every day, clear of Monday 03:00 to 05:00 in the container's time zone, UTC unless you set one, when the weekly catalog refresh writes the most and a backup would hold the database's write-ahead log back, and clear of 11:30 in the same time zone, when the daily price refresh starts writing the prices database, which a backup's read transaction holds back in the same way):
 
 ```cron
 17 2 * * * cd /srv/collector && docker compose run --rm -T -v /srv/collector-backups:/backups web bin/rails "backup:create[/backups]" >> /var/log/collector-backup.log 2>&1

app/jobs/prices/refresh_job.rb

@@ -0,0 +1,29 @@
+# Runs the daily price refresh for a price source, in the background (spec 089). Queued daily by config/recurring.yml,
+# or by hand from `bin/rails runner`; there is no button or task until the admin page (#367).
+#
+# No error is discarded: a failed job shows under Failed on `/admin/jobs`, which is how an operator learns of a guard that
+# fired before #367 shows the runs. A departure from `background-jobs.md`'s `discard_on` advice, whose example is a
+# `DeserializationError`: the argument is a string, and a discarded job would hide a guard that fired (spec 089 decision 7).
+#
+# @see file:prices.md
+class Prices::RefreshJob < ApplicationJob
+  # How long the queue holds a second refresh of the source back when the first never reports its end: a waiting refresh
+  # must never start while another's job may still be working (the catalog refresh's value, spec 089 decision 7). The
+  # run's own stall rule is `PriceRefreshRun::STALL_AFTER`, in minutes.
+  LOCK_FOR = 6.hours
+
+  queue_as :sync
+
+  limits_concurrency to: 1, key: ->(source_key, *) { source_key }, duration: LOCK_FOR
+
+  # The download's size error is retried too: a file republished mid-download gives exactly it, and the retry asks the
+  # index again (spec 089 decision 7).
+  retry_on Catalog::Sources::TransientError, Catalog::Sources::IntegrityError, ActiveRecord::StatementTimeout,
+    wait: :polynomially_longer, attempts: 3
+  # After retry_on, so that it is tried first: a rate-limited source is a TransientError too, but the job comes back when
+  # the source asked. It reports no progress: a rate limit ends a refresh before any row is written.
+  include Catalog::RateLimitRetry
+
+  # @param source_key [String] a key of `Prices.sources`
+  def perform(source_key) = Prices::Refresh.new(source_key, job_id:).call
+end

app/models/catalog/sources.rb

@@ -8,7 +8,9 @@
 #   .alternate_names(entries)           optional: [[identity id, name], …] the name index adds to identity names
 #   #languages                          sorted language codes to ingest (raises ConfigurationError)
 #   #current_version(languages:)        opaque String naming the current source data
-#   #download(version, dir:)            Pathname of the verified local copy (IntegrityError, TransientError)
+#   #download(version, dir:)            Pathname of the verified local copy (IntegrityError, TransientError). Two callers
+#                                       (the catalog refresh and a price refresh) may call it for one version at once, so it
+#                                       must tolerate concurrent calls: a partial download is never shared between calls
 #   #each_set { |SetRecord| }
 #   #each_entry(path, languages:) { |EntryRecord or Malformed| }   streamed, filtered to languages
 #   #reapply?                           optional: true to apply a version again although it was applied (asked before a skip)

app/models/current_price.rb

@@ -0,0 +1,11 @@
+# The latest known price of one printing in one series from one source, in the prices database (spec 081; spec 089).
+# Keyed by printing (a catalog entry's id, with no foreign key: `prices.md`), source and series; the table is
+# WITHOUT ROWID with no other index, so every lookup goes by printing, the key's first column.
+#
+# The daily refresh writes it with `upsert_all` and `delete_all` by key, which skip validations on purpose: each row comes
+# from a validated `Prices::Sources::Quote`, and the table's own checks refuse the rest.
+#
+# @see file:prices.md
+class CurrentPrice < PricesRecord
+  self.primary_key = %i[catalog_entry_id price_source_id price_series_id]
+end