From 039ac0d20978aa81c94f42ecb3cbba5e8f99e590 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:14:42 -0500 Subject: [PATCH 01/41] docs(specs): draft spec 090, every user has a household (#380) Draft v0.1.0. Decisions 2 to 6 are written on the recommendations put to the maintainer. Written by the main session on Sonnet 5.5. Co-Authored-By: Claude Sonnet 5.5 --- .../090-every-user-has-a-household/spec.md | 228 ++++++++++++++++++ docs/specs/README.md | 1 + 2 files changed, 229 insertions(+) create mode 100644 docs/specs/090-every-user-has-a-household/spec.md diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md new file mode 100644 index 00000000..068f23dd --- /dev/null +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -0,0 +1,228 @@ +# Feature 090: Every User Has a Household + +**Status:** Draft (decisions 2 to 6 are pending the maintainer's ruling; each is written on the recommendation) +**Version:** 0.1.0 +**Created:** 2026-11-13 +**Last Updated:** 2026-11-13 +**Depends on:** [spec 004](../004-design-system-collection/spec.md) (accounts, FR-2 to FR-5; frozen, by [tenancy and accounts](../../architecture/tenancy-and-accounts.md)), [spec 017](../017-export-format/spec.md) FR-9 (the coverage guard), [spec 031](../031-upgrade-from-previous-release/spec.md) (the upgrade proof), [spec 049](../049-session-idle-expiry/spec.md) (three migrations without a transaction, the precedent for decision 5) +**Branch:** `090-every-user-has-a-household` +**Issue:** [#380](https://github.com/plainprogrammer/Collector/issues/380) (the 1.0 milestone, households lane; the root of #381 to #389 and of the hosted lane's #393) +**ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md), which supersedes the one-user half of [ADR 0017](../../adr/0017-row-level-tenancy-one-user-per-account.md) +**Supersedes:** spec 004 FR-3 in part ("an account has exactly one user": an account is now in a household, and `users.account_id` stays unique) and [ADR 0017](../../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by ADR 0030; the area doc [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) records it +**Design:** [households and the hosted service](../../superpowers/specs/2026-11-13-households-and-hosted-service-design.md), decisions 9 to 11 (rulings 43 and 44 of the [roadmap](../../roadmap.md)) + +--- + +## Changelog + +| Version | Date | Change | +|---------|------|--------| +| 0.1.0 | 2026-11-13 | First draft from issue #380, the design and the coordinator's brief. Decisions 2 to 6 are the five scope questions put to the maintainer, written on their recommendations | + +## Problem Statement + +Every later piece of the households lane (joining, leaving, a shared collection, transfers, borrowing) and the hosted lane's plan per household needs one fact that does not exist: which household a user, and an account, belongs to. Adding it later, after more tables exist and after more instances hold data, only gets dearer (design, "Why"). The fact has to exist for every user on every instance, including the ones that are upgraded from a release that had no households, and it has to arrive without a collector seeing anything change: no page, no word, no new rule for them to learn. + +This feature adds the structure and nothing that uses it: a household with a name and its members, a role for each member, a kind for each account, a household made with every new user, and one made for every user that already exists on upgrade. + +> **Terms.** *Household*: a named group of accounts; today always one user's. *Member* (of a household): a user in it. *Manager*: a member with the manager role; *plain member*: a member with the member role. *Membership*: the row that puts one user in one household with a role. *Personal account*: an account of kind `personal`, a member's own and private (what every account is today). *Shared account*: an account of kind `shared`, owned by the household and used by no single user; none exists after this feature. *Household of one*: a household with one member. **"Member" is also spec 015's word for a user without the admin flag** (the admin users page's "Member"); the two are independent: the admin flag governs the instance, the role governs a household, and no page shows a role in this feature. + +## Goals + +- Every user belongs to exactly one household, as a manager or a member; a household always has a manager. +- A new user, by every way a user is made (first run, sign-up, an admin, the operator's command), starts as the manager of a household of one, made in the same write as the user and their account. +- An account knows its household and its kind (`personal` or `shared`); a user's account is personal and is in the user's household. +- An existing instance gains a household for every user when it upgrades, unattended, with no row lost and no collector signed out, and the upgrade proof shows it. +- Nothing in the app looks different: no new page, link, word or field, and every tenant-isolation check passes unchanged. +- The names the later issues build on are settled here: tables, columns, roles, kinds. + +## Non-Goals + +- Any page, link, form or word about households: no invitation, no join or leave, no household name field, no role shown, no switch. The admin users page does not change (#389). +- A shared account, a way to create one, or a collection in it (#384). The schema allows one; nothing makes one. +- A user in two households, or a household with no member. +- Moving a user between households (#381, #382) and handing the manager role over (#383). This feature makes the rule they must keep. +- A kid role, or any role beyond manager and member (design decision 11). +- `Current.household`, or any change to how a request resolves `Current.account` (decision 14 of the design is #384's). +- Raising the export format's version or carrying a household in an export (decision 4). +- A plan, a limit or hosted mode (the hosted lane, #392 to #394). + +## Users and Context + +**Primary users:** every collector, who sees no change; an operator upgrading an instance, who needs the upgrade to give households to existing users with no step to run. +**Secondary users:** the agents and maintainers of #381 to #389 and #393, who build on the names and rules here; the maintainer's own instance, upgraded from the previous release. +**Usage context:** a migration at the first start after an upgrade (`db:prepare` on boot), and a household made whenever a user is. +**User mental model:** none. A household of one is invisible (design decision 10); the collector's mental model is the one they have, a private collection. + +## Decisions + +1. **The names.** *(Settles design decisions 10 and 11 and the schema half of 9; no departure from the design's names.)* Tables `households` (`id`, `name`, timestamps) and `memberships` (`id`, `household_id`, `user_id`, `role`, timestamps); roles `manager` and `member` (`memberships.role`, an enum with explicit values, a `CHECK` naming both); two columns on `accounts`: `household_id` (a foreign key to `households`, indexed) and `kind` (`personal` or `shared`, default `personal`, a named `CHECK`). `users.account_id` stays, unique, and `users` gains nothing. A user's household is read through their membership, an account's through `accounts.household_id`. **Both exist on purpose:** a shared account has no user, so it can't be found through a membership, and a membership is where the role lives; the cost is one fact held twice for a personal account. Its rule is decision 6. + **Added to the design's model, each with its reason:** `memberships.user_id` is unique (one household per user, as a fact of the database, not only the model); `memberships.household_id` is indexed (the household's members are read by it); a partial unique index on `accounts (household_id) WHERE kind = 'shared'` (the design says "one shared Account"; stating it now means #384 can't build a second by accident, and no row can break it today). +2. **A household of one is named for its user.** *(Pending the maintainer's ruling; recommended over: a constant "Household".)* The name is stored text, the user's name followed by "'s household" ("Ada's household"), the user's name cut to fit so the whole is at most 100 characters; a household's name is present and at most 100 characters. It is stored, not derived, so a user's rename later changes nothing in the household, and a list of households (#389) tells them apart. No page shows or edits it in this feature; a manager renames it when a later issue gives them the page. +3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Pending the maintainer's ruling; recommended over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys, `NOT NULL` on every column. The model holds: a household is created with its first membership, a manager, in one transaction with the user; and a household with another member can't lose its last manager by demotion, by removal of the membership, or by deleting the user. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). +4. **The export does not carry the household, and the format stays version 4.** *(Pending the maintainer's ruling; recommended over: carrying the name in `profile` and raising to version 5 now.)* The export is one file per account and holds the collection and its profile (design decision 16). A household is a fact about a group, not about the collection; a household of one has a default name, and a person who imports their file onto another instance is made a household there by the same rule that makes every user one. The shared collection's file and its `profile` are #384's (the design's open question 4), and that is the issue that raises the version. The format document's Coverage tables list the new data as "Left out" with the reason (`memberships`, `accounts.household_id`, `accounts.kind`); `households` has neither `account_id` nor `user_id`, so the guard doesn't ask for it, and the format document says in one sentence where it is. +5. **Three migrations, as spec 049 did; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Pending the maintainer's ruling; recommended over: leaving the column nullable.)* + - **(a) Schema.** Creates `households` and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). Guarded, so a run that stopped part way is finished by the next boot. + - **(b) Backfill.** A data migration with no schema change. It reads the users that have no membership in batches of 500, and for each batch, in one short transaction, inserts a household for each user (decision 2's name), the manager membership and the account's `household_id`, with plain SQL and no application model. It is idempotent: it selects only users with no membership, so a second run, or a run that was stopped after some batches, makes exactly the rest and nothing twice; it takes no lock beyond each batch's own write transaction, so the instance's other work waits at most one batch. Cost: users, not lots; ten thousand users are twenty batches. + - **(c) Tighten.** Makes `accounts.household_id` `NOT NULL`, so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **What the copy risks.** SQLite can't add `NOT NULL` to a column in place; Rails copies `accounts` (create the new table, copy the rows, drop the old, rename). `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven have `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), `lots` has the default `RESTRICT` (the reason `Account` declares `lots` before `locations`, `spec/models/account_spec.rb`), and `users` has the default too; a cascade from one of the eleven reaches their own children in turn (a location's lots are refused by the restrict, a deck's zones and proxies go). With foreign keys enforced, the copy's `DROP TABLE accounts` runs those delete rules: the cascades would delete every collector's working state and decks, and the restrict would refuse the drop or, with enforcement off, leave the copy half made. Rails turns enforcement off for a copy only **outside a transaction**, so (c) runs with `disable_ddl_transaction!` (as `AddLocationToLots`, `AddDecks` and spec 049's third migration do), which opens the window that a statement fails part way with no rollback; every step is therefore guarded, and a run that stopped is finished by the next boot. **What proves no row is lost:** the upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and seeds a row in every table including each of those thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count is unchanged; and the copy's foreign keys and named checks are read back from the schema dump. + - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no no-transaction window, two migrations, nothing in this feature can lose a row. *Costs:* the database can't state "every account has a household", so a bug or a hand-written insert can leave an account with none and nothing but a guard spec and the upgrade proof would notice; a fresh volume (made from `schema.rb`) and an upgraded one agree only if the column is nullable in both, so the schema says nullable forever unless a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). The recommendation is (c): the copy has been done safely three times on this schema (`locations` twice, `sessions` once), the proof that guards it exists and this issue is the cheapest time to pay it. The ruling is the maintainer's; if the plan review finds the copy unsafe, the fallback is taken and this spec's version records it. + - **Reversible:** (c) down makes the column nullable again by the same copy; (b) down does nothing (the rows are removed with their tables, and a second run of its up finds nothing to do); (a) down drops the tables and the columns, and **refuses** (an `IrreversibleMigration` that says what exists) when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is simply removed with its name. +6. **A personal account is in its user's household.** For a user, `accounts.household_id` of their account equals `memberships.household_id` of their membership. The database can't compare two tables, so the rule is the model's (a user is built with both from one household; a later issue that moves a user changes both in one transaction), and a guard spec and the upgrade proof check it on real rows. A shared account has no user; a personal account has exactly one. +7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. A household left with no member is removed in the same delete; a household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). The delete order is the plan's: the membership's and the account's foreign keys decide it. +8. **A household is made by the user model, so every path gets one.** The one place an account is made from a user today (`before_validation :build_account`) also makes the household and the manager membership; the first-run sign-up, the open sign-up, the admin's add-a-user form and `bin/rails collector:user` all go through it, and a spec per path says so. Nothing in a request names a household, a role or a kind (`household_id`, `role` and `kind` are not permitted from parameters). +9. **"Nothing looks different" is checked by four things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), in the page or the head. (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. +10. **The upgrade proof gains a household check, and `expected_changes.yml` stays empty.** `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and no entry would match; the harness gains a check on the upgraded snapshot (and on the restore leg's) that every user has exactly one manager membership, every account a household, a personal account's household is its user's, every household has a manager, and the households' count equals the users' (the seed's two users make two). The seed for the next release (`script/upgrade/seeds/next.rb`) creates its users through the models, so the new tables hold rows, as the seed rule needs. +11. **The ADR is 0030, and the tenancy area doc changes with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 gets a status line saying it is superseded in part; the one-user text isn't rewritten. The doc [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) changes in the glossary, invariants, deletion and accepted limits, and the [`data-portability.md`](../../architecture/data-portability.md) and [export format](../../export-format/README.md) coverage lines change with decision 4. +12. **Hosted-door rules.** *Cost per household:* one household row, one membership row per member, no row per lot or per request. *Stored per account or user:* the household's name and members' roles are the household's, nothing else is stored about a collector's choices. *The operator is not the user:* no admin page changes, so none shows a collection; the household's fields are not on any page. + +## User Stories and Acceptance Criteria + +### Story 1: A new user starts in a household of one + +**As** an operator or a person signing up **I want** every user, however they are made, to be the manager of a household of one **so that** the later features have a household to work with from the first day. + +- [ ] **AC-1.1** Given an instance with no user When the first visitor signs up Then a user, their personal account, a household named "'s household" and a membership with role `manager` exist, the account's household is that household, and the page the visitor lands on is the collection as before. +- [ ] **AC-1.2** Given sign-up is open When a person signs up Then they have the same four records, as the manager of a household of their own (not the first user's). +- [ ] **AC-1.3** Given an admin on the users page When they add a user Then the user has the same four records, and the admin's own household is unchanged. +- [ ] **AC-1.4** Given the operator's command When it creates an admin by email Then the admin has the same four records. +- [ ] **AC-1.5** Given a user whose name is 100 characters When the user is created Then the household's name is at most 100 characters and begins with the start of the user's name; given a name of one character Then it is that character followed by "'s household". +- [ ] **AC-1.6** Given a sign-up that is invalid (a blank name, a used email) When it is submitted Then no household, membership or account exists afterwards, and a sign-up that loses the first-user race (spec 004 AC-3.4) leaves none either. +- [ ] **AC-1.7** Given any request that creates a user When its parameters hold `household_id`, `role` or `kind` Then they are ignored: the user's household is a new one, their role `manager` and their account `personal`. + +### Story 2: The rules of a household hold + +**As** a maintainer of the later households issues **I want** the structure to refuse what a household may not be **so that** a bug in a later page can't leave a household without a manager or a user in two. + +- [ ] **AC-2.1** Given a membership When its role is anything but `manager` or `member` Then the model refuses it, and an insert that bypasses the model is refused by the database. +- [ ] **AC-2.2** Given a user with a membership When a second membership is made for them, in the same or another household Then the model refuses it, and an insert that bypasses the model is refused by the database. +- [ ] **AC-2.3** Given a household with two members, one a manager When the manager's role is changed to member, or their membership is removed, or their user is deleted Then it is refused and nothing changes; given the same household with two managers When one is demoted Then it succeeds. +- [ ] **AC-2.4** Given a household of one When its only member's membership is removed through the model on its own Then it is refused (a household is never left with none by that route); deleting the user (AC-4.4) is the only way a household of one goes. +- [ ] **AC-2.5** Given an account When its kind is anything but `personal` or `shared` Then the model refuses it and the database refuses a bypassing insert; given two shared accounts in one household Then the second is refused by the model and by the database. +- [ ] **AC-2.6** Given a shared account When it has a user, or a personal account that has none Then the model refuses it. No shared account exists in any request or in the seed. +- [ ] **AC-2.7** Given a household name that is blank or over 100 characters When it is saved Then the model refuses it. +- [ ] **AC-2.8** Given every user in the test database after any example that makes users When the guard runs Then each has exactly one membership, each household has a manager, and each personal account's household is its user's household (a guard the upgrade proof repeats on real data). + +### Story 3: An existing instance gains households on upgrade + +**As** an operator **I want** every existing user to have a household after the upgrade **so that** nothing breaks and nothing is left for me to do. + +- [ ] **AC-3.1** Given a database as the previous release leaves it (users, accounts, lots, locations, decks, proxies, sessions, working state, the prices database untouched) When the migrations run Then every user has one household named "'s household", one membership with role `manager`, and an account whose household is it, every account kind is `personal`, and the rows of every old table are unchanged. +- [ ] **AC-3.2** Given the migrations have run When the backfill runs again (its migration is applied a second time, or the database is migrated from a state with some users done) Then no household or membership is added or changed and the count of each is the same. +- [ ] **AC-3.3** Given 1,201 users with no membership When the backfill runs Then it makes households in batches of at most 500, each batch one transaction, and every user has exactly one household afterwards; given it stops after the first batch When the migrations run again Then the rest are made and the first batch's are not made twice. +- [ ] **AC-3.4** Given an instance with no users (a fresh volume's first run, or an upgraded instance that never had one) When the migrations run Then they succeed and the first sign-up gets its household by AC-1.1. +- [ ] **AC-3.5** Given the three migrations When the database is rolled back through them Then it returns to the previous release's schema with every old row unchanged; given a household with two members or a shared account When (a) is rolled back Then it refuses, names what exists, and changes nothing. +- [ ] **AC-3.6** Given (c) has run on a database that holds rows in every table that references `accounts` When the counts are compared before and after Then each is equal, `PRAGMA foreign_key_check` and `integrity_check` report nothing, and the foreign keys, named checks and indexes of `accounts` and of its thirteen referencing tables are those of the schema dump. +- [ ] **AC-3.7** Given an older backup When it is restored onto the new code and the instance starts Then the migrations run as in AC-3.1 and every user has a household, and no session the restore didn't end is ended by the migration. +- [ ] **AC-3.8** Given sessions exist before the three migrations run When they have run Then every `sessions` row is unchanged, so a collector signed in before the upgrade is still signed in (spec 049's rule for upgrades; the upgrade proof's behaviour leg signs in with the old password and checks the same). + +### Story 4: Nothing looks different + +**As** a collector **I want** the app to be exactly what it was **so that** nothing I do or see has to be learned again. + +- [ ] **AC-4.1** Given a collector and an admin When each opens the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search Then no body contains "household" (case-insensitive), and each page's status and text equal what they were on `main` before this feature (the existing specs for each, unedited, pass). +- [ ] **AC-4.2** Given the application's routes When `bin/rails routes` on the branch is compared with the output on `main` before this feature Then the diff is empty (checked once at the implementation review and recorded in `verification.md`; not an RSpec example, decision 9). +- [ ] **AC-4.3** Given every existing request spec for another account's record When the suite runs Then each passes unchanged: another account's record is 404, and `Current.account` is still the signed-in user's own account. +- [ ] **AC-4.4** Given an admin deletes a user (spec 004 AC-5.4) When it completes Then the confirmation, the notice and the other accounts are as before, and the deleted user's household, membership and account are gone. +- [ ] **AC-4.5** Given the admin users page When it lists users Then each row, count and button are as before, and no query reads `households` or `memberships`. + +### Story 5: The export and the backup stay whole + +**As** a collector **I want** my export to round-trip and my instance's backup to keep working **so that** nothing about my data's safety changes. + +- [ ] **AC-5.1** Given the export format document When its Coverage tables are compared with the database Then `memberships` is a table "Left out" with its reason, `accounts.household_id` and `accounts.kind` are columns "Left out" with their reasons, the guard passes, and no new field is in the schema or the examples; the format version stays 4. +- [ ] **AC-5.2** Given a collector's export When it is applied to the same account, or to a fresh user's account on a fresh instance Then the lots, locations, decks, proxies and currency round-trip as before, and the account's household, membership and kind are unchanged by the apply. +- [ ] **AC-5.3** Given `backup:create` and `backup:restore` When they run on an instance that has households Then the restored instance has the same households and memberships (they are in the primary database, which the backup holds whole), and `spec/backup_guards_spec.rb` passes unedited. +- [ ] **AC-5.4** Given the format document When it is read Then one sentence says the household isn't in the file, why, and that the shared collection's file (#384) is where it is decided again. + +### Story 6: The upgrade proof covers the households + +**As** a maintainer **I want** the upgrade proof to fail if the backfill is wrong **so that** an upgrade that gives a user no household can't ship. + +- [ ] **AC-6.1** Given the seed of the previous release (`script/upgrade/seeds/0.3.0.rb`), upgraded by the code under test When the harness checks the upgraded snapshot Then every user has exactly one manager membership, every account a household of its user's, every household a manager, the households' count equals the users' and every account's kind is `personal`; a snapshot that breaks any of these fails the run naming it. +- [ ] **AC-6.2** Given the proof's backup leg (spec 031 AC-6.3) When the previous release's backup is restored into the new image Then the same check passes on its snapshot. +- [ ] **AC-6.3** Given the proof's comparison When it runs Then `expected_changes.yml` is empty (`changes: []`), no old table or column is reported changed, and the new tables and columns are not compared. +- [ ] **AC-6.4** Given the seed for the next release When it runs on a fresh instance of the code under test Then `households` and `memberships` each hold a row and no seed exception is added. +- [ ] **AC-6.5** Given a fixture snapshot with a user who has no membership, one with a household with no manager, and one whose account is in another household When the check runs on each Then it fails naming the rule broken (a spec of the harness, not of the application). +- [ ] **AC-6.6** Given `bin/upgrade-smoke` run against the previous release built from its tag and the code under test When it completes Then it passes with the check of AC-6.1 and AC-6.2 (evidence in `verification.md`; it needs the container engine and a slot). + +## Functional Requirements + +### FR-1: The household structure + +A household has a name; a membership puts one user in one household with a role; an account has a household and a kind (decisions 1, 2, 6). + +**Must:** keep `users.account_id` unique; keep every `account_id` rule of ADR 0017; give each personal account exactly one user; keep a shared account out of every request and every seed. +**Must not:** add a column to `users`; read or write a household in any controller, view or helper; add `Current.household`. + +### FR-2: Every user is made with their household + +The user model makes the household, the manager membership and the account together, in the transaction that saves the user (decision 8). + +**Must:** work for the four paths of Story 1; roll back with the user; take the household's name from decision 2. +**Must not:** accept a household, role or kind from a request; make a household for a user who already has one. + +### FR-3: The migrations + +Three migrations, in the order of decision 5. + +**Must:** run unattended from the previous release and from any older one; be guarded so a stopped run is finished by the next boot; run (c) without a transaction; touch no table the previous release's code reads other than `accounts`. +**Must not:** edit a released migration; reference an application model; sign anyone out; write a row per lot, per location or per request. + +### FR-4: The upgrade proof + +The harness checks the upgraded and the restored snapshots (decision 10). + +**Must:** fail naming the broken rule; leave `expected_changes.yml` empty. +**Must not:** weaken a comparison of an old table. + +### FR-5: Documentation + +The ADR, the tenancy area doc, the format document's Coverage and one sentence, `CLAUDE.md`'s Project Status line for accounts, the specs index (decision 11). + +**Must:** record the supersession in this spec, ADR 0030 and the index in both directions (`.claude/rules/documentation.md`). +**Must not:** rewrite ADR 0017's text, or restate an area's rules in `CLAUDE.md`. + +## Non-Functional Requirements + +### Performance + +- The backfill's batch is a short write transaction, so another request waits at most one batch's write; on 10,000 users it makes 20 batches and completes in seconds (measured by a `timing: true` example on the plan's larger count, not asserted as a hard bound). +- A request's queries are unchanged: no request reads a household. A new user costs two extra inserts (household, membership). + +### Security + +- `household_id`, `role` and `kind` are never permitted from parameters. Another account's record is still 404; no household rule gives one user a view of another's collection. +- The admin gate and the admin pages are unchanged; no admin page shows a household, a role or a collection. +- No migration, spec or seed runs the `sqlite3` command or a production-environment task. + +### Reliability + +- A migration that stops part way is finished by the next boot (guards, idempotent backfill); a failed (c) leaves `accounts` as it was or finished, never missing rows (AC-3.6). +- A restore of an older backup is migrated by the same migrations (AC-3.7). + +## Error Scenarios + +| Scenario | Expected Behavior | +|----------|-------------------| +| A user is created and the household or membership insert fails | The whole write rolls back; no user, account, household or membership remains; the sign-up or admin form shows the error it shows for any failed save | +| The backfill is stopped after some batches | The next boot's migration makes the households of the users that have none and no other; no user has two | +| The backfill meets a busy database | A batch's transaction waits on the busy timeout as every write does; a failure ends the migration, and the next boot resumes it | +| (c) stops between its steps | Its guards finish it on the next boot; the proof and AC-3.6 read the result | +| (a) is rolled back with a two-member household or a shared account | Refused: "N households with more than one member and M shared accounts exist; remove them before rolling back"; nothing changes | +| An admin deletes the last manager of a household with other members | Refused by the model; no page makes such a household yet (#382, #383) | +| A restored backup from before this feature | Migrated at the next start like an upgrade; households exist for every user | + +## Open Questions + +- [NEEDS CLARIFICATION: decision 2, the name of a household of one: "'s household" (recommended) or a constant "Household". The maintainer rules.] +- [NEEDS CLARIFICATION: decision 3, the "always has a manager" rule: model plus the database's checks (recommended) or a trigger as well (rejected here for the schema dump). The maintainer rules.] +- [NEEDS CLARIFICATION: decision 4, the export: not carried, version 4 stays (recommended) or the name carried in `profile` at version 5. The maintainer rules.] +- [NEEDS CLARIFICATION: decision 5, migration (c): the table copy making `accounts.household_id` `NOT NULL` (recommended) or the nullable fallback. The maintainer rules; the risks and costs of both are in the decision.] +- [NEEDS CLARIFICATION: decision 5's partial unique index on the shared account, kept or left to #384. Recommended: kept.] + +## Out of Scope (Future Considerations) + +- Joining, leaving, removing a member, handing the manager role over (#381 to #383); the working collection and the shared collection (#384); transfers and borrowing (#385 to #388); households on the admin users page (#389). +- A household's name shown or edited anywhere; the roles shown anywhere. +- A plan per household (#393). +- Rewriting ADR 0017's text. It stays as written, with its status line pointing to ADR 0030. diff --git a/docs/specs/README.md b/docs/specs/README.md index d911e5dc..58bf1093 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -103,3 +103,4 @@ A plan is deleted from the tree in a commit of its own after its feature's pull | [085 A Price Source Contract, and Scryfall's Prices Through It](085-price-source-contract/spec.md) | Frozen, by [prices](../architecture/prices.md) | [Prices](../architecture/prices.md) (the source contract, Scryfall's source, the source and series rows) | retired in `e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4`; `git show e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4^:docs/specs/085-price-source-contract/plan.md` | | | | [086 A Start Refuses a Volume That Holds a Prices Database and No Primary](086-start-refuses-orphan-prices/spec.md) | Frozen, by [release and deploy](../architecture/release-and-deploy.md) | [Release and deploy](../architecture/release-and-deploy.md) (the start check before `db:prepare`, its message, the restore's advice and the smoke leg); [prices](../architecture/prices.md) (a pointer only) | retired in `2c5939d4604cc1221534677c3e4bcc4aa5bdc916`; `git show 2c5939d4604cc1221534677c3e4bcc4aa5bdc916^:docs/specs/086-start-refuses-orphan-prices/plan.md` | In part, spec 081 decision 8's "Collector must never be started on such a volume" and AC-5.9's advice (the start now refuses it; the restore's advice says so) ([#375](https://github.com/plainprogrammer/Collector/issues/375)) | | | [087 The Licences of the Third-Party Software and Assets in the Image, Readable from the App](087-third-party-licences-in-app/spec.md) | Frozen, by [catalog core](../architecture/catalog-core.md) (the licence line's third sentence, the page, the reader, the manifest, the guard), [release and deploy](../architecture/release-and-deploy.md) (the smoke check) and [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint); the `Licences` pattern is in the [design system](../design-system/README.md) | [Catalog core](../architecture/catalog-core.md) (the footer and the third-party licences page); [release and deploy](../architecture/release-and-deploy.md) (the smoke check); [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint that skips sign-in); [design system](../design-system/README.md) (`Licences`, `Footer`) | retired in `11de80918d734d3b6e001d16b013f8a77c820ffc`; `git show 11de80918d734d3b6e001d16b013f8a77c820ffc^:docs/specs/087-third-party-licences-in-app/plan.md` | 084: AC-1.4's "no other part of the line is a link", the full wording of AC-2.1, AC-2.3, AC-2.4 and AC-2.14 and decision 2's wording table, AC-2.13's two links, FR-1's "with two links", the Accessibility NFR's and Users and Context's two links, decision 12's no-script notice (it gains the link), the Non-Goals "a licence page in the app" and "Listing third-party licences" ([#376](https://github.com/plainprogrammer/Collector/issues/376)) | | +| [090 Every User Has a Household](090-every-user-has-a-household/spec.md) | Draft (Gate A pending) | [Tenancy and accounts](../architecture/tenancy-and-accounts.md); [data portability](../architecture/data-portability.md) (coverage); [release and deploy](../architecture/release-and-deploy.md) (the upgrade proof) | present | [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md); spec 004 FR-3's "one user per account" in part (an account is now in a household) ([#380](https://github.com/plainprogrammer/Collector/issues/380)) | | From ffd5bb2527e1299e7d637f07b47d35b53a8722c5 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:14:54 -0500 Subject: [PATCH 02/41] docs(adr): 0030, a household is a group of accounts (#380) Supersedes the one-user half of ADR 0017. Proposed; decisions 5 and 6 await the maintainer's ruling. Written by the main session on Sonnet 5.5. Co-Authored-By: Claude Sonnet 5.5 --- ...-row-level-tenancy-one-user-per-account.md | 2 +- ...0030-a-household-is-a-group-of-accounts.md | 51 +++++++++++++++++++ docs/adr/README.md | 3 +- 3 files changed, 54 insertions(+), 2 deletions(-) create mode 100644 docs/adr/0030-a-household-is-a-group-of-accounts.md diff --git a/docs/adr/0017-row-level-tenancy-one-user-per-account.md b/docs/adr/0017-row-level-tenancy-one-user-per-account.md index f74f33a2..27336d15 100644 --- a/docs/adr/0017-row-level-tenancy-one-user-per-account.md +++ b/docs/adr/0017-row-level-tenancy-one-user-per-account.md @@ -2,7 +2,7 @@ ## Status -Accepted (recorded 2026-10-11, after the fact; decided in spec 004) +Accepted (recorded 2026-10-11, after the fact; decided in spec 004). The one-user half ("an account has exactly one user" as the shape of tenancy) is superseded by [ADR 0030](0030-a-household-is-a-group-of-accounts.md); the rest stands. **Date:** 2026-10-11 **Feature:** 004-design-system-collection diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md new file mode 100644 index 00000000..f8190dfc --- /dev/null +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -0,0 +1,51 @@ +# 0030: A household is a group of accounts; an account stays the unit of ownership and privacy + +## Status + +Proposed (2026-11-13), written with spec 090. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44). Its decisions 5 and 6 (the shape of the migrations and what the export carries) are the maintainer's to rule in spec 090's decisions 4 and 5, and are written here on the recommendation. + +**Date:** 2026-11-13 +**Feature:** 090-every-user-has-a-household + +## Context + +ADR 0017 fixed the tenancy rule: every collector-owned table carries `account_id`, a request resolves `Current.account`, every query goes through it, and another account's record is a 404. It also fixed that an account has exactly one user, and deferred personal and household collections side by side, to be decided with an export format bump and migrations when they came. + +They came: the maintainer moved households into 1.0 on 2026-11-13 ([design](../superpowers/specs/2026-11-13-households-and-hosted-service-design.md), rulings 43 and 44), because the family's collection is the 1.0 test and a family is several people, and because a change to who owns a row gets dearer with each table added (lots, locations, decks, proxies and prices all exist or are on the way). Each family member wants a private collection, and the household wants one it shares. The hosted lane then prices a plan per household. + +This ADR records the decision that is needed first, by every later issue of both lanes: what a household is in the schema, and what it leaves alone. The shared collection, joining, leaving and transfers are decided in their own specs (#381 to #389). + +## Options considered + +- **A. A household is a group of accounts (chosen; ruled 2026-11-13).** Each member keeps their own account, and the household may have one shared account. Everything that exists keeps going through `Current.account`; another account is a 404. A copy is in one account at a time, so a copy moves between a member's account and the household's by a transfer. +- **B. An owner on each row.** One account per household, and every place, deck and lot carries the member who owns it, or none for the household. Rejected by the design: every query gains "mine or the household's", a lot's key gains the owner in all five places that build it, the export is reshaped, and a missed condition shows a child's collection to the household. Its one gain, using a household copy in a member's deck with no transfer, is answered by borrowing (#388). +- **C. Keep one user per account and add nothing.** The 1.0 test fails: the family has no shared collection. + +**For the structure (spec 090 decisions 1 and 6):** + +- **A. A `households` table, a `memberships` table (user, household, role) and `accounts.household_id` with `accounts.kind` (chosen).** A personal account's household is found both through its user's membership and through the account; a shared account has no user, so only the account can say. The cost is one fact held twice for a personal account, kept in step by the model and checked by a guard. +- **B. `users.household_id` and a role on `users`, with accounts finding their household through their user.** One fact, not two, and no membership table. Rejected: a shared account has no user, so it would need its own column anyway, and a user's role is a fact of the membership, which later issues move and remove as a row. +- **C. `accounts.household_id` only, with a role on the account.** Rejected: a role belongs to a person, not to the collection they hold; a shared account has no person. + +## Decision + +1. **An account stays the unit of ownership and privacy.** Every collector-owned table keeps `account_id` and nothing in it changes. A household is a group of accounts: each member's own account (`kind` `personal`) and at most one shared account (`kind` `shared`, no user, one per household, enforced by a partial unique index). +2. **Every user belongs to exactly one household,** through a membership row whose `user_id` is unique. A membership has a role, `manager` or `member`, stated by a `CHECK`. A household always has a manager. There is no other role in 1.0 (no kid role: a child's collection is any member's). +3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL`. +4. **A household is made with its user, as the account is.** The user model makes the household, the manager membership and the account in one write, so first run, sign-up, an admin's form and the operator's command all give one. A household of one is named for its user and is invisible in the app. +5. **"A household always has a manager" is the model's rule, plus the database's checks.** There is no trigger, because `db/schema.rb` doesn't dump triggers and a volume made from the schema would differ from an upgraded one. The rule is checked on real rows by a guard spec and by the upgrade proof. +6. **The export doesn't carry the household, and the format stays version 4.** The export is one file per account. The household's name and members are a fact about a group; they are listed in the format document's Coverage tables as left out. The shared collection's own file, and the version raise it brings, are #384's. +7. **Three migrations carry it onto every instance:** the schema (new tables, two columns added without a table copy), a batched idempotent backfill that gives every existing user a household of one, and `NOT NULL` on `accounts.household_id` by a table copy run without a transaction. The upgrade proof checks the result on the upgraded and on the restored snapshot. The third migration's risk and its nullable fallback are weighed in spec 090 decision 5. + +What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolved once and failing closed, queries through the account, 404 for another account's record, catalog data global, jobs and broadcasts carrying the account, the admin flag giving no view into a collection, and "the operator is not the user". `Current.account` is still the signed-in user's own account; that a session may work in the household's shared account instead is #384's. + +## Consequences + +- **Good:** the isolation rule that the suite proves per resource is untouched, so a family member's private collection stays private by the rule that exists. The structure the later lanes need exists on every instance before any of them is built, and costs one household row and one membership row per member. +- **Good:** a household of one is the case every user is in on upgrade, so each later feature starts from a state that already exists on every instance, not from a migration of its own. +- **Cost:** one fact, a personal account's household, is held twice (the account's column and its user's membership). A later issue that moves a user between households changes both in one transaction; until then a guard and the upgrade proof check them. +- **Cost:** a copy is in one account at a time, so a household card in a personal deck means a transfer or a borrow (design decision 15, #385, #388). +- **Cost:** the third migration copies `accounts`, the most referenced table in the schema, with foreign keys switched off outside a transaction. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost. If the maintainer chooses the fallback, `accounts.household_id` is nullable in the database and the model alone says it is present. +- **Cost:** "member" now means two things: spec 015's user without the admin flag, and a household's non-manager. They are independent, and no page shows either yet; #389 and the pages of #381 onward choose their words with both in view. +- **Cost:** an older release can't hold a household with two members or a shared account, so rolling back the first migration refuses when either exists. +- **Follow-up:** ADR 0017's status line records that this supersedes its one-user half; its text isn't rewritten. The tenancy area doc, the format document's Coverage and `CLAUDE.md`'s status paragraph change with spec 090. diff --git a/docs/adr/README.md b/docs/adr/README.md index e78d3f7b..37d4976c 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -21,7 +21,8 @@ Add a line here with each new ADR. All are Accepted unless the line says otherwi ### Foundation - [0016](0016-sqlite-in-every-environment.md): SQLite in every environment, one file per role -- [0017](0017-row-level-tenancy-one-user-per-account.md): Row-level tenancy, one user per account +- [0017](0017-row-level-tenancy-one-user-per-account.md): Row-level tenancy, one user per account (the one-user half superseded by 0030) +- [0030](0030-a-household-is-a-group-of-accounts.md): A household is a group of accounts; an account stays the unit of ownership and privacy - [0018](0018-collectible-agnostic-catalog-core.md): A collectible-agnostic catalog core with per-type extensions - [0019](0019-lot-as-the-unit-of-ownership.md): The lot is the unit of ownership From 360ecec2aa65750c8a532bfdbaced0ebd1cf3cdf Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:22:01 -0500 Subject: [PATCH 03/41] docs(specs): spec 090 v0.2.0, rulings and review round 1 (#380) The maintainer ruled decisions 2, 3, 4 and 10. Review round 1 (Fable 5.2) fixed: delete rules, the copy's risks and the nullable fallback, all three migrations without a transaction, the harness check in two tiers, scope of shared-account rules. Written by the main session on Sonnet 5.5. Co-Authored-By: Claude Sonnet 5.5 --- ...0030-a-household-is-a-group-of-accounts.md | 8 +- .../090-every-user-has-a-household/spec.md | 86 ++++++++++--------- 2 files changed, 48 insertions(+), 46 deletions(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index f8190dfc..473976db 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -2,7 +2,7 @@ ## Status -Proposed (2026-11-13), written with spec 090. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44). Its decisions 5 and 6 (the shape of the migrations and what the export carries) are the maintainer's to rule in spec 090's decisions 4 and 5, and are written here on the recommendation. +Proposed (2026-11-13), written with spec 090; it becomes Accepted with the pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) awaits the maintainer's ruling on spec 090 decision 5. **Date:** 2026-11-13 **Feature:** 090-every-user-has-a-household @@ -29,13 +29,13 @@ This ADR records the decision that is needed first, by every later issue of both ## Decision -1. **An account stays the unit of ownership and privacy.** Every collector-owned table keeps `account_id` and nothing in it changes. A household is a group of accounts: each member's own account (`kind` `personal`) and at most one shared account (`kind` `shared`, no user, one per household, enforced by a partial unique index). +1. **An account stays the unit of ownership and privacy.** Every collector-owned table keeps `account_id` and nothing in it changes. A household is a group of accounts: each member's own account (`kind` `personal`) and, from #384, at most one shared account (`kind` `shared`, no user). The rules of the shared account (one per household, no user) are written by the issue that makes one, not here. The foreign keys: `accounts.household_id` and `memberships.household_id` are no action, so a household that holds an account or a member can't be deleted, and `memberships.user_id` cascades, so a user's delete removes their membership. 2. **Every user belongs to exactly one household,** through a membership row whose `user_id` is unique. A membership has a role, `manager` or `member`, stated by a `CHECK`. A household always has a manager. There is no other role in 1.0 (no kid role: a child's collection is any member's). 3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL`. 4. **A household is made with its user, as the account is.** The user model makes the household, the manager membership and the account in one write, so first run, sign-up, an admin's form and the operator's command all give one. A household of one is named for its user and is invisible in the app. 5. **"A household always has a manager" is the model's rule, plus the database's checks.** There is no trigger, because `db/schema.rb` doesn't dump triggers and a volume made from the schema would differ from an upgraded one. The rule is checked on real rows by a guard spec and by the upgrade proof. 6. **The export doesn't carry the household, and the format stays version 4.** The export is one file per account. The household's name and members are a fact about a group; they are listed in the format document's Coverage tables as left out. The shared collection's own file, and the version raise it brings, are #384's. -7. **Three migrations carry it onto every instance:** the schema (new tables, two columns added without a table copy), a batched idempotent backfill that gives every existing user a household of one, and `NOT NULL` on `accounts.household_id` by a table copy run without a transaction. The upgrade proof checks the result on the upgraded and on the restored snapshot. The third migration's risk and its nullable fallback are weighed in spec 090 decision 5. +7. **Three migrations carry it onto every instance, all run without a transaction:** the schema (new tables, two columns added without a table copy), a batched idempotent backfill that gives every existing user a household of one, and `NOT NULL` on `accounts.household_id` by a table copy that first gives any straggler a household. The upgrade proof checks the invariants on the upgraded, the restored and a fresh snapshot, and the backfill's own facts on the upgraded and restored ones. The third migration's risk and its nullable fallback are weighed in spec 090 decision 5, and the maintainer's ruling on it is recorded there. What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolved once and failing closed, queries through the account, 404 for another account's record, catalog data global, jobs and broadcasts carrying the account, the admin flag giving no view into a collection, and "the operator is not the user". `Current.account` is still the signed-in user's own account; that a session may work in the household's shared account instead is #384's. @@ -45,7 +45,7 @@ What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolve - **Good:** a household of one is the case every user is in on upgrade, so each later feature starts from a state that already exists on every instance, not from a migration of its own. - **Cost:** one fact, a personal account's household, is held twice (the account's column and its user's membership). A later issue that moves a user between households changes both in one transaction; until then a guard and the upgrade proof check them. - **Cost:** a copy is in one account at a time, so a household card in a personal deck means a transfer or a borrow (design decision 15, #385, #388). -- **Cost:** the third migration copies `accounts`, the most referenced table in the schema, with foreign keys switched off outside a transaction. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost. If the maintainer chooses the fallback, `accounts.household_id` is nullable in the database and the model alone says it is present. +- **Cost:** the third migration copies `accounts`, the most referenced table in the schema (eleven cascading children), with foreign keys switched off outside a transaction; inside one the copy would silently delete those children. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost; the copy also restores the table's id sequence. If the maintainer chooses the fallback, `accounts.household_id` is nullable in the database and the model alone says it is present. - **Cost:** "member" now means two things: spec 015's user without the admin flag, and a household's non-manager. They are independent, and no page shows either yet; #389 and the pages of #381 onward choose their words with both in view. - **Cost:** an older release can't hold a household with two members or a shared account, so rolling back the first migration refuses when either exists. - **Follow-up:** ADR 0017's status line records that this supersedes its one-user half; its text isn't rewritten. The tenancy area doc, the format document's Coverage and `CLAUDE.md`'s status paragraph change with spec 090. diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index 068f23dd..7dfdadff 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -1,7 +1,7 @@ # Feature 090: Every User Has a Household -**Status:** Draft (decisions 2 to 6 are pending the maintainer's ruling; each is written on the recommendation) -**Version:** 0.1.0 +**Status:** Draft (decisions 2, 3, 4 and 10 are ruled by the maintainer, 2026-11-13; decision 5's step (c) is held for the maintainer's ruling with Gate A) +**Version:** 0.2.0 **Created:** 2026-11-13 **Last Updated:** 2026-11-13 **Depends on:** [spec 004](../004-design-system-collection/spec.md) (accounts, FR-2 to FR-5; frozen, by [tenancy and accounts](../../architecture/tenancy-and-accounts.md)), [spec 017](../017-export-format/spec.md) FR-9 (the coverage guard), [spec 031](../031-upgrade-from-previous-release/spec.md) (the upgrade proof), [spec 049](../049-session-idle-expiry/spec.md) (three migrations without a transaction, the precedent for decision 5) @@ -17,6 +17,7 @@ | Version | Date | Change | |---------|------|--------| +| 0.2.0 | 2026-11-13 | The maintainer ruled decisions 2, 3, 4 and 10 as recommended. Spec review round 1 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 8 important, 11 minor) fixed: the foreign keys' delete rules are stated (decision 1, AC-2.9); decision 5 rewritten with the copy's exact risks (atomic outside a transaction, silent cascades and deferred keys inside one, `NO ACTION` not `RESTRICT` on `lots` and `users`, the id sequence, an older image on the upgraded volume), a corrected precedent claim, all three migrations without a transaction, (c)'s stragglers step (AC-3.7), the backfill's `updated_at` untouched, and the nullable fallback with its gains and costs; the harness check in two tiers (decision 10); the shared-account index and validations dropped to #384; the rule for removing a membership has one wording (decision 3, AC-2.4); the name's rule is exact (first 88 characters, decision 2); decision 9 gains a grep and a query count; the docs list is complete (decision 11) | | 0.1.0 | 2026-11-13 | First draft from issue #380, the design and the coordinator's brief. Decisions 2 to 6 are the five scope questions put to the maintainer, written on their recommendations | ## Problem Statement @@ -57,22 +58,26 @@ This feature adds the structure and nothing that uses it: a household with a nam ## Decisions 1. **The names.** *(Settles design decisions 10 and 11 and the schema half of 9; no departure from the design's names.)* Tables `households` (`id`, `name`, timestamps) and `memberships` (`id`, `household_id`, `user_id`, `role`, timestamps); roles `manager` and `member` (`memberships.role`, an enum with explicit values, a `CHECK` naming both); two columns on `accounts`: `household_id` (a foreign key to `households`, indexed) and `kind` (`personal` or `shared`, default `personal`, a named `CHECK`). `users.account_id` stays, unique, and `users` gains nothing. A user's household is read through their membership, an account's through `accounts.household_id`. **Both exist on purpose:** a shared account has no user, so it can't be found through a membership, and a membership is where the role lives; the cost is one fact held twice for a personal account. Its rule is decision 6. - **Added to the design's model, each with its reason:** `memberships.user_id` is unique (one household per user, as a fact of the database, not only the model); `memberships.household_id` is indexed (the household's members are read by it); a partial unique index on `accounts (household_id) WHERE kind = 'shared'` (the design says "one shared Account"; stating it now means #384 can't build a second by accident, and no row can break it today). -2. **A household of one is named for its user.** *(Pending the maintainer's ruling; recommended over: a constant "Household".)* The name is stored text, the user's name followed by "'s household" ("Ada's household"), the user's name cut to fit so the whole is at most 100 characters; a household's name is present and at most 100 characters. It is stored, not derived, so a user's rename later changes nothing in the household, and a list of households (#389) tells them apart. No page shows or edits it in this feature; a manager renames it when a later issue gives them the page. -3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Pending the maintainer's ruling; recommended over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys, `NOT NULL` on every column. The model holds: a household is created with its first membership, a manager, in one transaction with the user; and a household with another member can't lose its last manager by demotion, by removal of the membership, or by deleting the user. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). -4. **The export does not carry the household, and the format stays version 4.** *(Pending the maintainer's ruling; recommended over: carrying the name in `profile` and raising to version 5 now.)* The export is one file per account and holds the collection and its profile (design decision 16). A household is a fact about a group, not about the collection; a household of one has a default name, and a person who imports their file onto another instance is made a household there by the same rule that makes every user one. The shared collection's file and its `profile` are #384's (the design's open question 4), and that is the issue that raises the version. The format document's Coverage tables list the new data as "Left out" with the reason (`memberships`, `accounts.household_id`, `accounts.kind`); `households` has neither `account_id` nor `user_id`, so the guard doesn't ask for it, and the format document says in one sentence where it is. -5. **Three migrations, as spec 049 did; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Pending the maintainer's ruling; recommended over: leaving the column nullable.)* - - **(a) Schema.** Creates `households` and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). Guarded, so a run that stopped part way is finished by the next boot. - - **(b) Backfill.** A data migration with no schema change. It reads the users that have no membership in batches of 500, and for each batch, in one short transaction, inserts a household for each user (decision 2's name), the manager membership and the account's `household_id`, with plain SQL and no application model. It is idempotent: it selects only users with no membership, so a second run, or a run that was stopped after some batches, makes exactly the rest and nothing twice; it takes no lock beyond each batch's own write transaction, so the instance's other work waits at most one batch. Cost: users, not lots; ten thousand users are twenty batches. - - **(c) Tighten.** Makes `accounts.household_id` `NOT NULL`, so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **What the copy risks.** SQLite can't add `NOT NULL` to a column in place; Rails copies `accounts` (create the new table, copy the rows, drop the old, rename). `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven have `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), `lots` has the default `RESTRICT` (the reason `Account` declares `lots` before `locations`, `spec/models/account_spec.rb`), and `users` has the default too; a cascade from one of the eleven reaches their own children in turn (a location's lots are refused by the restrict, a deck's zones and proxies go). With foreign keys enforced, the copy's `DROP TABLE accounts` runs those delete rules: the cascades would delete every collector's working state and decks, and the restrict would refuse the drop or, with enforcement off, leave the copy half made. Rails turns enforcement off for a copy only **outside a transaction**, so (c) runs with `disable_ddl_transaction!` (as `AddLocationToLots`, `AddDecks` and spec 049's third migration do), which opens the window that a statement fails part way with no rollback; every step is therefore guarded, and a run that stopped is finished by the next boot. **What proves no row is lost:** the upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and seeds a row in every table including each of those thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count is unchanged; and the copy's foreign keys and named checks are read back from the schema dump. - - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no no-transaction window, two migrations, nothing in this feature can lose a row. *Costs:* the database can't state "every account has a household", so a bug or a hand-written insert can leave an account with none and nothing but a guard spec and the upgrade proof would notice; a fresh volume (made from `schema.rb`) and an upgraded one agree only if the column is nullable in both, so the schema says nullable forever unless a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). The recommendation is (c): the copy has been done safely three times on this schema (`locations` twice, `sessions` once), the proof that guards it exists and this issue is the cheapest time to pay it. The ruling is the maintainer's; if the plan review finds the copy unsafe, the fallback is taken and this spec's version records it. - - **Reversible:** (c) down makes the column nullable again by the same copy; (b) down does nothing (the rows are removed with their tables, and a second run of its up finds nothing to do); (a) down drops the tables and the columns, and **refuses** (an `IrreversibleMigration` that says what exists) when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is simply removed with its name. + **Added to the design's model, each with its reason:** `memberships.user_id` is unique (one household per user, as a fact of the database, not only the model); `memberships.household_id` is indexed (the household's members are read by it). **No** index or validation about shared accounts is added: the design's "one shared account", and that a shared account has no user, are #384's to state when it makes one (a partial unique index costs no table copy later, and a model rule that a personal account has a user can't hold at the account's own insert, which comes before its user's). + **The foreign keys' delete rules, which are schema and the dangerous part:** `accounts.household_id` → `households` is **no action** (never cascade: deleting a household must never reach an account, and through the eleven cascades a collection); `memberships.household_id` → `households` is **no action** (a household with a member can't be deleted); `memberships.user_id` → `users` is **cascade** (deleting a user removes their membership). A household that holds an account or a member can't be deleted, by the model and by the database (AC-2.9). +2. **A household of one is named for its user.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a constant "Household".)* The name is stored text, the user's name followed by the straight apostrophe (U+0027) and "s household" ("Ada's household"). That suffix is 12 characters, so the user's name is cut to its **first 88 characters** and the whole is at most 100; a household's name is present and at most 100 characters. One definition builds the name for the model and for the backfill (which does it in SQL with `substr`); a spec holds the two equal on the same names (as `Location::Name`'s parity spec does), including a 100-character name, one with an apostrophe and one with multi-byte characters. It is stored, not derived, so a user's rename later changes nothing in the household, and a list of households (#389) tells them apart. No page shows or edits it in this feature; a manager renames it when a later issue gives them the page. +3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys with decision 1's rules, `NOT NULL` on every column. The model holds: a household is created with its first membership, a manager, in one transaction with the user; a membership can't be removed on its own (the one wording of the rule: a membership goes only with its user's delete in this feature, and the issues that let a person leave or be removed, #382 and #383, change that rule where they need to); a manager can't be demoted when no other manager remains; and a user can't be deleted when they are the last manager of a household that has other members. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). +4. **The export does not carry the household, and the format stays version 4.** *(Ruled by the maintainer 2026-11-13, as recommended; over: carrying the name in `profile` and raising to version 5 now.)* The export is one file per account and holds the collection and its profile (design decision 16). A household is a fact about a group, not about the collection; a household of one has a default name, and a person who imports their file onto another instance is made a household there by the same rule that makes every user one. The shared collection's file and its `profile` are #384's (the design's open question 4), and that is the issue that raises the version. The format document's Coverage tables list the new data as "Left out" with the reason (`memberships`, `accounts.household_id`, `accounts.kind`); `households` has neither `account_id` nor `user_id`, so the guard doesn't ask for it, and the format document says in one sentence that the guard does not cover it and where the question is decided again (so #384 and #393 don't read the guard as covering it). The guard's own spec (`spec/export_format_document_spec.rb`) hard-codes the "Left out" tables and columns and gains `memberships`, `accounts.household_id` and `accounts.kind`; the format document's sentence that households "will raise the version (ADR 0017)" points to ADR 0030 and #384. +5. **Three migrations, as spec 049 did, all run without a transaction; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Held: the maintainer rules on (c) with Gate A, choosing between the copy and the nullable fallback below. (a) and (b) are proposed and not in question. Recommended over: leaving the column nullable.)* All three use `disable_ddl_transaction!` and guard each step, so a run that stopped part way is finished by the next boot: (b) must, to make each batch its own short transaction (spec 049's backfill does the same), and (a) and (c) must because their `down` and (c)'s `up` copy `accounts` (below). + - **(a) Schema.** Creates `households` and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). + - **(b) Backfill.** A data migration with no schema change. It reads the users that have no membership in batches of 500, and for each batch, in one short transaction, inserts a household for each user (decision 2's name), the manager membership and the account's `household_id`, with plain SQL and no application model. **The `UPDATE` of `accounts` sets `household_id` and nothing else, `updated_at` included**, because the upgrade proof compares `accounts`' rows over the columns both databases have and `expected_changes.yml` stays empty (decision 10). It is idempotent: it selects only users with no membership, so a second run, or a run that was stopped after some batches, makes exactly the rest and nothing twice; it holds no lock beyond each batch's own write transaction, so the instance's other work waits at most one batch. Cost: users, not lots; ten thousand users are twenty batches. + - **(c) Tighten.** First runs the same idempotent backfill for stragglers, then makes `accounts.household_id` `NOT NULL`, so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **The stragglers' step matters:** (b) is recorded in `schema_migrations` once it has run and never runs again, but a user can be made between (b) and (c) (an old container still serving during a Kamal deploy, or a boot that stopped between the two), and without the step (c) would fail on every boot after. Spec 049's third migration runs the same kind of safety-net `UPDATE` before its `change_column_null`. + **What the copy is.** SQLite can't add `NOT NULL` to a column in place. Rails (activerecord 8.1.4.1, the SQLite adapter's `alter_table`) creates a new `accounts` with the change, copies the rows, drops the old table and renames the new one, the two moves in one transaction inside `disable_referential_integrity`, so the copy itself is atomic when no outer transaction holds it. `accounts` holds one row per user, so the copy is small and short; the risk is its delete rules, not its volume. + **What the copy risks.** `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven are `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), and `lots` and `users` hold SQLite's default, `NO ACTION` (`db/schema.rb`; the `RESTRICT` that makes `Account` declare `lots` before `locations`, `spec/models/account_spec.rb`, is `lots → locations`). The copy's `DROP TABLE accounts` runs an implicit delete of every row, so with foreign keys enforced: the eleven cascades **delete every collector's decks, places, proxies and working state, silently**, and `lots` and `users` are deferred to the end of the transaction rather than refused. Rails turns enforcement off for a copy only **outside a transaction** (`PRAGMA foreign_keys = OFF` does nothing inside one, and Rails then defers the keys), so a copy that ran inside Rails' own migration transaction would lose that data with no error. That is why (c) runs without one, and the cost of that is the window the guards cover: a statement of (c) outside the copy can fail with no rollback of the ones before it. **Two more effects the proof must catch:** the copy drops and recreates the table, so the `sqlite_sequence` row for `accounts` restarts at the highest id present, and an id of a deleted account can be reused (`compare.rb` skips `sqlite_%` tables, so the proof can't see it: (c) reads the sequence before the copy and restores it after, and AC-3.6 asserts it); and an older release's image started on the upgraded volume can't create a user or account any more, since it doesn't know the column (the same of every added `NOT NULL` column; a downgrade is a restore from the backup taken before the upgrade, as the README says). The precedent is narrower than a pattern: the one shipped copy of a referenced table by an `up` is `sessions` (spec 049), with two cascading children against thirteen here; `AddDecks` and its kin add columns without a copy. + **What proves no row is lost.** The upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and its seed holds a row in every table, including each of the thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count, and the `accounts` sequence, is unchanged; and the copy's foreign keys, named checks and indexes are read back against the schema dump. The same holds for (c)'s and (a)'s `down`, which also copy `accounts` (AC-3.5). + - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no no-transaction window and no sequence to restore; two migrations, and nothing in this feature can lose a row; an older image on an upgraded volume still creates users (with an account that has no household). *Costs:* the database can't state "every account has a household", so a bug, an older image's insert or a hand-written one can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If a later review finds (c) unsafe, the fallback is taken and this spec's version records it. + - **Reversible:** (c)'s `down` makes the column nullable again by the same copy; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the columns by a copy of `accounts`) and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. 6. **A personal account is in its user's household.** For a user, `accounts.household_id` of their account equals `memberships.household_id` of their membership. The database can't compare two tables, so the rule is the model's (a user is built with both from one household; a later issue that moves a user changes both in one transaction), and a guard spec and the upgrade proof check it on real rows. A shared account has no user; a personal account has exactly one. -7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. A household left with no member is removed in the same delete; a household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). The delete order is the plan's: the membership's and the account's foreign keys decide it. +7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. The order is the foreign keys': the user's row (its membership goes with it by decision 1's cascade, **without** the membership's own refusal of decision 3, which guards a membership removed on its own; the user's delete has its own check for the last manager), then the account (`User` declares `belongs_to :account, dependent: :destroy`), then the household when no member and no account is left. A household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). 8. **A household is made by the user model, so every path gets one.** The one place an account is made from a user today (`before_validation :build_account`) also makes the household and the manager membership; the first-run sign-up, the open sign-up, the admin's add-a-user form and `bin/rails collector:user` all go through it, and a spec per path says so. Nothing in a request names a household, a role or a kind (`household_id`, `role` and `kind` are not permitted from parameters). -9. **"Nothing looks different" is checked by four things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), in the page or the head. (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. -10. **The upgrade proof gains a household check, and `expected_changes.yml` stays empty.** `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and no entry would match; the harness gains a check on the upgraded snapshot (and on the restore leg's) that every user has exactly one manager membership, every account a household, a personal account's household is its user's, every household has a manager, and the households' count equals the users' (the seed's two users make two). The seed for the next release (`script/upgrade/seeds/next.rb`) creates its users through the models, so the new tables hold rows, as the seed rule needs. -11. **The ADR is 0030, and the tenancy area doc changes with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 gets a status line saying it is superseded in part; the one-user text isn't rewritten. The doc [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) changes in the glossary, invariants, deletion and accepted limits, and the [`data-portability.md`](../../architecture/data-portability.md) and [export format](../../export-format/README.md) coverage lines change with decision 4. +9. **"Nothing looks different" is checked by five things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), and counts the queries of those requests that name `households` or `memberships` (none). (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) A one-off search of `app/views`, `app/helpers`, `app/controllers`, `app/javascript` and `config/locales` for "household" and "membership" finds nothing outside the models, recorded beside the routes diff. (v) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. +10. **The upgrade proof gains a household check in two tiers, and `expected_changes.yml` stays empty.** *(Ruled by the maintainer 2026-11-13, as recommended.)* `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and no entry would match; the harness gains a check on the snapshots it takes. **Tier 1, always** (it must stay true when #381 and #384 add joined households and shared accounts, so it names no count): every user has exactly one membership, every household has at least one manager, every account has a household, and a personal account's household is its user's. **Tier 2, only when the old snapshot has no `households` table** (the backfill's own facts): the households' count equals the users' (the seed's two users make two), every membership is `manager`, every account is `personal`, and every household's name is its user's name's form of decision 2. Tier 1 runs on the upgraded snapshot, on the restore leg's, and on the `next` snapshot of a fresh instance of the code under test (`next_release`), the one place the production image's creation path is proved. The seed for the next release (`script/upgrade/seeds/next.rb`) creates its users through the models, so the new tables hold rows, as the seed rule needs. +11. **The ADR is 0030, accepted with the pull request, and the area docs change with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 has a status line saying it is superseded in part; its one-user text isn't rewritten; ADR 0030 is Accepted in the same pull request (its status line already cites it). The docs that change: [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) (glossary, invariants, the delete, accepted limits, and a line that `households` and `memberships` are instance-level tables never scoped through `Current.account`), [`.claude/rules/multi-tenancy.md`](../../../.claude/rules/multi-tenancy.md) (the same line), [`data-portability.md`](../../architecture/data-portability.md) and the [export format](../../export-format/README.md) (coverage, decision 4, and the sentence at its version history), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) (the harness's check), the [specs index](../README.md) (this spec's row and spec 004's, both directions), and `CLAUDE.md`'s Project Status paragraph. 12. **Hosted-door rules.** *Cost per household:* one household row, one membership row per member, no row per lot or per request. *Stored per account or user:* the household's name and members' roles are the household's, nothing else is stored about a collector's choices. *The operator is not the user:* no admin page changes, so none shows a collection; the household's fields are not on any page. ## User Stories and Acceptance Criteria @@ -85,7 +90,7 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-1.2** Given sign-up is open When a person signs up Then they have the same four records, as the manager of a household of their own (not the first user's). - [ ] **AC-1.3** Given an admin on the users page When they add a user Then the user has the same four records, and the admin's own household is unchanged. - [ ] **AC-1.4** Given the operator's command When it creates an admin by email Then the admin has the same four records. -- [ ] **AC-1.5** Given a user whose name is 100 characters When the user is created Then the household's name is at most 100 characters and begins with the start of the user's name; given a name of one character Then it is that character followed by "'s household". +- [ ] **AC-1.5** Given a user whose name is 100 characters When the user is created Then the household's name is exactly the user's first 88 characters followed by "'s household" (100 characters); given a name of one character Then it is that character followed by "'s household"; and the backfill's SQL builds the same name as the model for a 100-character name, a name with an apostrophe and a name of multi-byte characters. - [ ] **AC-1.6** Given a sign-up that is invalid (a blank name, a used email) When it is submitted Then no household, membership or account exists afterwards, and a sign-up that loses the first-user race (spec 004 AC-3.4) leaves none either. - [ ] **AC-1.7** Given any request that creates a user When its parameters hold `household_id`, `role` or `kind` Then they are ignored: the user's household is a new one, their role `manager` and their account `personal`. @@ -95,24 +100,25 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-2.1** Given a membership When its role is anything but `manager` or `member` Then the model refuses it, and an insert that bypasses the model is refused by the database. - [ ] **AC-2.2** Given a user with a membership When a second membership is made for them, in the same or another household Then the model refuses it, and an insert that bypasses the model is refused by the database. -- [ ] **AC-2.3** Given a household with two members, one a manager When the manager's role is changed to member, or their membership is removed, or their user is deleted Then it is refused and nothing changes; given the same household with two managers When one is demoted Then it succeeds. -- [ ] **AC-2.4** Given a household of one When its only member's membership is removed through the model on its own Then it is refused (a household is never left with none by that route); deleting the user (AC-4.4) is the only way a household of one goes. -- [ ] **AC-2.5** Given an account When its kind is anything but `personal` or `shared` Then the model refuses it and the database refuses a bypassing insert; given two shared accounts in one household Then the second is refused by the model and by the database. -- [ ] **AC-2.6** Given a shared account When it has a user, or a personal account that has none Then the model refuses it. No shared account exists in any request or in the seed. +- [ ] **AC-2.3** Given a household with two members, one a manager When the manager's role is changed to member, or their user is deleted Then it is refused and nothing changes; given the same household with two managers When one is demoted Then it succeeds. +- [ ] **AC-2.4** Given any membership, a household of one's included When it is removed on its own, through the model, Then it is refused; the only way a membership goes in this feature is with its user's delete (AC-4.4), which for a household of one removes the household too. +- [ ] **AC-2.5** Given an account When its kind is anything but `personal` or `shared` Then the model refuses it and the database refuses a bypassing insert. No shared account exists in any request or in the seed, and no index or validation about one is added (decision 1). +- [ ] **AC-2.6** Given a user whose account is in a different household from their membership's When the user is validated Then it is refused (decision 6); given a user built through the model Then the two households are the same one. - [ ] **AC-2.7** Given a household name that is blank or over 100 characters When it is saved Then the model refuses it. -- [ ] **AC-2.8** Given every user in the test database after any example that makes users When the guard runs Then each has exactly one membership, each household has a manager, and each personal account's household is its user's household (a guard the upgrade proof repeats on real data). +- [ ] **AC-2.8** Given a spec helper that checks the invariants on the test database (each user has exactly one membership, each household has a manager, each account has a household, each personal account's household is its user's) When it is called after the examples that make users by each of the four paths of Story 1, delete a user and apply an import Then it passes each time (the same invariants as the upgrade proof's tier 1, decision 10). +- [ ] **AC-2.9** Given a household that holds an account or a member When it is destroyed through the model, or its row is deleted by a statement that bypasses the model Then it is refused and no account, membership or collection row changes (decision 1's no-action keys). ### Story 3: An existing instance gains households on upgrade **As** an operator **I want** every existing user to have a household after the upgrade **so that** nothing breaks and nothing is left for me to do. -- [ ] **AC-3.1** Given a database as the previous release leaves it (users, accounts, lots, locations, decks, proxies, sessions, working state, the prices database untouched) When the migrations run Then every user has one household named "'s household", one membership with role `manager`, and an account whose household is it, every account kind is `personal`, and the rows of every old table are unchanged. +- [ ] **AC-3.1** Given a database as the previous release leaves it (users, accounts, lots, locations, decks, proxies, sessions, working state, the prices database untouched) When the migrations run Then every user has one household named "'s household", one membership with role `manager`, and an account whose household is it, every account kind is `personal`, and the rows of every old table are unchanged, `accounts.updated_at` included (the backfill sets `household_id` only). - [ ] **AC-3.2** Given the migrations have run When the backfill runs again (its migration is applied a second time, or the database is migrated from a state with some users done) Then no household or membership is added or changed and the count of each is the same. - [ ] **AC-3.3** Given 1,201 users with no membership When the backfill runs Then it makes households in batches of at most 500, each batch one transaction, and every user has exactly one household afterwards; given it stops after the first batch When the migrations run again Then the rest are made and the first batch's are not made twice. -- [ ] **AC-3.4** Given an instance with no users (a fresh volume's first run, or an upgraded instance that never had one) When the migrations run Then they succeed and the first sign-up gets its household by AC-1.1. -- [ ] **AC-3.5** Given the three migrations When the database is rolled back through them Then it returns to the previous release's schema with every old row unchanged; given a household with two members or a shared account When (a) is rolled back Then it refuses, names what exists, and changes nothing. -- [ ] **AC-3.6** Given (c) has run on a database that holds rows in every table that references `accounts` When the counts are compared before and after Then each is equal, `PRAGMA foreign_key_check` and `integrity_check` report nothing, and the foreign keys, named checks and indexes of `accounts` and of its thirteen referencing tables are those of the schema dump. -- [ ] **AC-3.7** Given an older backup When it is restored onto the new code and the instance starts Then the migrations run as in AC-3.1 and every user has a household, and no session the restore didn't end is ended by the migration. +- [ ] **AC-3.4** Given an instance with no users When the migrations run on an empty database Then they succeed and the first sign-up gets its household by AC-1.1; and a database made from `db/schema.rb` (a fresh volume) and one migrated while empty have the same `accounts`, `households` and `memberships` columns, keys, checks and indexes. +- [ ] **AC-3.5** Given the three migrations have run on a database that holds rows in every table that references `accounts` When they are rolled back Then the database has the schema before (a) and the count of every one of those tables and every old row are unchanged; given a household with two members or a shared account When (a) is rolled back Then it refuses, names what exists, and changes nothing. +- [ ] **AC-3.6** Given (c) has run on a database that holds rows in every table that references `accounts` When the counts are compared before and after Then each is equal, the `sqlite_sequence` row of `accounts` is unchanged (a new account after (c) gets an id above the highest ever issued, not a reused one), `PRAGMA foreign_key_check` and `integrity_check` report nothing, and the foreign keys, named checks and indexes of `accounts` and of its thirteen referencing tables are those of the schema dump. +- [ ] **AC-3.7** Given a user made after (b) has run and before (c) When (c) runs Then the user is given a household by (c)'s stragglers step and (c) succeeds; given (c) is run again Then it changes nothing. - [ ] **AC-3.8** Given sessions exist before the three migrations run When they have run Then every `sessions` row is unchanged, so a collector signed in before the upgrade is still signed in (spec 049's rule for upgrades; the upgrade proof's behaviour leg signs in with the old password and checks the same). ### Story 4: Nothing looks different @@ -123,13 +129,13 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-4.2** Given the application's routes When `bin/rails routes` on the branch is compared with the output on `main` before this feature Then the diff is empty (checked once at the implementation review and recorded in `verification.md`; not an RSpec example, decision 9). - [ ] **AC-4.3** Given every existing request spec for another account's record When the suite runs Then each passes unchanged: another account's record is 404, and `Current.account` is still the signed-in user's own account. - [ ] **AC-4.4** Given an admin deletes a user (spec 004 AC-5.4) When it completes Then the confirmation, the notice and the other accounts are as before, and the deleted user's household, membership and account are gone. -- [ ] **AC-4.5** Given the admin users page When it lists users Then each row, count and button are as before, and no query reads `households` or `memberships`. +- [ ] **AC-4.5** Given the pages of decision 9 (i) When each is requested Then no query of the request names `households` or `memberships`, and the admin users page's rows, counts and buttons are as before. ### Story 5: The export and the backup stay whole **As** a collector **I want** my export to round-trip and my instance's backup to keep working **so that** nothing about my data's safety changes. -- [ ] **AC-5.1** Given the export format document When its Coverage tables are compared with the database Then `memberships` is a table "Left out" with its reason, `accounts.household_id` and `accounts.kind` are columns "Left out" with their reasons, the guard passes, and no new field is in the schema or the examples; the format version stays 4. +- [ ] **AC-5.1** Given the export format document When its Coverage tables are compared with the database Then `memberships` is a table "Left out" with its reason, `accounts.household_id` and `accounts.kind` are columns "Left out" with their reasons, `households` is not a row (it has neither `account_id` nor `user_id`), the guard's own spec lists the three additions, the guard passes, and no new field is in the schema or the examples; the format version stays 4. - [ ] **AC-5.2** Given a collector's export When it is applied to the same account, or to a fresh user's account on a fresh instance Then the lots, locations, decks, proxies and currency round-trip as before, and the account's household, membership and kind are unchanged by the apply. - [ ] **AC-5.3** Given `backup:create` and `backup:restore` When they run on an instance that has households Then the restored instance has the same households and memberships (they are in the primary database, which the backup holds whole), and `spec/backup_guards_spec.rb` passes unedited. - [ ] **AC-5.4** Given the format document When it is read Then one sentence says the household isn't in the file, why, and that the shared collection's file (#384) is where it is decided again. @@ -138,11 +144,11 @@ This feature adds the structure and nothing that uses it: a household with a nam **As** a maintainer **I want** the upgrade proof to fail if the backfill is wrong **so that** an upgrade that gives a user no household can't ship. -- [ ] **AC-6.1** Given the seed of the previous release (`script/upgrade/seeds/0.3.0.rb`), upgraded by the code under test When the harness checks the upgraded snapshot Then every user has exactly one manager membership, every account a household of its user's, every household a manager, the households' count equals the users' and every account's kind is `personal`; a snapshot that breaks any of these fails the run naming it. -- [ ] **AC-6.2** Given the proof's backup leg (spec 031 AC-6.3) When the previous release's backup is restored into the new image Then the same check passes on its snapshot. +- [ ] **AC-6.1** Given the seed of the previous release (`script/upgrade/seeds/0.3.0.rb`, whose snapshot has no `households` table), upgraded by the code under test When the harness checks the upgraded snapshot Then tier 1 holds (every user has exactly one membership, every household a manager, every account a household, a personal account's household its user's) and tier 2 holds (the households' count equals the users', each membership is `manager`, each account `personal`, each name decision 2's form); a snapshot that breaks any of these fails the run naming it. +- [ ] **AC-6.2** Given the proof's backup leg (spec 031 AC-6.3) When the previous release's backup is restored into the new image Then tiers 1 and 2 pass on its snapshot; given the `next` snapshot of a fresh instance of the code under test Then tier 1 passes on it (tier 2 is not run: its old snapshot has `households`). - [ ] **AC-6.3** Given the proof's comparison When it runs Then `expected_changes.yml` is empty (`changes: []`), no old table or column is reported changed, and the new tables and columns are not compared. - [ ] **AC-6.4** Given the seed for the next release When it runs on a fresh instance of the code under test Then `households` and `memberships` each hold a row and no seed exception is added. -- [ ] **AC-6.5** Given a fixture snapshot with a user who has no membership, one with a household with no manager, and one whose account is in another household When the check runs on each Then it fails naming the rule broken (a spec of the harness, not of the application). +- [ ] **AC-6.5** Given a fixture snapshot with a user who has no membership, one with a household with no manager, one whose account is in another household and one (tier 2) whose household name differs from decision 2's When the check runs on each Then it fails naming the rule broken; and a snapshot with a joined household or a shared account passes tier 1 (a spec of the harness, not of the application). - [ ] **AC-6.6** Given `bin/upgrade-smoke` run against the previous release built from its tag and the code under test When it completes Then it passes with the check of AC-6.1 and AC-6.2 (evidence in `verification.md`; it needs the container engine and a slot). ## Functional Requirements @@ -165,19 +171,19 @@ The user model makes the household, the manager membership and the account toget Three migrations, in the order of decision 5. -**Must:** run unattended from the previous release and from any older one; be guarded so a stopped run is finished by the next boot; run (c) without a transaction; touch no table the previous release's code reads other than `accounts`. +**Must:** run unattended from the previous release and from any older one; be guarded so a stopped run is finished by the next boot; run all three without a transaction (decision 5); give a user made between (b) and (c) a household before (c) tightens; keep `accounts`' id sequence across the copy; touch no table the previous release's code reads other than `accounts`. **Must not:** edit a released migration; reference an application model; sign anyone out; write a row per lot, per location or per request. ### FR-4: The upgrade proof The harness checks the upgraded and the restored snapshots (decision 10). -**Must:** fail naming the broken rule; leave `expected_changes.yml` empty. +**Must:** fail naming the broken rule; check tier 1 always and tier 2 only on a backfill (decision 10); leave `expected_changes.yml` empty. **Must not:** weaken a comparison of an old table. ### FR-5: Documentation -The ADR, the tenancy area doc, the format document's Coverage and one sentence, `CLAUDE.md`'s Project Status line for accounts, the specs index (decision 11). +The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/multi-tenancy.md`, the format document's Coverage and sentences, the release-and-deploy area doc's upgrade-proof paragraph, `CLAUDE.md`'s Project Status line for accounts, the specs index in both directions (decision 11). **Must:** record the supersession in this spec, ADR 0030 and the index in both directions (`.claude/rules/documentation.md`). **Must not:** rewrite ADR 0017's text, or restate an area's rules in `CLAUDE.md`. @@ -197,8 +203,8 @@ The ADR, the tenancy area doc, the format document's Coverage and one sentence, ### Reliability -- A migration that stops part way is finished by the next boot (guards, idempotent backfill); a failed (c) leaves `accounts` as it was or finished, never missing rows (AC-3.6). -- A restore of an older backup is migrated by the same migrations (AC-3.7). +- A migration that stops part way is finished by the next boot (guards, idempotent backfill); a failed (c) leaves `accounts` as it was or finished, never missing rows (AC-3.6, AC-3.7). +- A restore of an older backup is migrated by the same migrations (AC-6.2's restore leg). ## Error Scenarios @@ -214,11 +220,7 @@ The ADR, the tenancy area doc, the format document's Coverage and one sentence, ## Open Questions -- [NEEDS CLARIFICATION: decision 2, the name of a household of one: "'s household" (recommended) or a constant "Household". The maintainer rules.] -- [NEEDS CLARIFICATION: decision 3, the "always has a manager" rule: model plus the database's checks (recommended) or a trigger as well (rejected here for the schema dump). The maintainer rules.] -- [NEEDS CLARIFICATION: decision 4, the export: not carried, version 4 stays (recommended) or the name carried in `profile` at version 5. The maintainer rules.] -- [NEEDS CLARIFICATION: decision 5, migration (c): the table copy making `accounts.household_id` `NOT NULL` (recommended) or the nullable fallback. The maintainer rules; the risks and costs of both are in the decision.] -- [NEEDS CLARIFICATION: decision 5's partial unique index on the shared account, kept or left to #384. Recommended: kept.] +- [NEEDS CLARIFICATION: decision 5, migration (c): the table copy making `accounts.household_id` `NOT NULL` (recommended) or the nullable fallback. The maintainer rules; the risks and costs of both are in the decision. Decisions 2, 3, 4 and 10 are ruled (2026-11-13, as recommended); (a) and (b) are not in question.] ## Out of Scope (Future Considerations) From dde41669ba677b297a6052c1252db60893790c86 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:27:26 -0500 Subject: [PATCH 04/41] docs(specs): spec 090 v0.3.0, review round 2 (#380) Review round 2 (Fable 5.2) fixed: migration (c) is one transaction around its stragglers step, the copy and the sequence restore; the fallback comparison now states its stragglers hole and four missing facts; the README claim is corrected; the suite's bare accounts are named allowed edits. Written by the main session on Sonnet 5.5. Co-Authored-By: Claude Sonnet 5.5 --- ...0030-a-household-is-a-group-of-accounts.md | 2 +- .../090-every-user-has-a-household/spec.md | 40 ++++++++++--------- 2 files changed, 22 insertions(+), 20 deletions(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index 473976db..4092c4f6 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -31,7 +31,7 @@ This ADR records the decision that is needed first, by every later issue of both 1. **An account stays the unit of ownership and privacy.** Every collector-owned table keeps `account_id` and nothing in it changes. A household is a group of accounts: each member's own account (`kind` `personal`) and, from #384, at most one shared account (`kind` `shared`, no user). The rules of the shared account (one per household, no user) are written by the issue that makes one, not here. The foreign keys: `accounts.household_id` and `memberships.household_id` are no action, so a household that holds an account or a member can't be deleted, and `memberships.user_id` cascades, so a user's delete removes their membership. 2. **Every user belongs to exactly one household,** through a membership row whose `user_id` is unique. A membership has a role, `manager` or `member`, stated by a `CHECK`. A household always has a manager. There is no other role in 1.0 (no kid role: a child's collection is any member's). -3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL`. +3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL` (subject to the maintainer's ruling on spec 090 decision 5, which weighs a nullable fallback). 4. **A household is made with its user, as the account is.** The user model makes the household, the manager membership and the account in one write, so first run, sign-up, an admin's form and the operator's command all give one. A household of one is named for its user and is invisible in the app. 5. **"A household always has a manager" is the model's rule, plus the database's checks.** There is no trigger, because `db/schema.rb` doesn't dump triggers and a volume made from the schema would differ from an upgraded one. The rule is checked on real rows by a guard spec and by the upgrade proof. 6. **The export doesn't carry the household, and the format stays version 4.** The export is one file per account. The household's name and members are a fact about a group; they are listed in the format document's Coverage tables as left out. The shared collection's own file, and the version raise it brings, are #384's. diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index 7dfdadff..fc37df29 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -1,7 +1,7 @@ # Feature 090: Every User Has a Household **Status:** Draft (decisions 2, 3, 4 and 10 are ruled by the maintainer, 2026-11-13; decision 5's step (c) is held for the maintainer's ruling with Gate A) -**Version:** 0.2.0 +**Version:** 0.3.0 **Created:** 2026-11-13 **Last Updated:** 2026-11-13 **Depends on:** [spec 004](../004-design-system-collection/spec.md) (accounts, FR-2 to FR-5; frozen, by [tenancy and accounts](../../architecture/tenancy-and-accounts.md)), [spec 017](../017-export-format/spec.md) FR-9 (the coverage guard), [spec 031](../031-upgrade-from-previous-release/spec.md) (the upgrade proof), [spec 049](../049-session-idle-expiry/spec.md) (three migrations without a transaction, the precedent for decision 5) @@ -17,6 +17,7 @@ | Version | Date | Change | |---------|------|--------| +| 0.3.0 | 2026-11-13 | Spec review round 2 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 5 important, 8 minor) fixed: (c) is one transaction around its stragglers step, the copy and the sequence restore (upsert), so a failed (c) changes nothing; the README claim is corrected (no downgrade is documented) and `docs/releasing.md`'s Upgrade notes join the docs; the fallback's stragglers hole is stated and what it would need; four facts added to the comparison (a failed (c) stops the boot, the deploy window, fresh volumes and backups, three copies on a full rollback); the suite's bare accounts are named allowed edits; the name's length `CHECK`; an empty household left by an older container is removed by (c); AC-3.4 by `PRAGMA`, AC-3.5's sequence, AC-3.6's `:committed` example and its fault case; #381 and #384 revisit decision 7 | | 0.2.0 | 2026-11-13 | The maintainer ruled decisions 2, 3, 4 and 10 as recommended. Spec review round 1 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 8 important, 11 minor) fixed: the foreign keys' delete rules are stated (decision 1, AC-2.9); decision 5 rewritten with the copy's exact risks (atomic outside a transaction, silent cascades and deferred keys inside one, `NO ACTION` not `RESTRICT` on `lots` and `users`, the id sequence, an older image on the upgraded volume), a corrected precedent claim, all three migrations without a transaction, (c)'s stragglers step (AC-3.7), the backfill's `updated_at` untouched, and the nullable fallback with its gains and costs; the harness check in two tiers (decision 10); the shared-account index and validations dropped to #384; the rule for removing a membership has one wording (decision 3, AC-2.4); the name's rule is exact (first 88 characters, decision 2); decision 9 gains a grep and a query count; the docs list is complete (decision 11) | | 0.1.0 | 2026-11-13 | First draft from issue #380, the design and the coordinator's brief. Decisions 2 to 6 are the five scope questions put to the maintainer, written on their recommendations | @@ -61,23 +62,23 @@ This feature adds the structure and nothing that uses it: a household with a nam **Added to the design's model, each with its reason:** `memberships.user_id` is unique (one household per user, as a fact of the database, not only the model); `memberships.household_id` is indexed (the household's members are read by it). **No** index or validation about shared accounts is added: the design's "one shared account", and that a shared account has no user, are #384's to state when it makes one (a partial unique index costs no table copy later, and a model rule that a personal account has a user can't hold at the account's own insert, which comes before its user's). **The foreign keys' delete rules, which are schema and the dangerous part:** `accounts.household_id` → `households` is **no action** (never cascade: deleting a household must never reach an account, and through the eleven cascades a collection); `memberships.household_id` → `households` is **no action** (a household with a member can't be deleted); `memberships.user_id` → `users` is **cascade** (deleting a user removes their membership). A household that holds an account or a member can't be deleted, by the model and by the database (AC-2.9). 2. **A household of one is named for its user.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a constant "Household".)* The name is stored text, the user's name followed by the straight apostrophe (U+0027) and "s household" ("Ada's household"). That suffix is 12 characters, so the user's name is cut to its **first 88 characters** and the whole is at most 100; a household's name is present and at most 100 characters. One definition builds the name for the model and for the backfill (which does it in SQL with `substr`); a spec holds the two equal on the same names (as `Location::Name`'s parity spec does), including a 100-character name, one with an apostrophe and one with multi-byte characters. It is stored, not derived, so a user's rename later changes nothing in the household, and a list of households (#389) tells them apart. No page shows or edits it in this feature; a manager renames it when a later issue gives them the page. -3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys with decision 1's rules, `NOT NULL` on every column. The model holds: a household is created with its first membership, a manager, in one transaction with the user; a membership can't be removed on its own (the one wording of the rule: a membership goes only with its user's delete in this feature, and the issues that let a person leave or be removed, #382 and #383, change that rule where they need to); a manager can't be demoted when no other manager remains; and a user can't be deleted when they are the last manager of a household that has other members. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). +3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys with decision 1's rules, `NOT NULL` on every column of `households` and `memberships`, and the name's length `CHECK` (decision 5 (a)). The model holds: a household is created with its first membership, a manager, in one transaction with the user; a membership can't be removed on its own (the one wording of the rule: a membership goes only with its user's delete in this feature, and the issues that let a person leave or be removed, #382 and #383, change that rule where they need to); a manager can't be demoted when no other manager remains; and a user can't be deleted when they are the last manager of a household that has other members. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). 4. **The export does not carry the household, and the format stays version 4.** *(Ruled by the maintainer 2026-11-13, as recommended; over: carrying the name in `profile` and raising to version 5 now.)* The export is one file per account and holds the collection and its profile (design decision 16). A household is a fact about a group, not about the collection; a household of one has a default name, and a person who imports their file onto another instance is made a household there by the same rule that makes every user one. The shared collection's file and its `profile` are #384's (the design's open question 4), and that is the issue that raises the version. The format document's Coverage tables list the new data as "Left out" with the reason (`memberships`, `accounts.household_id`, `accounts.kind`); `households` has neither `account_id` nor `user_id`, so the guard doesn't ask for it, and the format document says in one sentence that the guard does not cover it and where the question is decided again (so #384 and #393 don't read the guard as covering it). The guard's own spec (`spec/export_format_document_spec.rb`) hard-codes the "Left out" tables and columns and gains `memberships`, `accounts.household_id` and `accounts.kind`; the format document's sentence that households "will raise the version (ADR 0017)" points to ADR 0030 and #384. -5. **Three migrations, as spec 049 did, all run without a transaction; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Held: the maintainer rules on (c) with Gate A, choosing between the copy and the nullable fallback below. (a) and (b) are proposed and not in question. Recommended over: leaving the column nullable.)* All three use `disable_ddl_transaction!` and guard each step, so a run that stopped part way is finished by the next boot: (b) must, to make each batch its own short transaction (spec 049's backfill does the same), and (a) and (c) must because their `down` and (c)'s `up` copy `accounts` (below). - - **(a) Schema.** Creates `households` and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). +5. **Three migrations, as spec 049 did, all run without a transaction; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Held: the maintainer rules on (c) with Gate A, choosing between the copy and the nullable fallback below. (a) and (b) are proposed and not in question. Recommended over: leaving the column nullable.)* All three use `disable_ddl_transaction!` and guard each step, so a run that stopped part way is finished by the next boot: (b) must, to make each batch its own short transaction (spec 049's backfill does the same), and (a) and (c) must because their `down` and (c)'s `up` copy `accounts` (below). **(c) is the exception to "guarded steps":** it wraps its stragglers step, the copy and the sequence restore in one transaction of its own, opened with enforcement already off (`disable_referential_integrity { transaction { … } }`; the copy's own inner transaction joins it), so the three commit together or not at all and a failed (c) changes nothing. + - **(a) Schema.** Creates `households` (its `name` with a named `CHECK (length(name) BETWEEN 1 AND 100)`, as `locations_name_length` does for a location) and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). - **(b) Backfill.** A data migration with no schema change. It reads the users that have no membership in batches of 500, and for each batch, in one short transaction, inserts a household for each user (decision 2's name), the manager membership and the account's `household_id`, with plain SQL and no application model. **The `UPDATE` of `accounts` sets `household_id` and nothing else, `updated_at` included**, because the upgrade proof compares `accounts`' rows over the columns both databases have and `expected_changes.yml` stays empty (decision 10). It is idempotent: it selects only users with no membership, so a second run, or a run that was stopped after some batches, makes exactly the rest and nothing twice; it holds no lock beyond each batch's own write transaction, so the instance's other work waits at most one batch. Cost: users, not lots; ten thousand users are twenty batches. - - **(c) Tighten.** First runs the same idempotent backfill for stragglers, then makes `accounts.household_id` `NOT NULL`, so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **The stragglers' step matters:** (b) is recorded in `schema_migrations` once it has run and never runs again, but a user can be made between (b) and (c) (an old container still serving during a Kamal deploy, or a boot that stopped between the two), and without the step (c) would fail on every boot after. Spec 049's third migration runs the same kind of safety-net `UPDATE` before its `change_column_null`. + - **(c) Tighten.** In one transaction (above): first runs the same idempotent backfill for stragglers (and removes a household with no member and no account, which an older container's delete of a user leaves behind: it removes the account but knows nothing of the membership the cascade already took), then makes `accounts.household_id` `NOT NULL`, then restores the `accounts` sequence as an upsert (a copy of an empty table leaves no `sqlite_sequence` row to update), so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **The stragglers' step matters:** (b) is recorded in `schema_migrations` once it has run and never runs again, but a user can be made between (b) and (c) (an old container still serving during a Kamal deploy, or a boot that stopped between the two), and without the step (c) would fail on every boot after. Spec 049's third migration runs the same kind of safety-net `UPDATE` before its `change_column_null`. **What the copy is.** SQLite can't add `NOT NULL` to a column in place. Rails (activerecord 8.1.4.1, the SQLite adapter's `alter_table`) creates a new `accounts` with the change, copies the rows, drops the old table and renames the new one, the two moves in one transaction inside `disable_referential_integrity`, so the copy itself is atomic when no outer transaction holds it. `accounts` holds one row per user, so the copy is small and short; the risk is its delete rules, not its volume. - **What the copy risks.** `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven are `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), and `lots` and `users` hold SQLite's default, `NO ACTION` (`db/schema.rb`; the `RESTRICT` that makes `Account` declare `lots` before `locations`, `spec/models/account_spec.rb`, is `lots → locations`). The copy's `DROP TABLE accounts` runs an implicit delete of every row, so with foreign keys enforced: the eleven cascades **delete every collector's decks, places, proxies and working state, silently**, and `lots` and `users` are deferred to the end of the transaction rather than refused. Rails turns enforcement off for a copy only **outside a transaction** (`PRAGMA foreign_keys = OFF` does nothing inside one, and Rails then defers the keys), so a copy that ran inside Rails' own migration transaction would lose that data with no error. That is why (c) runs without one, and the cost of that is the window the guards cover: a statement of (c) outside the copy can fail with no rollback of the ones before it. **Two more effects the proof must catch:** the copy drops and recreates the table, so the `sqlite_sequence` row for `accounts` restarts at the highest id present, and an id of a deleted account can be reused (`compare.rb` skips `sqlite_%` tables, so the proof can't see it: (c) reads the sequence before the copy and restores it after, and AC-3.6 asserts it); and an older release's image started on the upgraded volume can't create a user or account any more, since it doesn't know the column (the same of every added `NOT NULL` column; a downgrade is a restore from the backup taken before the upgrade, as the README says). The precedent is narrower than a pattern: the one shipped copy of a referenced table by an `up` is `sessions` (spec 049), with two cascading children against thirteen here; `AddDecks` and its kin add columns without a copy. + **What the copy risks.** `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven are `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), and `lots` and `users` hold SQLite's default, `NO ACTION` (`db/schema.rb`; the `RESTRICT` that makes `Account` declare `lots` before `locations`, `spec/models/account_spec.rb`, is `lots → locations`). The copy's `DROP TABLE accounts` runs an implicit delete of every row, so with foreign keys enforced: the eleven cascades **delete every collector's decks, places, proxies and working state, silently**, and `lots` and `users` are deferred to the end of the transaction rather than refused. Rails turns enforcement off for a copy only **outside a transaction** (`PRAGMA foreign_keys = OFF` does nothing inside one, and Rails then defers the keys), so a copy that ran inside Rails' own migration transaction would lose that data with no error. That is why (c) runs without Rails' migration transaction, and opens its own with enforcement already off: outside any transaction the steps would each commit alone, and a (c) that stopped after the copy and before the sequence's restore would never get it back, since the next boot finds the column already `NOT NULL`. **Two more effects the proof must catch:** the copy drops and recreates the table, so the `sqlite_sequence` row for `accounts` restarts at the highest id present, and an id of a deleted account can be reused (`compare.rb` skips `sqlite_%` tables, so the proof can't see it: (c) reads the sequence before the copy and restores it after, in the same transaction, and AC-3.6 asserts it); and an older release's image started on the upgraded volume can't create a user or account any more, since it doesn't know the column (the same of every added `NOT NULL` column). **Collector documents no downgrade:** the README says to back up before every upgrade, and going back means restoring that backup onto a new volume under the old image and losing what was written since; `docs/releasing.md`'s Upgrade notes carry this migration's warning. Also: **a failed (c) stops the boot** (Compose is down until it is fixed; Kamal keeps the old container serving), and during a Kamal deploy an old container's sign-up or add-a-user fails with a 500 for the seconds between (c) and the switch. The precedent is narrower than a pattern: the one shipped copy of a referenced table by an `up` is `sessions` (spec 049), with two cascading children against thirteen here; `AddDecks` and its kin add columns without a copy. **What proves no row is lost.** The upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and its seed holds a row in every table, including each of the thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count, and the `accounts` sequence, is unchanged; and the copy's foreign keys, named checks and indexes are read back against the schema dump. The same holds for (c)'s and (a)'s `down`, which also copy `accounts` (AC-3.5). - - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no no-transaction window and no sequence to restore; two migrations, and nothing in this feature can lose a row; an older image on an upgraded volume still creates users (with an account that has no household). *Costs:* the database can't state "every account has a household", so a bug, an older image's insert or a hand-written one can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If a later review finds (c) unsafe, the fallback is taken and this spec's version records it. - - **Reversible:** (c)'s `down` makes the column nullable again by the same copy; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the columns by a copy of `accounts`) and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. + - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no sequence to restore, no step that can stop a boot, and nothing in this feature can lose a row; (a)'s `down` still copies `accounts`, once. *Costs:* an older image on an upgraded volume (the Kamal deploy window) still creates users, **each with no membership and an account with no household, and nothing ever repairs it, because (b) is recorded and never runs again**: under the fallback the Goal "every user in exactly one household" can be broken permanently by one deploy. To be sound the fallback needs a repair that runs on every boot (the stragglers step run from `db:prepare`'s boot path) or a household made on first load; neither is in this spec, so the fallback as written does not meet the Goal. The database also can't state "every account has a household", so a bug or a hand-written insert can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). **The same under both options:** a fresh volume loads `db/schema.rb` and runs no copy; an older backup restored is migrated like an upgrade; a backup taken after the upgrade is refused by the older release either way. The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If a later review finds (c) unsafe, the fallback is taken and this spec's version records it. + - **Reversible:** (c)'s `down` makes the column nullable again by the same copy, restoring the sequence as its `up` does; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the `kind` check before its column, as `AddDecks`' `down` does, so a full rollback copies `accounts` three times: (c)'s, then two in (a)), keeping the sequence at each copy and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. 6. **A personal account is in its user's household.** For a user, `accounts.household_id` of their account equals `memberships.household_id` of their membership. The database can't compare two tables, so the rule is the model's (a user is built with both from one household; a later issue that moves a user changes both in one transaction), and a guard spec and the upgrade proof check it on real rows. A shared account has no user; a personal account has exactly one. -7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. The order is the foreign keys': the user's row (its membership goes with it by decision 1's cascade, **without** the membership's own refusal of decision 3, which guards a membership removed on its own; the user's delete has its own check for the last manager), then the account (`User` declares `belongs_to :account, dependent: :destroy`), then the household when no member and no account is left. A household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). +7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. The order is the foreign keys': the user's row (its membership goes with it by decision 1's cascade, **without** the membership's own refusal of decision 3, which guards a membership removed on its own; the user's delete has its own check for the last manager), then the account (`User` declares `belongs_to :account, dependent: :destroy`), then the household when no member and no account is left. A household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). The removal of a household "when no member and no account is left" is revisited by #381 (a joiner leaves a household of one behind) and #384 (a shared account is an account of the household). 8. **A household is made by the user model, so every path gets one.** The one place an account is made from a user today (`before_validation :build_account`) also makes the household and the manager membership; the first-run sign-up, the open sign-up, the admin's add-a-user form and `bin/rails collector:user` all go through it, and a spec per path says so. Nothing in a request names a household, a role or a kind (`household_id`, `role` and `kind` are not permitted from parameters). -9. **"Nothing looks different" is checked by five things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), and counts the queries of those requests that name `households` or `memberships` (none). (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) A one-off search of `app/views`, `app/helpers`, `app/controllers`, `app/javascript` and `config/locales` for "household" and "membership" finds nothing outside the models, recorded beside the routes diff. (v) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. +9. **"Nothing looks different" is checked by five things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), and counts the queries of those requests that name `households` or `memberships` (none). (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. **The allowed edits to the suite's support code:** `factory :account` (used by `spec/models/account_spec.rb`, `spec/helpers/scanners_helper_spec.rb` and `spec/lib/collector/scanner_findings/art_sitting_report_spec.rb`) gains a household, a bare one with no member (those examples have no user, and the helper of AC-2.8 isn't called after them); and the two places that insert an account by SQL (`spec/support/backup_helpers.rb`, `spec/lib/collector/backup/snapshot_spec.rb`) insert a household first. Nothing else in `spec/` changes for the tenancy checks. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) A one-off search of `app/views`, `app/helpers`, `app/controllers`, `app/javascript` and `config/locales` for "household" and "membership" finds nothing outside the models, recorded beside the routes diff. (v) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. 10. **The upgrade proof gains a household check in two tiers, and `expected_changes.yml` stays empty.** *(Ruled by the maintainer 2026-11-13, as recommended.)* `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and no entry would match; the harness gains a check on the snapshots it takes. **Tier 1, always** (it must stay true when #381 and #384 add joined households and shared accounts, so it names no count): every user has exactly one membership, every household has at least one manager, every account has a household, and a personal account's household is its user's. **Tier 2, only when the old snapshot has no `households` table** (the backfill's own facts): the households' count equals the users' (the seed's two users make two), every membership is `manager`, every account is `personal`, and every household's name is its user's name's form of decision 2. Tier 1 runs on the upgraded snapshot, on the restore leg's, and on the `next` snapshot of a fresh instance of the code under test (`next_release`), the one place the production image's creation path is proved. The seed for the next release (`script/upgrade/seeds/next.rb`) creates its users through the models, so the new tables hold rows, as the seed rule needs. -11. **The ADR is 0030, accepted with the pull request, and the area docs change with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 has a status line saying it is superseded in part; its one-user text isn't rewritten; ADR 0030 is Accepted in the same pull request (its status line already cites it). The docs that change: [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) (glossary, invariants, the delete, accepted limits, and a line that `households` and `memberships` are instance-level tables never scoped through `Current.account`), [`.claude/rules/multi-tenancy.md`](../../../.claude/rules/multi-tenancy.md) (the same line), [`data-portability.md`](../../architecture/data-portability.md) and the [export format](../../export-format/README.md) (coverage, decision 4, and the sentence at its version history), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) (the harness's check), the [specs index](../README.md) (this spec's row and spec 004's, both directions), and `CLAUDE.md`'s Project Status paragraph. +11. **The ADR is 0030, accepted with the pull request, and the area docs change with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 has a status line saying it is superseded in part; its one-user text isn't rewritten; ADR 0030 is Accepted in the same pull request (its status line already cites it). The docs that change: [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) (glossary, invariants, the delete, accepted limits, and a line that `households` and `memberships` are instance-level tables never scoped through `Current.account`), [`.claude/rules/multi-tenancy.md`](../../../.claude/rules/multi-tenancy.md) (the same line), [`data-portability.md`](../../architecture/data-portability.md) and the [export format](../../export-format/README.md) (coverage, decision 4, and the sentence at its version history), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) (the harness's check), `docs/releasing.md`'s Upgrade notes (the migration's entry and warning), the [specs index](../README.md) (this spec's row and spec 004's, both directions), and `CLAUDE.md`'s Project Status paragraph. 12. **Hosted-door rules.** *Cost per household:* one household row, one membership row per member, no row per lot or per request. *Stored per account or user:* the household's name and members' roles are the household's, nothing else is stored about a collector's choices. *The operator is not the user:* no admin page changes, so none shows a collection; the household's fields are not on any page. ## User Stories and Acceptance Criteria @@ -104,7 +105,7 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-2.4** Given any membership, a household of one's included When it is removed on its own, through the model, Then it is refused; the only way a membership goes in this feature is with its user's delete (AC-4.4), which for a household of one removes the household too. - [ ] **AC-2.5** Given an account When its kind is anything but `personal` or `shared` Then the model refuses it and the database refuses a bypassing insert. No shared account exists in any request or in the seed, and no index or validation about one is added (decision 1). - [ ] **AC-2.6** Given a user whose account is in a different household from their membership's When the user is validated Then it is refused (decision 6); given a user built through the model Then the two households are the same one. -- [ ] **AC-2.7** Given a household name that is blank or over 100 characters When it is saved Then the model refuses it. +- [ ] **AC-2.7** Given a household name that is blank or over 100 characters When it is saved Then the model refuses it, and an insert that bypasses the model is refused by the database's named `CHECK`. - [ ] **AC-2.8** Given a spec helper that checks the invariants on the test database (each user has exactly one membership, each household has a manager, each account has a household, each personal account's household is its user's) When it is called after the examples that make users by each of the four paths of Story 1, delete a user and apply an import Then it passes each time (the same invariants as the upgrade proof's tier 1, decision 10). - [ ] **AC-2.9** Given a household that holds an account or a member When it is destroyed through the model, or its row is deleted by a statement that bypasses the model Then it is refused and no account, membership or collection row changes (decision 1's no-action keys). @@ -115,10 +116,10 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-3.1** Given a database as the previous release leaves it (users, accounts, lots, locations, decks, proxies, sessions, working state, the prices database untouched) When the migrations run Then every user has one household named "'s household", one membership with role `manager`, and an account whose household is it, every account kind is `personal`, and the rows of every old table are unchanged, `accounts.updated_at` included (the backfill sets `household_id` only). - [ ] **AC-3.2** Given the migrations have run When the backfill runs again (its migration is applied a second time, or the database is migrated from a state with some users done) Then no household or membership is added or changed and the count of each is the same. - [ ] **AC-3.3** Given 1,201 users with no membership When the backfill runs Then it makes households in batches of at most 500, each batch one transaction, and every user has exactly one household afterwards; given it stops after the first batch When the migrations run again Then the rest are made and the first batch's are not made twice. -- [ ] **AC-3.4** Given an instance with no users When the migrations run on an empty database Then they succeed and the first sign-up gets its household by AC-1.1; and a database made from `db/schema.rb` (a fresh volume) and one migrated while empty have the same `accounts`, `households` and `memberships` columns, keys, checks and indexes. -- [ ] **AC-3.5** Given the three migrations have run on a database that holds rows in every table that references `accounts` When they are rolled back Then the database has the schema before (a) and the count of every one of those tables and every old row are unchanged; given a household with two members or a shared account When (a) is rolled back Then it refuses, names what exists, and changes nothing. -- [ ] **AC-3.6** Given (c) has run on a database that holds rows in every table that references `accounts` When the counts are compared before and after Then each is equal, the `sqlite_sequence` row of `accounts` is unchanged (a new account after (c) gets an id above the highest ever issued, not a reused one), `PRAGMA foreign_key_check` and `integrity_check` report nothing, and the foreign keys, named checks and indexes of `accounts` and of its thirteen referencing tables are those of the schema dump. -- [ ] **AC-3.7** Given a user made after (b) has run and before (c) When (c) runs Then the user is given a household by (c)'s stragglers step and (c) succeeds; given (c) is run again Then it changes nothing. +- [ ] **AC-3.4** Given an instance with no users When the migrations run on an empty database Then they succeed and the first sign-up gets its household by AC-1.1; and a database made from `db/schema.rb` (a fresh volume) and one migrated while empty have the same `accounts`, `households` and `memberships` columns, foreign keys, checks and indexes as read by `PRAGMA` (not as the table's SQL text, which a column-level check before a copy and a table-level one after it state differently). +- [ ] **AC-3.5** Given the three migrations have run on a database that holds rows in every table that references `accounts` When they are rolled back Then the database has the schema before (a) and the count of every one of those tables, every old row and the `accounts` sequence are unchanged; given a household with two members or a shared account When (a) is rolled back Then it refuses, names what exists, and changes nothing. +- [ ] **AC-3.6** Given (c) has run on a database that holds rows in every table that references `accounts` When the counts are compared before and after Then each is equal, the `sqlite_sequence` row of `accounts` is unchanged (a new account after (c) gets an id above the highest ever issued, not a reused one), `PRAGMA foreign_key_check` and `integrity_check` report nothing, and the foreign keys, named checks and indexes of `accounts` and of its thirteen referencing tables are those of the schema dump; given the copy raises part way (a fault injected after the stragglers step) Then `accounts`, the thirteen tables, the households and the sequence are as before (nothing committed) and a second run succeeds. The example runs `:committed` through the migrator outside any example transaction (inside one, enforcement stays on and the cascades fire), as `spec/migrations/add_currency_to_accounts_spec.rb` does. +- [ ] **AC-3.7** Given a user made after (b) has run and before (c) When (c) runs Then the user is given a household by (c)'s stragglers step and (c) succeeds; given a household left with no member and no account by an older container's delete of a user Then (c) removes it; given (c) is run again Then it changes nothing. - [ ] **AC-3.8** Given sessions exist before the three migrations run When they have run Then every `sessions` row is unchanged, so a collector signed in before the upgrade is still signed in (spec 049's rule for upgrades; the upgrade proof's behaviour leg signs in with the old password and checks the same). ### Story 4: Nothing looks different @@ -148,7 +149,7 @@ This feature adds the structure and nothing that uses it: a household with a nam - [ ] **AC-6.2** Given the proof's backup leg (spec 031 AC-6.3) When the previous release's backup is restored into the new image Then tiers 1 and 2 pass on its snapshot; given the `next` snapshot of a fresh instance of the code under test Then tier 1 passes on it (tier 2 is not run: its old snapshot has `households`). - [ ] **AC-6.3** Given the proof's comparison When it runs Then `expected_changes.yml` is empty (`changes: []`), no old table or column is reported changed, and the new tables and columns are not compared. - [ ] **AC-6.4** Given the seed for the next release When it runs on a fresh instance of the code under test Then `households` and `memberships` each hold a row and no seed exception is added. -- [ ] **AC-6.5** Given a fixture snapshot with a user who has no membership, one with a household with no manager, one whose account is in another household and one (tier 2) whose household name differs from decision 2's When the check runs on each Then it fails naming the rule broken; and a snapshot with a joined household or a shared account passes tier 1 (a spec of the harness, not of the application). +- [ ] **AC-6.5** Given a fixture snapshot with a user who has no membership, one with a household with no manager, one whose account is in another household and one (tier 2) whose household name differs from decision 2's (the fixture uses AC-1.5's three names, so the harness's name check and the model and the backfill agree on the same cases) When the check runs on each Then it fails naming the rule broken; and a snapshot with a joined household or a shared account passes tier 1 (a spec of the harness, not of the application). - [ ] **AC-6.6** Given `bin/upgrade-smoke` run against the previous release built from its tag and the code under test When it completes Then it passes with the check of AC-6.1 and AC-6.2 (evidence in `verification.md`; it needs the container engine and a slot). ## Functional Requirements @@ -183,7 +184,7 @@ The harness checks the upgraded and the restored snapshots (decision 10). ### FR-5: Documentation -The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/multi-tenancy.md`, the format document's Coverage and sentences, the release-and-deploy area doc's upgrade-proof paragraph, `CLAUDE.md`'s Project Status line for accounts, the specs index in both directions (decision 11). +The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/multi-tenancy.md`, the format document's Coverage and sentences, the release-and-deploy area doc's upgrade-proof paragraph, `docs/releasing.md`'s Upgrade notes (this migration's entry and its warning), `CLAUDE.md`'s Project Status line for accounts, the specs index in both directions (decision 11). **Must:** record the supersession in this spec, ADR 0030 and the index in both directions (`.claude/rules/documentation.md`). **Must not:** rewrite ADR 0017's text, or restate an area's rules in `CLAUDE.md`. @@ -213,7 +214,8 @@ The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/m | A user is created and the household or membership insert fails | The whole write rolls back; no user, account, household or membership remains; the sign-up or admin form shows the error it shows for any failed save | | The backfill is stopped after some batches | The next boot's migration makes the households of the users that have none and no other; no user has two | | The backfill meets a busy database | A batch's transaction waits on the busy timeout as every write does; a failure ends the migration, and the next boot resumes it | -| (c) stops between its steps | Its guards finish it on the next boot; the proof and AC-3.6 read the result | +| (c) fails (a fault, a busy database past its timeout) | Nothing changed: its stragglers step, copy and sequence restore are one transaction, so the next boot runs it whole; the boot stops until it does (Kamal keeps the old container) | +| An older container sign-up or add-a-user after (c) | Fails with a 500 until the switch; documented in the Upgrade notes | | (a) is rolled back with a two-member household or a shared account | Refused: "N households with more than one member and M shared accounts exist; remove them before rolling back"; nothing changes | | An admin deletes the last manager of a household with other members | Refused by the model; no page makes such a household yet (#382, #383) | | A restored backup from before this feature | Migrated at the next start like an upgrade; households exist for every user | From df8c0978ca257c14c93a2f90a0166a8722b75258 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:36:05 -0500 Subject: [PATCH 05/41] docs(specs): spec 090 v0.3.2, the maintainer rules migration (c) (#380) Take the table copy of accounts, not the nullable fallback, on two conditions: the plan review traces it against Rails 8.1.4.1, and bin/upgrade-smoke runs from the previous release built from its tag before the push. Co-Authored-By: Claude Opus 5.5 --- docs/adr/0030-a-household-is-a-group-of-accounts.md | 8 ++++---- docs/specs/090-every-user-has-a-household/spec.md | 12 +++++++----- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index 4092c4f6..8682280c 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -2,7 +2,7 @@ ## Status -Proposed (2026-11-13), written with spec 090; it becomes Accepted with the pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) awaits the maintainer's ruling on spec 090 decision 5. +Proposed (2026-11-13), written with spec 090; it becomes Accepted with the pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) was ruled by the maintainer the same day: the table copy, not the nullable fallback, on two conditions (the plan review traces it against Rails 8.1.4.1's `alter_table`, and the upgrade proof runs from the previous release built from its tag). **Date:** 2026-11-13 **Feature:** 090-every-user-has-a-household @@ -31,11 +31,11 @@ This ADR records the decision that is needed first, by every later issue of both 1. **An account stays the unit of ownership and privacy.** Every collector-owned table keeps `account_id` and nothing in it changes. A household is a group of accounts: each member's own account (`kind` `personal`) and, from #384, at most one shared account (`kind` `shared`, no user). The rules of the shared account (one per household, no user) are written by the issue that makes one, not here. The foreign keys: `accounts.household_id` and `memberships.household_id` are no action, so a household that holds an account or a member can't be deleted, and `memberships.user_id` cascades, so a user's delete removes their membership. 2. **Every user belongs to exactly one household,** through a membership row whose `user_id` is unique. A membership has a role, `manager` or `member`, stated by a `CHECK`. A household always has a manager. There is no other role in 1.0 (no kid role: a child's collection is any member's). -3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL` (subject to the maintainer's ruling on spec 090 decision 5, which weighs a nullable fallback). +3. **`users.account_id` stays unique.** A user has one account, their own, and a personal account has one user. What changes is that an account belongs to a household, through `accounts.household_id`, which is `NOT NULL` (ruled by the maintainer, 2026-11-13, over a nullable fallback; spec 090 decision 5 weighs both). 4. **A household is made with its user, as the account is.** The user model makes the household, the manager membership and the account in one write, so first run, sign-up, an admin's form and the operator's command all give one. A household of one is named for its user and is invisible in the app. 5. **"A household always has a manager" is the model's rule, plus the database's checks.** There is no trigger, because `db/schema.rb` doesn't dump triggers and a volume made from the schema would differ from an upgraded one. The rule is checked on real rows by a guard spec and by the upgrade proof. 6. **The export doesn't carry the household, and the format stays version 4.** The export is one file per account. The household's name and members are a fact about a group; they are listed in the format document's Coverage tables as left out. The shared collection's own file, and the version raise it brings, are #384's. -7. **Three migrations carry it onto every instance, all run without a transaction:** the schema (new tables, two columns added without a table copy), a batched idempotent backfill that gives every existing user a household of one, and `NOT NULL` on `accounts.household_id` by a table copy that first gives any straggler a household. The upgrade proof checks the invariants on the upgraded, the restored and a fresh snapshot, and the backfill's own facts on the upgraded and restored ones. The third migration's risk and its nullable fallback are weighed in spec 090 decision 5, and the maintainer's ruling on it is recorded there. +7. **Three migrations carry it onto every instance, all run without a transaction:** the schema (new tables, two columns added without a table copy), a batched idempotent backfill that gives every existing user a household of one, and `NOT NULL` on `accounts.household_id` by a table copy that first gives any straggler a household. The upgrade proof checks the invariants on the upgraded, the restored and a fresh snapshot, and the backfill's own facts on the upgraded and restored ones. The third migration's risk and the nullable fallback it was ruled over are weighed in spec 090 decision 5. What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolved once and failing closed, queries through the account, 404 for another account's record, catalog data global, jobs and broadcasts carrying the account, the admin flag giving no view into a collection, and "the operator is not the user". `Current.account` is still the signed-in user's own account; that a session may work in the household's shared account instead is #384's. @@ -45,7 +45,7 @@ What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolve - **Good:** a household of one is the case every user is in on upgrade, so each later feature starts from a state that already exists on every instance, not from a migration of its own. - **Cost:** one fact, a personal account's household, is held twice (the account's column and its user's membership). A later issue that moves a user between households changes both in one transaction; until then a guard and the upgrade proof check them. - **Cost:** a copy is in one account at a time, so a household card in a personal deck means a transfer or a borrow (design decision 15, #385, #388). -- **Cost:** the third migration copies `accounts`, the most referenced table in the schema (eleven cascading children), with foreign keys switched off outside a transaction; inside one the copy would silently delete those children. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost; the copy also restores the table's id sequence. If the maintainer chooses the fallback, `accounts.household_id` is nullable in the database and the model alone says it is present. +- **Cost:** the third migration copies `accounts`, the most referenced table in the schema (eleven cascading children), with foreign keys switched off outside a transaction; inside one the copy would silently delete those children. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost; the copy also restores the table's id sequence. Had the fallback been chosen, `accounts.household_id` would be nullable in the database and the model alone would say it is present. - **Cost:** "member" now means two things: spec 015's user without the admin flag, and a household's non-manager. They are independent, and no page shows either yet; #389 and the pages of #381 onward choose their words with both in view. - **Cost:** an older release can't hold a household with two members or a shared account, so rolling back the first migration refuses when either exists. - **Follow-up:** ADR 0017's status line records that this supersedes its one-user half; its text isn't rewritten. The tenancy area doc, the format document's Coverage and `CLAUDE.md`'s status paragraph change with spec 090. diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index fc37df29..82a367f2 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -1,7 +1,7 @@ # Feature 090: Every User Has a Household -**Status:** Draft (decisions 2, 3, 4 and 10 are ruled by the maintainer, 2026-11-13; decision 5's step (c) is held for the maintainer's ruling with Gate A) -**Version:** 0.3.0 +**Status:** Approved (2026-11-13, Gate A: the coordinator under the maintainer's delegation, v0.3.1; the maintainer ruled decision 5's step (c) on 2026-11-13, v0.3.2) +**Version:** 0.3.2 **Created:** 2026-11-13 **Last Updated:** 2026-11-13 **Depends on:** [spec 004](../004-design-system-collection/spec.md) (accounts, FR-2 to FR-5; frozen, by [tenancy and accounts](../../architecture/tenancy-and-accounts.md)), [spec 017](../017-export-format/spec.md) FR-9 (the coverage guard), [spec 031](../031-upgrade-from-previous-release/spec.md) (the upgrade proof), [spec 049](../049-session-idle-expiry/spec.md) (three migrations without a transaction, the precedent for decision 5) @@ -17,6 +17,8 @@ | Version | Date | Change | |---------|------|--------| +| 0.3.2 | 2026-11-13 | **The maintainer ruled decision 5's step (c): take the table copy, not the nullable fallback**, on the two conditions the decision records. No requirement changed; the open question is closed | +| 0.3.1 | 2026-11-13 | **Gate A: approved** by the coordinator under the maintainer's delegation (2026-11-13): two Fable rounds, nothing blocking after round 2, every finding fixed; decision 5's step (c) and the nullable fallback stay with the maintainer, whose ruling the coordinator sends. The plan makes (c) and everything that exists only for it one phase of its own, last among the migrations, with (a) and (b) complete and green without it. No requirement changed | | 0.3.0 | 2026-11-13 | Spec review round 2 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 5 important, 8 minor) fixed: (c) is one transaction around its stragglers step, the copy and the sequence restore (upsert), so a failed (c) changes nothing; the README claim is corrected (no downgrade is documented) and `docs/releasing.md`'s Upgrade notes join the docs; the fallback's stragglers hole is stated and what it would need; four facts added to the comparison (a failed (c) stops the boot, the deploy window, fresh volumes and backups, three copies on a full rollback); the suite's bare accounts are named allowed edits; the name's length `CHECK`; an empty household left by an older container is removed by (c); AC-3.4 by `PRAGMA`, AC-3.5's sequence, AC-3.6's `:committed` example and its fault case; #381 and #384 revisit decision 7 | | 0.2.0 | 2026-11-13 | The maintainer ruled decisions 2, 3, 4 and 10 as recommended. Spec review round 1 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 8 important, 11 minor) fixed: the foreign keys' delete rules are stated (decision 1, AC-2.9); decision 5 rewritten with the copy's exact risks (atomic outside a transaction, silent cascades and deferred keys inside one, `NO ACTION` not `RESTRICT` on `lots` and `users`, the id sequence, an older image on the upgraded volume), a corrected precedent claim, all three migrations without a transaction, (c)'s stragglers step (AC-3.7), the backfill's `updated_at` untouched, and the nullable fallback with its gains and costs; the harness check in two tiers (decision 10); the shared-account index and validations dropped to #384; the rule for removing a membership has one wording (decision 3, AC-2.4); the name's rule is exact (first 88 characters, decision 2); decision 9 gains a grep and a query count; the docs list is complete (decision 11) | | 0.1.0 | 2026-11-13 | First draft from issue #380, the design and the coordinator's brief. Decisions 2 to 6 are the five scope questions put to the maintainer, written on their recommendations | @@ -64,14 +66,14 @@ This feature adds the structure and nothing that uses it: a household with a nam 2. **A household of one is named for its user.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a constant "Household".)* The name is stored text, the user's name followed by the straight apostrophe (U+0027) and "s household" ("Ada's household"). That suffix is 12 characters, so the user's name is cut to its **first 88 characters** and the whole is at most 100; a household's name is present and at most 100 characters. One definition builds the name for the model and for the backfill (which does it in SQL with `substr`); a spec holds the two equal on the same names (as `Location::Name`'s parity spec does), including a 100-character name, one with an apostrophe and one with multi-byte characters. It is stored, not derived, so a user's rename later changes nothing in the household, and a list of households (#389) tells them apart. No page shows or edits it in this feature; a manager renames it when a later issue gives them the page. 3. **"A household always has a manager" is enforced by the model and by the parts of the database SQLite can state.** *(Ruled by the maintainer 2026-11-13, as recommended; over: a database trigger.)* The database holds: the role `CHECK`, `memberships.user_id` unique, both foreign keys with decision 1's rules, `NOT NULL` on every column of `households` and `memberships`, and the name's length `CHECK` (decision 5 (a)). The model holds: a household is created with its first membership, a manager, in one transaction with the user; a membership can't be removed on its own (the one wording of the rule: a membership goes only with its user's delete in this feature, and the issues that let a person leave or be removed, #382 and #383, change that rule where they need to); a manager can't be demoted when no other manager remains; and a user can't be deleted when they are the last manager of a household that has other members. **No trigger:** `db/schema.rb` doesn't dump triggers, so a volume made from the schema would lack the rule that an upgraded one has. A spec and the upgrade proof check the rule on real rows (a household without a manager is a failure of both). 4. **The export does not carry the household, and the format stays version 4.** *(Ruled by the maintainer 2026-11-13, as recommended; over: carrying the name in `profile` and raising to version 5 now.)* The export is one file per account and holds the collection and its profile (design decision 16). A household is a fact about a group, not about the collection; a household of one has a default name, and a person who imports their file onto another instance is made a household there by the same rule that makes every user one. The shared collection's file and its `profile` are #384's (the design's open question 4), and that is the issue that raises the version. The format document's Coverage tables list the new data as "Left out" with the reason (`memberships`, `accounts.household_id`, `accounts.kind`); `households` has neither `account_id` nor `user_id`, so the guard doesn't ask for it, and the format document says in one sentence that the guard does not cover it and where the question is decided again (so #384 and #393 don't read the guard as covering it). The guard's own spec (`spec/export_format_document_spec.rb`) hard-codes the "Left out" tables and columns and gains `memberships`, `accounts.household_id` and `accounts.kind`; the format document's sentence that households "will raise the version (ADR 0017)" points to ADR 0030 and #384. -5. **Three migrations, as spec 049 did, all run without a transaction; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Held: the maintainer rules on (c) with Gate A, choosing between the copy and the nullable fallback below. (a) and (b) are proposed and not in question. Recommended over: leaving the column nullable.)* All three use `disable_ddl_transaction!` and guard each step, so a run that stopped part way is finished by the next boot: (b) must, to make each batch its own short transaction (spec 049's backfill does the same), and (a) and (c) must because their `down` and (c)'s `up` copy `accounts` (below). **(c) is the exception to "guarded steps":** it wraps its stragglers step, the copy and the sequence restore in one transaction of its own, opened with enforcement already off (`disable_referential_integrity { transaction { … } }`; the copy's own inner transaction joins it), so the three commit together or not at all and a failed (c) changes nothing. +5. **Three migrations, as spec 049 did, all run without a transaction; the third makes `accounts.household_id` `NOT NULL` by a table copy.** *(Ruled by the maintainer 2026-11-13: (c) as written, the table copy, over the nullable fallback below, which is kept as the record of the alternative and its cost. The two conditions of the ruling: the plan review traces (c) line by line against Rails 8.1.4.1's `alter_table` and the thirteen referencing tables, and `bin/upgrade-smoke` runs from the previous release built from its tag before the push.)* All three use `disable_ddl_transaction!` and guard each step, so a run that stopped part way is finished by the next boot: (b) must, to make each batch its own short transaction (spec 049's backfill does the same), and (a) and (c) must because their `down` and (c)'s `up` copy `accounts` (below). **(c) is the exception to "guarded steps":** it wraps its stragglers step, the copy and the sequence restore in one transaction of its own, opened with enforcement already off (`disable_referential_integrity { transaction { … } }`; the copy's own inner transaction joins it), so the three commit together or not at all and a failed (c) changes nothing. - **(a) Schema.** Creates `households` (its `name` with a named `CHECK (length(name) BETWEEN 1 AND 100)`, as `locations_name_length` does for a location) and `memberships`; adds `accounts.household_id` and `accounts.kind` with `ALTER TABLE … ADD COLUMN`, which copies no table (a `REFERENCES` column is accepted because its default is `NULL`; a `CHECK` stated on the column is tested against every row and is kept by the schema dump when named; the precedent is `AddDecks`). - **(b) Backfill.** A data migration with no schema change. It reads the users that have no membership in batches of 500, and for each batch, in one short transaction, inserts a household for each user (decision 2's name), the manager membership and the account's `household_id`, with plain SQL and no application model. **The `UPDATE` of `accounts` sets `household_id` and nothing else, `updated_at` included**, because the upgrade proof compares `accounts`' rows over the columns both databases have and `expected_changes.yml` stays empty (decision 10). It is idempotent: it selects only users with no membership, so a second run, or a run that was stopped after some batches, makes exactly the rest and nothing twice; it holds no lock beyond each batch's own write transaction, so the instance's other work waits at most one batch. Cost: users, not lots; ten thousand users are twenty batches. - **(c) Tighten.** In one transaction (above): first runs the same idempotent backfill for stragglers (and removes a household with no member and no account, which an older container's delete of a user leaves behind: it removes the account but knows nothing of the membership the cascade already took), then makes `accounts.household_id` `NOT NULL`, then restores the `accounts` sequence as an upsert (a copy of an empty table leaves no `sqlite_sequence` row to update), so "every account has a household" is a fact of the database, as `users.account_id`'s `NOT NULL` is. **The stragglers' step matters:** (b) is recorded in `schema_migrations` once it has run and never runs again, but a user can be made between (b) and (c) (an old container still serving during a Kamal deploy, or a boot that stopped between the two), and without the step (c) would fail on every boot after. Spec 049's third migration runs the same kind of safety-net `UPDATE` before its `change_column_null`. **What the copy is.** SQLite can't add `NOT NULL` to a column in place. Rails (activerecord 8.1.4.1, the SQLite adapter's `alter_table`) creates a new `accounts` with the change, copies the rows, drops the old table and renames the new one, the two moves in one transaction inside `disable_referential_integrity`, so the copy itself is atomic when no outer transaction holds it. `accounts` holds one row per user, so the copy is small and short; the risk is its delete rules, not its volume. **What the copy risks.** `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven are `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), and `lots` and `users` hold SQLite's default, `NO ACTION` (`db/schema.rb`; the `RESTRICT` that makes `Account` declare `lots` before `locations`, `spec/models/account_spec.rb`, is `lots → locations`). The copy's `DROP TABLE accounts` runs an implicit delete of every row, so with foreign keys enforced: the eleven cascades **delete every collector's decks, places, proxies and working state, silently**, and `lots` and `users` are deferred to the end of the transaction rather than refused. Rails turns enforcement off for a copy only **outside a transaction** (`PRAGMA foreign_keys = OFF` does nothing inside one, and Rails then defers the keys), so a copy that ran inside Rails' own migration transaction would lose that data with no error. That is why (c) runs without Rails' migration transaction, and opens its own with enforcement already off: outside any transaction the steps would each commit alone, and a (c) that stopped after the copy and before the sequence's restore would never get it back, since the next boot finds the column already `NOT NULL`. **Two more effects the proof must catch:** the copy drops and recreates the table, so the `sqlite_sequence` row for `accounts` restarts at the highest id present, and an id of a deleted account can be reused (`compare.rb` skips `sqlite_%` tables, so the proof can't see it: (c) reads the sequence before the copy and restores it after, in the same transaction, and AC-3.6 asserts it); and an older release's image started on the upgraded volume can't create a user or account any more, since it doesn't know the column (the same of every added `NOT NULL` column). **Collector documents no downgrade:** the README says to back up before every upgrade, and going back means restoring that backup onto a new volume under the old image and losing what was written since; `docs/releasing.md`'s Upgrade notes carry this migration's warning. Also: **a failed (c) stops the boot** (Compose is down until it is fixed; Kamal keeps the old container serving), and during a Kamal deploy an old container's sign-up or add-a-user fails with a 500 for the seconds between (c) and the switch. The precedent is narrower than a pattern: the one shipped copy of a referenced table by an `up` is `sessions` (spec 049), with two cascading children against thirteen here; `AddDecks` and its kin add columns without a copy. **What proves no row is lost.** The upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and its seed holds a row in every table, including each of the thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count, and the `accounts` sequence, is unchanged; and the copy's foreign keys, named checks and indexes are read back against the schema dump. The same holds for (c)'s and (a)'s `down`, which also copy `accounts` (AC-3.5). - - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no sequence to restore, no step that can stop a boot, and nothing in this feature can lose a row; (a)'s `down` still copies `accounts`, once. *Costs:* an older image on an upgraded volume (the Kamal deploy window) still creates users, **each with no membership and an account with no household, and nothing ever repairs it, because (b) is recorded and never runs again**: under the fallback the Goal "every user in exactly one household" can be broken permanently by one deploy. To be sound the fallback needs a repair that runs on every boot (the stragglers step run from `db:prepare`'s boot path) or a household made on first load; neither is in this spec, so the fallback as written does not meet the Goal. The database also can't state "every account has a household", so a bug or a hand-written insert can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). **The same under both options:** a fresh volume loads `db/schema.rb` and runs no copy; an older backup restored is migrated like an upgrade; a backup taken after the upgrade is refused by the older release either way. The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If a later review finds (c) unsafe, the fallback is taken and this spec's version records it. + - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no sequence to restore, no step that can stop a boot, and nothing in this feature can lose a row; (a)'s `down` still copies `accounts`, once. *Costs:* an older image on an upgraded volume (the Kamal deploy window) still creates users, **each with no membership and an account with no household, and nothing ever repairs it, because (b) is recorded and never runs again**: under the fallback the Goal "every user in exactly one household" can be broken permanently by one deploy. To be sound the fallback needs a repair that runs on every boot (the stragglers step run from `db:prepare`'s boot path) or a household made on first load; neither is in this spec, so the fallback as written does not meet the Goal. The database also can't state "every account has a household", so a bug or a hand-written insert can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). **The same under both options:** a fresh volume loads `db/schema.rb` and runs no copy; an older backup restored is migrated like an upgrade; a backup taken after the upgrade is refused by the older release either way. The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If the plan review's trace finds (c) unsafe, the maintainer is asked again before the fallback is taken, and this spec's version records the outcome. - **Reversible:** (c)'s `down` makes the column nullable again by the same copy, restoring the sequence as its `up` does; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the `kind` check before its column, as `AddDecks`' `down` does, so a full rollback copies `accounts` three times: (c)'s, then two in (a)), keeping the sequence at each copy and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. 6. **A personal account is in its user's household.** For a user, `accounts.household_id` of their account equals `memberships.household_id` of their membership. The database can't compare two tables, so the rule is the model's (a user is built with both from one household; a later issue that moves a user changes both in one transaction), and a guard spec and the upgrade proof check it on real rows. A shared account has no user; a personal account has exactly one. 7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. The order is the foreign keys': the user's row (its membership goes with it by decision 1's cascade, **without** the membership's own refusal of decision 3, which guards a membership removed on its own; the user's delete has its own check for the last manager), then the account (`User` declares `belongs_to :account, dependent: :destroy`), then the household when no member and no account is left. A household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). The removal of a household "when no member and no account is left" is revisited by #381 (a joiner leaves a household of one behind) and #384 (a shared account is an account of the household). @@ -222,7 +224,7 @@ The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/m ## Open Questions -- [NEEDS CLARIFICATION: decision 5, migration (c): the table copy making `accounts.household_id` `NOT NULL` (recommended) or the nullable fallback. The maintainer rules; the risks and costs of both are in the decision. Decisions 2, 3, 4 and 10 are ruled (2026-11-13, as recommended); (a) and (b) are not in question.] +- None: every question is ruled (decisions 2, 3, 4, 5 and 10 by the maintainer on 2026-11-13). ## Out of Scope (Future Considerations) From d8bac5280e70746c980dd907d8018b4206b76108 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 19:36:16 -0500 Subject: [PATCH 06/41] docs(specs): plan for spec 090, every user has a household (#380) Seven phases: (a) the schema, the models, (b) the backfill, nothing looks different, the upgrade proof's households check, (c) the NOT NULL copy as its own last migration phase, then documentation and evidence. Co-Authored-By: Claude Opus 5.5 --- .../090-every-user-has-a-household/plan.md | 1348 +++++++++++++++++ 1 file changed, 1348 insertions(+) create mode 100644 docs/specs/090-every-user-has-a-household/plan.md diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md new file mode 100644 index 00000000..0d8a9fe5 --- /dev/null +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -0,0 +1,1348 @@ +# Plan 090: Every User Has a Household + +**Spec:** [spec.md](spec.md) v0.3.1 (Approved by the coordinator under the maintainer's delegation, 2026-11-13; the maintainer ruled step (c) the same day: take the copy). **ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md). **Issue:** #380. **Branch:** `090-every-user-has-a-household`. +**Written by:** the main session (Opus 5.5 from this step; the spec and ADR were written on Sonnet 5.5). Implementers: Sonnet 5.5 (`model: "sonnet"`), 2 to 4 consecutive tasks each, test first, one commit per task, reading this file by line range; the main session checks each batch's diff. + +## Global Constraints + +- **No agent runs the `sqlite3` command**, `bin/rails console`, `db:rollback`, `db:migrate:down`, `db:seed`, anything with `RAILS_ENV=production`, `bundle update`, or anything else that prompts. Migrations go down and up **inside specs, through the Migrator** (`MigrationContext#run`), as `spec/migrations/add_decks_spec.rb` does. `bin/rails db:migrate` (development) writes `db/schema.rb`; `bin/rails db:test:prepare` then loads it into the test database. +- Targeted `bin/rspec ` runs need no slot. Never two test runs at once (one shared test database). No full `bin/ci` and no push without a slot from the coordinator. +- Released migrations are never edited. The three new files are `20261113120001_create_households_and_memberships.rb`, `20261113120002_backfill_households.rb`, `20261113120003_make_account_household_required.rb` (phase 6; the only one that exists only if the maintainer rules (c)). None references an application model; all use `disable_ddl_transaction!` and guard each step. +- Every public method added in `app/models/`, `lib/`, `script/upgrade/` or `spec/support/` has a YARD summary (the comment ratchet); no `{…}` in comments; no collectible named in a core file's comments. +- `household_id`, `role` and `kind` are never permitted from parameters (no controller changes at all: Story 4). +- Commits: Conventional Commits, one per task, stage and commit in separate shell calls, ending with the trailer the session reminder gives. +- `bin/rubocop -a` runs on each Ruby file after an edit (hook); remaining offenses are fixed before the commit. + +## Goal + +Every user has one household and one manager membership, made with the user; an account has a household and a kind; an upgraded instance backfills them unattended; nothing visible changes; the upgrade proof checks it. (a) and (b) are complete and green without (c); (c) is phase 6 and removable. + +## Gates + +- **Simplicity:** three new models' worth of code (`Household`, `Membership`, `Account` and `User` edits), three migrations, one harness module. No gem, no controller, no view, no route. +- **Anti-abstraction:** Active Record directly; migrations in plain SQL; the name's rule has one Ruby home (`Household.name_for`) and one SQL home (the backfill), held equal by a spec. +- **Integration-first:** the migration specs (phase 1, 3, 6) and the harness fixture spec (phase 5) are written before the code they test. Complexity Tracking: phase 6 exists only for the held ruling (see its note). + +## Files + +| File | Responsibility | Phase | +|---|---|---| +| `db/migrate/20261113120001_create_households_and_memberships.rb` | (a) tables, columns, keys, checks | 1 | +| `db/schema.rb` | dumped by `db:migrate` | 1, 6 | +| `app/models/household.rb`, `app/models/membership.rb` | the new models | 2 | +| `app/models/account.rb`, `app/models/user.rb` | `belongs_to :household`, `kind`; creation and deletion | 2 | +| `spec/factories/accounts.rb` | `:household`, `:account` gets one | 2 | +| `spec/support/household_invariants.rb` | the helper of AC-2.8 | 2 | +| `db/migrate/20261113120002_backfill_households.rb` | (b) backfill | 3 | +| `spec/requests/households_*_spec.rb`, `spec/models/household_spec.rb`, `membership_spec.rb`, `collection_import/household_untouched_spec.rb` | the specs of Stories 1, 2, 4, 5 | 2, 4 | +| `docs/export-format/README.md`, `spec/export_format_document_spec.rb` | coverage | 4 | +| `script/upgrade/households.rb`, `script/upgrade/harness.rb` | the two-tier check | 5 | +| `db/migrate/20261113120003_make_account_household_required.rb` | (c) | 6 | +| area docs, rules, `docs/releasing.md`, `CLAUDE.md`, indexes, `verification.md` | documentation | 7 | + +--- + +## Phase 0: Baseline + +**Implements:** none | **Satisfies:** none | **Interfaces:** none + +- [ ] **0.1** `bin/rails db:prepare` then `bin/rspec spec/models/account_spec.rb spec/models/user_spec.rb spec/script/upgrade spec/export_format_document_spec.rb`. Expected: all pass (record the count in `verification.md` later). + +--- + +## Phase 1: Migration (a), the schema + +**Implements:** FR-1, FR-3, decision 1, decision 5 (a) | **Satisfies:** AC-2.1 (database half), AC-2.2 (database half), AC-2.5 (database half), AC-2.7 (database half), AC-2.9 (database half), AC-3.5 (the (a) half), AC-3.4 (shape) | **Interfaces:** tables `households` (`id`, `name` NOT NULL with check `households_name_length`, timestamps) and `memberships` (`id`, `household_id` NOT NULL no-action FK indexed, `user_id` NOT NULL cascade FK unique-indexed, `role` NOT NULL with check `memberships_role_known`, timestamps); columns `accounts.household_id` (integer, nullable, `REFERENCES households (id)`, indexed) and `accounts.kind` (`NOT NULL DEFAULT 'personal'`, check `accounts_kind_known`). + +- [ ] **1.1 Write the failing spec** `spec/migrations/create_households_and_memberships_spec.rb`: + +```ruby +require "rails_helper" +require Rails.root.join("db/migrate/20261113120001_create_households_and_memberships") + +# Spec 090 decision 1, decision 5 (a): the tables, the two columns of accounts, their keys and checks. The migration runs outside a +# transaction (disable_ddl_transaction!), so the examples commit their rows (:committed empties the database around them) and go +# down and up through the Migrator, as db:rollback does (the precedent is spec/migrations/add_decks_spec.rb). Rows are made with +# SQL, so the examples don't depend on the models. +RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + let(:stamp) { "2026-11-13 12:00:00" } + + # Every version of the households migrations that exists, in order. Phase 6 adds the third. + def versions = [ 2026_11_13_120001 ] + + def context = ActiveRecord::Base.connection_pool.migration_context + + def down_all = versions.reverse_each { |version| context.run(:down, version) } + + def up_all = versions.each { |version| context.run(:up, version) } + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + def sequence = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + + # What PRAGMA says about a table: columns, foreign keys, indexes, checks (not the table's SQL text). + def shape(table) + [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }, + connection.foreign_keys(table).map { [ it.to_table, it.column, it.on_delete ] }.sort_by(&:to_s), + connection.indexes(table).map { [ it.name, it.columns, it.unique, it.where ] }.sort_by(&:to_s), + connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] + end + + def insert(sql, *binds) = connection.insert(ActiveRecord::Base.sanitize_sql_array([ sql, *binds ])) + + def insert_household(name = "Ann's household") = insert("INSERT INTO households (name, created_at, updated_at) VALUES (?, ?, ?)", name, stamp, stamp) + + def insert_membership(household, user, role) = insert("INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", household, user, role, stamp, stamp) + + # A user with an account in a household of its own; answers [user id, account id, household id]. + def insert_user(number = 1) + household = insert_household("User #{number}'s household") + account = insert("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (?, ?, ?)", stamp, stamp, household) + user = insert("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (?, ?, ?, 'x', ?, ?)", account, "User #{number}", "u#{number}@example.test", stamp, stamp) + [ user, account, household ] + end + + around do |example| + ActiveRecord::Migration.verbose = false + @before = shape(:accounts) + example.run + ensure + up_all unless connection.table_exists?(:households) + Account.reset_column_information + ActiveRecord::Migration.verbose = true + end + + it "turns the migration's transaction off, so its table copy runs with foreign keys off" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "creates the two tables with their keys, indexes and checks", :aggregate_failures do + expect(connection.columns(:households).map(&:name)).to eq(%w[id name created_at updated_at]) + expect(connection.columns(:memberships).map(&:name)).to eq(%w[id household_id user_id role created_at updated_at]) + expect(connection.check_constraints(:households).map(&:name)).to eq([ "households_name_length" ]) + expect(connection.check_constraints(:memberships).map(&:name)).to eq([ "memberships_role_known" ]) + expect(connection.foreign_keys(:memberships).to_h { [ it.to_table, it.on_delete ] }).to eq("households" => nil, "users" => :cascade) + expect(connection.indexes(:memberships).map { [ it.columns, it.unique ] }).to contain_exactly([ %w[household_id], false ], [ %w[user_id], true ]) + end + + it "adds household_id and kind to accounts, with a no-action key, an index and a named check", :aggregate_failures do + expect(connection.foreign_keys(:accounts).map { [ it.to_table, it.column, it.on_delete ] }).to include([ "households", "household_id", nil ]) + expect(connection.index_exists?(:accounts, :household_id)).to be(true) + kind = connection.columns(:accounts).find { it.name == "kind" } + expect([ kind.null, kind.default ]).to eq([ false, "personal" ]) + expect(connection.check_constraints(:accounts).map(&:name)).to include("accounts_kind_known") + end + + it "refuses a role that is neither manager nor member, and a second membership for a user (AC-2.1, AC-2.2)", :aggregate_failures do + user, _account, household = insert_user + expect { insert_membership(household, user, "boss") }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) + insert_membership(household, user, "manager") + expect { insert_membership(insert_household("Other"), user, "member") }.to raise_error(ActiveRecord::RecordNotUnique) + end + + it "refuses an account kind that is neither personal nor shared (AC-2.5)" do + _user, account, = insert_user + expect { connection.execute("UPDATE accounts SET kind = 'other' WHERE id = #{account}") }.to raise_error(ActiveRecord::StatementInvalid, /accounts_kind_known/) + end + + it "refuses a household name that is blank or over 100 characters, and keeps one of exactly 100 (AC-2.7)", :aggregate_failures do + expect { insert_household("") }.to raise_error(ActiveRecord::StatementInvalid, /households_name_length/) + expect { insert_household("a" * 101) }.to raise_error(ActiveRecord::StatementInvalid, /households_name_length/) + expect { insert_household("a" * 100) }.not_to raise_error + end + + it "refuses to delete a household that holds an account or a member, and deletes a membership with its user (AC-2.9)", :aggregate_failures do + user, _account, household = insert_user + insert_membership(household, user, "manager") + expect { connection.execute("DELETE FROM households WHERE id = #{household}") }.to raise_error(ActiveRecord::InvalidForeignKey) + connection.execute("DELETE FROM users WHERE id = #{user}") + expect([ count("memberships"), count("households") ]).to eq([ 0, 1 ]) + end + + it "runs again with nothing left to do", :aggregate_failures do + insert_user + before = [ count("households"), count("accounts") ] + expect { described_class.new.migrate(:up) }.not_to raise_error + expect([ count("households"), count("accounts") ]).to eq(before) + end + + it "returns to the schema before it going down, and keeps every account row and its id sequence going down and up (AC-3.5)", :aggregate_failures do + insert_user(1) + insert_user(2) + _user, last_account, = insert_user(3) + connection.execute("DELETE FROM users WHERE account_id = #{last_account}") + connection.execute("DELETE FROM accounts WHERE id = #{last_account}") + rows = connection.select_rows("SELECT id, created_at, updated_at, currency FROM accounts ORDER BY id") + seq = sequence + expect(seq).to be > rows.last.first # a copy without a restore would lower it + + down_all + expect(connection.columns(:accounts).map(&:name)).to eq(%w[id created_at updated_at currency]) + expect([ connection.table_exists?(:households), connection.table_exists?(:memberships) ]).to eq([ false, false ]) + expect(sequence).to eq(seq) + up_all + + expect(connection.select_rows("SELECT id, created_at, updated_at, currency FROM accounts ORDER BY id")).to eq(rows) + expect(sequence).to eq(seq) + end + + it "leaves accounts, after down and up, in the shape the schema file gives it (AC-3.4)" do + down_all + up_all + expect(shape(:accounts)).to eq(@before) + end + + it "refuses to go down while a household has two members or a shared account exists, and changes nothing (AC-3.5)", :aggregate_failures do + one, _a, first = insert_user(1) + two, second_account, second = insert_user(2) + insert_membership(first, one, "manager") + insert_membership(second, two, "manager") + connection.execute("UPDATE memberships SET household_id = #{first} WHERE user_id = #{two}") + + expect { context.run(:down, 2026_11_13_120001) }.to raise_error(StandardError, /1 household with more than one member/) + expect(connection.table_exists?(:households)).to be(true) + + connection.execute("UPDATE memberships SET household_id = #{second} WHERE user_id = #{two}") + connection.execute("UPDATE accounts SET kind = 'shared' WHERE id = #{second_account}") + expect { context.run(:down, 2026_11_13_120001) }.to raise_error(StandardError, /1 shared account/) + expect(connection.column_exists?(:accounts, :kind)).to be(true) + end +end +``` + +- [ ] **1.2 Run it to see it fail:** `bin/rspec spec/migrations/create_households_and_memberships_spec.rb`. Expected: a `LoadError` (the migration file does not exist), 0 examples run. +- [ ] **1.3 Write the migration** `db/migrate/20261113120001_create_households_and_memberships.rb`: + +```ruby +# A household and its members, and the two columns that put an account in one (spec 090 decisions 1 and 5 (a), ADR 0030). +# Without a transaction, as AddDecks explains: dropping the two columns on the way down copies accounts, and with foreign keys +# enforced inside a transaction the copy's DROP TABLE would run the delete rules of the thirteen tables that reference it. Every +# step is guarded, so a run that stopped part way is finished by the next boot. Going down refuses while a household has two +# members or a shared account exists (an older release can't hold either), and keeps accounts' id sequence across each copy. +class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + def up + create_table :households, if_not_exists: true do |t| + t.string :name, null: false + t.timestamps + t.check_constraint "length(name) BETWEEN 1 AND 100", name: "households_name_length" + end + create_table :memberships, if_not_exists: true do |t| + t.references :household, null: false, foreign_key: true + t.references :user, null: false, index: { unique: true }, foreign_key: { on_delete: :cascade } + t.string :role, null: false + t.timestamps + t.check_constraint "role IN ('manager', 'member')", name: "memberships_role_known" + end + # ADD COLUMN copies no table: REFERENCES is accepted because the default is NULL, and a named CHECK on the column is + # tested against every row and kept by the schema dump (the precedent is AddDecks). + unless column_exists?(:accounts, :household_id) + execute "ALTER TABLE accounts ADD COLUMN household_id integer REFERENCES households (id)" + end + add_index :accounts, :household_id unless index_exists?(:accounts, :household_id) + unless column_exists?(:accounts, :kind) + execute "ALTER TABLE accounts ADD COLUMN kind varchar NOT NULL DEFAULT 'personal' " \ + "CONSTRAINT accounts_kind_known CHECK (kind IN ('personal', 'shared'))" + end + end + + def down + refuse_what_the_older_release_cannot_hold + connection.disable_referential_integrity do + connection.transaction do + keeping_sequence do + remove_check_constraint :accounts, name: "accounts_kind_known" if check_constraint_exists?(:accounts, name: "accounts_kind_known") + remove_column :accounts, :kind if column_exists?(:accounts, :kind) + end + keeping_sequence do + remove_index :accounts, :household_id if index_exists?(:accounts, :household_id) + remove_column :accounts, :household_id if column_exists?(:accounts, :household_id) + end + end + end + drop_table :memberships, if_exists: true + drop_table :households, if_exists: true + end + + private + def refuse_what_the_older_release_cannot_hold + shared_households = table_exists?(:memberships) ? select_value("SELECT COUNT(*) FROM (SELECT 1 FROM memberships GROUP BY household_id HAVING COUNT(*) > 1)").to_i : 0 + shared_accounts = column_exists?(:accounts, :kind) ? select_value("SELECT COUNT(*) FROM accounts WHERE kind = 'shared'").to_i : 0 + return if shared_households.zero? && shared_accounts.zero? + + raise ActiveRecord::IrreversibleMigration, + "#{shared_households} #{"household".pluralize(shared_households)} with more than one member and " \ + "#{shared_accounts} shared #{"account".pluralize(shared_accounts)} exist; remove them before rolling back" + end + + # A table copy drops and recreates accounts, which resets its row of sqlite_sequence to the highest id present, so the id of a + # deleted account could be issued again. The sequence is read before and restored after, never lowered, and the row is made + # when the copy left none (an empty table has none). + def keeping_sequence + seq = select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + yield + return unless seq + + execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" + end +end +``` + +- [ ] **1.4 Dump the schema and prepare the test database:** `bin/rails db:migrate` then `bin/rails db:test:prepare`. Check `git diff db/schema.rb`: the two tables, `accounts.household_id` and `accounts.kind` with their key, index and check, `add_foreign_key "accounts", "households"`, `add_foreign_key "memberships", "households"`, `add_foreign_key "memberships", "users", on_delete: :cascade`, and the version `2026_11_13_120001`. The prices structure file is untouched. +- [ ] **1.5 Run the spec to see it pass:** `bin/rspec spec/migrations/create_households_and_memberships_spec.rb`. Expected: 10 examples, 0 failures. If the shape example fails on a check's expression text, print both shapes in the failure and report it: do not weaken the example. +- [ ] **1.6 Commit:** `feat(db): households, memberships and the account's household and kind (spec 090 phase 1)`. + +--- + +## Phase 2: The models, and a household with every user + +**Implements:** FR-1, FR-2, decisions 3, 6, 7, 8 | **Satisfies:** AC-1.1 to AC-1.7, AC-2.1 to AC-2.9, AC-4.4 | **Interfaces:** `Household` (`name_for(user_name)`, `memberships`, `accounts`, `users`; refuses destroy while held); `Membership` (`role` enum `manager`/`member`, `last_manager?`; refuses a direct destroy and a demotion of the last manager); `Account#household`, `Account#kind` (enum `personal`/`shared`); `User#membership`; a user is made with an account in a new household and a manager membership; a user's delete removes the membership, the account, and the household when empty. + +- [ ] **2.1 Factories and the invariants helper.** In `spec/factories/accounts.rb` replace `factory :account` by: + +```ruby + factory :household do + sequence(:name) { |n| "Household #{n}" } + end + + factory :account do + household + end +``` +Create `spec/support/household_invariants.rb`: + +```ruby +# The invariants of a household, checked on the test database (spec 090 AC-2.8): the same four the upgrade proof's tier 1 checks. +module HouseholdInvariants + # Fails the example, naming the broken rule, unless every user has exactly one membership, every household with a member has a + # manager, every account has a household and a personal account's household is its user's. + def expect_household_invariants! + connection = ActiveRecord::Base.connection + expect(connection.select_values("SELECT users.id FROM users LEFT JOIN memberships ON memberships.user_id = users.id GROUP BY users.id HAVING COUNT(memberships.id) <> 1")).to be_empty, "a user without exactly one membership" + expect(connection.select_values("SELECT households.id FROM households WHERE EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id AND m.role = 'manager')")).to be_empty, "a household without a manager" + expect(connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL")).to be_empty, "an account without a household" + expect(connection.select_values("SELECT accounts.id FROM accounts JOIN users ON users.account_id = accounts.id JOIN memberships ON memberships.user_id = users.id WHERE accounts.kind = 'personal' AND accounts.household_id <> memberships.household_id")).to be_empty, "a personal account outside its user's household" + end +end + +RSpec.configure { |config| config.include HouseholdInvariants } +``` +- [ ] **2.2 Write the failing model specs.** `spec/models/household_spec.rb`: + +```ruby +require "rails_helper" + +RSpec.describe Household, type: :model do + describe ".name_for" do + it "is the user's name and \"'s household\", the name cut to its first 88 characters (spec 090 decision 2, AC-1.5)", :aggregate_failures do + expect(described_class.name_for("Ada")).to eq("Ada's household") + expect(described_class.name_for("A")).to eq("A's household") + long = described_class.name_for("é" * 100) + expect([ long.length, long ]).to eq([ 100, "#{"é" * 88}'s household" ]) + expect(described_class.name_for("O'Neil")).to eq("O'Neil's household") + end + end + + it "needs a name of 1 to 100 characters (AC-2.7)", :aggregate_failures do + expect(build(:household, name: "").tap(&:validate).errors[:name]).to be_present + expect(build(:household, name: "a" * 101).tap(&:validate).errors[:name]).to be_present + expect(build(:household, name: "a" * 100)).to be_valid + end + + it "can't be destroyed while it holds an account or a member, by the model or by the database (AC-2.9)", :aggregate_failures do + user = create(:user) + household = user.membership.household + + expect(household.destroy).to be(false) + expect(described_class.exists?(household.id)).to be(true) + expect { described_class.connection.execute("DELETE FROM households WHERE id = #{household.id}") }.to raise_error(ActiveRecord::InvalidForeignKey) + expect(create(:household).destroy).to be_truthy + end +end +``` +`spec/models/membership_spec.rb`: + +```ruby +require "rails_helper" + +RSpec.describe Membership, type: :model do + # Puts +member+ into +manager+'s household as a plain member, the way a later issue's join will, by SQL (spec only). + def join(member, manager) + described_class.where(user_id: member.id).update_all(household_id: manager.membership.household_id, role: "member") + Account.where(id: member.account_id).update_all(household_id: manager.membership.household_id) + member.reload + end + + let(:manager) { create(:user) } + let(:member) { join(create(:user), manager) } + + it "has a role of manager or member only (AC-2.1)", :aggregate_failures do + expect(described_class.new(role: "boss").tap(&:validate).errors[:role]).to be_present + expect { described_class.insert!({ household_id: manager.membership.household_id, user_id: create(:user).id, role: "boss" }) }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) + end + + it "is one per user (AC-2.2)", :aggregate_failures do + second = described_class.new(household: create(:household), user: manager, role: "member") + expect(second).not_to be_valid + expect { described_class.insert!({ household_id: create(:household).id, user_id: manager.id, role: "member" }) }.to raise_error(ActiveRecord::RecordNotUnique) + end + + it "can't be removed on its own, a household of one's included (AC-2.4)", :aggregate_failures do + expect(manager.membership.destroy).to be(false) + expect(member.membership.destroy).to be(false) + expect(described_class.count).to eq(2) + end + + it "keeps a manager: the last one can't be demoted, one of two can (AC-2.3)", :aggregate_failures do + member # joins manager's household + expect(manager.membership.update(role: "member")).to be(false) + expect(manager.membership.errors[:role]).to be_present + member.membership.update!(role: "manager") + expect(manager.membership.reload.update(role: "member")).to be(true) + end + + it "keeps a user from being deleted when they are the last manager of a household with others (AC-2.3, decision 7)", :aggregate_failures do + member + expect { manager.destroy! }.to raise_error(ActiveRecord::RecordNotDestroyed) + expect([ User.exists?(manager.id), described_class.exists?(user_id: manager.id) ]).to eq([ true, true ]) + end + + it "lets a plain member's user be deleted, leaving the household and its manager (decision 7)", :aggregate_failures do + household = member.membership.household + member.destroy! + expect([ Household.exists?(household.id), described_class.where(household_id: household.id).pluck(:user_id) ]).to eq([ true, [ manager.id ] ]) + end +end +``` +`spec/models/account_spec.rb`: append inside the top `RSpec.describe` before its last `end`: + +```ruby + describe "household and kind (spec 090)" do + it "needs a household (decision 8)" do + expect(described_class.new.tap(&:validate).errors[:household]).to be_present + end + + it "is personal by default and shared or personal only (AC-2.5)", :aggregate_failures do + expect(create(:account).kind).to eq("personal") + expect(build(:account, kind: "other")).not_to be_valid + expect { described_class.connection.execute("UPDATE accounts SET kind = 'other' WHERE id = #{create(:account).id}") }.to raise_error(ActiveRecord::StatementInvalid, /accounts_kind_known/) + end + end +``` +Append to `spec/models/user_spec.rb` (inside the describe): + +```ruby + describe "household (spec 090)" do + it "is made with a household named for it, an account in it and a manager membership (AC-1.1)", :aggregate_failures do + user = create(:user, name: "Ada") + + expect(user.membership).to have_attributes(role: "manager", household: have_attributes(name: "Ada's household")) + expect(user.account).to have_attributes(kind: "personal", household: user.membership.household) + expect(Household.count).to eq(1) + end + + it "gives a user with 100 characters a name of exactly 100 (AC-1.5)" do + expect(create(:user, name: "n" * 100).membership.household.name).to eq("#{"n" * 88}'s household") + end + + it "makes nothing for a user that is invalid (AC-1.6)", :aggregate_failures do + expect { build(:user, name: "").save }.not_to change { [ Household.count, Membership.count, Account.count ] } + end + + it "is refused when its membership is in another household than its account's (AC-2.6)", :aggregate_failures do + user = create(:user) + user.account.household = create(:household) + + expect(user).not_to be_valid + expect(user.errors[:membership]).to be_present + expect(create(:user)).to be_valid + end + + it "removes its membership, its account and its household when it is deleted (AC-4.4)", :aggregate_failures do + user = create(:user) + household = user.membership.household + + user.destroy! + + expect([ Household.exists?(household.id), Membership.exists?(user_id: user.id), Account.exists?(user.account_id) ]).to eq([ false, false, false ]) + end + + it "keeps the invariants however it is made and deleted (AC-2.8)" do + create(:user) + create(:admin).destroy! + expect_household_invariants! + end + end +``` +- [ ] **2.3 Write the failing request spec** `spec/requests/households_creation_spec.rb` (Story 1's four paths; AC-1.1 to AC-1.4, AC-1.6, AC-1.7, AC-4.4): + +```ruby +require "rails_helper" + +RSpec.describe "A household with every user (spec 090 Story 1)", type: :request do + let(:password) { "correct horse battery" } + + def sign_up(**extra) = post(registration_path, params: { user: { name: "Ann", email_address: "ann@example.test", password:, password_confirmation: password, **extra } }) + + def expect_a_household_of_one(user, named: "#{user.name}'s household") + expect(user.membership).to have_attributes(role: "manager", household: have_attributes(name: named)) + expect(user.account).to have_attributes(kind: "personal", household: user.membership.household) + expect(user.membership.household.memberships.count).to eq(1) + end + + it "gives the first visitor a household of one and lands them in the collection (AC-1.1)", :aggregate_failures do + sign_up + expect(response).to redirect_to(collection_path) + expect_a_household_of_one(User.sole) + expect_household_invariants! + end + + it "gives a person who signs up while sign-up is open their own household (AC-1.2)", :aggregate_failures do + first = create(:admin) + InstanceSetting.current.update!(sign_up_open: true) + sign_up + expect_a_household_of_one(User.find_by!(email_address: "ann@example.test")) + expect(first.membership.household.memberships.count).to eq(1) + end + + it "gives a user an admin adds a household of one and leaves the admin's own alone (AC-1.3)", :aggregate_failures do + admin = sign_in_as(create(:admin)) + household = admin.membership.household + post admin_users_path, params: { user: { name: "Bo", email_address: "bo@example.test", password: } } + expect(response).to redirect_to(admin_users_path) + expect_a_household_of_one(User.find_by!(email_address: "bo@example.test")) + expect(admin.reload.membership.household).to eq(household) + end + + it "gives the operator command's admin a household of one (AC-1.4)", :aggregate_failures do + user, = User::Command.call(email: "ops@example.test", password:) + expect_a_household_of_one(user, named: "ops's household") + end + + it "makes nothing for a sign-up that is invalid or loses the first-user race (AC-1.6)", :aggregate_failures do + create(:admin) + expect { post registration_path, params: { user: { name: "", email_address: "x@example.test", password:, password_confirmation: password } } }.not_to change { [ Household.count, Membership.count, Account.count ] } + end + + it "ignores a household, role or kind in the parameters of a sign-up and of an admin's form (AC-1.7)", :aggregate_failures do + other = create(:household) + sign_up(household_id: other.id, role: "member", kind: "shared") + ann = User.find_by!(email_address: "ann@example.test") + expect_a_household_of_one(ann) + expect(ann.membership.household).not_to eq(other) + + sign_in_as(create(:admin)) + post admin_users_path, params: { user: { name: "Bo", email_address: "bo@example.test", password:, household_id: other.id, role: "member", kind: "shared" } } + expect_a_household_of_one(User.find_by!(email_address: "bo@example.test")) + end + + it "removes a deleted user's household, membership and account, and no other (AC-4.4)", :aggregate_failures do + admin = sign_in_as(create(:admin)) + bo = create(:user, name: "Bo") + delete admin_user_path(bo) + expect(response).to redirect_to(admin_users_path) + expect([ Household.where(name: "Bo's household").count, Membership.where(user_id: bo.id).count, Account.where(id: bo.account_id).count ]).to eq([ 0, 0, 0 ]) + expect(admin.reload.membership).to be_present + expect_household_invariants! + end +end +``` +- [ ] **2.4 Run the new specs to see them fail:** `bin/rspec spec/models/household_spec.rb spec/models/membership_spec.rb spec/models/account_spec.rb spec/models/user_spec.rb spec/requests/households_creation_spec.rb`. Expected: failures (`uninitialized constant Household`, undefined methods). +- [ ] **2.5 Write the models.** `app/models/household.rb`: + +```ruby +# A named group of accounts: today always one user's, from a later issue also a family's (spec 090, ADR 0030). It is the +# household's, not the collector's, so it is never scoped through `Current.account` and no request names one. +class Household < ApplicationRecord + NAME_LIMIT = 100 + # The name of a household of one is the user's name followed by this: 12 characters, so the user's name is cut to 88. + SUFFIX = "'s household".freeze + + has_many :memberships, dependent: nil # the foreign key has no action: a household that holds a member can't be deleted + has_many :users, through: :memberships + has_many :accounts, dependent: nil # likewise, so a delete can never reach an account and, through its cascades, a collection + + validates :name, presence: true, length: { maximum: NAME_LIMIT } + + before_destroy :refuse_while_held, prepend: true + + # The name a household of one gets (spec 090 decision 2): the user's first 88 characters and "'s household", at most 100. + # `script/upgrade/households.rb` and the backfill migration build the same name in SQL with `substr`; a spec holds the three equal. + # + # @param user_name [String] the user's name + # @return [String] + def self.name_for(user_name) = "#{user_name.to_s.slice(0, NAME_LIMIT - SUFFIX.length)}#{SUFFIX}" + + private + def refuse_while_held + throw :abort if memberships.exists? || accounts.exists? + end +end +``` +`app/models/membership.rb`: + +```ruby +# One user in one household, as a manager or a member (spec 090, ADR 0030). A household always has a manager: the model +# refuses to demote the last one and to remove a membership on its own; a membership goes with its user (`User` deletes it +# without these callbacks, and the database cascades). The role check and the unique user are the database's too. +class Membership < ApplicationRecord + belongs_to :household + belongs_to :user + + enum :role, { manager: "manager", member: "member" }, validate: true + + validates :user_id, uniqueness: true, allow_nil: true + validate :keeps_a_manager, on: :update, if: -> { role_changed?(from: "manager") } + + before_destroy { throw :abort } + + # Whether this is its household's only manager. + # + # @return [Boolean] + def last_manager? = manager? && !household.memberships.manager.where.not(id:).exists? + + private + def keeps_a_manager + errors.add(:role, "can't change: a household needs a manager.") unless household.memberships.manager.where.not(id:).exists? + end +end +``` +Edit `app/models/account.rb`: after `has_one :user, dependent: nil` add `belongs_to :household` with a comment, and after `validates :currency …` add the kind: + +```ruby + # The household this account is in: a user's own account (personal) is in that user's household, and a shared account (none + # exists yet, #384) is the household's. Required, and `NOT NULL` once spec 090's third migration has run. + belongs_to :household + enum :kind, { personal: "personal", shared: "shared" }, default: :personal, validate: true +``` +Edit `app/models/user.rb`: replace `belongs_to :account, dependent: :destroy` through the callbacks as follows (keep everything else): + +```ruby + has_secure_password + # Deleted without callbacks (Membership refuses a direct destroy; the database would cascade it anyway), before the user's row. + has_one :membership, dependent: :delete, inverse_of: :user + belongs_to :account, dependent: :destroy + has_many :sessions, dependent: :delete_all + ... + validates :membership, presence: true, on: :create + validate :membership_is_in_the_accounts_household + + before_validation :build_account_and_household, on: :create, unless: :account + before_destroy :keep_a_manager_for_the_others, prepend: true + after_destroy :remove_an_empty_household # after the account's, which `belongs_to … dependent: :destroy` registers above +``` +and in `private`: + +```ruby + # A user is made with its account and its household of one, as the manager (spec 090 decision 8): one write, so a failed + # save leaves none of the three. + def build_account_and_household + household = Household.new(name: Household.name_for(name)) + build_account(household:) + build_membership(household:, role: :manager) + end + + def membership_is_in_the_accounts_household + return unless membership && account + + errors.add(:membership, "must be in the household of the account") unless membership.household == account.household + end + + # A household can't lose its last manager by deleting the user while others are in it (spec 090 decision 7); a page that + # makes such a household (#382, #383) gives it a message. Remembers the household for the delete's last step. + def keep_a_manager_for_the_others + @leaving_household = membership&.household + return unless membership&.last_manager? && membership.household.memberships.where.not(id: membership.id).exists? + + errors.add(:base, "is the last manager of a household that has other members.") + throw :abort + end + + # The household of a user whose delete left no member and no account goes with them; `Household` refuses while it holds either. + def remove_an_empty_household = @leaving_household&.destroy +``` +- [ ] **2.6 Run the specs to see them pass:** same command as 2.4. Expected: 0 failures. Then `bin/rspec spec/requests spec/models` (no slot needed) to see nothing else broke; fix a factory-made account in any spec that needs the household (only the three named in the spec's decision 9 (ii) use `create(:account)`, now with a household). +- [ ] **2.7 Commit** in two commits: `feat(models): household and membership, an account's household and kind (spec 090 phase 2)` (models, factories, helper, model specs) and `feat(auth): every way of making a user makes a household of one (spec 090 phase 2)` if the request spec was written after; otherwise one. + +--- + +## Phase 3: Migration (b), the backfill + +**Implements:** FR-3, decision 5 (b) | **Satisfies:** AC-3.1, AC-3.2, AC-3.3, AC-3.4, AC-3.8, AC-1.5 (the SQL half) | **Interfaces:** an idempotent data migration; batches of 500 users; each batch one transaction; sets `accounts.household_id` only. + +- [ ] **3.1 Write the failing spec** `spec/migrations/backfill_households_spec.rb`: + +```ruby +require "rails_helper" +require Rails.root.join("db/migrate/20261113120002_backfill_households") + +# Spec 090 decision 5 (b), AC-3.1 to AC-3.3, AC-3.8, AC-1.5: a household of one for every user that has none, in batches, idempotent, +# touching accounts' household_id only. The users are made with the models and their household rows then removed by SQL, which is the +# state a database from the previous release is in once (a) has run. +RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + + def run_up = described_class.new.tap { it.verbose = false }.migrate(:up) + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + # Takes every household away, as the previous release leaves users and accounts. + def forget_households + connection.execute("DELETE FROM memberships") + connection.execute("UPDATE accounts SET household_id = NULL") + connection.execute("DELETE FROM households") + end + + def pairs = connection.select_rows("SELECT users.id, households.name, memberships.role, accounts.kind FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id JOIN accounts ON accounts.id = users.account_id AND accounts.household_id = households.id ORDER BY users.id") + + it "turns the migration's transaction off, so each batch is its own short write" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "gives every user a household named for them, a manager membership and an account in it, and changes no other column (AC-3.1)", :aggregate_failures do + users = Array.new(3) { |i| create(:user, name: "Collector #{i}") } + lot = create(:lot, account: users.first.account) + accounts = connection.select_rows("SELECT id, created_at, updated_at, currency, kind FROM accounts ORDER BY id") + forget_households + + run_up + + expect(pairs).to eq(users.map { [ it.id, "#{it.name}'s household", "manager", "personal" ] }) + expect(connection.select_rows("SELECT id, created_at, updated_at, currency, kind FROM accounts ORDER BY id")).to eq(accounts) + expect(count("lots")).to eq(1) + expect(Lot.find(lot.id).account_id).to eq(users.first.account_id) + end + + it "builds the name the model builds, for 100 characters, an apostrophe and multi-byte characters (AC-1.5)", :aggregate_failures do + names = [ "n" * 100, "O'Neil", "é" * 100, "A" ] + users = names.each_with_index.map { |name, i| create(:user, name:, email_address: "u#{i}@example.test") } + forget_households + + run_up + + expect(connection.select_values("SELECT households.name FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id ORDER BY users.id")) + .to eq(users.map { Household.name_for(it.name) }) + end + + it "adds nothing the second time and makes only the rest after a stop (AC-3.2)", :aggregate_failures do + first, second = Array.new(2) { create(:user) } + forget_households + # the state of a run that stopped after its first batch: the first user's household exists + household = connection.insert("INSERT INTO households (name, created_at, updated_at) VALUES ('kept', datetime('now'), datetime('now'))") + connection.execute("INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household}, #{first.id}, 'manager', datetime('now'), datetime('now'))") + connection.execute("UPDATE accounts SET household_id = #{household} WHERE id = #{first.account_id}") + + run_up + after = [ count("households"), count("memberships"), pairs ] + run_up + + expect([ count("households"), count("memberships"), pairs ]).to eq(after) + expect(connection.select_value("SELECT name FROM households WHERE id = #{household}")).to eq("kept") + expect(connection.select_value("SELECT COUNT(*) FROM memberships WHERE user_id = #{second.id}")).to eq(1) + end + + it "works in batches of at most 500, each its own transaction (AC-3.3)", :aggregate_failures do + expect(described_class::BATCH).to eq(500) + now = "2026-11-13 12:00:00" + stub_const("BackfillHouseholds::BATCH", 2) # 5 users make 3 batches + 5.times { |i| connection.execute("INSERT INTO accounts (created_at, updated_at) VALUES ('#{now}', '#{now}')") ; connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (#{connection.select_value("SELECT MAX(id) FROM accounts")}, 'U#{i}', 'u#{i}@example.test', 'x', '#{now}', '#{now}')") } + forget_households + batches = 0 + counter = ->(*, payload) { batches += 1 if payload[:sql].match?(/\ASELECT .*FROM users/i) } + + ActiveSupport::Notifications.subscribed(counter, "sql.active_record") { run_up } + + expect(count("households")).to eq(5) + expect(batches).to eq(4) # three batches with rows, one empty read that ends the loop + end + + it "succeeds on a database with no user (AC-3.4)" do + expect { run_up }.not_to raise_error + end + + it "leaves every session row as it was (AC-3.8)" do + user = create(:user) + session = user.sessions.create! + forget_households + rows = connection.select_rows("SELECT * FROM sessions ORDER BY id") + + run_up + + expect(connection.select_rows("SELECT * FROM sessions ORDER BY id")).to eq(rows) + expect(session.reload).to be_present + end +end +``` +Note for the implementer: the batch-count example reads `SELECT … FROM users` once per batch; if the migration's read differs, fix the example's regexp to count the reads of the unhoused users, not the migration to fit it. The example's `accounts` inserts rely on `household_id` being nullable and `kind` defaulted: in phase 6 (c) makes `household_id` NOT NULL, and phase 6 edits this one example (step 6.6). +- [ ] **3.2 Run it to see it fail** (`LoadError`), then **3.3 write the migration** `db/migrate/20261113120002_backfill_households.rb`: + +```ruby +# Gives every user that has no household one of their own, as its manager (spec 090 decision 5 (b), decision 2). Plain SQL and no +# application model (.claude/rules/sqlite-and-migrations.md). Without a transaction, so each batch of users is its own short write +# transaction: a large instance never holds the write lock for more than one batch, and a run that stopped is finished by the next +# boot. Idempotent: it reads only users with no membership. It sets accounts.household_id and nothing else, updated_at included, so +# the upgrade proof's comparison of accounts needs no expected change. The name is the user's first 88 characters and "'s household" +# (`Household.name_for`; a spec holds the two equal). Nothing to undo going down: the tables go with the first migration. +class BackfillHouseholds < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + BATCH = 500 + + def up + now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) + loop do + users = select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") + break if users.empty? + + connection.transaction do + users.each do |user_id, account_id| + execute "INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}" + household = select_value("SELECT last_insert_rowid()") + execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" + execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" + end + end + end + end + + def down; end +end +``` +(`BATCH.to_i`, `user_id.to_i` are integers from the database or a constant: the interpolations are safe and say so in a trailing comment as `BackfillSessionsLastUsedAt` does.) +- [ ] **3.4 Run the spec:** `bin/rspec spec/migrations/backfill_households_spec.rb`. Expected: 7 examples, 0 failures. Then `bin/rails db:migrate` (the development database holds users: check `bin/rails runner 'puts [User.count, Household.count].inspect'` prints equal counts) and `git diff db/schema.rb` (only the version line changes). +- [ ] **3.5 Commit:** `feat(db): backfill a household of one for every existing user (spec 090 phase 3)`. + +--- + +## Phase 4: Nothing looks different, the export and the round trip + +**Implements:** FR-5 (format document), decision 4, decision 9 | **Satisfies:** AC-4.1, AC-4.3, AC-4.5, AC-5.1, AC-5.2, AC-5.3, AC-5.4 | **Interfaces:** the Coverage tables list `memberships` (table, "Left out"), `accounts.household_id` and `accounts.kind` (columns, "Left out"); the format stays version 4. + +- [ ] **4.1 Failing spec for the invisible change.** Create `spec/requests/households_invisible_spec.rb`: + +```ruby +require "rails_helper" + +# Spec 090 Story 4, decision 9 (i): no page says "household", and no request of these pages reads a household or a membership. +RSpec.describe "A household is invisible (spec 090 Story 4)", type: :request do + let!(:admin) { create(:admin, name: "Ann") } + let(:paths) { [ collection_path, admin_users_path, more_path, catalog_entries_path ] } + + def statements_naming(pattern, &) + seen = [] + ActiveSupport::Notifications.subscribed(->(*, payload) { seen << payload[:sql] if payload[:sql].match?(pattern) }, "sql.active_record", &) + seen + end + + it "shows no page of a collector or an admin the word, and runs no query naming the new tables (AC-4.1, AC-4.5)", :aggregate_failures do + sign_in_as(admin) + seen = statements_naming(/\b(households|memberships)\b/i) do + paths.each do |path| + get path + expect(response).to have_http_status(:ok), "#{path} answered #{response.status}" + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + expect(seen).to be_empty + end + + it "shows no signed-out page the word (AC-4.1)", :aggregate_failures do + [ new_session_path, new_registration_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + + it "lists users as before: a row, its copies and the same buttons (AC-4.5)", :aggregate_failures do + sign_in_as(admin) + bo = create(:user, name: "Bo") + create(:lot, account: bo.account, quantity: 3) + get admin_users_path + expect(response.body).to include("Bo", '3') + expect(response.body.downcase).not_to include("household") + end +end +``` +Run it: it should pass at once (nothing visible was added); that is the point of this check. Record in `verification.md` that it passed on the first run. Then **4.2 the one-off checks** (recorded in `verification.md`, no commit): `git grep -n -i 'household\|membership' -- app/views app/helpers app/controllers app/javascript config/locales ':!.kamal/secrets'` prints nothing; `bin/rails routes > "$TMPDIR/routes_branch.txt"`, then the same on `main` in the main checkout is not allowed (read-only worktree rule): compare with `git show main:config/routes.rb` against `config/routes.rb` (`git diff main -- config/routes.rb` is empty) and `bin/rails routes | wc -l` equal to the line count recorded from the first `bin/rails routes` run in phase 0 (add that run to step 0.1's record). +- [ ] **4.3 Failing spec edits for the coverage guard** (`spec/export_format_document_spec.rb`): in the `"marks each table as the spec rules"` example add `memberships` to the `%w[...]` list of "Left out" tables (alphabetical: `… import_uploads memberships number_entries …`); in the columns example add `accounts.household_id accounts.kind` after `accounts.updated_at` in the left-out list. Run `bin/rspec spec/export_format_document_spec.rb`: the coverage examples fail (the document lacks the rows). +- [ ] **4.4 Edit `docs/export-format/README.md`.** In the Coverage tables: add the row `| \`memberships\` | Left out | A household's membership of a user: a fact about a group, not about the collection. A household of one is made with every user, so a person who imports their file is given one by the same rule (spec 090 decision 4). |` between `import_uploads` and `number_entries` (the table is alphabetical after `accounts`, `lots`, ... as the file lists; keep its order of "Exported" rows first, then "Left out" alphabetically); in the Column table add after `accounts.updated_at`: `| \`accounts.household_id\` | Left out | An internal id of a group. A file is one account's collection, and the household is not in it. |` and `| \`accounts.kind\` | Left out | Whether the account is a member's own or the household's. A file is one account's collection, and every account that has a file is personal until the shared collection (#384) decides its own file. |`. After the Coverage tables add the sentence: "`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4)." Replace the sentence at the Versions history (line 336 area) "One change is already expected: collections shared by a household will change who owns a lot and will raise the version ([ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md))." by "One change is already expected: the household's shared collection has its own file and will raise the version ([ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md), issue #384). A household itself, one per user from spec 090, is not in the file and raised no version." Edit `docs/architecture/data-portability.md` line 47's sentence to add: "`memberships` and `accounts.household_id` and `.kind` were listed as left out with spec 090; `households` has neither key and is outside the guard." Run `bin/rspec spec/export_format_document_spec.rb` (expected green) and `bin/rspec spec/models/export_format spec/models/collection_export_spec.rb` (green). +- [ ] **4.5 Failing spec for the round trip and the backup.** Create `spec/models/collection_import/household_untouched_spec.rb`: + +```ruby +require "rails_helper" + +# Spec 090 AC-5.2: an apply writes the collection and the currency and never the account's household, kind or membership. +RSpec.describe "Applying an export and the household (spec 090 AC-5.2)", type: :model do + let(:source) { create(:user, name: "Source", email_address: "source@example.test") } + let(:target) { create(:user, name: "Target", email_address: "target@example.test") } + + def export(user) = JSON.parse(CollectionExport.new(account: user.account, at: Time.utc(2026, 11, 13, 12)).to_json) + + def apply!(user, data) + upload = store_import!(account: user.account, data:) + CollectionImport::Apply.new(account: user.account.reload, fingerprint: upload&.fingerprint, confirmed: true).call + end + + def household_facts(user) + user.reload + [ user.membership.household_id, user.membership.role, user.account.household_id, user.account.kind, Household.pluck(:id, :name).sort ] + end + + before { held_lot(source.account, import_printing, quantity: 3, finish: "foil") } + + it "changes no household, role or kind, whether the file restores a collection or replaces one (AC-5.2)", :aggregate_failures do + document = export(source) + expect(document.keys).not_to include("household", "households", "memberships") + before = [ household_facts(source), household_facts(target) ] + + apply!(target, document) + apply!(source, document.merge("lots" => [])) + + expect([ household_facts(source), household_facts(target) ]).to eq(before) + expect(target.account.lots.sum(:quantity)).to eq(3) + end +end +``` +It passes as soon as written (the apply never wrote them); record that in `verification.md` (a guard, not a fix). **Backup (AC-5.3):** `bin/rspec spec/backup_guards_spec.rb spec/backup_upgrade_spec.rb spec/lib/collector/backup` expected green and unedited (the primary database is held whole; the households are in it). If an example inserts accounts by SQL and fails on the model's new rules, note it for phase 6 (the SQL inserts at `spec/support/backup_helpers.rb:40` and `spec/lib/collector/backup/snapshot_spec.rb:12` need no change while `household_id` is nullable). +- [ ] **4.6 Run the tenancy guard:** `bin/rspec spec/requests` (no edits to any existing request spec; expected: 0 failures), then `bin/rspec spec/system -e "sign" --dry-run` is not needed: system specs run in the one CI run. Commit: `test(households): nothing looks different, the format's coverage and the round trip (spec 090 phase 4)` and, separately, `docs(export-format): list the household's tables as left out (spec 090 phase 4)`. + +--- + +## Phase 5: The upgrade proof checks the households + +**Implements:** FR-4, decision 10 | **Satisfies:** AC-6.1 to AC-6.5, AC-3.8 (the proof's behaviour leg) | **Interfaces:** `Upgrade::Households.problems(db, backfill: false)` answers a list of strings, empty when the snapshot holds the invariants; the harness raises `Failure` with `AC-6.1` on problems in `compare` (upgraded and restored snapshots) and in `next_release`. + +- [ ] **5.1 Failing spec** `spec/script/upgrade/households_spec.rb`: + +```ruby +require "rails_helper" +require Rails.root.join("script/upgrade/households").to_s + +# Spec 090 decision 10 and AC-6.5: the two tiers of the check on fixture snapshots, never on the application. +RSpec.describe Upgrade::Households, :upgrade do + # The three names of AC-1.5, so this check, the model and the backfill agree on the same cases. + let(:names) { [ "n" * 100, "O'Neil", "é" * 100 ] } + + def snapshot(users:, memberships: nil, households: nil, accounts: nil) + path = File.join(upgrade_dir, "snap#{SecureRandom.hex(3)}.sqlite3") + build_database(path) do |db| + db.execute_batch(<<~SQL) + CREATE TABLE households (id INTEGER PRIMARY KEY, name TEXT); + CREATE TABLE memberships (id INTEGER PRIMARY KEY, household_id INTEGER, user_id INTEGER, role TEXT); + CREATE TABLE accounts (id INTEGER PRIMARY KEY, household_id INTEGER, kind TEXT); + CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, account_id INTEGER); + SQL + users.each_with_index do |name, i| + id = i + 1 + db.execute("INSERT INTO households VALUES (?, ?)", [ id, "#{name[0, 88]}'s household" ]) unless households == :none + db.execute("INSERT INTO accounts VALUES (?, ?, 'personal')", [ id, id ]) + db.execute("INSERT INTO users VALUES (?, ?, ?)", [ id, name, id ]) + db.execute("INSERT INTO memberships VALUES (?, ?, ?, 'manager')", [ id, id, id ]) unless memberships == :none + end + yield db if block_given? + end + open_database(path) + end + + it "passes a clean snapshot in both tiers (AC-6.1)" do + expect(described_class.problems(snapshot(users: names), backfill: true)).to be_empty + end + + it "names a user with no membership (AC-6.5)" do + expect(described_class.problems(snapshot(users: names, memberships: :none))).to include(a_string_matching(/user .* membership/)) + end + + it "names a household with no manager (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE memberships SET role = 'member' WHERE id = 1") } + expect(described_class.problems(db)).to include(a_string_matching(/household 1 .*manager/)) + end + + it "names an account in another user's household (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE accounts SET household_id = 2 WHERE id = 1") } + expect(described_class.problems(db)).to include(a_string_matching(/account 1 .*household/)) + end + + it "names an account with no household (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE accounts SET household_id = NULL WHERE id = 3") } + expect(described_class.problems(db)).to include(a_string_matching(/account 3 .*no household/)) + end + + it "checks the backfill's facts only when asked: a name that differs, a count that differs, a role or a kind that differs (AC-6.5)", :aggregate_failures do + renamed = snapshot(users: names) { it.execute("UPDATE households SET name = 'Someone else' WHERE id = 2") } + expect(described_class.problems(renamed)).to be_empty + expect(described_class.problems(renamed, backfill: true)).to include(a_string_matching(/name/)) + extra = snapshot(users: names) { it.execute("INSERT INTO households VALUES (9, 'x'); INSERT INTO memberships VALUES (9, 9, 99, 'manager')") } + expect(described_class.problems(extra, backfill: true)).to include(a_string_matching(/count/)) + member = snapshot(users: names) { it.execute("INSERT INTO memberships VALUES (10, 1, 98, 'member')") } + expect(described_class.problems(member, backfill: true)).to include(a_string_matching(/role/)) + shared = snapshot(users: names) { it.execute("UPDATE accounts SET kind = 'shared' WHERE id = 1") } + expect(described_class.problems(shared, backfill: true)).to include(a_string_matching(/kind/)) + end + + it "lets a joined household and a shared account pass tier 1 (it must outlive #381 and #384)" do + db = snapshot(users: names) do + it.execute("UPDATE memberships SET household_id = 1, role = 'member' WHERE user_id = 2") + it.execute("UPDATE accounts SET household_id = 1 WHERE id = 2") + it.execute("INSERT INTO accounts VALUES (10, 1, 'shared')") + end + expect(described_class.problems(db)).to be_empty + end + + it "fails a snapshot with no households table (AC-6.1)" do + path = File.join(upgrade_dir, "old.sqlite3") + build_database(path) { it.execute_batch("CREATE TABLE users (id INTEGER PRIMARY KEY)") } + expect(described_class.problems(open_database(path))).to eq([ "the snapshot has no households or memberships table" ]) + end +end +``` +(The block form of `snapshot` runs inside `build_database`'s open connection so the examples can adjust rows; `it` in a block is Ruby 3.4's implicit parameter, the repository's Ruby 4.0.7 accepts it.) +- [ ] **5.2 Run it to see it fail** (`LoadError`), then **5.3 write** `script/upgrade/households.rb`: + +```ruby +# The household facts of a snapshot of the primary database (spec 090 decision 10, ADR 0030). Tier 1 is the invariants of a +# household, true on every snapshot of every release from this one on, so it names no count: it must stay true when joined households +# and shared accounts exist (#381, #384). Tier 2 is the backfill's own facts, checked only when the old snapshot had no households. +# It reads the snapshot with SQLite directly, never through the application, as `compare.rb` does. +require_relative "compare" + +module Upgrade + module Households + SUFFIX = "'s household".freeze + + # What is wrong with the households of +db+, as sentences naming the rule and the first row. + # + # @param db [SQLite3::Database] a snapshot, opened read-only + # @param backfill [Boolean] also check the facts a backfill of a database with no households leaves (decision 10, tier 2) + # @return [Array] empty when the snapshot holds every rule checked + def self.problems(db, backfill: false) + tables = Compare.tables(db) + return [ "the snapshot has no households or memberships table" ] unless (%w[households memberships] - tables).empty? + + found = invariants(db) + found += backfilled(db) if backfill + found + end + + def self.invariants(db) + found = [] + db.execute("SELECT users.id, COUNT(memberships.id) FROM users LEFT JOIN memberships ON memberships.user_id = users.id GROUP BY users.id HAVING COUNT(memberships.id) <> 1 LIMIT 3").each do |id, count| + found << "user #{id} has #{count} memberships, not exactly one" + end + db.execute("SELECT id FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE household_id = households.id AND role = 'manager') LIMIT 3").flatten.each do |id| + found << "household #{id} has no manager" + end + db.execute("SELECT id FROM accounts WHERE household_id IS NULL OR household_id NOT IN (SELECT id FROM households) LIMIT 3").flatten.each do |id| + found << "account #{id} has no household" + end + db.execute("SELECT accounts.id FROM accounts JOIN users ON users.account_id = accounts.id JOIN memberships ON memberships.user_id = users.id WHERE accounts.kind = 'personal' AND accounts.household_id <> memberships.household_id LIMIT 3").flatten.each do |id| + found << "account #{id} is in a household other than its user's" + end + found + end + private_class_method :invariants + + def self.backfilled(db) + found = [] + users = db.get_first_value("SELECT COUNT(*) FROM users") + households = db.get_first_value("SELECT COUNT(*) FROM households") + found << "the households' count #{households} is not the users' #{users}" unless users == households + found << "a membership's role is not manager" if db.get_first_value("SELECT COUNT(*) FROM memberships WHERE role <> 'manager'").positive? + found << "an account's kind is not personal" if db.get_first_value("SELECT COUNT(*) FROM accounts WHERE kind <> 'personal'").positive? + db.execute("SELECT users.id FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id WHERE households.name <> substr(users.name, 1, 88) || '#{SUFFIX.gsub("'", "''")}' LIMIT 3").flatten.each do |id| + found << "the name of user #{id}'s household is not the form of spec 090 decision 2" + end + found + end + private_class_method :backfilled + end +end +``` +- [ ] **5.4 Run the spec** (`bin/rspec spec/script/upgrade/households_spec.rb`, 8 examples, 0 failures), then **5.5 failing harness examples** in `spec/script/upgrade/harness_spec.rb`: (1) change `seeded` (line 62 area) to take `households: true` and build the household tables and the extra columns: + +```ruby + def seeded(name, versions, lots: 2000, extra: "", households: true) + accounts = households ? "id INTEGER PRIMARY KEY, household_id INTEGER, kind TEXT DEFAULT 'personal'" : "id INTEGER PRIMARY KEY" + users = households ? "id INTEGER PRIMARY KEY, password_digest TEXT, name TEXT, account_id INTEGER" : "id INTEGER PRIMARY KEY, password_digest TEXT" + build_database(File.join(dir, "#{name}.sqlite3")) do |db| + db.execute_batch(<<~SQL) + CREATE TABLE accounts (#{accounts}); CREATE TABLE users (#{users}); + CREATE TABLE lots (id INTEGER PRIMARY KEY, account_id INTEGER); CREATE TABLE instance_settings (id INTEGER PRIMARY KEY, sign_up_open BOOLEAN); + CREATE TABLE schema_migrations (version TEXT PRIMARY KEY); + #{households ? "INSERT INTO accounts VALUES (1, 1, 'personal'), (2, 2, 'personal'); INSERT INTO users VALUES (1, 'x', 'Ada', 1), (2, 'y', 'Grace', 2);" : "INSERT INTO accounts VALUES (1), (2); INSERT INTO users VALUES (1, 'x'), (2, 'y');"} + INSERT INTO instance_settings VALUES (1, 1); + #{households ? "CREATE TABLE households (id INTEGER PRIMARY KEY, name TEXT); CREATE TABLE memberships (id INTEGER PRIMARY KEY, household_id INTEGER, user_id INTEGER, role TEXT); INSERT INTO households VALUES (1, 'Ada''s household'), (2, 'Grace''s household'); INSERT INTO memberships VALUES (1, 1, 1, 'manager'), (2, 2, 2, 'manager');" : ""} + WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < #{lots}) INSERT INTO lots SELECT i, 1 FROM n; + #{versions.map { "INSERT INTO schema_migrations VALUES ('#{it}');" }.join} + #{extra} + SQL + end + end +``` +(2) add three examples next to "proves a clean upgrade": one where the old is `seeded(..., households: false)` and the upgraded `seeded(...)` (a backfill: expect OK and the line `households: ok`), one where the upgraded's membership is deleted by `extra: "DELETE FROM memberships WHERE id = 2;"` (expect exit 1 and output matching `AC-6.1`), one where only the `next` snapshot breaks the rule (expect `AC-6.1`). Copy the structure of the nearest existing example that makes the upgraded database differ (find it with `grep -n "differs" spec/script/upgrade/harness_spec.rb`), including how it passes `databases:` to `FakeEngine`; use `pair(...)` for the prices files as the others do. Run: they fail (the harness does not check). +- [ ] **5.6 Wire the harness.** In `script/upgrade/harness.rb` add `require_relative "households"` next to the other requires; in `compare` after the `Compare.integrity` lines: + +```ruby + problems = Households.problems(upgraded, backfill: !Compare.tables(old).include?("households")) + raise Failure, "AC-6.1: #{problems.first(3).join("; ")}" if problems.any? + @out.puts "households: ok#{" (backfilled)" unless Compare.tables(old).include?("households")}" +``` +and in `next_release` after `Compare.seed_problems(database)`: + +```ruby + households = Households.problems(database) + raise Failure, "AC-6.1: the next snapshot: #{households.first(3).join("; ")}" if households.any? +``` +Run `bin/rspec spec/script/upgrade` (expected: all green, the three new examples included). Then in `spec/script/upgrade/seeds_spec.rb` add to the `next.rb` describe: + +```ruby + it "gives every seeded user a household of one, so households and memberships hold rows (spec 090 AC-6.4)", :aggregate_failures do + run_seed("next.rb") + expect([ Household.count, Membership.count ]).to eq([ User.count, User.count ]) + expect(Upgrade::SeedExceptions.load(seeds.join("../seed_exceptions.yml"), "next").entries).to be_empty + end +``` +Run `bin/rspec spec/script/upgrade/seeds_spec.rb`. `script/upgrade/expected_changes.yml` stays `changes: []` (AC-6.3: confirm with `git diff --stat -- script/upgrade/expected_changes.yml` empty). +- [ ] **5.7 Commit:** `test(upgrade): the proof checks every user's household on the upgraded, restored and next snapshots (spec 090 phase 5)`; docs of the harness in the commit body. Stop here for the phases that exist whatever the maintainer rules: **(a), (b), the models, the proof and their specs are complete and green without (c).** + +--- + +## Phase 6: Migration (c), `accounts.household_id` NOT NULL (ruled: take it) + +**Implements:** decision 5 (c) | **Satisfies:** AC-3.4, AC-3.5, AC-3.6, AC-3.7 | **Interfaces:** `accounts.household_id` is `NOT NULL`; a stragglers step, the copy and the sequence's restore are one transaction opened with enforcement already off. + +**Complexity Tracking:** the maintainer ruled (c) on 2026-11-13 on two conditions: the plan review's trace below, and `bin/upgrade-smoke` from the previous release built from its tag before the push (step 7.3). The phase stays the last of the migrations and nothing in phases 1 to 5 depends on it; if the trace finds the copy unsafe, stop and ask the coordinator before the nullable fallback is taken. **Trace to do in the plan review and again before step 6.3:** read `activerecord-8.1.4.1/lib/active_record/connection_adapters/sqlite3_adapter.rb` (`alter_table`, `copy_table`, `change_column_null`, `disable_referential_integrity`) and `.../sqlite3/database_statements.rb` (transaction mode), and confirm, line by line against the code below: (1) `connection.disable_referential_integrity { connection.transaction { … } }` sets `PRAGMA foreign_keys = OFF` before the transaction opens (it is a no-op after); (2) `alter_table`'s own `disable_referential_integrity` and `transaction` nest inside ours without re-enabling enforcement or committing early; (3) the copy preserves the named checks, the foreign key to `households`, the index and the unique and partial indexes of `accounts`; (4) the children's own keys are untouched (no table rename of a child); (5) `lots → accounts` and `users → accounts` are `NO ACTION` and the eleven others `CASCADE` (`db/schema.rb`); (6) what `PRAGMA foreign_key_check` Rails runs after the copy, if any, and that it passes with the stragglers step done. + +- [ ] **6.1 Failing spec** `spec/migrations/make_account_household_required_spec.rb`. Copy the model calls of `script/upgrade/seeds/next.rb` lines 33 to 60 (scanner sitting, bulk selection and marks, bulk removal, locations, deck, lot in a zone, proxy, number entry, import upload) into a `fill_every_referencing_table(user)` helper that makes a row in each of the thirteen tables: + +```ruby +require "rails_helper" +require Rails.root.join("db/migrate/20261113120003_make_account_household_required") + +# Spec 090 decision 5 (c), AC-3.4 to AC-3.7: the copy of accounts must lose no row of the thirteen tables that reference it, keep their +# keys, keep the id sequence, give a straggler a household, and change nothing when it fails. It commits its rows and runs through the +# Migrator OUTSIDE any example transaction (inside one, enforcement stays on and the cascades would fire). +RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + let(:tables) { %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users] } + + def context = ActiveRecord::Base.connection_pool.migration_context + + def down = context.run(:down, 2026_11_13_120003) + + def up = context.run(:up, 2026_11_13_120003) + + def nullable? = connection.columns(:accounts).find { it.name == "household_id" }.null + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + def counts = (tables + %w[accounts households memberships sessions]).to_h { [ it, count(it) ] } + + def sequence = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + + def shapes = (%w[accounts] + tables).to_h { [ it, shape(it) ] } + + def shape(table) + [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }, + connection.foreign_keys(table).map { [ it.to_table, it.column, it.on_delete ] }.sort_by(&:to_s), + connection.indexes(table).map { [ it.name, it.columns, it.unique, it.where ] }.sort_by(&:to_s), + connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] + end + + # One row in each of the thirteen tables, as the upgrade seed makes them (script/upgrade/seeds/next.rb). + def fill_every_referencing_table(user) + # the plan's implementer copies the model calls of next.rb lines 33 to 60 here, with `ada.account` as `user.account` + end + + around do |example| + ActiveRecord::Migration.verbose = false + example.run + ensure + up if nullable? + Account.reset_column_information + ActiveRecord::Migration.verbose = true + end + + it "turns the migration's transaction off, so the copy runs with foreign keys off, and opens its own" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "makes the column NOT NULL and keeps every row of the thirteen tables, going down and up (AC-3.6)", :aggregate_failures do + fill_every_referencing_table(create(:user)) + before = counts + expect(before.values_at(*tables)).to all(be >= 1) + + down + expect(nullable?).to be(true) + expect(counts).to eq(before) + up + + expect(nullable?).to be(false) + expect(counts).to eq(before) + expect(connection.select_rows("PRAGMA foreign_key_check")).to be_empty + expect(connection.select_values("PRAGMA integrity_check")).to eq([ "ok" ]) + end + + it "keeps the foreign keys, checks and indexes of accounts and the thirteen tables, going down and up (AC-3.6)" do + before = shapes + down + up + expect(shapes).to eq(before) + end + + it "keeps the id sequence of accounts, so a deleted account's id is not issued again (AC-3.6)", :aggregate_failures do + create(:user) + gone = create(:user) + connection.execute("DELETE FROM memberships WHERE user_id = #{gone.id}") + gone_account = gone.account_id + connection.execute("DELETE FROM users WHERE id = #{gone.id}") + connection.execute("DELETE FROM accounts WHERE id = #{gone_account}") + seq = sequence + expect(seq).to be >= gone_account + + down + expect(sequence).to eq(seq) + up + expect(sequence).to eq(seq) + expect(connection.insert("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (datetime('now'), datetime('now'), #{create(:household).id})")).to be > seq + end + + it "gives a user made between (b) and (c) a household, removes an emptied household, and succeeds (AC-3.7)", :aggregate_failures do + straggler = create(:user, name: "Late") + down + connection.execute("DELETE FROM memberships WHERE user_id = #{straggler.id}") + connection.execute("UPDATE accounts SET household_id = NULL WHERE id = #{straggler.account_id}") + emptied = connection.insert("INSERT INTO households (name, created_at, updated_at) VALUES ('Left behind', datetime('now'), datetime('now'))") + + up + + expect(connection.select_value("SELECT households.name FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id WHERE users.id = #{straggler.id}")).to eq("Late's household") + expect(connection.select_value("SELECT COUNT(*) FROM households WHERE id = #{emptied}")).to eq(0) + expect { up }.not_to raise_error # a second run changes nothing + end + + it "changes nothing when it fails, and succeeds on the next run (AC-3.6, the Error Scenarios row)", :aggregate_failures do + create(:user) + straggler = create(:user, name: "Late") + down + connection.execute("DELETE FROM memberships WHERE user_id = #{straggler.id}") + connection.execute("UPDATE accounts SET household_id = NULL WHERE id = #{straggler.account_id}") + orphan = connection.insert("INSERT INTO accounts (created_at, updated_at) VALUES (datetime('now'), datetime('now'))") # no user: no step can house it + before = counts.merge("sequence" => sequence) + + expect { up }.to raise_error(StandardError, /accounts? .*#{orphan}.* no user/m) + + expect(nullable?).to be(true) + expect(counts.merge("sequence" => sequence)).to eq(before) + expect(connection.select_value("SELECT COUNT(*) FROM memberships WHERE user_id = #{straggler.id}")).to eq(0) # the stragglers step was rolled back + connection.execute("DELETE FROM accounts WHERE id = #{orphan}") + expect { up }.not_to raise_error + expect(nullable?).to be(false) + end +end +``` +- [ ] **6.2 Run it to see it fail** (`LoadError`). **6.3 Re-trace the Rails code** as the Complexity Tracking note says, and record the six findings in the commit body. **6.4 Write the migration** `db/migrate/20261113120003_make_account_household_required.rb`: + +```ruby +# Makes accounts.household_id NOT NULL, so "every account has a household" is a fact of the database (spec 090 decision 5 (c)). +# SQLite can't add NOT NULL in place; Rails copies accounts (create the new table, copy the rows, drop the old, rename). accounts is +# referenced by thirteen tables, eleven of them ON DELETE CASCADE (lots and users are NO ACTION), so a copy run with foreign keys +# enforced deletes every collector's decks, places, proxies and working state, silently. Rails turns enforcement off for a copy only +# outside a transaction, so the migration runs without Rails' own (`disable_ddl_transaction!`) and opens one itself with enforcement +# already off: the stragglers step, the copy and the sequence's restore commit together or not at all, so a failed run changes nothing +# and the next boot runs it whole. The stragglers step is spec 049's safety-net UPDATE: BackfillHouseholds is recorded once it has +# run, but a user can be made after it (an older container serving during a deploy), and without this the NOT NULL would fail on +# every boot after. It also removes a household an older container's delete of a user left with no member and no account. An +# account with no user and no household can't be housed by any step: the migration says so and changes nothing. +# @see file:tenancy-and-accounts.md +class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + BATCH = 500 + + def up + return unless household_nullable? + + connection.disable_referential_integrity do + connection.transaction do + house_stragglers + remove_empty_households + refuse_accounts_nobody_can_house + keeping_sequence { change_column_null :accounts, :household_id, false } + end + end + end + + def down + return if household_nullable? + + connection.disable_referential_integrity do + connection.transaction { keeping_sequence { change_column_null :accounts, :household_id, true } } + end + end + + private + def household_nullable? = columns(:accounts).find { it.name == "household_id" }.null + + # The same SQL as BackfillHouseholds (migrations don't share code: a released one is never edited). + def house_stragglers + now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) + loop do + users = select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") + break if users.empty? + + users.each do |user_id, account_id| + execute "INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}" + household = select_value("SELECT last_insert_rowid()") + execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" + execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" + end + end + end + + def remove_empty_households + execute "DELETE FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM accounts WHERE accounts.household_id = households.id)" + end + + def refuse_accounts_nobody_can_house + orphans = select_values("SELECT id FROM accounts WHERE household_id IS NULL") + return if orphans.empty? + + raise ActiveRecord::MigrationError, "accounts #{orphans.first(5).join(", ")} have no household and no user, so no household can be made for them; " \ + "remove them (an account is made with its user and is deleted with it) and start again; nothing was changed" + end + + # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). + def keeping_sequence + seq = select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + yield + return unless seq + + execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" + end +end +``` +- [ ] **6.5 Dump and prepare:** `bin/rails db:migrate` and `bin/rails db:test:prepare`; `git diff db/schema.rb` shows `t.integer "household_id", null: false` and the new version `2026_11_13_120003`, and no other change to `accounts`, its foreign keys or the thirteen tables (any other line is a failure of the copy: stop and report). Run `bin/rspec spec/migrations/make_account_household_required_spec.rb` (6 examples, 0 failures). +- [ ] **6.6 Make the earlier specs follow the final shape.** In `spec/migrations/create_households_and_memberships_spec.rb` and `spec/migrations/backfill_households_spec.rb`, extend the versions so a spec's down and up cover (c): in the first, `def versions = [ 2026_11_13_120001, 2026_11_13_120002, 2026_11_13_120003 ]` and add `require` lines for the other two migrations; in the second, add to the `around` an `ensure` that reruns `context.run(:up, 2026_11_13_120003)` when `accounts.household_id` is nullable after an example that goes down. In `backfill_households_spec.rb`'s batch example, the raw `INSERT INTO accounts` rows get a household: insert a household first and pass `household_id` (the column is NOT NULL now); `forget_households` sets `household_id = NULL`, which the copy refuses: have it first `down` (c) when the column is NOT NULL (`context.run(:down, 2026_11_13_120003)`) and `up` it in `ensure`. In `spec/support/backup_helpers.rb:40` and `spec/lib/collector/backup/snapshot_spec.rb:12` insert a household and give the account its id (`INSERT INTO households (name, created_at, updated_at) VALUES ('h', ?, ?)`, then `accounts (created_at, updated_at, household_id)`). Run `bin/rspec spec/migrations spec/lib/collector/backup spec/backup_guards_spec.rb spec/backup_upgrade_spec.rb` (green). +- [ ] **6.7 Commit:** `feat(db): accounts.household_id is NOT NULL, by a copy that loses no row (spec 090 phase 6)` and `test(db): the earlier households specs follow the final shape (spec 090 phase 6)`. + +--- + +## Phase 7: Documentation, evidence and the finish + +**Implements:** FR-5, decision 11, Story 6's evidence | **Satisfies:** AC-4.2, AC-6.6, the documentation criteria (verified by reading, `verification.md`) | **Interfaces:** none. + +- [ ] **7.1 `docs/architecture/tenancy-and-accounts.md`** (the source of each statement inline): in the Scope bullet "The tenant and its holder" add `Household`, `Membership` (spec 090; ADR 0030); Glossary: replace the Account entry's "created with its user and deleted with it" tail with "created with its user and deleted with it, and in a household (`accounts.household_id`) as a personal account (`accounts.kind`)", add **Household**, **Membership (manager, member)**, **Personal and shared account** entries and the note that "member" also means an admin page's non-admin (spec 090 Terms); the Tenant-owned list gains `memberships` (user-owned, instance-level) and says `households` and `memberships` carry no `account_id`, are never scoped through `Current.account` and no request names one; Invariants: change "One user per account" to the new rule ("`users.account_id` stays unique; a household is a group of accounts; a household always has a manager; a personal account is in its user's household; a household is made with its user by `User#build_account_and_household`" with `spec 090`, ADR 0030), add the cross-account reads sentence (the household tables are read by the model and the proof only); the "Deleting a user" paragraph gains the order (membership, user, account, household when empty) and the last-manager refusal; "Accepted limits" replaces "An account has exactly one user. Personal and household collections side by side will supersede that half of ADR 0017" with what exists now (a household of one for every user; the rest is #381 to #389); Sources gains spec 090, ADR 0030 and #380; Distillation log gains a row once the PR number exists (the distill-feature step writes it). +- [ ] **7.2** `.claude/rules/multi-tenancy.md`: add one line under the ✓ list: "✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request." `docs/architecture/release-and-deploy.md`: in the upgrade proof's paragraph add the two-tier households check (decision 10; `Upgrade::Households`; spec 090 AC-6.1 to AC-6.5) and that `expected_changes.yml` stays empty because `compare.rb` reads only the old database's columns. `docs/releasing.md`'s Upgrade notes: the entry for this migration with its warning (an older image can't create a user after it; going back is a restore of the backup taken before; a failed third migration stops the boot and changes nothing; Kamal's old container fails sign-up for seconds). `CLAUDE.md`: in the Project Status accounts sentence add "every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389)". `docs/adr/0030`: Status becomes Accepted. `docs/specs/README.md`: spec 004's row gains "Superseded by 090 in part (FR-3's one user per account)" and 090's row Status "Approved". +- [ ] **7.3 Evidence for `verification.md`** (create it with the headings the other specs use; write each result as it is produced): the phase 0 and each phase's spec runs with counts; the grep and the routes diff of step 4.2; **`bin/upgrade-smoke`**: build the previous release from its tag as `.claude/memory/container-engine-colima-macos.md` says (`git archive` of `v0.3.0` into its own directory, three `--build-arg` flags for the image: `README.md`, Building from source), build the code under test the same way, then `bin/upgrade-smoke ` with `TMPDIR` under the worktree and `/opt/homebrew/bin` appended to `PATH`; expected: the run ends `OK: v0.3.0` and prints `households: ok (backfilled)` for the upgraded and the restored snapshot and `households: ok` for the next one. **Ask the coordinator for the container engine and a slot before this step.** A failure is read, not re-run blindly (`.claude/memory/reproduce-ci-with-seed.md` for CI). +- [ ] **7.4 `bin/ci checks`** (a documentation-and-code branch: it runs RuboCop, YARD, the comment ratchet, Brakeman, bundler-audit, importmap audit and the migration guard): expected green; the comment ratchet needs a summary on each new public method (the models and `Upgrade::Households` have them). Fix and re-run. +- [ ] **7.5 The implementation review** (`sdd-review` Mode B, read-only, Opus 5.5) recorded in `verification.md` under `## Implementation review` with the verdict first; then `sdd-superpowers:session-wrap` and `distill-feature` (digest to the coordinator before commit); then one slot for the one full `bin/ci`; push; draft PR closing #380; CI green; `gh pr ready`; `gh pr edit --add-assignee @me`; `worker_done`. + +## Self-review + +- **FR coverage:** FR-1 (phases 1, 2), FR-2 (phase 2), FR-3 (phases 1, 3, 6), FR-4 (phase 5), FR-5 (phases 4, 7). +- **AC coverage:** AC-1.1 to 1.7 (2.2, 2.3, 3.1); AC-2.1 to 2.9 (1.1, 2.2); AC-3.1 to 3.3, 3.8 (3.1); AC-3.4 to 3.7 (1.1, 6.1); AC-4.1, 4.3, 4.5 (4.1, 4.6); AC-4.2 (4.2); AC-4.4 (2.2, 2.3); AC-5.1 to 5.4 (4.3 to 4.5); AC-6.1 to 6.5 (5.1 to 5.6); AC-6.6 (7.3). AC-3.4's "empty database" half is 3.1's last example. +- **Names:** `Household.name_for`, `Membership#last_manager?`, `Upgrade::Households.problems(db, backfill:)`, `expect_household_invariants!`, `BackfillHouseholds::BATCH` are used with the same signatures wherever they appear. From 85804585074d58a6b8a180dd44c9e49c5689675c Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 20:55:12 -0500 Subject: [PATCH 07/41] docs(specs): plan 090, plan review round 1 fixes (#380) The Fable review traced migration (c) against Rails 8.1.4.1's alter_table and found the copy safe. Fixed: the user's membership check no longer reads a household on an update; (c) checks enforcement is off and that no key is broken, and has a fault test after the DROP; the shape example compares literals; phases 1 and 2 are one batch; the backfill uses the connection, one insert id and counts its transactions; the upgrade smoke runs on the commit to be pushed. Co-Authored-By: Claude Sonnet 5.5 --- .../090-every-user-has-a-household/plan.md | 190 +++++++++++++----- .../090-every-user-has-a-household/spec.md | 2 +- 2 files changed, 138 insertions(+), 54 deletions(-) diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md index 0d8a9fe5..64ce7845 100644 --- a/docs/specs/090-every-user-has-a-household/plan.md +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -1,13 +1,13 @@ # Plan 090: Every User Has a Household -**Spec:** [spec.md](spec.md) v0.3.1 (Approved by the coordinator under the maintainer's delegation, 2026-11-13; the maintainer ruled step (c) the same day: take the copy). **ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md). **Issue:** #380. **Branch:** `090-every-user-has-a-household`. +**Spec:** [spec.md](spec.md) v0.3.2 (Approved by the coordinator under the maintainer's delegation, 2026-11-13; the maintainer ruled step (c) the same day: take the copy). **ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md). **Issue:** #380. **Branch:** `090-every-user-has-a-household`. **Written by:** the main session (Opus 5.5 from this step; the spec and ADR were written on Sonnet 5.5). Implementers: Sonnet 5.5 (`model: "sonnet"`), 2 to 4 consecutive tasks each, test first, one commit per task, reading this file by line range; the main session checks each batch's diff. ## Global Constraints - **No agent runs the `sqlite3` command**, `bin/rails console`, `db:rollback`, `db:migrate:down`, `db:seed`, anything with `RAILS_ENV=production`, `bundle update`, or anything else that prompts. Migrations go down and up **inside specs, through the Migrator** (`MigrationContext#run`), as `spec/migrations/add_decks_spec.rb` does. `bin/rails db:migrate` (development) writes `db/schema.rb`; `bin/rails db:test:prepare` then loads it into the test database. - Targeted `bin/rspec ` runs need no slot. Never two test runs at once (one shared test database). No full `bin/ci` and no push without a slot from the coordinator. -- Released migrations are never edited. The three new files are `20261113120001_create_households_and_memberships.rb`, `20261113120002_backfill_households.rb`, `20261113120003_make_account_household_required.rb` (phase 6; the only one that exists only if the maintainer rules (c)). None references an application model; all use `disable_ddl_transaction!` and guard each step. +- Released migrations are never edited. The three new files are `20261113120001_create_households_and_memberships.rb`, `20261113120002_backfill_households.rb`, `20261113120003_make_account_household_required.rb` (phase 6; ruled by the maintainer, 2026-11-13, and the last of the migrations). None references an application model; all use `disable_ddl_transaction!` and guard each step. - Every public method added in `app/models/`, `lib/`, `script/upgrade/` or `spec/support/` has a YARD summary (the comment ratchet); no `{…}` in comments; no collectible named in a core file's comments. - `household_id`, `role` and `kind` are never permitted from parameters (no controller changes at all: Story 4). - Commits: Conventional Commits, one per task, stage and commit in separate shell calls, ending with the trailer the session reminder gives. @@ -83,7 +83,7 @@ RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model # What PRAGMA says about a table: columns, foreign keys, indexes, checks (not the table's SQL text). def shape(table) - [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }, + [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }.sort_by(&:first), # by name: a copy moves a column connection.foreign_keys(table).map { [ it.to_table, it.column, it.on_delete ] }.sort_by(&:to_s), connection.indexes(table).map { [ it.name, it.columns, it.unique, it.where ] }.sort_by(&:to_s), connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] @@ -105,10 +105,9 @@ RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model around do |example| ActiveRecord::Migration.verbose = false - @before = shape(:accounts) example.run ensure - up_all unless connection.table_exists?(:households) + up_all # the Migrator skips a version that is recorded Account.reset_column_information ActiveRecord::Migration.verbose = true end @@ -187,10 +186,19 @@ RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model expect(sequence).to eq(seq) end - it "leaves accounts, after down and up, in the shape the schema file gives it (AC-3.4)" do + it "leaves accounts, households and memberships, after down and up, as the schema states them (AC-3.4)", :aggregate_failures do down_all up_all - expect(shape(:accounts)).to eq(@before) + + # The literals are what db/schema.rb dumps for the migration; if a check's expression text differs, correct the literal to the + # dumped text and say so, don't loosen the example. + expect(shape(:accounts)[1..]).to eq([ [ [ "households", "household_id", nil ] ], + [ [ "index_accounts_on_household_id", %w[household_id], false, nil ] ], [ [ "accounts_kind_known", "kind IN ('personal', 'shared')" ] ] ]) + expect(connection.columns(:accounts).map(&:name).sort).to eq(%w[created_at currency household_id id kind updated_at]) + expect(shape(:households)[1..]).to eq([ [], [], [ [ "households_name_length", "length(name) BETWEEN 1 AND 100" ] ] ]) + expect(shape(:memberships)[1..]).to eq([ [ [ "households", "household_id", nil ], [ "users", "user_id", :cascade ] ], + [ [ "index_memberships_on_household_id", %w[household_id], false, nil ], [ "index_memberships_on_user_id", %w[user_id], true, nil ] ], + [ [ "memberships_role_known", "role IN ('manager', 'member')" ] ] ]) end it "refuses to go down while a household has two members or a shared account exists, and changes nothing (AC-3.5)", :aggregate_failures do @@ -239,11 +247,11 @@ class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] # ADD COLUMN copies no table: REFERENCES is accepted because the default is NULL, and a named CHECK on the column is # tested against every row and kept by the schema dump (the precedent is AddDecks). unless column_exists?(:accounts, :household_id) - execute "ALTER TABLE accounts ADD COLUMN household_id integer REFERENCES households (id)" + connection.execute "ALTER TABLE accounts ADD COLUMN household_id integer REFERENCES households (id)" end add_index :accounts, :household_id unless index_exists?(:accounts, :household_id) unless column_exists?(:accounts, :kind) - execute "ALTER TABLE accounts ADD COLUMN kind varchar NOT NULL DEFAULT 'personal' " \ + connection.execute "ALTER TABLE accounts ADD COLUMN kind varchar NOT NULL DEFAULT 'personal' " \ "CONSTRAINT accounts_kind_known CHECK (kind IN ('personal', 'shared'))" end end @@ -268,8 +276,8 @@ class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] private def refuse_what_the_older_release_cannot_hold - shared_households = table_exists?(:memberships) ? select_value("SELECT COUNT(*) FROM (SELECT 1 FROM memberships GROUP BY household_id HAVING COUNT(*) > 1)").to_i : 0 - shared_accounts = column_exists?(:accounts, :kind) ? select_value("SELECT COUNT(*) FROM accounts WHERE kind = 'shared'").to_i : 0 + shared_households = table_exists?(:memberships) ? connection.select_value("SELECT COUNT(*) FROM (SELECT 1 FROM memberships GROUP BY household_id HAVING COUNT(*) > 1)").to_i : 0 + shared_accounts = column_exists?(:accounts, :kind) ? connection.select_value("SELECT COUNT(*) FROM accounts WHERE kind = 'shared'").to_i : 0 return if shared_households.zero? && shared_accounts.zero? raise ActiveRecord::IrreversibleMigration, @@ -281,18 +289,18 @@ class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] # deleted account could be issued again. The sequence is read before and restored after, never lowered, and the row is made # when the copy left none (an empty table has none). def keeping_sequence - seq = select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + seq = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") yield return unless seq - execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" - execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" + connection.execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + connection.execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" end end ``` -- [ ] **1.4 Dump the schema and prepare the test database:** `bin/rails db:migrate` then `bin/rails db:test:prepare`. Check `git diff db/schema.rb`: the two tables, `accounts.household_id` and `accounts.kind` with their key, index and check, `add_foreign_key "accounts", "households"`, `add_foreign_key "memberships", "households"`, `add_foreign_key "memberships", "users", on_delete: :cascade`, and the version `2026_11_13_120001`. The prices structure file is untouched. -- [ ] **1.5 Run the spec to see it pass:** `bin/rspec spec/migrations/create_households_and_memberships_spec.rb`. Expected: 10 examples, 0 failures. If the shape example fails on a check's expression text, print both shapes in the failure and report it: do not weaken the example. +- [ ] **1.4 Dump the schema and prepare the test database:** `bin/rails db:migrate` then `bin/rails db:test:prepare`. Also do steps 4.3 and 4.4 now, before the commit (they are phase 1's: without them `spec/export_format_document_spec.rb` fails). `spec/script/upgrade/seeds_spec.rb`'s example "fills every table db/schema.rb creates" stays red until step 2.5 (the seed makes users through the old model, which makes no household): phases 1 and 2 are one implementer's batch and the suite is green again at 2.6. Check `git diff db/schema.rb`: the two tables, `accounts.household_id` and `accounts.kind` with their key, index and check, `add_foreign_key "accounts", "households"`, `add_foreign_key "memberships", "households"`, `add_foreign_key "memberships", "users", on_delete: :cascade`, and the version `2026_11_13_120001`. The prices structure file is untouched. +- [ ] **1.5 Run the spec to see it pass:** `bin/rspec spec/migrations/create_households_and_memberships_spec.rb`. Expected: 11 examples, 0 failures. If the shape example fails on a check's expression text, print both shapes in the failure and report it: do not weaken the example. - [ ] **1.6 Commit:** `feat(db): households, memberships and the account's household and kind (spec 090 phase 1)`. --- @@ -317,12 +325,12 @@ Create `spec/support/household_invariants.rb`: ```ruby # The invariants of a household, checked on the test database (spec 090 AC-2.8): the same four the upgrade proof's tier 1 checks. module HouseholdInvariants - # Fails the example, naming the broken rule, unless every user has exactly one membership, every household with a member has a + # Fails the example, naming the broken rule, unless every user has exactly one membership, every household has a # manager, every account has a household and a personal account's household is its user's. def expect_household_invariants! connection = ActiveRecord::Base.connection expect(connection.select_values("SELECT users.id FROM users LEFT JOIN memberships ON memberships.user_id = users.id GROUP BY users.id HAVING COUNT(memberships.id) <> 1")).to be_empty, "a user without exactly one membership" - expect(connection.select_values("SELECT households.id FROM households WHERE EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id AND m.role = 'manager')")).to be_empty, "a household without a manager" + expect(connection.select_values("SELECT households.id FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id AND m.role = 'manager')")).to be_empty, "a household without a manager" expect(connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL")).to be_empty, "an account without a household" expect(connection.select_values("SELECT accounts.id FROM accounts JOIN users ON users.account_id = accounts.id JOIN memberships ON memberships.user_id = users.id WHERE accounts.kind = 'personal' AND accounts.household_id <> memberships.household_id")).to be_empty, "a personal account outside its user's household" end @@ -381,7 +389,9 @@ RSpec.describe Membership, type: :model do it "has a role of manager or member only (AC-2.1)", :aggregate_failures do expect(described_class.new(role: "boss").tap(&:validate).errors[:role]).to be_present - expect { described_class.insert!({ household_id: manager.membership.household_id, user_id: create(:user).id, role: "boss" }) }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) + bare = create(:user) + described_class.where(user_id: bare.id).delete_all # a user with no membership, by SQL + expect { described_class.insert!({ household_id: manager.membership.household_id, user_id: bare.id, role: "boss" }) }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) end it "is one per user (AC-2.2)", :aggregate_failures do @@ -491,6 +501,7 @@ RSpec.describe "A household with every user (spec 090 Story 1)", type: :request expect(user.membership).to have_attributes(role: "manager", household: have_attributes(name: named)) expect(user.account).to have_attributes(kind: "personal", household: user.membership.household) expect(user.membership.household.memberships.count).to eq(1) + expect_household_invariants! end it "gives the first visitor a household of one and lands them in the collection (AC-1.1)", :aggregate_failures do @@ -522,13 +533,24 @@ RSpec.describe "A household with every user (spec 090 Story 1)", type: :request expect_a_household_of_one(user, named: "ops's household") end - it "makes nothing for a sign-up that is invalid or loses the first-user race (AC-1.6)", :aggregate_failures do + it "makes nothing for a sign-up that is invalid: a blank name on an empty instance, a used email while sign-up is open (AC-1.6)", :aggregate_failures do + expect { sign_up(name: "") }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) + create(:admin, email_address: "ann@example.test") + InstanceSetting.current.update!(sign_up_open: true) + expect { sign_up }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) + end + + it "makes nothing for a sign-up that finds a user already there, as when it loses the first-user race (AC-1.6)", :aggregate_failures do + get new_registration_path create(:admin) - expect { post registration_path, params: { user: { name: "", email_address: "x@example.test", password:, password_confirmation: password } } }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect { sign_up }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) end it "ignores a household, role or kind in the parameters of a sign-up and of an admin's form (AC-1.7)", :aggregate_failures do - other = create(:household) + other = create(:user).membership.household # a household with a manager, so the invariants helper holds sign_up(household_id: other.id, role: "member", kind: "shared") ann = User.find_by!(email_address: "ann@example.test") expect_a_household_of_one(ann) @@ -623,12 +645,14 @@ Edit `app/models/user.rb`: replace `belongs_to :account, dependent: :destroy` th ```ruby has_secure_password # Deleted without callbacks (Membership refuses a direct destroy; the database would cascade it anyway), before the user's row. - has_one :membership, dependent: :delete, inverse_of: :user + has_one :membership, dependent: :delete, inverse_of: :user, autosave: true belongs_to :account, dependent: :destroy has_many :sessions, dependent: :delete_all ... validates :membership, presence: true, on: :create - validate :membership_is_in_the_accounts_household + # Only when both are loaded (a new user, or a user a caller loaded both for): an update of a signed-in user, whose account the + # session loads, must not read a household (spec 090 Story 4, AC-4.5). + validate :membership_is_in_the_accounts_household, if: -> { association(:membership).loaded? && association(:account).loaded? } before_validation :build_account_and_household, on: :create, unless: :account before_destroy :keep_a_manager_for_the_others, prepend: true @@ -664,7 +688,7 @@ and in `private`: # The household of a user whose delete left no member and no account goes with them; `Household` refuses while it holds either. def remove_an_empty_household = @leaving_household&.destroy ``` -- [ ] **2.6 Run the specs to see them pass:** same command as 2.4. Expected: 0 failures. Then `bin/rspec spec/requests spec/models` (no slot needed) to see nothing else broke; fix a factory-made account in any spec that needs the household (only the three named in the spec's decision 9 (ii) use `create(:account)`, now with a household). +- [ ] **2.6 Run the specs to see them pass:** same command as 2.4. Expected: 0 failures. Then `bin/rspec spec/requests spec/models` (no slot needed) to see nothing else broke; edit `spec/models/account_spec.rb` line 42 (`expect(described_class.new(currency: "SEK")).to be_valid` becomes `expect(build(:account, currency: "SEK")).to be_valid`: an account needs its household now; change nothing in the model to keep the old line passing); fix a factory-made account in any other spec that needs the household (only the three named in the spec's decision 9 (ii) use `create(:account)`, now with a household). - [ ] **2.7 Commit** in two commits: `feat(models): household and membership, an account's household and kind (spec 090 phase 2)` (models, factories, helper, model specs) and `feat(auth): every way of making a user makes a household of one (spec 090 phase 2)` if the request spec was written after; otherwise one. --- @@ -685,7 +709,14 @@ require Rails.root.join("db/migrate/20261113120002_backfill_households") RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do let(:connection) { ActiveRecord::Base.connection } - def run_up = described_class.new.tap { it.verbose = false }.migrate(:up) + around do |example| + ActiveRecord::Migration.verbose = false + example.run + ensure + ActiveRecord::Migration.verbose = true + end + + def run_up = described_class.new.migrate(:up) def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec @@ -750,13 +781,21 @@ RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do stub_const("BackfillHouseholds::BATCH", 2) # 5 users make 3 batches 5.times { |i| connection.execute("INSERT INTO accounts (created_at, updated_at) VALUES ('#{now}', '#{now}')") ; connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (#{connection.select_value("SELECT MAX(id) FROM accounts")}, 'U#{i}', 'u#{i}@example.test', 'x', '#{now}', '#{now}')") } forget_households - batches = 0 - counter = ->(*, payload) { batches += 1 if payload[:sql].match?(/\ASELECT .*FROM users/i) } + begins = 0 + counter = ->(*, payload) { begins += 1 if payload[:sql].match?(/\ABEGIN/i) } ActiveSupport::Notifications.subscribed(counter, "sql.active_record") { run_up } expect(count("households")).to eq(5) - expect(batches).to eq(4) # three batches with rows, one empty read that ends the loop + expect(begins).to eq(3) # one transaction for each of the three batches; the empty read that ends the loop opens none + end + + it "makes 10,000 households in 20 batches, and says how long it took (the NFR)", timing: true do + connection.execute("WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < 10000) INSERT INTO accounts (created_at, updated_at) SELECT datetime('now'), datetime('now') FROM n") + connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) SELECT id, 'U' || id, 'u' || id || '@example.test', 'x', datetime('now'), datetime('now') FROM accounts") + seconds = Benchmark.realtime { run_up } + expect(count("households")).to eq(10_000) + puts "backfill of 10,000 users: #{seconds.round(2)} s" end it "succeeds on a database with no user (AC-3.4)" do @@ -776,7 +815,7 @@ RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do end end ``` -Note for the implementer: the batch-count example reads `SELECT … FROM users` once per batch; if the migration's read differs, fix the example's regexp to count the reads of the unhoused users, not the migration to fit it. The example's `accounts` inserts rely on `household_id` being nullable and `kind` defaulted: in phase 6 (c) makes `household_id` NOT NULL, and phase 6 edits this one example (step 6.6). +Note for the implementer: the batch example counts the `BEGIN` statements of the run (one for each batch); `run_up` below is wrapped in the `around` that sets `ActiveRecord::Migration.verbose = false` and restores it. The example's `accounts` inserts rely on `household_id` being nullable and `kind` defaulted: in phase 6 (c) makes `household_id` NOT NULL, and phase 6 edits this one example (step 6.6). - [ ] **3.2 Run it to see it fail** (`LoadError`), then **3.3 write the migration** `db/migrate/20261113120002_backfill_households.rb`: ```ruby @@ -794,15 +833,15 @@ class BackfillHouseholds < ActiveRecord::Migration[8.1] def up now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) loop do - users = select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") + users = connection.select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") break if users.empty? connection.transaction do users.each do |user_id, account_id| - execute "INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}" - household = select_value("SELECT last_insert_rowid()") - execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" - execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" + household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless household + connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" + connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" end end end @@ -829,7 +868,7 @@ require "rails_helper" # Spec 090 Story 4, decision 9 (i): no page says "household", and no request of these pages reads a household or a membership. RSpec.describe "A household is invisible (spec 090 Story 4)", type: :request do let!(:admin) { create(:admin, name: "Ann") } - let(:paths) { [ collection_path, admin_users_path, more_path, catalog_entries_path ] } + let(:paths) { [ collection_path, collection_path(view: "table"), admin_users_path, more_path, catalog_entries_path ] } def statements_naming(pattern, &) seen = [] @@ -849,6 +888,14 @@ RSpec.describe "A household is invisible (spec 090 Story 4)", type: :request do expect(seen).to be_empty end + it "shows a collector who is not an admin no page with the word (AC-4.1)", :aggregate_failures do + sign_in_as(create(:user)) + [ collection_path, collection_path(view: "table"), more_path, catalog_entries_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + it "shows no signed-out page the word (AC-4.1)", :aggregate_failures do [ new_session_path, new_registration_path ].each do |path| get path @@ -867,8 +914,8 @@ RSpec.describe "A household is invisible (spec 090 Story 4)", type: :request do end ``` Run it: it should pass at once (nothing visible was added); that is the point of this check. Record in `verification.md` that it passed on the first run. Then **4.2 the one-off checks** (recorded in `verification.md`, no commit): `git grep -n -i 'household\|membership' -- app/views app/helpers app/controllers app/javascript config/locales ':!.kamal/secrets'` prints nothing; `bin/rails routes > "$TMPDIR/routes_branch.txt"`, then the same on `main` in the main checkout is not allowed (read-only worktree rule): compare with `git show main:config/routes.rb` against `config/routes.rb` (`git diff main -- config/routes.rb` is empty) and `bin/rails routes | wc -l` equal to the line count recorded from the first `bin/rails routes` run in phase 0 (add that run to step 0.1's record). -- [ ] **4.3 Failing spec edits for the coverage guard** (`spec/export_format_document_spec.rb`): in the `"marks each table as the spec rules"` example add `memberships` to the `%w[...]` list of "Left out" tables (alphabetical: `… import_uploads memberships number_entries …`); in the columns example add `accounts.household_id accounts.kind` after `accounts.updated_at` in the left-out list. Run `bin/rspec spec/export_format_document_spec.rb`: the coverage examples fail (the document lacks the rows). -- [ ] **4.4 Edit `docs/export-format/README.md`.** In the Coverage tables: add the row `| \`memberships\` | Left out | A household's membership of a user: a fact about a group, not about the collection. A household of one is made with every user, so a person who imports their file is given one by the same rule (spec 090 decision 4). |` between `import_uploads` and `number_entries` (the table is alphabetical after `accounts`, `lots`, ... as the file lists; keep its order of "Exported" rows first, then "Left out" alphabetically); in the Column table add after `accounts.updated_at`: `| \`accounts.household_id\` | Left out | An internal id of a group. A file is one account's collection, and the household is not in it. |` and `| \`accounts.kind\` | Left out | Whether the account is a member's own or the household's. A file is one account's collection, and every account that has a file is personal until the shared collection (#384) decides its own file. |`. After the Coverage tables add the sentence: "`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4)." Replace the sentence at the Versions history (line 336 area) "One change is already expected: collections shared by a household will change who owns a lot and will raise the version ([ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md))." by "One change is already expected: the household's shared collection has its own file and will raise the version ([ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md), issue #384). A household itself, one per user from spec 090, is not in the file and raised no version." Edit `docs/architecture/data-portability.md` line 47's sentence to add: "`memberships` and `accounts.household_id` and `.kind` were listed as left out with spec 090; `households` has neither key and is outside the guard." Run `bin/rspec spec/export_format_document_spec.rb` (expected green) and `bin/rspec spec/models/export_format spec/models/collection_export_spec.rb` (green). +- [ ] **4.3 (done in step 1.4) Failing spec edits for the coverage guard** (`spec/export_format_document_spec.rb`): in the `"marks each table as the spec rules"` example add `memberships` to the `%w[...]` list of "Left out" tables (alphabetical: `… import_uploads memberships number_entries …`); in the columns example add `accounts.household_id accounts.kind` after `accounts.updated_at` in the left-out list. Run `bin/rspec spec/export_format_document_spec.rb`: the coverage examples fail (the document lacks the rows). +- [ ] **4.4 (done in step 1.4) Edit `docs/export-format/README.md`.** In the Coverage tables: add the row `| \`memberships\` | Left out | A household's membership of a user: a fact about a group, not about the collection. A household of one is made with every user, so a person who imports their file is given one by the same rule (spec 090 decision 4). |` between `import_uploads` and `number_entries` (the table is alphabetical after `accounts`, `lots`, ... as the file lists; keep its order of "Exported" rows first, then "Left out" alphabetically); in the Column table add after `accounts.updated_at`: `| \`accounts.household_id\` | Left out | An internal id of a group. A file is one account's collection, and the household is not in it. |` and `| \`accounts.kind\` | Left out | Whether the account is a member's own or the household's. A file is one account's collection, and every account that has a file is personal until the shared collection (#384) decides its own file. |`. After the Coverage tables add the sentence: "`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4)." Replace the sentence at the Versions history (line 336 area) "One change is already expected: collections shared by a household will change who owns a lot and will raise the version ([ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md))." by "One change is already expected: the household's shared collection has its own file and will raise the version ([ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md), issue #384). A household itself, one per user from spec 090, is not in the file and raised no version." Edit `docs/architecture/data-portability.md` line 47's sentence to add: "`memberships` and `accounts.household_id` and `.kind` were listed as left out with spec 090; `households` has neither key and is outside the guard." Run `bin/rspec spec/export_format_document_spec.rb` (expected green) and `bin/rspec spec/models/export_format spec/models/collection_export_spec.rb` (green). - [ ] **4.5 Failing spec for the round trip and the backup.** Create `spec/models/collection_import/household_untouched_spec.rb`: ```ruby @@ -1223,7 +1270,24 @@ RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model d expect { up }.not_to raise_error # a second run changes nothing end - it "changes nothing when it fails, and succeeds on the next run (AC-3.6, the Error Scenarios row)", :aggregate_failures do + it "undoes a failure of the copy itself, after the old table was dropped: nothing is lost (AC-3.6)", :aggregate_failures do + create(:user) + down + orphan = connection.insert("INSERT INTO accounts (created_at, updated_at) VALUES (datetime('now'), datetime('now'))") + before = counts.merge("sequence" => sequence) + # Fault injection: the refusal that would stop the run before any DDL is switched off, so the copy-back's NOT NULL fails. + migration = Class.new(described_class) { private def refuse_accounts_nobody_can_house = nil }.new + + expect { migration.migrate(:up) }.to raise_error(ActiveRecord::StatementInvalid, /NOT NULL/) + + expect([ nullable?, connection.select_value("PRAGMA foreign_keys") ]).to eq([ true, 1 ]) + expect(counts.merge("sequence" => sequence)).to eq(before) + expect(connection.select_value("SELECT COUNT(*) FROM accounts WHERE id = #{orphan}")).to eq(1) + connection.execute("DELETE FROM accounts WHERE id = #{orphan}") + expect { up }.not_to raise_error + end + + it "changes nothing when it refuses, and succeeds on the next run (AC-3.6, the Error Scenarios row)", :aggregate_failures do create(:user) straggler = create(:user, name: "Late") down @@ -1235,6 +1299,7 @@ RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model d expect { up }.to raise_error(StandardError, /accounts? .*#{orphan}.* no user/m) expect(nullable?).to be(true) + expect(connection.select_value("PRAGMA foreign_keys")).to eq(1) expect(counts.merge("sequence" => sequence)).to eq(before) expect(connection.select_value("SELECT COUNT(*) FROM memberships WHERE user_id = #{straggler.id}")).to eq(0) # the stragglers step was rolled back connection.execute("DELETE FROM accounts WHERE id = #{orphan}") @@ -1267,10 +1332,12 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] connection.disable_referential_integrity do connection.transaction do + require_enforcement_off house_stragglers remove_empty_households refuse_accounts_nobody_can_house keeping_sequence { change_column_null :accounts, :household_id, false } + require_no_broken_keys end end end @@ -1279,7 +1346,11 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] return if household_nullable? connection.disable_referential_integrity do - connection.transaction { keeping_sequence { change_column_null :accounts, :household_id, true } } + connection.transaction do + require_enforcement_off + keeping_sequence { change_column_null :accounts, :household_id, true } + require_no_broken_keys + end end end @@ -1290,24 +1361,37 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] def house_stragglers now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) loop do - users = select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") + users = connection.select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") break if users.empty? users.each do |user_id, account_id| - execute "INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}" - household = select_value("SELECT last_insert_rowid()") - execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" - execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" + household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless household + connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" + connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" end end end + # Rails runs no check after a copy and, inside a transaction, enforcement can't be switched off: if either went wrong the copy would + # have cascaded silently, so a run that finds enforcement on, or rows left pointing at accounts or households, stops and rolls back. + def require_enforcement_off + return if connection.select_value("PRAGMA foreign_keys").to_i.zero? + + raise ActiveRecord::MigrationError, "foreign keys are enforced inside the table copy, which would delete the collections that reference accounts; nothing was changed" + end + + def require_no_broken_keys + broken = connection.select_rows("PRAGMA foreign_key_check").select { |_table, _row, parent, *| %w[accounts households].include?(parent) } + raise ActiveRecord::MigrationError, "the copy left rows pointing at accounts or households (#{broken.first(3).inspect}); nothing was changed" if broken.any? + end + def remove_empty_households - execute "DELETE FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM accounts WHERE accounts.household_id = households.id)" + connection.execute "DELETE FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM accounts WHERE accounts.household_id = households.id)" end def refuse_accounts_nobody_can_house - orphans = select_values("SELECT id FROM accounts WHERE household_id IS NULL") + orphans = connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL") return if orphans.empty? raise ActiveRecord::MigrationError, "accounts #{orphans.first(5).join(", ")} have no household and no user, so no household can be made for them; " \ @@ -1316,17 +1400,17 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). def keeping_sequence - seq = select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + seq = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") yield return unless seq - execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" - execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" + connection.execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + connection.execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" end end ``` - [ ] **6.5 Dump and prepare:** `bin/rails db:migrate` and `bin/rails db:test:prepare`; `git diff db/schema.rb` shows `t.integer "household_id", null: false` and the new version `2026_11_13_120003`, and no other change to `accounts`, its foreign keys or the thirteen tables (any other line is a failure of the copy: stop and report). Run `bin/rspec spec/migrations/make_account_household_required_spec.rb` (6 examples, 0 failures). -- [ ] **6.6 Make the earlier specs follow the final shape.** In `spec/migrations/create_households_and_memberships_spec.rb` and `spec/migrations/backfill_households_spec.rb`, extend the versions so a spec's down and up cover (c): in the first, `def versions = [ 2026_11_13_120001, 2026_11_13_120002, 2026_11_13_120003 ]` and add `require` lines for the other two migrations; in the second, add to the `around` an `ensure` that reruns `context.run(:up, 2026_11_13_120003)` when `accounts.household_id` is nullable after an example that goes down. In `backfill_households_spec.rb`'s batch example, the raw `INSERT INTO accounts` rows get a household: insert a household first and pass `household_id` (the column is NOT NULL now); `forget_households` sets `household_id = NULL`, which the copy refuses: have it first `down` (c) when the column is NOT NULL (`context.run(:down, 2026_11_13_120003)`) and `up` it in `ensure`. In `spec/support/backup_helpers.rb:40` and `spec/lib/collector/backup/snapshot_spec.rb:12` insert a household and give the account its id (`INSERT INTO households (name, created_at, updated_at) VALUES ('h', ?, ?)`, then `accounts (created_at, updated_at, household_id)`). Run `bin/rspec spec/migrations spec/lib/collector/backup spec/backup_guards_spec.rb spec/backup_upgrade_spec.rb` (green). +- [ ] **6.6 Make the earlier specs follow the final shape.** In `spec/migrations/create_households_and_memberships_spec.rb` and `spec/migrations/backfill_households_spec.rb`, extend the versions so a spec's down and up cover (c): in the first, `def versions = [ 2026_11_13_120001, 2026_11_13_120002, 2026_11_13_120003 ]` and add `require` lines for the other two migrations; in the second, add to the `around` an `ensure` that reruns `context.run(:up, 2026_11_13_120003)` when `accounts.household_id` is nullable after an example that goes down. In `backfill_households_spec.rb`'s batch example and its 10,000-user timing example (both insert accounts by SQL), the raw `INSERT INTO accounts` rows get a household (the timing example inserts one household first and gives every account that id, or runs with (c) down as `forget_households` does): insert a household first and pass `household_id` (the column is NOT NULL now); `forget_households` sets `household_id = NULL`, which the copy refuses: have it first `down` (c) when the column is NOT NULL (`context.run(:down, 2026_11_13_120003)`) and `up` it in `ensure`. In `spec/support/backup_helpers.rb:40` and `spec/lib/collector/backup/snapshot_spec.rb:12` insert a household and give the account its id (`INSERT INTO households (name, created_at, updated_at) VALUES ('h', ?, ?)`, then `accounts (created_at, updated_at, household_id)`). Run `bin/rspec spec/migrations spec/lib/collector/backup spec/backup_guards_spec.rb spec/backup_upgrade_spec.rb` (green). - [ ] **6.7 Commit:** `feat(db): accounts.household_id is NOT NULL, by a copy that loses no row (spec 090 phase 6)` and `test(db): the earlier households specs follow the final shape (spec 090 phase 6)`. --- @@ -1336,8 +1420,8 @@ end **Implements:** FR-5, decision 11, Story 6's evidence | **Satisfies:** AC-4.2, AC-6.6, the documentation criteria (verified by reading, `verification.md`) | **Interfaces:** none. - [ ] **7.1 `docs/architecture/tenancy-and-accounts.md`** (the source of each statement inline): in the Scope bullet "The tenant and its holder" add `Household`, `Membership` (spec 090; ADR 0030); Glossary: replace the Account entry's "created with its user and deleted with it" tail with "created with its user and deleted with it, and in a household (`accounts.household_id`) as a personal account (`accounts.kind`)", add **Household**, **Membership (manager, member)**, **Personal and shared account** entries and the note that "member" also means an admin page's non-admin (spec 090 Terms); the Tenant-owned list gains `memberships` (user-owned, instance-level) and says `households` and `memberships` carry no `account_id`, are never scoped through `Current.account` and no request names one; Invariants: change "One user per account" to the new rule ("`users.account_id` stays unique; a household is a group of accounts; a household always has a manager; a personal account is in its user's household; a household is made with its user by `User#build_account_and_household`" with `spec 090`, ADR 0030), add the cross-account reads sentence (the household tables are read by the model and the proof only); the "Deleting a user" paragraph gains the order (membership, user, account, household when empty) and the last-manager refusal; "Accepted limits" replaces "An account has exactly one user. Personal and household collections side by side will supersede that half of ADR 0017" with what exists now (a household of one for every user; the rest is #381 to #389); Sources gains spec 090, ADR 0030 and #380; Distillation log gains a row once the PR number exists (the distill-feature step writes it). -- [ ] **7.2** `.claude/rules/multi-tenancy.md`: add one line under the ✓ list: "✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request." `docs/architecture/release-and-deploy.md`: in the upgrade proof's paragraph add the two-tier households check (decision 10; `Upgrade::Households`; spec 090 AC-6.1 to AC-6.5) and that `expected_changes.yml` stays empty because `compare.rb` reads only the old database's columns. `docs/releasing.md`'s Upgrade notes: the entry for this migration with its warning (an older image can't create a user after it; going back is a restore of the backup taken before; a failed third migration stops the boot and changes nothing; Kamal's old container fails sign-up for seconds). `CLAUDE.md`: in the Project Status accounts sentence add "every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389)". `docs/adr/0030`: Status becomes Accepted. `docs/specs/README.md`: spec 004's row gains "Superseded by 090 in part (FR-3's one user per account)" and 090's row Status "Approved". -- [ ] **7.3 Evidence for `verification.md`** (create it with the headings the other specs use; write each result as it is produced): the phase 0 and each phase's spec runs with counts; the grep and the routes diff of step 4.2; **`bin/upgrade-smoke`**: build the previous release from its tag as `.claude/memory/container-engine-colima-macos.md` says (`git archive` of `v0.3.0` into its own directory, three `--build-arg` flags for the image: `README.md`, Building from source), build the code under test the same way, then `bin/upgrade-smoke ` with `TMPDIR` under the worktree and `/opt/homebrew/bin` appended to `PATH`; expected: the run ends `OK: v0.3.0` and prints `households: ok (backfilled)` for the upgraded and the restored snapshot and `households: ok` for the next one. **Ask the coordinator for the container engine and a slot before this step.** A failure is read, not re-run blindly (`.claude/memory/reproduce-ci-with-seed.md` for CI). +- [ ] **7.2** `.claude/rules/multi-tenancy.md`: add one line under the ✓ list: "✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request." `docs/architecture/release-and-deploy.md`: in the upgrade proof's paragraph add the two-tier households check (decision 10; `Upgrade::Households`; spec 090 AC-6.1 to AC-6.5) and that `expected_changes.yml` stays empty because `compare.rb` reads only the old database's columns. `docs/releasing.md`'s Upgrade notes: the entry for this migration with its warning (an older image can't create a user after it; going back is a restore of the backup taken before; a failed third migration stops the boot and changes nothing; Kamal's old container fails sign-up for seconds). `CLAUDE.md`: in the Project Status accounts sentence add "every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389)". `docs/adr/0030`: Status becomes Accepted (ADR 0017's "superseded in part by 0030" status line and both ADR index rows were written with the ADR: verify them). `docs/specs/README.md`: spec 004's row gains "Superseded by 090 in part (FR-3's one user per account)" and 090's row Status "Approved". +- [ ] **7.3 Evidence for `verification.md`** (create it with the headings the other specs use; write each result as it is produced): the phase 0 and each phase's spec runs with counts; the grep and the routes diff of step 4.2; **`bin/upgrade-smoke`**: build the previous release from its tag as `.claude/memory/container-engine-colima-macos.md` says (`git archive` of `v0.3.0` into its own directory, three `--build-arg` flags for the image: `README.md`, Building from source), build the code under test the same way, then `bin/upgrade-smoke ` with `TMPDIR` under the worktree and `/opt/homebrew/bin` appended to `PATH`; expected: the run ends `OK: v0.3.0` and prints `households: ok (backfilled)` for the upgraded and the restored snapshot and `households: ok` for the next one. **Run it on the final commit that will be pushed: after 7.4 and after the implementation review's fixes of 7.5; if code changes after a run, run it again, and record the SHA it ran on in `verification.md`.** **Ask the coordinator for the container engine and a slot before this step.** A failure is read, not re-run blindly (`.claude/memory/reproduce-ci-with-seed.md` for CI). - [ ] **7.4 `bin/ci checks`** (a documentation-and-code branch: it runs RuboCop, YARD, the comment ratchet, Brakeman, bundler-audit, importmap audit and the migration guard): expected green; the comment ratchet needs a summary on each new public method (the models and `Upgrade::Households` have them). Fix and re-run. - [ ] **7.5 The implementation review** (`sdd-review` Mode B, read-only, Opus 5.5) recorded in `verification.md` under `## Implementation review` with the verdict first; then `sdd-superpowers:session-wrap` and `distill-feature` (digest to the coordinator before commit); then one slot for the one full `bin/ci`; push; draft PR closing #380; CI green; `gh pr ready`; `gh pr edit --add-assignee @me`; `worker_done`. diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index 82a367f2..0a9e4aaa 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -74,7 +74,7 @@ This feature adds the structure and nothing that uses it: a household with a nam **What the copy risks.** `accounts` is the most referenced table in the schema: thirteen tables hold a foreign key to it. Eleven are `ON DELETE CASCADE` (`bulk_removals`, `bulk_selection_marks`, `bulk_selections`, `deck_proxies`, `decks`, `import_uploads`, `locations`, `number_entries`, `number_entry_lines`, `scanner_sitting_entries`, `scanner_sittings`), and `lots` and `users` hold SQLite's default, `NO ACTION` (`db/schema.rb`; the `RESTRICT` that makes `Account` declare `lots` before `locations`, `spec/models/account_spec.rb`, is `lots → locations`). The copy's `DROP TABLE accounts` runs an implicit delete of every row, so with foreign keys enforced: the eleven cascades **delete every collector's decks, places, proxies and working state, silently**, and `lots` and `users` are deferred to the end of the transaction rather than refused. Rails turns enforcement off for a copy only **outside a transaction** (`PRAGMA foreign_keys = OFF` does nothing inside one, and Rails then defers the keys), so a copy that ran inside Rails' own migration transaction would lose that data with no error. That is why (c) runs without Rails' migration transaction, and opens its own with enforcement already off: outside any transaction the steps would each commit alone, and a (c) that stopped after the copy and before the sequence's restore would never get it back, since the next boot finds the column already `NOT NULL`. **Two more effects the proof must catch:** the copy drops and recreates the table, so the `sqlite_sequence` row for `accounts` restarts at the highest id present, and an id of a deleted account can be reused (`compare.rb` skips `sqlite_%` tables, so the proof can't see it: (c) reads the sequence before the copy and restores it after, in the same transaction, and AC-3.6 asserts it); and an older release's image started on the upgraded volume can't create a user or account any more, since it doesn't know the column (the same of every added `NOT NULL` column). **Collector documents no downgrade:** the README says to back up before every upgrade, and going back means restoring that backup onto a new volume under the old image and losing what was written since; `docs/releasing.md`'s Upgrade notes carry this migration's warning. Also: **a failed (c) stops the boot** (Compose is down until it is fixed; Kamal keeps the old container serving), and during a Kamal deploy an old container's sign-up or add-a-user fails with a 500 for the seconds between (c) and the switch. The precedent is narrower than a pattern: the one shipped copy of a referenced table by an `up` is `sessions` (spec 049), with two cascading children against thirteen here; `AddDecks` and its kin add columns without a copy. **What proves no row is lost.** The upgrade proof compares every table of the old database with the upgraded one over the columns both have, row by row (`script/upgrade/compare.rb`), and its seed holds a row in every table, including each of the thirteen; `PRAGMA foreign_key_check` and `integrity_check` run on the result; a migration spec runs (c) on a database holding rows in each referencing table and asserts every count, and the `accounts` sequence, is unchanged; and the copy's foreign keys, named checks and indexes are read back against the schema dump. The same holds for (c)'s and (a)'s `down`, which also copy `accounts` (AC-3.5). - **The nullable fallback (a real option).** Skip (c): `accounts.household_id` stays nullable in the database and the model validates it present. *Gains:* no copy of `accounts`, no sequence to restore, no step that can stop a boot, and nothing in this feature can lose a row; (a)'s `down` still copies `accounts`, once. *Costs:* an older image on an upgraded volume (the Kamal deploy window) still creates users, **each with no membership and an account with no household, and nothing ever repairs it, because (b) is recorded and never runs again**: under the fallback the Goal "every user in exactly one household" can be broken permanently by one deploy. To be sound the fallback needs a repair that runs on every boot (the stragglers step run from `db:prepare`'s boot path) or a household made on first load; neither is in this spec, so the fallback as written does not meet the Goal. The database also can't state "every account has a household", so a bug or a hand-written insert can leave an account with none, and only a guard spec and the upgrade proof would notice; and a fresh volume (built from `schema.rb`) and an upgraded one agree only while the column is nullable in both, so the schema says nullable until a later migration pays the copy then, with more tables referencing `accounts` and more instances to carry through it (the design's own reason for doing this now). **The same under both options:** a fresh volume loads `db/schema.rb` and runs no copy; an older backup restored is migrated like an upgrade; a backup taken after the upgrade is refused by the older release either way. The recommendation is (c), for those costs and because the proof that guards it exists and this issue is the cheapest time to pay it; but the copy is the one step of this feature that can lose data if its guard is wrong, and the maintainer rules between them. If the plan review's trace finds (c) unsafe, the maintainer is asked again before the fallback is taken, and this spec's version records the outcome. - - **Reversible:** (c)'s `down` makes the column nullable again by the same copy, restoring the sequence as its `up` does; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the `kind` check before its column, as `AddDecks`' `down` does, so a full rollback copies `accounts` three times: (c)'s, then two in (a)), keeping the sequence at each copy and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. + - **Reversible:** (c)'s `down` makes the column nullable again by the same copy, restoring the sequence as its `up` does; (b)'s `down` does nothing (a second run of its `up` finds nothing to do); (a)'s `down` drops the tables and the two columns (the `kind` check before its column, as `AddDecks`' `down` does, so (a)'s `down` copies `accounts` three times (the check's removal, `kind`'s and `household_id`'s) and a full rollback four times with (c)'s), keeping the sequence at each copy and **refuses**, with an `IrreversibleMigration` that says what exists, when any household has more than one member or any shared account exists, since the older code can't hold either; a household of one is removed with its name. 6. **A personal account is in its user's household.** For a user, `accounts.household_id` of their account equals `memberships.household_id` of their membership. The database can't compare two tables, so the rule is the model's (a user is built with both from one household; a later issue that moves a user changes both in one transaction), and a guard spec and the upgrade proof check it on real rows. A shared account has no user; a personal account has exactly one. 7. **Deleting a user removes their membership, their account, and, when no one is left, their household.** An admin's delete of a user (spec 004 AC-5.4) works as before. The order is the foreign keys': the user's row (its membership goes with it by decision 1's cascade, **without** the membership's own refusal of decision 3, which guards a membership removed on its own; the user's delete has its own check for the last manager), then the account (`User` declares `belongs_to :account, dependent: :destroy`), then the household when no member and no account is left. A household left with other members keeps them, and the model refuses the delete when the user is the household's last manager and others remain (no page makes such a household in this feature, so only a model spec reaches it; #382 and #383 give it a page and a message). The removal of a household "when no member and no account is left" is revisited by #381 (a joiner leaves a household of one behind) and #384 (a shared account is an account of the household). 8. **A household is made by the user model, so every path gets one.** The one place an account is made from a user today (`before_validation :build_account`) also makes the household and the manager membership; the first-run sign-up, the open sign-up, the admin's add-a-user form and `bin/rails collector:user` all go through it, and a spec per path says so. Nothing in a request names a household, a role or a kind (`household_id`, `role` and `kind` are not permitted from parameters). From 4ffe33290f8ab70678dc411766d14fb21a3942c2 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 20:55:58 -0500 Subject: [PATCH 08/41] docs(specs): record Gate B for plan 090 and the condition on phase 6 (#380) Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/plan.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md index 64ce7845..82401b66 100644 --- a/docs/specs/090-every-user-has-a-household/plan.md +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -1,6 +1,7 @@ # Plan 090: Every User Has a Household **Spec:** [spec.md](spec.md) v0.3.2 (Approved by the coordinator under the maintainer's delegation, 2026-11-13; the maintainer ruled step (c) the same day: take the copy). **ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md). **Issue:** #380. **Branch:** `090-every-user-has-a-household`. +**Gate B:** approved by the coordinator under the maintainer's delegation (2026-11-13) for phases 1 to 5 and 7. **Condition on phase 6:** its batch does not start until a second read-only Fable review of phase 6 as it now stands (with `require_enforcement_off`, `require_no_broken_keys`, the fault-after-DROP spec and the corrected `down`) and of the changed `User` validation (it can never leave a user without a household and never reads one on an ordinary request) has been sent to the coordinator with its verdict. **Written by:** the main session (Opus 5.5 from this step; the spec and ADR were written on Sonnet 5.5). Implementers: Sonnet 5.5 (`model: "sonnet"`), 2 to 4 consecutive tasks each, test first, one commit per task, reading this file by line range; the main session checks each batch's diff. ## Global Constraints From 3c01b6a3151bc29a23012a718acbd0603ef61df8 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:01:44 -0500 Subject: [PATCH 09/41] feat(db): households, memberships and the account's household and kind (spec 090 phase 1) Adds the migration, its spec, the schema dump, and the export format's coverage rows (plan steps 4.3 and 4.4, done here as step 1.4 says). Changed from the plan's code: the migration's keeping_sequence and the spec's sequence helper read and write main.sqlite_sequence, not the bare name. Rails' table copy builds a TEMP table with AUTOINCREMENT, which gives the connection a temp.sqlite_sequence that a bare name then resolves to, so the plan's restore wrote to the temp table and the spec read the same wrong one (it failed on the first run against a fresh test database, whose seed 34601 order puts a copy first). The assertions are unchanged. Co-Authored-By: Claude Sonnet 5.5 --- ...20001_create_households_and_memberships.rb | 75 ++++++++ db/schema.rb | 27 ++- docs/architecture/data-portability.md | 2 +- docs/export-format/README.md | 7 +- spec/export_format_document_spec.rb | 4 +- .../create_households_and_memberships_spec.rb | 160 ++++++++++++++++++ 6 files changed, 270 insertions(+), 5 deletions(-) create mode 100644 db/migrate/20261113120001_create_households_and_memberships.rb create mode 100644 spec/migrations/create_households_and_memberships_spec.rb diff --git a/db/migrate/20261113120001_create_households_and_memberships.rb b/db/migrate/20261113120001_create_households_and_memberships.rb new file mode 100644 index 00000000..83d6d470 --- /dev/null +++ b/db/migrate/20261113120001_create_households_and_memberships.rb @@ -0,0 +1,75 @@ +# A household and its members, and the two columns that put an account in one (spec 090 decisions 1 and 5 (a), ADR 0030). +# Without a transaction, as AddDecks explains: dropping the two columns on the way down copies accounts, and with foreign keys +# enforced inside a transaction the copy's DROP TABLE would run the delete rules of the thirteen tables that reference it. Every +# step is guarded, so a run that stopped part way is finished by the next boot. Going down refuses while a household has two +# members or a shared account exists (an older release can't hold either), and keeps accounts' id sequence across each copy. +class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + def up + create_table :households, if_not_exists: true do |t| + t.string :name, null: false + t.timestamps + t.check_constraint "length(name) BETWEEN 1 AND 100", name: "households_name_length" + end + create_table :memberships, if_not_exists: true do |t| + t.references :household, null: false, foreign_key: true + t.references :user, null: false, index: { unique: true }, foreign_key: { on_delete: :cascade } + t.string :role, null: false + t.timestamps + t.check_constraint "role IN ('manager', 'member')", name: "memberships_role_known" + end + # ADD COLUMN copies no table: REFERENCES is accepted because the default is NULL, and a named CHECK on the column is + # tested against every row and kept by the schema dump (the precedent is AddDecks). + unless column_exists?(:accounts, :household_id) + connection.execute "ALTER TABLE accounts ADD COLUMN household_id integer REFERENCES households (id)" + end + add_index :accounts, :household_id unless index_exists?(:accounts, :household_id) + unless column_exists?(:accounts, :kind) + connection.execute "ALTER TABLE accounts ADD COLUMN kind varchar NOT NULL DEFAULT 'personal' " \ + "CONSTRAINT accounts_kind_known CHECK (kind IN ('personal', 'shared'))" + end + end + + def down + refuse_what_the_older_release_cannot_hold + connection.disable_referential_integrity do + connection.transaction do + keeping_sequence do + remove_check_constraint :accounts, name: "accounts_kind_known" if check_constraint_exists?(:accounts, name: "accounts_kind_known") + remove_column :accounts, :kind if column_exists?(:accounts, :kind) + end + keeping_sequence do + remove_index :accounts, :household_id if index_exists?(:accounts, :household_id) + remove_column :accounts, :household_id if column_exists?(:accounts, :household_id) + end + end + end + drop_table :memberships, if_exists: true + drop_table :households, if_exists: true + end + + private + def refuse_what_the_older_release_cannot_hold + shared_households = table_exists?(:memberships) ? connection.select_value("SELECT COUNT(*) FROM (SELECT 1 FROM memberships GROUP BY household_id HAVING COUNT(*) > 1)").to_i : 0 + shared_accounts = column_exists?(:accounts, :kind) ? connection.select_value("SELECT COUNT(*) FROM accounts WHERE kind = 'shared'").to_i : 0 + return if shared_households.zero? && shared_accounts.zero? + + raise ActiveRecord::IrreversibleMigration, + "#{shared_households} #{"household".pluralize(shared_households)} with more than one member and " \ + "#{shared_accounts} shared #{"account".pluralize(shared_accounts)} exist; remove them before rolling back" + end + + # A table copy drops and recreates accounts, which resets its row of sqlite_sequence to the highest id present, so the id of a + # deleted account could be issued again. The sequence is read before and restored after, never lowered, and the row is made + # when the copy left none (an empty table has none). The table is always `main.sqlite_sequence`: the copy builds a TEMP table + # with AUTOINCREMENT, which gives the connection a `temp.sqlite_sequence` that a bare name then resolves to. + def keeping_sequence + seq = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") + yield + return unless seq + + connection.execute "UPDATE main.sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + connection.execute "INSERT INTO main.sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM main.sqlite_sequence WHERE name = 'accounts')" + end +end diff --git a/db/schema.rb b/db/schema.rb index bb587d5e..66cbd090 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -10,11 +10,15 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema[8.1].define(version: 2026_11_11_150001) do +ActiveRecord::Schema[8.1].define(version: 2026_11_13_120001) do create_table "accounts", force: :cascade do |t| t.datetime "created_at", null: false t.datetime "updated_at", null: false t.string "currency", default: "USD", null: false + t.integer "household_id" + t.string "kind", default: "personal", null: false + t.index ["household_id"], name: "index_accounts_on_household_id" + t.check_constraint "kind IN ('personal', 'shared')", name: "accounts_kind_known" end create_table "active_storage_attachments", force: :cascade do |t| @@ -185,6 +189,13 @@ t.index ["location_id"], name: "index_decks_on_location_id", unique: true end + create_table "households", force: :cascade do |t| + t.string "name", null: false + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.check_constraint "length(name) BETWEEN 1 AND 100", name: "households_name_length" + end + create_table "import_uploads", force: :cascade do |t| t.integer "account_id", null: false t.string "fingerprint", null: false @@ -239,6 +250,17 @@ t.check_constraint "quantity BETWEEN 1 AND 9999", name: "lots_quantity_range" end + create_table "memberships", force: :cascade do |t| + t.integer "household_id", null: false + t.integer "user_id", null: false + t.string "role", null: false + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index ["household_id"], name: "index_memberships_on_household_id" + t.index ["user_id"], name: "index_memberships_on_user_id", unique: true + t.check_constraint "role IN ('manager', 'member')", name: "memberships_role_known" + end + create_table "mtg_art_builds", force: :cascade do |t| t.string "status", null: false t.string "job_id" @@ -379,6 +401,7 @@ t.index ["email_address"], name: "index_users_on_email_address", unique: true end + add_foreign_key "accounts", "households" add_foreign_key "active_storage_attachments", "active_storage_blobs", column: "blob_id" add_foreign_key "active_storage_variant_records", "active_storage_blobs", column: "blob_id" add_foreign_key "bulk_removals", "accounts", on_delete: :cascade @@ -402,6 +425,8 @@ add_foreign_key "lots", "accounts" add_foreign_key "lots", "catalog_entries" add_foreign_key "lots", "locations", on_delete: :restrict + add_foreign_key "memberships", "households" + add_foreign_key "memberships", "users", on_delete: :cascade add_foreign_key "mtg_artworks", "catalog_entries" add_foreign_key "mtg_cards", "catalog_identities" add_foreign_key "mtg_printings", "catalog_entries" diff --git a/docs/architecture/data-portability.md b/docs/architecture/data-portability.md index 7b624643..835ad905 100644 --- a/docs/architecture/data-portability.md +++ b/docs/architecture/data-portability.md @@ -44,7 +44,7 @@ How a collector's collection leaves and enters Collector: the export format, the - **The format document is the format's definition.** The app's schema copies are byte-identical to the ones in `docs/export-format/` (one per version), and the suite fails if they differ (ADR 0021; spec 029 AC-7.6). - **No internal id appears in a file.** A lot points at its entry by collectible type and source id, a place and a deck are their names, a zone is its deck's name and its key, a proxy is its deck's name, its zone's key and its entry, and a file carries no account or user id (spec 017 FR-1, FR-6, AC-1.4; spec 029 AC-6.5; spec 062 AC-6.5). - **The meaning of a released version never changes.** A version rises when an older reader would misread a file or lose collector-owned data; a new field of collector-owned data raises it, an information-only field doesn't (spec 017 FR-8, decision 14; ADR 0020; ADR 0023). Version 2 added locations; version 3 added each lot's `location` (spec 032, ADR 0025); version 4 adds `decks`, a lot's zone and `proxies` (spec 062, ADR 0028; spec 068: a file's `proxies` is required, and version 4 was unreleased when it was added). -- **A new table with `account_id` or `user_id`, or a new column on `accounts`, `lots`, `users` or `locations`, fails the suite** until the format document's Coverage tables list it; `decks` and `locations.kind`, `parent_id` and `zone` were listed with version 4, and `deck_proxies` with its proxies (spec 017 FR-9, AC-3.4, AC-3.5; ADR 0020; spec 062 AC-9.10; spec 068 AC-7.3). +- **A new table with `account_id` or `user_id`, or a new column on `accounts`, `lots`, `users` or `locations`, fails the suite** until the format document's Coverage tables list it; `decks` and `locations.kind`, `parent_id` and `zone` were listed with version 4, and `deck_proxies` with its proxies (spec 017 FR-9, AC-3.4, AC-3.5; ADR 0020; spec 062 AC-9.10; spec 068 AC-7.3). `memberships` and `accounts.household_id` and `.kind` were listed as left out with spec 090; `households` has neither key and is outside the guard. - **The core names no collectible.** The export's, the preview's and the apply's code contain neither `MTG` nor `Scryfall`; the export never asks the catalog registry about a type, so a lot of a type the instance no longer has still exports (spec 022 FR-6, decision 4; spec 025 FR-9; spec 028 FR-8; spec 029 FR-2). Checked by `spec/collection_export_core_spec.rb`, `spec/collection_import_core_spec.rb`, `spec/import_apply_core_spec.rb`; the last also lists the location files, and `spec/deck_core_spec.rb` the import's deck files (`Decks`, `DeckChanges`, the preview's deck partial), which also name no format and no zone key but `considering` (spec 062 FR-3, AC-9.11). - **Everything runs through `Current.account`.** No parameter names an account, an upload or a lot; the one parameter besides the confirmation is the fingerprint (spec 022 Security; spec 025 FR-5; spec 028 FR-5, AC-4.2). Another account's data never appears in a response (spec 022 AC-2.1; spec 025 AC-4.1; spec 028 AC-4.1). - **An export is built in the request and stores nothing.** No job, no row, no file, no cache entry (spec 022 decision 1, FR-3). diff --git a/docs/export-format/README.md b/docs/export-format/README.md index 22de93f6..0eee8a63 100644 --- a/docs/export-format/README.md +++ b/docs/export-format/README.md @@ -333,7 +333,7 @@ The meaning of a version never changes once it has been released. Version 4 gained `proxies` before any release wrote a version 4 file, so it is one version (the maintainer's ruling of 2026-11-11); a version 4 file written by a development build before that has no `proxies` and isn't a well-formed version 4 file. -One change is already expected: collections shared by a household will change who owns a lot and will raise the version ([ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)). A currency for each account (issue #64, spec 038) moved where the code the file carries comes from, from the instance to the account, without a version, because the field's meaning didn't change. +One change is already expected: the household's shared collection has its own file and will raise the version ([ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md), issue #384). A household itself, one per user from spec 090, is not in the file and raised no version. A currency for each account (issue #64, spec 038) moved where the code the file carries comes from, from the instance to the account, without a version, because the field's meaning didn't change. ## Extending the format @@ -463,6 +463,7 @@ Every table in Collector's database that belongs to an account or to a user is l | `bulk_selection_marks` | Left out | Working state that lasts for one bulk edit. | | `bulk_selections` | Left out | Working state that lasts for one bulk edit. | | `import_uploads` | Left out | Transient working state: the previewed file, kept for an hour so it can be applied. The file is the collector's own export, already restored into the lots by applying it. | +| `memberships` | Left out | A household's membership of a user: a fact about a group, not about the collection. A household of one is made with every user, so a person who imports their file is given one by the same rule (spec 090 decision 4). | | `number_entries` | Left out | Working state of one run of adding cards by number. What it adds is already in the lots. | | `number_entry_lines` | Left out | Working state of one run of adding cards by number. What it adds is already in the lots. | | `scanner_sitting_entries` | Left out | Working state of one sitting at the scanner. What a sitting adds is already in the lots. | @@ -474,6 +475,8 @@ Every table in Collector's database that belongs to an account or to a user is l | `accounts.id` | Left out | An internal id. | | `accounts.created_at` | Left out | A fact about the instance, not about the collection. | | `accounts.updated_at` | Left out | A fact about the instance, not about the collection. | +| `accounts.household_id` | Left out | An internal id of a group. A file is one account's collection, and the household is not in it. | +| `accounts.kind` | Left out | Whether the account is a member's own or the household's. A file is one account's collection, and every account that has a file is personal until the shared collection (#384) decides its own file. | | `accounts.currency` | `currency` | | | `lots.id` | Left out | An internal id. | | `lots.account_id` | Left out | An internal id. A file is one account's collection. | @@ -524,6 +527,8 @@ Every table in Collector's database that belongs to an account or to a user is l `currency` is `accounts.currency`. +`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4). + ## Examples | Folder | Holds | diff --git a/spec/export_format_document_spec.rb b/spec/export_format_document_spec.rb index 8a195fa3..77000149 100644 --- a/spec/export_format_document_spec.rb +++ b/spec/export_format_document_spec.rb @@ -76,7 +76,7 @@ def columns_of(tables) expect(status).to include("lots" => "Exported", "users" => "Exported", "locations" => "Exported", "accounts" => "Exported", "deck_proxies" => "Exported") expect(status.select { |_table, mark| mark == "Left out" }.keys) - .to match_array(%w[bulk_removals bulk_selection_marks bulk_selections import_uploads number_entries number_entry_lines scanner_sitting_entries scanner_sittings sessions]) + .to match_array(%w[bulk_removals bulk_selection_marks bulk_selections import_uploads memberships number_entries number_entry_lines scanner_sitting_entries scanner_sittings sessions]) expect(covered_tables.map { |row| row["Why"] }).to all(match(/\S/)) end @@ -86,7 +86,7 @@ def columns_of(tables) expect(left_out.map { |row| row["Why left out"] }).to all(match(/\S/)) expect(mapped.map { |row| row["In the file"] } - fields).to be_empty expect(left_out.map { |row| row["Column"] }).to match_array(%w[ - accounts.id accounts.created_at accounts.updated_at + accounts.id accounts.created_at accounts.updated_at accounts.household_id accounts.kind locations.id locations.account_id locations.name_key locations.created_at locations.updated_at locations.kind locations.parent_id decks.id decks.account_id decks.location_id decks.created_at decks.updated_at deck_proxies.id deck_proxies.account_id deck_proxies.created_at deck_proxies.updated_at diff --git a/spec/migrations/create_households_and_memberships_spec.rb b/spec/migrations/create_households_and_memberships_spec.rb new file mode 100644 index 00000000..b0ad1dc1 --- /dev/null +++ b/spec/migrations/create_households_and_memberships_spec.rb @@ -0,0 +1,160 @@ +require "rails_helper" +require Rails.root.join("db/migrate/20261113120001_create_households_and_memberships") + +# Spec 090 decision 1, decision 5 (a): the tables, the two columns of accounts, their keys and checks. The migration runs outside a +# transaction (disable_ddl_transaction!), so the examples commit their rows (:committed empties the database around them) and go +# down and up through the Migrator, as db:rollback does (the precedent is spec/migrations/add_decks_spec.rb). Rows are made with +# SQL, so the examples don't depend on the models. +RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + let(:stamp) { "2026-11-13 12:00:00" } + + # Every version of the households migrations that exists, in order. Phase 6 adds the third. + def versions = [ 2026_11_13_120001 ] + + def context = ActiveRecord::Base.connection_pool.migration_context + + def down_all = versions.reverse_each { |version| context.run(:down, version) } + + def up_all = versions.each { |version| context.run(:up, version) } + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + def sequence = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") + + # What PRAGMA says about a table: columns, foreign keys, indexes, checks (not the table's SQL text). + def shape(table) + [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }.sort_by(&:first), # by name: a copy moves a column + connection.foreign_keys(table).map { [ it.to_table, it.column, it.on_delete ] }.sort_by(&:to_s), + connection.indexes(table).map { [ it.name, it.columns, it.unique, it.where ] }.sort_by(&:to_s), + connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] + end + + def insert(sql, *binds) = connection.insert(ActiveRecord::Base.sanitize_sql_array([ sql, *binds ])) + + def insert_household(name = "Ann's household") = insert("INSERT INTO households (name, created_at, updated_at) VALUES (?, ?, ?)", name, stamp, stamp) + + def insert_membership(household, user, role) = insert("INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", household, user, role, stamp, stamp) + + # A user with an account in a household of its own; answers [user id, account id, household id]. + def insert_user(number = 1) + household = insert_household("User #{number}'s household") + account = insert("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (?, ?, ?)", stamp, stamp, household) + user = insert("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (?, ?, ?, 'x', ?, ?)", account, "User #{number}", "u#{number}@example.test", stamp, stamp) + [ user, account, household ] + end + + around do |example| + ActiveRecord::Migration.verbose = false + example.run + ensure + up_all # the Migrator skips a version that is recorded + Account.reset_column_information + ActiveRecord::Migration.verbose = true + end + + it "turns the migration's transaction off, so its table copy runs with foreign keys off" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "creates the two tables with their keys, indexes and checks", :aggregate_failures do + expect(connection.columns(:households).map(&:name)).to eq(%w[id name created_at updated_at]) + expect(connection.columns(:memberships).map(&:name)).to eq(%w[id household_id user_id role created_at updated_at]) + expect(connection.check_constraints(:households).map(&:name)).to eq([ "households_name_length" ]) + expect(connection.check_constraints(:memberships).map(&:name)).to eq([ "memberships_role_known" ]) + expect(connection.foreign_keys(:memberships).to_h { [ it.to_table, it.on_delete ] }).to eq("households" => nil, "users" => :cascade) + expect(connection.indexes(:memberships).map { [ it.columns, it.unique ] }).to contain_exactly([ %w[household_id], false ], [ %w[user_id], true ]) + end + + it "adds household_id and kind to accounts, with a no-action key, an index and a named check", :aggregate_failures do + expect(connection.foreign_keys(:accounts).map { [ it.to_table, it.column, it.on_delete ] }).to include([ "households", "household_id", nil ]) + expect(connection.index_exists?(:accounts, :household_id)).to be(true) + kind = connection.columns(:accounts).find { it.name == "kind" } + expect([ kind.null, kind.default ]).to eq([ false, "personal" ]) + expect(connection.check_constraints(:accounts).map(&:name)).to include("accounts_kind_known") + end + + it "refuses a role that is neither manager nor member, and a second membership for a user (AC-2.1, AC-2.2)", :aggregate_failures do + user, _account, household = insert_user + expect { insert_membership(household, user, "boss") }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) + insert_membership(household, user, "manager") + expect { insert_membership(insert_household("Other"), user, "member") }.to raise_error(ActiveRecord::RecordNotUnique) + end + + it "refuses an account kind that is neither personal nor shared (AC-2.5)" do + _user, account, = insert_user + expect { connection.execute("UPDATE accounts SET kind = 'other' WHERE id = #{account}") }.to raise_error(ActiveRecord::StatementInvalid, /accounts_kind_known/) + end + + it "refuses a household name that is blank or over 100 characters, and keeps one of exactly 100 (AC-2.7)", :aggregate_failures do + expect { insert_household("") }.to raise_error(ActiveRecord::StatementInvalid, /households_name_length/) + expect { insert_household("a" * 101) }.to raise_error(ActiveRecord::StatementInvalid, /households_name_length/) + expect { insert_household("a" * 100) }.not_to raise_error + end + + it "refuses to delete a household that holds an account or a member, and deletes a membership with its user (AC-2.9)", :aggregate_failures do + user, _account, household = insert_user + insert_membership(household, user, "manager") + expect { connection.execute("DELETE FROM households WHERE id = #{household}") }.to raise_error(ActiveRecord::InvalidForeignKey) + connection.execute("DELETE FROM users WHERE id = #{user}") + expect([ count("memberships"), count("households") ]).to eq([ 0, 1 ]) + end + + it "runs again with nothing left to do", :aggregate_failures do + insert_user + before = [ count("households"), count("accounts") ] + expect { described_class.new.migrate(:up) }.not_to raise_error + expect([ count("households"), count("accounts") ]).to eq(before) + end + + it "returns to the schema before it going down, and keeps every account row and its id sequence going down and up (AC-3.5)", :aggregate_failures do + insert_user(1) + insert_user(2) + _user, last_account, = insert_user(3) + connection.execute("DELETE FROM users WHERE account_id = #{last_account}") + connection.execute("DELETE FROM accounts WHERE id = #{last_account}") + rows = connection.select_rows("SELECT id, created_at, updated_at, currency FROM accounts ORDER BY id") + seq = sequence + expect(seq).to be > rows.last.first # a copy without a restore would lower it + + down_all + expect(connection.columns(:accounts).map(&:name)).to eq(%w[id created_at updated_at currency]) + expect([ connection.table_exists?(:households), connection.table_exists?(:memberships) ]).to eq([ false, false ]) + expect(sequence).to eq(seq) + up_all + + expect(connection.select_rows("SELECT id, created_at, updated_at, currency FROM accounts ORDER BY id")).to eq(rows) + expect(sequence).to eq(seq) + end + + it "leaves accounts, households and memberships, after down and up, as the schema states them (AC-3.4)", :aggregate_failures do + down_all + up_all + + # The literals are what db/schema.rb dumps for the migration; if a check's expression text differs, correct the literal to the + # dumped text and say so, don't loosen the example. + expect(shape(:accounts)[1..]).to eq([ [ [ "households", "household_id", nil ] ], + [ [ "index_accounts_on_household_id", %w[household_id], false, nil ] ], [ [ "accounts_kind_known", "kind IN ('personal', 'shared')" ] ] ]) + expect(connection.columns(:accounts).map(&:name).sort).to eq(%w[created_at currency household_id id kind updated_at]) + expect(shape(:households)[1..]).to eq([ [], [], [ [ "households_name_length", "length(name) BETWEEN 1 AND 100" ] ] ]) + expect(shape(:memberships)[1..]).to eq([ [ [ "households", "household_id", nil ], [ "users", "user_id", :cascade ] ], + [ [ "index_memberships_on_household_id", %w[household_id], false, nil ], [ "index_memberships_on_user_id", %w[user_id], true, nil ] ], + [ [ "memberships_role_known", "role IN ('manager', 'member')" ] ] ]) + end + + it "refuses to go down while a household has two members or a shared account exists, and changes nothing (AC-3.5)", :aggregate_failures do + one, _a, first = insert_user(1) + two, second_account, second = insert_user(2) + insert_membership(first, one, "manager") + insert_membership(second, two, "manager") + connection.execute("UPDATE memberships SET household_id = #{first} WHERE user_id = #{two}") + + expect { context.run(:down, 2026_11_13_120001) }.to raise_error(StandardError, /1 household with more than one member/) + expect(connection.table_exists?(:households)).to be(true) + + connection.execute("UPDATE memberships SET household_id = #{second} WHERE user_id = #{two}") + connection.execute("UPDATE accounts SET kind = 'shared' WHERE id = #{second_account}") + expect { context.run(:down, 2026_11_13_120001) }.to raise_error(StandardError, /1 shared account/) + expect(connection.column_exists?(:accounts, :kind)).to be(true) + end +end From f822679d327a732cc2b04d8249ed9cb0c378908b Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:03:56 -0500 Subject: [PATCH 10/41] docs(specs): plan 090, phase 6 review fixes (#380) The second Fable review (phase 6 and the User validation) found no blocking issue. Fixed: the check after the copy compares row counts and new broken keys against the state before it, so a dangling row an operator left does not stop every boot; both guards have examples; the refusal checks there is no user; the validation's lambda no longer creates the association instance; the insert guard reads changes(). Co-Authored-By: Claude Sonnet 5.5 --- .../090-every-user-has-a-household/plan.md | 80 +++++++++++++++---- 1 file changed, 66 insertions(+), 14 deletions(-) diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md index 82401b66..9f29afba 100644 --- a/docs/specs/090-every-user-has-a-household/plan.md +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -481,7 +481,18 @@ Append to `spec/models/user_spec.rb` (inside the describe): expect([ Household.exists?(household.id), Membership.exists?(user_id: user.id), Account.exists?(user.account_id) ]).to eq([ false, false, false ]) end - it "keeps the invariants however it is made and deleted (AC-2.8)" do + it "reads no household or membership when a user loaded as a session loads it is validated, twice (AC-4.5)", :aggregate_failures do + user = create(:user) + loaded = Session.includes(user: :account).find(user.sessions.create!.id).user + seen = [] + ActiveSupport::Notifications.subscribed(->(*, payload) { seen << payload[:sql] if payload[:sql].match?(/\b(households|memberships)\b/i) }, "sql.active_record") do + 2.times { expect(loaded).to be_valid } + loaded.update!(collection_view: "table") + end + expect(seen).to be_empty + end + + it "keeps the invariants however it is made and deleted (AC-2.8)" do create(:user) create(:admin).destroy! expect_household_invariants! @@ -652,8 +663,9 @@ Edit `app/models/user.rb`: replace `belongs_to :account, dependent: :destroy` th ... validates :membership, presence: true, on: :create # Only when both are loaded (a new user, or a user a caller loaded both for): an update of a signed-in user, whose account the - # session loads, must not read a household (spec 090 Story 4, AC-4.5). - validate :membership_is_in_the_accounts_household, if: -> { association(:membership).loaded? && association(:account).loaded? } + # session loads, must not read a household (spec 090 Story 4, AC-4.5). `association_cached?` first: `association(:membership)` would + # otherwise create the association instance, and the next validation's autosave check would then read it. + validate :membership_is_in_the_accounts_household, if: -> { association_cached?(:membership) && association(:membership).loaded? && association(:account).loaded? } before_validation :build_account_and_household, on: :create, unless: :account before_destroy :keep_a_manager_for_the_others, prepend: true @@ -840,7 +852,7 @@ class BackfillHouseholds < ActiveRecord::Migration[8.1] connection.transaction do users.each do |user_id, account_id| household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") - raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless household + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless connection.select_value("SELECT changes()").to_i == 1 connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" end @@ -1271,6 +1283,27 @@ RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model d expect { up }.not_to raise_error # a second run changes nothing end + it "refuses to run when foreign keys are enforced, and changes nothing (AC-3.6)", :aggregate_failures do + create(:user) + down + before = counts + + connection.transaction { expect { described_class.new.migrate(:up) }.to raise_error(ActiveRecord::MigrationError, /enforced/) } + + expect([ nullable?, counts ]).to eq([ true, before ]) + end + + it "is not stopped by a dangling row the database already held, and keeps it (AC-3.6)", :aggregate_failures do + create(:user) + down + connection.disable_referential_integrity do # as an operator's sqlite3 session does + connection.execute("INSERT INTO locations (account_id, name, name_key, created_at, updated_at) VALUES (999999, 'Dangling', 'dangling', datetime('now'), datetime('now'))") + end + + expect { up }.not_to raise_error + expect(connection.select_value("SELECT COUNT(*) FROM locations WHERE account_id = 999999")).to eq(1) + end + it "undoes a failure of the copy itself, after the old table was dropped: nothing is lost (AC-3.6)", :aggregate_failures do create(:user) down @@ -1337,8 +1370,9 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] house_stragglers remove_empty_households refuse_accounts_nobody_can_house + before = state keeping_sequence { change_column_null :accounts, :household_id, false } - require_no_broken_keys + require_nothing_lost(before) end end end @@ -1349,14 +1383,15 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] connection.disable_referential_integrity do connection.transaction do require_enforcement_off + before = state keeping_sequence { change_column_null :accounts, :household_id, true } - require_no_broken_keys + require_nothing_lost(before) end end end private - def household_nullable? = columns(:accounts).find { it.name == "household_id" }.null + def household_nullable? = connection.columns(:accounts).find { it.name == "household_id" }&.null # The same SQL as BackfillHouseholds (migrations don't share code: a released one is never edited). def house_stragglers @@ -1367,7 +1402,7 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] users.each do |user_id, account_id| household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") - raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless household + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless connection.select_value("SELECT changes()").to_i == 1 connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" end @@ -1375,16 +1410,31 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] end # Rails runs no check after a copy and, inside a transaction, enforcement can't be switched off: if either went wrong the copy would - # have cascaded silently, so a run that finds enforcement on, or rows left pointing at accounts or households, stops and rolls back. + # have cascaded silently, so a run that finds enforcement on stops and rolls back, and so does one after which a table that + # references accounts holds fewer rows, or a key points at a missing account or household that did not before (a database an + # operator edited with enforcement off may hold dangling rows already; they are not this run's). def require_enforcement_off return if connection.select_value("PRAGMA foreign_keys").to_i.zero? raise ActiveRecord::MigrationError, "foreign keys are enforced inside the table copy, which would delete the collections that reference accounts; nothing was changed" end - def require_no_broken_keys + # Safe: literal table names. The thirteen tables that reference accounts; accounts itself is counted with them. + REFERENCING = %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users].freeze + + def state + rows = ([ "accounts" ] + REFERENCING).to_h { [ it, connection.select_value("SELECT COUNT(*) FROM #{it}").to_i ] } broken = connection.select_rows("PRAGMA foreign_key_check").select { |_table, _row, parent, *| %w[accounts households].include?(parent) } - raise ActiveRecord::MigrationError, "the copy left rows pointing at accounts or households (#{broken.first(3).inspect}); nothing was changed" if broken.any? + { rows:, broken: } + end + + def require_nothing_lost(before) + after = state + lost = after[:rows].reject { |table, count| count == before[:rows][table] } + new_broken = after[:broken] - before[:broken] + return if lost.empty? && new_broken.empty? + + raise ActiveRecord::MigrationError, "the table copy changed row counts #{lost.inspect} or broke keys #{new_broken.first(3).inspect}; nothing was changed" end def remove_empty_households @@ -1392,11 +1442,13 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] end def refuse_accounts_nobody_can_house - orphans = connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL") + # an account whose user has a membership but which was left with no household (a database edited by hand) is put in it + connection.execute "UPDATE accounts SET household_id = (SELECT memberships.household_id FROM users JOIN memberships ON memberships.user_id = users.id WHERE users.account_id = accounts.id) WHERE household_id IS NULL" + orphans = connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL AND NOT EXISTS (SELECT 1 FROM users WHERE users.account_id = accounts.id)") return if orphans.empty? raise ActiveRecord::MigrationError, "accounts #{orphans.first(5).join(", ")} have no household and no user, so no household can be made for them; " \ - "remove them (an account is made with its user and is deleted with it) and start again; nothing was changed" + "remove them with `bin/rails runner 'Account.find(ID).destroy'` (which deletes what they own; a bare DELETE with the sqlite3 command leaves rows behind, because it does not enforce keys) and start again; nothing was changed" end # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). @@ -1410,7 +1462,7 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] end end ``` -- [ ] **6.5 Dump and prepare:** `bin/rails db:migrate` and `bin/rails db:test:prepare`; `git diff db/schema.rb` shows `t.integer "household_id", null: false` and the new version `2026_11_13_120003`, and no other change to `accounts`, its foreign keys or the thirteen tables (any other line is a failure of the copy: stop and report). Run `bin/rspec spec/migrations/make_account_household_required_spec.rb` (6 examples, 0 failures). +- [ ] **6.5 Dump and prepare:** `bin/rails db:migrate` and `bin/rails db:test:prepare`; `git diff db/schema.rb` shows `t.integer "household_id", null: false` and the new version `2026_11_13_120003`, and no other change to `accounts`, its foreign keys or the thirteen tables (any other line is a failure of the copy: stop and report). Run `bin/rspec spec/migrations/make_account_household_required_spec.rb` (every example green; the count is whatever the file holds, record it). - [ ] **6.6 Make the earlier specs follow the final shape.** In `spec/migrations/create_households_and_memberships_spec.rb` and `spec/migrations/backfill_households_spec.rb`, extend the versions so a spec's down and up cover (c): in the first, `def versions = [ 2026_11_13_120001, 2026_11_13_120002, 2026_11_13_120003 ]` and add `require` lines for the other two migrations; in the second, add to the `around` an `ensure` that reruns `context.run(:up, 2026_11_13_120003)` when `accounts.household_id` is nullable after an example that goes down. In `backfill_households_spec.rb`'s batch example and its 10,000-user timing example (both insert accounts by SQL), the raw `INSERT INTO accounts` rows get a household (the timing example inserts one household first and gives every account that id, or runs with (c) down as `forget_households` does): insert a household first and pass `household_id` (the column is NOT NULL now); `forget_households` sets `household_id = NULL`, which the copy refuses: have it first `down` (c) when the column is NOT NULL (`context.run(:down, 2026_11_13_120003)`) and `up` it in `ensure`. In `spec/support/backup_helpers.rb:40` and `spec/lib/collector/backup/snapshot_spec.rb:12` insert a household and give the account its id (`INSERT INTO households (name, created_at, updated_at) VALUES ('h', ?, ?)`, then `accounts (created_at, updated_at, household_id)`). Run `bin/rspec spec/migrations spec/lib/collector/backup spec/backup_guards_spec.rb spec/backup_upgrade_spec.rb` (green). - [ ] **6.7 Commit:** `feat(db): accounts.household_id is NOT NULL, by a copy that loses no row (spec 090 phase 6)` and `test(db): the earlier households specs follow the final shape (spec 090 phase 6)`. From f001fc798254e2d4133d08b70cbece85ef587348 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:05:28 -0500 Subject: [PATCH 11/41] docs(specs): plan 090, no released path leaves an account with no user (#380) Checked in v0.1.0, v0.2.0 and v0.3.0: an account is inserted and deleted by the same transaction as its user, so only a hand edit leaves one. The refusal stands and the Upgrade notes say how to clear it. Phase 6 gains an example that changes the count of accounts and of each of the thirteen tables in turn. Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/plan.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md index 9f29afba..daf913e0 100644 --- a/docs/specs/090-every-user-has-a-household/plan.md +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -1173,6 +1173,8 @@ Run `bin/rspec spec/script/upgrade/seeds_spec.rb`. `script/upgrade/expected_chan **Implements:** decision 5 (c) | **Satisfies:** AC-3.4, AC-3.5, AC-3.6, AC-3.7 | **Interfaces:** `accounts.household_id` is `NOT NULL`; a stragglers step, the copy and the sequence's restore are one transaction opened with enforcement already off. +**Can an account with no user exist without a hand edit? No (checked against v0.1.0, v0.2.0 and v0.3.0, the three released versions).** In each, `User` is `belongs_to :account, dependent: :destroy` with `before_validation :build_account`, so an account is inserted by the same `save` that inserts its user (Rails wraps a save in one transaction, and `users.account_id` makes the account come first), and `Registration#save`, the admin's `User#save` and `User::Command` each run in a transaction of their own; a user's delete is `destroy!`, which deletes the user's row and then the account in one transaction, so an interrupted first run, a crashed delete or an interrupted import leaves both or neither (an import never creates an account). Only a database edited by hand, for instance with the `sqlite3` command, which does not enforce foreign keys, can leave one. So `refuse_accounts_nobody_can_house` stands: it refuses with its advice and changes nothing, and phase 7 says so in `docs/releasing.md`'s Upgrade notes and the release notes (step 7.2): an instance holding an account with no user will not start until it is removed with `bin/rails runner 'Account.find(ID).destroy'` (run with the previous release's image, or after setting the database back from a backup, since the new release stops at that migration). + **Complexity Tracking:** the maintainer ruled (c) on 2026-11-13 on two conditions: the plan review's trace below, and `bin/upgrade-smoke` from the previous release built from its tag before the push (step 7.3). The phase stays the last of the migrations and nothing in phases 1 to 5 depends on it; if the trace finds the copy unsafe, stop and ask the coordinator before the nullable fallback is taken. **Trace to do in the plan review and again before step 6.3:** read `activerecord-8.1.4.1/lib/active_record/connection_adapters/sqlite3_adapter.rb` (`alter_table`, `copy_table`, `change_column_null`, `disable_referential_integrity`) and `.../sqlite3/database_statements.rb` (transaction mode), and confirm, line by line against the code below: (1) `connection.disable_referential_integrity { connection.transaction { … } }` sets `PRAGMA foreign_keys = OFF` before the transaction opens (it is a no-op after); (2) `alter_table`'s own `disable_referential_integrity` and `transaction` nest inside ours without re-enabling enforcement or committing early; (3) the copy preserves the named checks, the foreign key to `households`, the index and the unique and partial indexes of `accounts`; (4) the children's own keys are untouched (no table rename of a child); (5) `lots → accounts` and `users → accounts` are `NO ACTION` and the eleven others `CASCADE` (`db/schema.rb`); (6) what `PRAGMA foreign_key_check` Rails runs after the copy, if any, and that it passes with the stragglers step done. - [ ] **6.1 Failing spec** `spec/migrations/make_account_household_required_spec.rb`. Copy the model calls of `script/upgrade/seeds/next.rb` lines 33 to 60 (scanner sitting, bulk selection and marks, bulk removal, locations, deck, lot in a zone, proxy, number entry, import upload) into a `fill_every_referencing_table(user)` helper that makes a row in each of the thirteen tables: @@ -1283,6 +1285,17 @@ RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model d expect { up }.not_to raise_error # a second run changes nothing end + it "stops and rolls back when a count of accounts or of any of the thirteen referencing tables differs after the copy (AC-3.6)", :aggregate_failures do + create(:user) + migration = described_class.new + before = migration.send(:state) # the check's own example: each table's count, changed in turn, is a loss it must name + ([ "accounts" ] + described_class::REFERENCING).each do |table| + lost = before.merge(rows: before[:rows].merge(table => before[:rows][table] + 1)) + expect { migration.send(:require_nothing_lost, lost) }.to raise_error(ActiveRecord::MigrationError, /#{table}/), "a changed count of #{table} was not caught" + end + expect { migration.send(:require_nothing_lost, before) }.not_to raise_error + end + it "refuses to run when foreign keys are enforced, and changes nothing (AC-3.6)", :aggregate_failures do create(:user) down @@ -1473,7 +1486,7 @@ end **Implements:** FR-5, decision 11, Story 6's evidence | **Satisfies:** AC-4.2, AC-6.6, the documentation criteria (verified by reading, `verification.md`) | **Interfaces:** none. - [ ] **7.1 `docs/architecture/tenancy-and-accounts.md`** (the source of each statement inline): in the Scope bullet "The tenant and its holder" add `Household`, `Membership` (spec 090; ADR 0030); Glossary: replace the Account entry's "created with its user and deleted with it" tail with "created with its user and deleted with it, and in a household (`accounts.household_id`) as a personal account (`accounts.kind`)", add **Household**, **Membership (manager, member)**, **Personal and shared account** entries and the note that "member" also means an admin page's non-admin (spec 090 Terms); the Tenant-owned list gains `memberships` (user-owned, instance-level) and says `households` and `memberships` carry no `account_id`, are never scoped through `Current.account` and no request names one; Invariants: change "One user per account" to the new rule ("`users.account_id` stays unique; a household is a group of accounts; a household always has a manager; a personal account is in its user's household; a household is made with its user by `User#build_account_and_household`" with `spec 090`, ADR 0030), add the cross-account reads sentence (the household tables are read by the model and the proof only); the "Deleting a user" paragraph gains the order (membership, user, account, household when empty) and the last-manager refusal; "Accepted limits" replaces "An account has exactly one user. Personal and household collections side by side will supersede that half of ADR 0017" with what exists now (a household of one for every user; the rest is #381 to #389); Sources gains spec 090, ADR 0030 and #380; Distillation log gains a row once the PR number exists (the distill-feature step writes it). -- [ ] **7.2** `.claude/rules/multi-tenancy.md`: add one line under the ✓ list: "✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request." `docs/architecture/release-and-deploy.md`: in the upgrade proof's paragraph add the two-tier households check (decision 10; `Upgrade::Households`; spec 090 AC-6.1 to AC-6.5) and that `expected_changes.yml` stays empty because `compare.rb` reads only the old database's columns. `docs/releasing.md`'s Upgrade notes: the entry for this migration with its warning (an older image can't create a user after it; going back is a restore of the backup taken before; a failed third migration stops the boot and changes nothing; Kamal's old container fails sign-up for seconds). `CLAUDE.md`: in the Project Status accounts sentence add "every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389)". `docs/adr/0030`: Status becomes Accepted (ADR 0017's "superseded in part by 0030" status line and both ADR index rows were written with the ADR: verify them). `docs/specs/README.md`: spec 004's row gains "Superseded by 090 in part (FR-3's one user per account)" and 090's row Status "Approved". +- [ ] **7.2** `.claude/rules/multi-tenancy.md`: add one line under the ✓ list: "✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request." `docs/architecture/release-and-deploy.md`: in the upgrade proof's paragraph add the two-tier households check (decision 10; `Upgrade::Households`; spec 090 AC-6.1 to AC-6.5) and that `expected_changes.yml` stays empty because `compare.rb` reads only the old database's columns. `docs/releasing.md`'s Upgrade notes: the entry for this migration with its warning, and in it that an instance holding an account with no user (possible only by editing the database by hand) will not start until it is removed with `bin/rails runner 'Account.find(ID).destroy'` (the migration names the ids; the release notes of the version say the same) (an older image can't create a user after it; going back is a restore of the backup taken before; a failed third migration stops the boot and changes nothing; Kamal's old container fails sign-up for seconds). `CLAUDE.md`: in the Project Status accounts sentence add "every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389)". `docs/adr/0030`: Status becomes Accepted (ADR 0017's "superseded in part by 0030" status line and both ADR index rows were written with the ADR: verify them). `docs/specs/README.md`: spec 004's row gains "Superseded by 090 in part (FR-3's one user per account)" and 090's row Status "Approved". - [ ] **7.3 Evidence for `verification.md`** (create it with the headings the other specs use; write each result as it is produced): the phase 0 and each phase's spec runs with counts; the grep and the routes diff of step 4.2; **`bin/upgrade-smoke`**: build the previous release from its tag as `.claude/memory/container-engine-colima-macos.md` says (`git archive` of `v0.3.0` into its own directory, three `--build-arg` flags for the image: `README.md`, Building from source), build the code under test the same way, then `bin/upgrade-smoke ` with `TMPDIR` under the worktree and `/opt/homebrew/bin` appended to `PATH`; expected: the run ends `OK: v0.3.0` and prints `households: ok (backfilled)` for the upgraded and the restored snapshot and `households: ok` for the next one. **Run it on the final commit that will be pushed: after 7.4 and after the implementation review's fixes of 7.5; if code changes after a run, run it again, and record the SHA it ran on in `verification.md`.** **Ask the coordinator for the container engine and a slot before this step.** A failure is read, not re-run blindly (`.claude/memory/reproduce-ci-with-seed.md` for CI). - [ ] **7.4 `bin/ci checks`** (a documentation-and-code branch: it runs RuboCop, YARD, the comment ratchet, Brakeman, bundler-audit, importmap audit and the migration guard): expected green; the comment ratchet needs a summary on each new public method (the models and `Upgrade::Households` have them). Fix and re-run. - [ ] **7.5 The implementation review** (`sdd-review` Mode B, read-only, Opus 5.5) recorded in `verification.md` under `## Implementation review` with the verdict first; then `sdd-superpowers:session-wrap` and `distill-feature` (digest to the coordinator before commit); then one slot for the one full `bin/ci`; push; draft PR closing #380; CI green; `gh pr ready`; `gh pr edit --add-assignee @me`; `worker_done`. From 4203d9001b6486340708828651a194603b6a34a5 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:07:57 -0500 Subject: [PATCH 12/41] feat(models): household and membership, an account's household and kind (spec 090 phase 2) Household, Membership, Account#household and #kind, and a user made with an account in a new household and a manager membership. The membership validation in User guards on association_cached?(:membership) first, as the coordinator's amendment to step 2.5 says, and user_spec gains the AC-4.5 example (it failed against the previous lambda, which let a second validation read memberships and households, and passes with the new one). Co-Authored-By: Claude Sonnet 5.5 --- app/models/account.rb | 4 +++ app/models/household.rb | 27 ++++++++++++++ app/models/membership.rb | 24 +++++++++++++ app/models/user.rb | 39 +++++++++++++++++++- spec/factories/accounts.rb | 8 ++++- spec/models/account_spec.rb | 14 +++++++- spec/models/household_spec.rb | 29 +++++++++++++++ spec/models/membership_spec.rb | 52 +++++++++++++++++++++++++++ spec/models/user_spec.rb | 53 ++++++++++++++++++++++++++++ spec/support/household_invariants.rb | 14 ++++++++ 10 files changed, 261 insertions(+), 3 deletions(-) create mode 100644 app/models/household.rb create mode 100644 app/models/membership.rb create mode 100644 spec/models/household_spec.rb create mode 100644 spec/models/membership_spec.rb create mode 100644 spec/support/household_invariants.rb diff --git a/app/models/account.rb b/app/models/account.rb index c59f5771..bb46560f 100644 --- a/app/models/account.rb +++ b/app/models/account.rb @@ -1,6 +1,9 @@ # The tenant: owns every piece of collection data. One per user (spec 004 FR-3). class Account < ApplicationRecord has_one :user, dependent: nil + # The household this account is in: a user's own account (personal) is in that user's household, and a shared account (none + # exists yet, #384) is the household's. Required, and `NOT NULL` once spec 090's third migration has run. + belongs_to :household # Order matters: the foreign key from a lot to its location refuses deleting a location a lot is in, so the lots go first. # spec/models/account_spec.rb guards it (spec 032 AC-6.6). has_many :lots, dependent: :delete_all @@ -16,6 +19,7 @@ class Account < ApplicationRecord # The currency every price of the account is shown in; a new account starts at the instance default (spec 038 decision 1–2). attribute :currency, :string, default: -> { Rails.configuration.x.default_currency.code } validates :currency, inclusion: { in: Collector::Currency::SYMBOLS.keys, message: "Choose one of the listed currencies." } + enum :kind, { personal: "personal", shared: "shared" }, default: :personal, validate: true def copies_count = lots.sum(:quantity) diff --git a/app/models/household.rb b/app/models/household.rb new file mode 100644 index 00000000..ca1ab1e4 --- /dev/null +++ b/app/models/household.rb @@ -0,0 +1,27 @@ +# A named group of accounts: today always one user's, from a later issue also a family's (spec 090, ADR 0030). It is the +# household's, not the collector's, so it is never scoped through `Current.account` and no request names one. +class Household < ApplicationRecord + NAME_LIMIT = 100 + # The name of a household of one is the user's name followed by this: 12 characters, so the user's name is cut to 88. + SUFFIX = "'s household".freeze + + has_many :memberships, dependent: nil # the foreign key has no action: a household that holds a member can't be deleted + has_many :users, through: :memberships + has_many :accounts, dependent: nil # likewise, so a delete can never reach an account and, through its cascades, a collection + + validates :name, presence: true, length: { maximum: NAME_LIMIT } + + before_destroy :refuse_while_held, prepend: true + + # The name a household of one gets (spec 090 decision 2): the user's first 88 characters and "'s household", at most 100. + # `script/upgrade/households.rb` and the backfill migration build the same name in SQL with `substr`; a spec holds the three equal. + # + # @param user_name [String] the user's name + # @return [String] + def self.name_for(user_name) = "#{user_name.to_s.slice(0, NAME_LIMIT - SUFFIX.length)}#{SUFFIX}" + + private + def refuse_while_held + throw :abort if memberships.exists? || accounts.exists? + end +end diff --git a/app/models/membership.rb b/app/models/membership.rb new file mode 100644 index 00000000..e5f077bc --- /dev/null +++ b/app/models/membership.rb @@ -0,0 +1,24 @@ +# One user in one household, as a manager or a member (spec 090, ADR 0030). A household always has a manager: the model +# refuses to demote the last one and to remove a membership on its own; a membership goes with its user (`User` deletes it +# without these callbacks, and the database cascades). The role check and the unique user are the database's too. +class Membership < ApplicationRecord + belongs_to :household + belongs_to :user + + enum :role, { manager: "manager", member: "member" }, validate: true + + validates :user_id, uniqueness: true, allow_nil: true + validate :keeps_a_manager, on: :update, if: -> { role_changed?(from: "manager") } + + before_destroy { throw :abort } + + # Whether this is its household's only manager. + # + # @return [Boolean] + def last_manager? = manager? && !household.memberships.manager.where.not(id:).exists? + + private + def keeps_a_manager + errors.add(:role, "can't change: a household needs a manager.") unless household.memberships.manager.where.not(id:).exists? + end +end diff --git a/app/models/user.rb b/app/models/user.rb index 3b1aad25..a52134e6 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -5,6 +5,8 @@ class User < ApplicationRecord COLLECTION_VIEWS = %w[grid table].freeze has_secure_password + # Deleted without callbacks (Membership refuses a direct destroy; the database would cascade it anyway), before the user's row. + has_one :membership, dependent: :delete, inverse_of: :user, autosave: true belongs_to :account, dependent: :destroy has_many :sessions, dependent: :delete_all @@ -18,7 +20,15 @@ class User < ApplicationRecord allow_nil: true validates :collection_view, inclusion: { in: COLLECTION_VIEWS } - before_validation :build_account, on: :create, unless: :account + validates :membership, presence: true, on: :create + # Only when both are loaded (a new user, or a user a caller loaded both for): an update of a signed-in user, whose account the + # session loads, must not read a household (spec 090 Story 4, AC-4.5). `association_cached?` first: `association(:membership)` would + # otherwise create the association instance, and the next validation's autosave check would then read it. + validate :membership_is_in_the_accounts_household, if: -> { association_cached?(:membership) && association(:membership).loaded? && association(:account).loaded? } + + before_validation :build_account_and_household, on: :create, unless: :account + before_destroy :keep_a_manager_for_the_others, prepend: true + after_destroy :remove_an_empty_household # after the account's, which `belongs_to … dependent: :destroy` registers above validate :keeps_an_admin, on: :update, if: -> { admin_changed?(from: true, to: false) } @@ -29,6 +39,33 @@ def initial = name.to_s.first.to_s.upcase def end_sessions! = sessions.delete_all private + # A user is made with its account and its household of one, as the manager (spec 090 decision 8): one write, so a failed + # save leaves none of the three. + def build_account_and_household + household = Household.new(name: Household.name_for(name)) + build_account(household:) + build_membership(household:, role: :manager) + end + + def membership_is_in_the_accounts_household + return unless membership && account + + errors.add(:membership, "must be in the household of the account") unless membership.household == account.household + end + + # A household can't lose its last manager by deleting the user while others are in it (spec 090 decision 7); a page that + # makes such a household (#382, #383) gives it a message. Remembers the household for the delete's last step. + def keep_a_manager_for_the_others + @leaving_household = membership&.household + return unless membership&.last_manager? && membership.household.memberships.where.not(id: membership.id).exists? + + errors.add(:base, "is the last manager of a household that has other members.") + throw :abort + end + + # The household of a user whose delete left no member and no account goes with them; `Household` refuses while it holds either. + def remove_an_empty_household = @leaving_household&.destroy + def keeps_an_admin errors.add(:admin, "can't be removed: Collector needs at least one admin.") if User.admins.where.not(id:).none? end diff --git a/spec/factories/accounts.rb b/spec/factories/accounts.rb index e2406375..8a026fb2 100644 --- a/spec/factories/accounts.rb +++ b/spec/factories/accounts.rb @@ -1,5 +1,11 @@ FactoryBot.define do - factory :account + factory :household do + sequence(:name) { |n| "Household #{n}" } + end + + factory :account do + household + end factory :user do sequence(:name) { |n| "Collector #{n}" } diff --git a/spec/models/account_spec.rb b/spec/models/account_spec.rb index 7f63e0de..fb2aef2b 100644 --- a/spec/models/account_spec.rb +++ b/spec/models/account_spec.rb @@ -39,7 +39,7 @@ def default_to(code) = allow(Rails.configuration.x).to receive(:default_currency [ "XYZ", "eur", "", nil, "EUR " ].each do |code| expect(described_class.new(currency: code).tap(&:validate).errors[:currency]).to eq([ "Choose one of the listed currencies." ]) end - expect(described_class.new(currency: "SEK")).to be_valid + expect(build(:account, currency: "SEK")).to be_valid end it "answers its currency as a unit" do @@ -56,4 +56,16 @@ def default_to(code) = allow(Rails.configuration.x).to receive(:default_currency expect(account.reload.currency).to eq("USD") end end + + describe "household and kind (spec 090)" do + it "needs a household (decision 8)" do + expect(described_class.new.tap(&:validate).errors[:household]).to be_present + end + + it "is personal by default and shared or personal only (AC-2.5)", :aggregate_failures do + expect(create(:account).kind).to eq("personal") + expect(build(:account, kind: "other")).not_to be_valid + expect { described_class.connection.execute("UPDATE accounts SET kind = 'other' WHERE id = #{create(:account).id}") }.to raise_error(ActiveRecord::StatementInvalid, /accounts_kind_known/) + end + end end diff --git a/spec/models/household_spec.rb b/spec/models/household_spec.rb new file mode 100644 index 00000000..8e61f0d9 --- /dev/null +++ b/spec/models/household_spec.rb @@ -0,0 +1,29 @@ +require "rails_helper" + +RSpec.describe Household, type: :model do + describe ".name_for" do + it "is the user's name and \"'s household\", the name cut to its first 88 characters (spec 090 decision 2, AC-1.5)", :aggregate_failures do + expect(described_class.name_for("Ada")).to eq("Ada's household") + expect(described_class.name_for("A")).to eq("A's household") + long = described_class.name_for("é" * 100) + expect([ long.length, long ]).to eq([ 100, "#{"é" * 88}'s household" ]) + expect(described_class.name_for("O'Neil")).to eq("O'Neil's household") + end + end + + it "needs a name of 1 to 100 characters (AC-2.7)", :aggregate_failures do + expect(build(:household, name: "").tap(&:validate).errors[:name]).to be_present + expect(build(:household, name: "a" * 101).tap(&:validate).errors[:name]).to be_present + expect(build(:household, name: "a" * 100)).to be_valid + end + + it "can't be destroyed while it holds an account or a member, by the model or by the database (AC-2.9)", :aggregate_failures do + user = create(:user) + household = user.membership.household + + expect(household.destroy).to be(false) + expect(described_class.exists?(household.id)).to be(true) + expect { described_class.connection.execute("DELETE FROM households WHERE id = #{household.id}") }.to raise_error(ActiveRecord::InvalidForeignKey) + expect(create(:household).destroy).to be_truthy + end +end diff --git a/spec/models/membership_spec.rb b/spec/models/membership_spec.rb new file mode 100644 index 00000000..ce3e4fa8 --- /dev/null +++ b/spec/models/membership_spec.rb @@ -0,0 +1,52 @@ +require "rails_helper" + +RSpec.describe Membership, type: :model do + # Puts +member+ into +manager+'s household as a plain member, the way a later issue's join will, by SQL (spec only). + def join(member, manager) + described_class.where(user_id: member.id).update_all(household_id: manager.membership.household_id, role: "member") + Account.where(id: member.account_id).update_all(household_id: manager.membership.household_id) + member.reload + end + + let(:manager) { create(:user) } + let(:member) { join(create(:user), manager) } + + it "has a role of manager or member only (AC-2.1)", :aggregate_failures do + expect(described_class.new(role: "boss").tap(&:validate).errors[:role]).to be_present + bare = create(:user) + described_class.where(user_id: bare.id).delete_all # a user with no membership, by SQL + expect { described_class.insert!({ household_id: manager.membership.household_id, user_id: bare.id, role: "boss" }) }.to raise_error(ActiveRecord::StatementInvalid, /memberships_role_known/) + end + + it "is one per user (AC-2.2)", :aggregate_failures do + second = described_class.new(household: create(:household), user: manager, role: "member") + expect(second).not_to be_valid + expect { described_class.insert!({ household_id: create(:household).id, user_id: manager.id, role: "member" }) }.to raise_error(ActiveRecord::RecordNotUnique) + end + + it "can't be removed on its own, a household of one's included (AC-2.4)", :aggregate_failures do + expect(manager.membership.destroy).to be(false) + expect(member.membership.destroy).to be(false) + expect(described_class.count).to eq(2) + end + + it "keeps a manager: the last one can't be demoted, one of two can (AC-2.3)", :aggregate_failures do + member # joins manager's household + expect(manager.membership.update(role: "member")).to be(false) + expect(manager.membership.errors[:role]).to be_present + member.membership.update!(role: "manager") + expect(manager.membership.reload.update(role: "member")).to be(true) + end + + it "keeps a user from being deleted when they are the last manager of a household with others (AC-2.3, decision 7)", :aggregate_failures do + member + expect { manager.destroy! }.to raise_error(ActiveRecord::RecordNotDestroyed) + expect([ User.exists?(manager.id), described_class.exists?(user_id: manager.id) ]).to eq([ true, true ]) + end + + it "lets a plain member's user be deleted, leaving the household and its manager (decision 7)", :aggregate_failures do + household = member.membership.household + member.destroy! + expect([ Household.exists?(household.id), described_class.where(household_id: household.id).pluck(:user_id) ]).to eq([ true, [ manager.id ] ]) + end +end diff --git a/spec/models/user_spec.rb b/spec/models/user_spec.rb index ddc2d79f..53203e59 100644 --- a/spec/models/user_spec.rb +++ b/spec/models/user_spec.rb @@ -35,4 +35,57 @@ it "uses the first letter of the name as the avatar initial" do expect(build(:user, name: "james").initial).to eq("J") end + + describe "household (spec 090)" do + it "is made with a household named for it, an account in it and a manager membership (AC-1.1)", :aggregate_failures do + user = create(:user, name: "Ada") + + expect(user.membership).to have_attributes(role: "manager", household: have_attributes(name: "Ada's household")) + expect(user.account).to have_attributes(kind: "personal", household: user.membership.household) + expect(Household.count).to eq(1) + end + + it "gives a user with 100 characters a name of exactly 100 (AC-1.5)" do + expect(create(:user, name: "n" * 100).membership.household.name).to eq("#{"n" * 88}'s household") + end + + it "makes nothing for a user that is invalid (AC-1.6)", :aggregate_failures do + expect { build(:user, name: "").save }.not_to change { [ Household.count, Membership.count, Account.count ] } + end + + it "is refused when its membership is in another household than its account's (AC-2.6)", :aggregate_failures do + user = create(:user) + user.account.household = create(:household) + + expect(user).not_to be_valid + expect(user.errors[:membership]).to be_present + expect(create(:user)).to be_valid + end + + it "removes its membership, its account and its household when it is deleted (AC-4.4)", :aggregate_failures do + user = create(:user) + household = user.membership.household + + user.destroy! + + expect([ Household.exists?(household.id), Membership.exists?(user_id: user.id), Account.exists?(user.account_id) ]).to eq([ false, false, false ]) + end + + it "reads no household or membership when a user loaded as a session loads it is validated, twice (AC-4.5)", :aggregate_failures do + user = create(:user) + loaded = Session.includes(user: :account).find(user.sessions.create!.id).user + seen = [] + ActiveSupport::Notifications.subscribed(->(*, payload) { seen << payload[:sql] if payload[:sql].match?(/\b(households|memberships)\b/i) }, "sql.active_record") do + 2.times { expect(loaded).to be_valid } + loaded.update!(collection_view: "table") + end + expect(seen).to be_empty + end + + it "keeps the invariants however it is made and deleted (AC-2.8)" do + create(:user) + create(:admin).destroy! + expect_household_invariants! + end + end end diff --git a/spec/support/household_invariants.rb b/spec/support/household_invariants.rb new file mode 100644 index 00000000..6ea423da --- /dev/null +++ b/spec/support/household_invariants.rb @@ -0,0 +1,14 @@ +# The invariants of a household, checked on the test database (spec 090 AC-2.8): the same four the upgrade proof's tier 1 checks. +module HouseholdInvariants + # Fails the example, naming the broken rule, unless every user has exactly one membership, every household has a + # manager, every account has a household and a personal account's household is its user's. + def expect_household_invariants! + connection = ActiveRecord::Base.connection + expect(connection.select_values("SELECT users.id FROM users LEFT JOIN memberships ON memberships.user_id = users.id GROUP BY users.id HAVING COUNT(memberships.id) <> 1")).to be_empty, "a user without exactly one membership" + expect(connection.select_values("SELECT households.id FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships m WHERE m.household_id = households.id AND m.role = 'manager')")).to be_empty, "a household without a manager" + expect(connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL")).to be_empty, "an account without a household" + expect(connection.select_values("SELECT accounts.id FROM accounts JOIN users ON users.account_id = accounts.id JOIN memberships ON memberships.user_id = users.id WHERE accounts.kind = 'personal' AND accounts.household_id <> memberships.household_id")).to be_empty, "a personal account outside its user's household" + end +end + +RSpec.configure { |config| config.include HouseholdInvariants } From b4505ca0907a266196d2726eaebcc3895f642436 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:08:07 -0500 Subject: [PATCH 13/41] feat(auth): every way of making a user makes a household of one (spec 090 phase 2) Request spec for Story 1: first visitor, open sign-up, an admin's form, the operator command, invalid sign-ups, ignored household/role/kind parameters, and a deleted user's household. Changed from the plan's code, in the AC-1.7 example: the instance already has a user, so it opens sign-up (sign_up_open) before the sign-up post, and signs Ann out (delete session_path) before signing in as the admin, because SessionsController redirects a signed-in visitor. Assertions are unchanged. Co-Authored-By: Claude Sonnet 5.5 --- spec/requests/households_creation_spec.rb | 83 +++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 spec/requests/households_creation_spec.rb diff --git a/spec/requests/households_creation_spec.rb b/spec/requests/households_creation_spec.rb new file mode 100644 index 00000000..73173d2a --- /dev/null +++ b/spec/requests/households_creation_spec.rb @@ -0,0 +1,83 @@ +require "rails_helper" + +RSpec.describe "A household with every user (spec 090 Story 1)", type: :request do + let(:password) { "correct horse battery" } + + def sign_up(**extra) = post(registration_path, params: { user: { name: "Ann", email_address: "ann@example.test", password:, password_confirmation: password, **extra } }) + + def expect_a_household_of_one(user, named: "#{user.name}'s household") + expect(user.membership).to have_attributes(role: "manager", household: have_attributes(name: named)) + expect(user.account).to have_attributes(kind: "personal", household: user.membership.household) + expect(user.membership.household.memberships.count).to eq(1) + expect_household_invariants! + end + + it "gives the first visitor a household of one and lands them in the collection (AC-1.1)", :aggregate_failures do + sign_up + expect(response).to redirect_to(collection_path) + expect_a_household_of_one(User.sole) + expect_household_invariants! + end + + it "gives a person who signs up while sign-up is open their own household (AC-1.2)", :aggregate_failures do + first = create(:admin) + InstanceSetting.current.update!(sign_up_open: true) + sign_up + expect_a_household_of_one(User.find_by!(email_address: "ann@example.test")) + expect(first.membership.household.memberships.count).to eq(1) + end + + it "gives a user an admin adds a household of one and leaves the admin's own alone (AC-1.3)", :aggregate_failures do + admin = sign_in_as(create(:admin)) + household = admin.membership.household + post admin_users_path, params: { user: { name: "Bo", email_address: "bo@example.test", password: } } + expect(response).to redirect_to(admin_users_path) + expect_a_household_of_one(User.find_by!(email_address: "bo@example.test")) + expect(admin.reload.membership.household).to eq(household) + end + + it "gives the operator command's admin a household of one (AC-1.4)", :aggregate_failures do + user, = User::Command.call(email: "ops@example.test", password:) + expect_a_household_of_one(user, named: "ops's household") + end + + it "makes nothing for a sign-up that is invalid: a blank name on an empty instance, a used email while sign-up is open (AC-1.6)", :aggregate_failures do + expect { sign_up(name: "") }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) + create(:admin, email_address: "ann@example.test") + InstanceSetting.current.update!(sign_up_open: true) + expect { sign_up }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) + end + + it "makes nothing for a sign-up that finds a user already there, as when it loses the first-user race (AC-1.6)", :aggregate_failures do + get new_registration_path + create(:admin) + expect { sign_up }.not_to change { [ Household.count, Membership.count, Account.count ] } + expect(response).to have_http_status(:unprocessable_content) + end + + it "ignores a household, role or kind in the parameters of a sign-up and of an admin's form (AC-1.7)", :aggregate_failures do + other = create(:user).membership.household # a household with a manager, so the invariants helper holds + InstanceSetting.current.update!(sign_up_open: true) # an instance with a user refuses a sign-up unless it is open + sign_up(household_id: other.id, role: "member", kind: "shared") + ann = User.find_by!(email_address: "ann@example.test") + expect_a_household_of_one(ann) + expect(ann.membership.household).not_to eq(other) + + delete session_path # the sign-up signed Ann in, and a signed-in visitor can't sign in as someone else + sign_in_as(create(:admin)) + post admin_users_path, params: { user: { name: "Bo", email_address: "bo@example.test", password:, household_id: other.id, role: "member", kind: "shared" } } + expect_a_household_of_one(User.find_by!(email_address: "bo@example.test")) + end + + it "removes a deleted user's household, membership and account, and no other (AC-4.4)", :aggregate_failures do + admin = sign_in_as(create(:admin)) + bo = create(:user, name: "Bo") + delete admin_user_path(bo) + expect(response).to redirect_to(admin_users_path) + expect([ Household.where(name: "Bo's household").count, Membership.where(user_id: bo.id).count, Account.where(id: bo.account_id).count ]).to eq([ 0, 0, 0 ]) + expect(admin.reload.membership).to be_present + expect_household_invariants! + end +end From 0204408dd27274417b3b1686e5aeb2874697abee Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:08:42 -0500 Subject: [PATCH 14/41] docs(specs): plan 090, phase 6 reads and writes main.sqlite_sequence (#380) Phase 1's implementer found that a table copy builds a TEMP table with AUTOINCREMENT, so a bare sqlite_sequence resolves to temp.sqlite_sequence. Phase 6 had the same bug. Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/plan.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/specs/090-every-user-has-a-household/plan.md b/docs/specs/090-every-user-has-a-household/plan.md index daf913e0..99a70c2f 100644 --- a/docs/specs/090-every-user-has-a-household/plan.md +++ b/docs/specs/090-every-user-has-a-household/plan.md @@ -1202,7 +1202,7 @@ RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model d def counts = (tables + %w[accounts households memberships sessions]).to_h { [ it, count(it) ] } - def sequence = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + def sequence = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") def shapes = (%w[accounts] + tables).to_h { [ it, shape(it) ] } @@ -1465,13 +1465,14 @@ class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] end # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). + # Always `main.sqlite_sequence`: the copy builds a TEMP table with AUTOINCREMENT, so a bare name resolves to `temp.sqlite_sequence` (found in phase 1). def keeping_sequence - seq = connection.select_value("SELECT seq FROM sqlite_sequence WHERE name = 'accounts'") + seq = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") yield return unless seq - connection.execute "UPDATE sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" - connection.execute "INSERT INTO sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM sqlite_sequence WHERE name = 'accounts')" + connection.execute "UPDATE main.sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + connection.execute "INSERT INTO main.sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM main.sqlite_sequence WHERE name = 'accounts')" end end ``` From 327983a7a2997f278fa0d6c933894bd2e484c9d2 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:10:24 -0500 Subject: [PATCH 15/41] feat(db): backfill a household of one for every existing user (spec 090 phase 3) Migration (b): every user with no membership gets a household named for them, a manager membership and their account's household_id, in batches of 500 users, each batch its own transaction. Idempotent, plain SQL, sets accounts.household_id only. Changes from the plan's code, none to an assertion: - spec: the timing example reports through RSpec.configuration.reporter (rubocop RSpec/Output forbids puts; the repo's timing specs use the reporter) and times with Process.clock_gettime, because Benchmark is not loaded under Ruby 4.0. - spec: the session example gets :aggregate_failures (it has two expectations; RSpec/MultipleExpectations). - migration: trailing "Safe:" comments on the interpolated SQL lines, as BackfillSessionsLastUsedAt does. Co-Authored-By: Claude Sonnet 5.5 --- .../20261113120002_backfill_households.rb | 31 +++++ db/schema.rb | 2 +- spec/migrations/backfill_households_spec.rb | 116 ++++++++++++++++++ 3 files changed, 148 insertions(+), 1 deletion(-) create mode 100644 db/migrate/20261113120002_backfill_households.rb create mode 100644 spec/migrations/backfill_households_spec.rb diff --git a/db/migrate/20261113120002_backfill_households.rb b/db/migrate/20261113120002_backfill_households.rb new file mode 100644 index 00000000..ab5bf6fa --- /dev/null +++ b/db/migrate/20261113120002_backfill_households.rb @@ -0,0 +1,31 @@ +# Gives every user that has no household one of their own, as its manager (spec 090 decision 5 (b), decision 2). Plain SQL and no +# application model (.claude/rules/sqlite-and-migrations.md). Without a transaction, so each batch of users is its own short write +# transaction: a large instance never holds the write lock for more than one batch, and a run that stopped is finished by the next +# boot. Idempotent: it reads only users with no membership. It sets accounts.household_id and nothing else, updated_at included, so +# the upgrade proof's comparison of accounts needs no expected change. The name is the user's first 88 characters and "'s household" +# (`Household.name_for`; a spec holds the two equal). Nothing to undo going down: the tables go with the first migration. +class BackfillHouseholds < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + BATCH = 500 + + def up + now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) + loop do + users = connection.select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") # Safe: an integer constant + break if users.empty? + + connection.transaction do + users.each do |user_id, account_id| + household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") # Safe: integer ids from the query above + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless connection.select_value("SELECT changes()").to_i == 1 + + connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" # Safe: integers + connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" # Safe: integers + end + end + end + end + + def down; end +end diff --git a/db/schema.rb b/db/schema.rb index 66cbd090..bda15c18 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -10,7 +10,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema[8.1].define(version: 2026_11_13_120001) do +ActiveRecord::Schema[8.1].define(version: 2026_11_13_120002) do create_table "accounts", force: :cascade do |t| t.datetime "created_at", null: false t.datetime "updated_at", null: false diff --git a/spec/migrations/backfill_households_spec.rb b/spec/migrations/backfill_households_spec.rb new file mode 100644 index 00000000..604bbdc5 --- /dev/null +++ b/spec/migrations/backfill_households_spec.rb @@ -0,0 +1,116 @@ +require "rails_helper" +require Rails.root.join("db/migrate/20261113120002_backfill_households") + +# Spec 090 decision 5 (b), AC-3.1 to AC-3.3, AC-3.8, AC-1.5: a household of one for every user that has none, in batches, idempotent, +# touching accounts' household_id only. The users are made with the models and their household rows then removed by SQL, which is the +# state a database from the previous release is in once (a) has run. +RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + + around do |example| + ActiveRecord::Migration.verbose = false + example.run + ensure + ActiveRecord::Migration.verbose = true + end + + def run_up = described_class.new.migrate(:up) + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + # Takes every household away, as the previous release leaves users and accounts. + def forget_households + connection.execute("DELETE FROM memberships") + connection.execute("UPDATE accounts SET household_id = NULL") + connection.execute("DELETE FROM households") + end + + def pairs = connection.select_rows("SELECT users.id, households.name, memberships.role, accounts.kind FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id JOIN accounts ON accounts.id = users.account_id AND accounts.household_id = households.id ORDER BY users.id") + + it "turns the migration's transaction off, so each batch is its own short write" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "gives every user a household named for them, a manager membership and an account in it, and changes no other column (AC-3.1)", :aggregate_failures do + users = Array.new(3) { |i| create(:user, name: "Collector #{i}") } + lot = create(:lot, account: users.first.account) + accounts = connection.select_rows("SELECT id, created_at, updated_at, currency, kind FROM accounts ORDER BY id") + forget_households + + run_up + + expect(pairs).to eq(users.map { [ it.id, "#{it.name}'s household", "manager", "personal" ] }) + expect(connection.select_rows("SELECT id, created_at, updated_at, currency, kind FROM accounts ORDER BY id")).to eq(accounts) + expect(count("lots")).to eq(1) + expect(Lot.find(lot.id).account_id).to eq(users.first.account_id) + end + + it "builds the name the model builds, for 100 characters, an apostrophe and multi-byte characters (AC-1.5)", :aggregate_failures do + names = [ "n" * 100, "O'Neil", "é" * 100, "A" ] + users = names.each_with_index.map { |name, i| create(:user, name:, email_address: "u#{i}@example.test") } + forget_households + + run_up + + expect(connection.select_values("SELECT households.name FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id ORDER BY users.id")) + .to eq(users.map { Household.name_for(it.name) }) + end + + it "adds nothing the second time and makes only the rest after a stop (AC-3.2)", :aggregate_failures do + first, second = Array.new(2) { create(:user) } + forget_households + # the state of a run that stopped after its first batch: the first user's household exists + household = connection.insert("INSERT INTO households (name, created_at, updated_at) VALUES ('kept', datetime('now'), datetime('now'))") + connection.execute("INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household}, #{first.id}, 'manager', datetime('now'), datetime('now'))") + connection.execute("UPDATE accounts SET household_id = #{household} WHERE id = #{first.account_id}") + + run_up + after = [ count("households"), count("memberships"), pairs ] + run_up + + expect([ count("households"), count("memberships"), pairs ]).to eq(after) + expect(connection.select_value("SELECT name FROM households WHERE id = #{household}")).to eq("kept") + expect(connection.select_value("SELECT COUNT(*) FROM memberships WHERE user_id = #{second.id}")).to eq(1) + end + + it "works in batches of at most 500, each its own transaction (AC-3.3)", :aggregate_failures do + expect(described_class::BATCH).to eq(500) + now = "2026-11-13 12:00:00" + stub_const("BackfillHouseholds::BATCH", 2) # 5 users make 3 batches + 5.times { |i| connection.execute("INSERT INTO accounts (created_at, updated_at) VALUES ('#{now}', '#{now}')") ; connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (#{connection.select_value("SELECT MAX(id) FROM accounts")}, 'U#{i}', 'u#{i}@example.test', 'x', '#{now}', '#{now}')") } + forget_households + begins = 0 + counter = ->(*, payload) { begins += 1 if payload[:sql].match?(/\ABEGIN/i) } + + ActiveSupport::Notifications.subscribed(counter, "sql.active_record") { run_up } + + expect(count("households")).to eq(5) + expect(begins).to eq(3) # one transaction for each of the three batches; the empty read that ends the loop opens none + end + + it "makes 10,000 households in 20 batches, and says how long it took (the NFR)", :timing do + connection.execute("WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < 10000) INSERT INTO accounts (created_at, updated_at) SELECT datetime('now'), datetime('now') FROM n") + connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) SELECT id, 'U' || id, 'u' || id || '@example.test', 'x', datetime('now'), datetime('now') FROM accounts") + started = Process.clock_gettime(Process::CLOCK_MONOTONIC) + run_up + seconds = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started + RSpec.configuration.reporter.message "backfill of 10,000 users: #{seconds.round(2)} s" + expect(count("households")).to eq(10_000) + end + + it "succeeds on a database with no user (AC-3.4)" do + expect { run_up }.not_to raise_error + end + + it "leaves every session row as it was (AC-3.8)", :aggregate_failures do + user = create(:user) + session = user.sessions.create! + forget_households + rows = connection.select_rows("SELECT * FROM sessions ORDER BY id") + + run_up + + expect(connection.select_rows("SELECT * FROM sessions ORDER BY id")).to eq(rows) + expect(session.reload).to be_present + end +end From 3300ae71a5f73ece431bba305cfecab620ceafab Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:13:29 -0500 Subject: [PATCH 16/41] test(households): nothing looks different, the format's coverage and the round trip (spec 090 phase 4) households_invisible_spec passed on its first run (nothing visible was added; it is a guard). household_untouched_spec: its first run failed on a spec bug (target was a lazy let, created after the first household_facts call, so Household.pluck differed); target is now let!. No assertion was changed. Coverage (4.3, 4.4) was done in phase 1. Co-Authored-By: Claude Sonnet 5.5 --- .../household_untouched_spec.rb | 33 +++++++++++++ spec/requests/households_invisible_spec.rb | 49 +++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 spec/models/collection_import/household_untouched_spec.rb create mode 100644 spec/requests/households_invisible_spec.rb diff --git a/spec/models/collection_import/household_untouched_spec.rb b/spec/models/collection_import/household_untouched_spec.rb new file mode 100644 index 00000000..997af10b --- /dev/null +++ b/spec/models/collection_import/household_untouched_spec.rb @@ -0,0 +1,33 @@ +require "rails_helper" + +# Spec 090 AC-5.2: an apply writes the collection and the currency and never the account's household, kind or membership. +RSpec.describe "Applying an export and the household (spec 090 AC-5.2)", type: :model do + let(:source) { create(:user, name: "Source", email_address: "source@example.test") } + let!(:target) { create(:user, name: "Target", email_address: "target@example.test") } + + def export(user) = JSON.parse(CollectionExport.new(account: user.account, at: Time.utc(2026, 11, 13, 12)).to_json) + + def apply!(user, data) + upload = store_import!(account: user.account, data:) + CollectionImport::Apply.new(account: user.account.reload, fingerprint: upload&.fingerprint, confirmed: true).call + end + + def household_facts(user) + user.reload + [ user.membership.household_id, user.membership.role, user.account.household_id, user.account.kind, Household.pluck(:id, :name).sort ] + end + + before { held_lot(source.account, import_printing, quantity: 3, finish: "foil") } + + it "changes no household, role or kind, whether the file restores a collection or replaces one (AC-5.2)", :aggregate_failures do + document = export(source) + expect(document.keys).not_to include("household", "households", "memberships") + before = [ household_facts(source), household_facts(target) ] + + apply!(target, document) + apply!(source, document.merge("lots" => [])) + + expect([ household_facts(source), household_facts(target) ]).to eq(before) + expect(target.account.lots.sum(:quantity)).to eq(3) + end +end diff --git a/spec/requests/households_invisible_spec.rb b/spec/requests/households_invisible_spec.rb new file mode 100644 index 00000000..0073b6c8 --- /dev/null +++ b/spec/requests/households_invisible_spec.rb @@ -0,0 +1,49 @@ +require "rails_helper" + +# Spec 090 Story 4, decision 9 (i): no page says "household", and no request of these pages reads a household or a membership. +RSpec.describe "A household is invisible (spec 090 Story 4)", type: :request do + let!(:admin) { create(:admin, name: "Ann") } + let(:paths) { [ collection_path, collection_path(view: "table"), admin_users_path, more_path, catalog_entries_path ] } + + def statements_naming(pattern, &) + seen = [] + ActiveSupport::Notifications.subscribed(->(*, payload) { seen << payload[:sql] if payload[:sql].match?(pattern) }, "sql.active_record", &) + seen + end + + it "shows no page of a collector or an admin the word, and runs no query naming the new tables (AC-4.1, AC-4.5)", :aggregate_failures do + sign_in_as(admin) + seen = statements_naming(/\b(households|memberships)\b/i) do + paths.each do |path| + get path + expect(response).to have_http_status(:ok), "#{path} answered #{response.status}" + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + expect(seen).to be_empty + end + + it "shows a collector who is not an admin no page with the word (AC-4.1)", :aggregate_failures do + sign_in_as(create(:user)) + [ collection_path, collection_path(view: "table"), more_path, catalog_entries_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + + it "shows no signed-out page the word (AC-4.1)", :aggregate_failures do + [ new_session_path, new_registration_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end + end + + it "lists users as before: a row, its copies and the same buttons (AC-4.5)", :aggregate_failures do + sign_in_as(admin) + bo = create(:user, name: "Bo") + create(:lot, account: bo.account, quantity: 3) + get admin_users_path + expect(response.body).to include("Bo", '3') + expect(response.body.downcase).not_to include("household") + end +end From 3c145daeaf470233d858051182902934a905256a Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:16:52 -0500 Subject: [PATCH 17/41] test(upgrade): the proof checks every user's household on the upgraded, restored and next snapshots (spec 090 phase 5) Upgrade::Households (script/upgrade/households.rb) reads a snapshot with SQLite directly in two tiers: invariants true on every snapshot from this release on (one membership per user, a manager per household, every account in a household, a personal account in its user's household), and the backfill's own facts (count, role, kind, name), checked only when the old snapshot had no households table. The harness runs it in `compare` (the upgraded and the restored snapshots, failing with AC-6.1) and in `next_release`. Prints `households: ok` or `households: ok (backfilled)`. expected_changes.yml stays `changes: []` and seed_exceptions.yml is unchanged. Changes from the plan: in households_spec the "joined household" fixture also deletes household 2 (user 2 leaves its household of one when it joins household 1; otherwise tier 1 rightly names a household with no manager); the assertion is unchanged. The harness examples are in a new describe block with its own run_with. The seeds_spec example passed on its first run (a guard: the seed's users are made through the models, which make the households). Co-Authored-By: Claude Sonnet 5.5 --- script/upgrade/harness.rb | 9 +++ script/upgrade/households.rb | 63 ++++++++++++++++++++ spec/script/upgrade/harness_spec.rb | 40 +++++++++++-- spec/script/upgrade/households_spec.rb | 80 ++++++++++++++++++++++++++ spec/script/upgrade/seeds_spec.rb | 6 ++ 5 files changed, 194 insertions(+), 4 deletions(-) create mode 100644 script/upgrade/households.rb create mode 100644 spec/script/upgrade/households_spec.rb diff --git a/script/upgrade/harness.rb b/script/upgrade/harness.rb index 3072c071..5b1b4914 100644 --- a/script/upgrade/harness.rb +++ b/script/upgrade/harness.rb @@ -12,6 +12,7 @@ require_relative "release" require_relative "snapshot" require_relative "compare" +require_relative "households" require_relative "rules" require_relative "engine" require_relative "web" @@ -331,6 +332,11 @@ def compare(old, upgraded, optional: [], emptied: []) problems = Compare.integrity(upgraded) raise Failure, "AC-3.5: #{problems.first(3).join("; ")}" if problems.any? + problems = Households.problems(upgraded, backfill: !Compare.tables(old).include?("households")) + raise Failure, "AC-6.1: #{problems.first(3).join("; ")}" if problems.any? + + @out.puts "households: ok#{" (backfilled)" unless Compare.tables(old).include?("households")}" + missing = (working_versions + @release.migration_versions(@plan[:ref])) - upgraded.execute("SELECT version FROM schema_migrations").flatten raise Failure, "AC-3.2: schema_migrations of the upgraded database lacks #{missing}" if missing.any? @@ -503,6 +509,9 @@ def next_release problems = Compare.seed_problems(database) raise Failure, problems.join("\n") if problems.any? + + households = Households.problems(database) + raise Failure, "AC-6.1: the next snapshot: #{households.first(3).join("; ")}" if households.any? end # -- containers ---------------------------------------------------------------------------------------------- diff --git a/script/upgrade/households.rb b/script/upgrade/households.rb new file mode 100644 index 00000000..8fa3c9ff --- /dev/null +++ b/script/upgrade/households.rb @@ -0,0 +1,63 @@ +# The household facts of a snapshot of the primary database (spec 090 decision 10, ADR 0030). Tier 1 is the invariants of a +# household, true on every snapshot of every release from this one on, so it names no count: it must stay true when joined households +# and shared accounts exist (#381, #384). Tier 2 is the backfill's own facts, checked only when the old snapshot had no households. +# It reads the snapshot with SQLite directly, never through the application, as `compare.rb` does. +require_relative "compare" + +module Upgrade + module Households + SUFFIX = "'s household".freeze + + # What is wrong with the households of +db+, as sentences naming the rule and the first row. + # + # @param db [SQLite3::Database] a snapshot, opened read-only + # @param backfill [Boolean] also check the facts a backfill of a database with no households leaves (decision 10, tier 2) + # @return [Array] empty when the snapshot holds every rule checked + def self.problems(db, backfill: false) + tables = Compare.tables(db) + return [ "the snapshot has no households or memberships table" ] unless (%w[households memberships] - tables).empty? + + found = invariants(db) + found += backfilled(db) if backfill + found + end + + # Tier 1: the rules every household snapshot holds, whatever joined households or shared accounts exist later. + # + # @api private + def self.invariants(db) + found = [] + db.execute("SELECT users.id, COUNT(memberships.id) FROM users LEFT JOIN memberships ON memberships.user_id = users.id GROUP BY users.id HAVING COUNT(memberships.id) <> 1 LIMIT 3").each do |id, count| + found << "user #{id} has #{count} memberships, not exactly one" + end + db.execute("SELECT id FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE household_id = households.id AND role = 'manager') LIMIT 3").flatten.each do |id| + found << "household #{id} has no manager" + end + db.execute("SELECT id FROM accounts WHERE household_id IS NULL OR household_id NOT IN (SELECT id FROM households) LIMIT 3").flatten.each do |id| + found << "account #{id} has no household" + end + db.execute("SELECT accounts.id FROM accounts JOIN users ON users.account_id = accounts.id JOIN memberships ON memberships.user_id = users.id WHERE accounts.kind = 'personal' AND accounts.household_id <> memberships.household_id LIMIT 3").flatten.each do |id| + found << "account #{id} is in a household other than its user's" + end + found + end + private_class_method :invariants + + # Tier 2: the facts a backfill of a database that had no households leaves (one household of one per user, named by the rule). + # + # @api private + def self.backfilled(db) + found = [] + users = db.get_first_value("SELECT COUNT(*) FROM users") + households = db.get_first_value("SELECT COUNT(*) FROM households") + found << "the households' count #{households} is not the users' #{users}" unless users == households + found << "a membership's role is not manager" if db.get_first_value("SELECT COUNT(*) FROM memberships WHERE role <> 'manager'").positive? + found << "an account's kind is not personal" if db.get_first_value("SELECT COUNT(*) FROM accounts WHERE kind <> 'personal'").positive? + db.execute("SELECT users.id FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id WHERE households.name <> substr(users.name, 1, 88) || '#{SUFFIX.gsub("'", "''")}' LIMIT 3").flatten.each do |id| + found << "the name of user #{id}'s household is not the form of spec 090 decision 2" + end + found + end + private_class_method :backfilled + end +end diff --git a/spec/script/upgrade/harness_spec.rb b/spec/script/upgrade/harness_spec.rb index be9e3dfc..239f93eb 100644 --- a/spec/script/upgrade/harness_spec.rb +++ b/spec/script/upgrade/harness_spec.rb @@ -58,13 +58,18 @@ def pair(primary, versions: prices_migrations, **options) end # A seeded database as a release makes it (accounts, users, 2,000 lots, sign-up open), with the given migrations recorded. - def seeded(name, versions, lots: 2000, extra: "") + # `households: false` is a release from before spec 090: no households or memberships tables and no household columns. + def seeded(name, versions, lots: 2000, extra: "", households: true) + accounts = households ? "id INTEGER PRIMARY KEY, household_id INTEGER, kind TEXT DEFAULT 'personal'" : "id INTEGER PRIMARY KEY" + users = households ? "id INTEGER PRIMARY KEY, password_digest TEXT, name TEXT, account_id INTEGER" : "id INTEGER PRIMARY KEY, password_digest TEXT" build_database(File.join(dir, "#{name}.sqlite3")) do |db| db.execute_batch(<<~SQL) - CREATE TABLE accounts (id INTEGER PRIMARY KEY); CREATE TABLE users (id INTEGER PRIMARY KEY, password_digest TEXT); + CREATE TABLE accounts (#{accounts}); CREATE TABLE users (#{users}); CREATE TABLE lots (id INTEGER PRIMARY KEY, account_id INTEGER); CREATE TABLE instance_settings (id INTEGER PRIMARY KEY, sign_up_open BOOLEAN); CREATE TABLE schema_migrations (version TEXT PRIMARY KEY); - INSERT INTO accounts VALUES (1), (2); INSERT INTO users VALUES (1, 'x'), (2, 'y'); INSERT INTO instance_settings VALUES (1, 1); + #{households ? "INSERT INTO accounts VALUES (1, 1, 'personal'), (2, 2, 'personal'); INSERT INTO users VALUES (1, 'x', 'Ada', 1), (2, 'y', 'Grace', 2);" : "INSERT INTO accounts VALUES (1), (2); INSERT INTO users VALUES (1, 'x'), (2, 'y');"} + INSERT INTO instance_settings VALUES (1, 1); + #{households ? "CREATE TABLE households (id INTEGER PRIMARY KEY, name TEXT); CREATE TABLE memberships (id INTEGER PRIMARY KEY, household_id INTEGER, user_id INTEGER, role TEXT); INSERT INTO households VALUES (1, 'Ada''s household'), (2, 'Grace''s household'); INSERT INTO memberships VALUES (1, 1, 1, 'manager'), (2, 2, 2, 'manager');" : ""} WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < #{lots}) INSERT INTO lots SELECT i, 1 FROM n; #{versions.map { "INSERT INTO schema_migrations VALUES ('#{it}');" }.join} #{extra} @@ -72,7 +77,6 @@ def seeded(name, versions, lots: 2000, extra: "") end end - def with_tmpdir(path) original = ENV.fetch("TMPDIR", nil) ENV["TMPDIR"] = path @@ -95,6 +99,34 @@ def harness(previous: "auto", engine: self.engine, env: { "GITHUB_ACTIONS" => "t expect(out.string).to include("== upgrade", "== snapshot and compare", "== behaviour", "skipped").and match(/^OK: v0\.2\.1/) end + describe "the households (spec 090 AC-6.1, decision 10)" do + def run_with(old: seeded("old", old_migrations), upgraded: pair(seeded("upgraded", all_migrations)), nxt: pair(seeded("next", all_migrations))) + harness(engine: FakeEngine.new(databases: [ old, upgraded, nxt ], log:)).call + end + + it "checks the backfill's facts when the previous release had no households, and says so (AC-6.1)", :aggregate_failures do + expect(run_with(old: seeded("old", old_migrations, households: false))).to eq(0) + expect(out.string).to include("households: ok (backfilled)").and match(/^OK: v0\.2\.1/) + end + + it "checks only the invariants when the previous release already had them, and says so (AC-6.1)", :aggregate_failures do + expect(run_with).to eq(0) + expect(out.string).to match(/^households: ok$/) + end + + it "fails naming AC-6.1 when the upgrade leaves a user with no membership (AC-6.1, AC-6.5)", :aggregate_failures do + upgraded = pair(seeded("upgraded", all_migrations, extra: "DELETE FROM memberships WHERE id = 2;")) + expect(run_with(old: seeded("old", old_migrations, households: false), upgraded:)).to eq(1) + expect(out.string).to include("AC-6.1", "user 2 has 0 memberships") + end + + it "fails naming AC-6.1 and the next snapshot when only the next release's seed breaks the rule (AC-6.1, AC-6.4)", :aggregate_failures do + nxt = pair(seeded("next", all_migrations, extra: "DELETE FROM memberships WHERE id = 2;")) + expect(run_with(nxt:)).to eq(1) + expect(out.string).to include("AC-6.1: the next snapshot", "user 2 has 0 memberships") + end + end + describe "the prices database (spec 081 AC-6.4, AC-6.5)" do let(:prices_line) { "prices: 5 tables, schema 20261113100001, WITHOUT ROWID on current_prices and price_changes" } diff --git a/spec/script/upgrade/households_spec.rb b/spec/script/upgrade/households_spec.rb new file mode 100644 index 00000000..7a72185c --- /dev/null +++ b/spec/script/upgrade/households_spec.rb @@ -0,0 +1,80 @@ +require "rails_helper" +require Rails.root.join("script/upgrade/households").to_s + +# Spec 090 decision 10 and AC-6.5: the two tiers of the check on fixture snapshots, never on the application. +RSpec.describe Upgrade::Households, :upgrade do + # The three names of AC-1.5, so this check, the model and the backfill agree on the same cases. + let(:names) { [ "n" * 100, "O'Neil", "é" * 100 ] } + + def snapshot(users:, memberships: nil, households: nil, accounts: nil) + path = File.join(upgrade_dir, "snap#{SecureRandom.hex(3)}.sqlite3") + build_database(path) do |db| + db.execute_batch(<<~SQL) + CREATE TABLE households (id INTEGER PRIMARY KEY, name TEXT); + CREATE TABLE memberships (id INTEGER PRIMARY KEY, household_id INTEGER, user_id INTEGER, role TEXT); + CREATE TABLE accounts (id INTEGER PRIMARY KEY, household_id INTEGER, kind TEXT); + CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, account_id INTEGER); + SQL + users.each_with_index do |name, i| + id = i + 1 + db.execute("INSERT INTO households VALUES (?, ?)", [ id, "#{name[0, 88]}'s household" ]) unless households == :none + db.execute("INSERT INTO accounts VALUES (?, ?, 'personal')", [ id, id ]) + db.execute("INSERT INTO users VALUES (?, ?, ?)", [ id, name, id ]) + db.execute("INSERT INTO memberships VALUES (?, ?, ?, 'manager')", [ id, id, id ]) unless memberships == :none + end + yield db if block_given? + end + open_database(path) + end + + it "passes a clean snapshot in both tiers (AC-6.1)" do + expect(described_class.problems(snapshot(users: names), backfill: true)).to be_empty + end + + it "names a user with no membership (AC-6.5)" do + expect(described_class.problems(snapshot(users: names, memberships: :none))).to include(a_string_matching(/user .* membership/)) + end + + it "names a household with no manager (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE memberships SET role = 'member' WHERE id = 1") } + expect(described_class.problems(db)).to include(a_string_matching(/household 1 .*manager/)) + end + + it "names an account in another user's household (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE accounts SET household_id = 2 WHERE id = 1") } + expect(described_class.problems(db)).to include(a_string_matching(/account 1 .*household/)) + end + + it "names an account with no household (AC-6.5)" do + db = snapshot(users: names) { it.execute("UPDATE accounts SET household_id = NULL WHERE id = 3") } + expect(described_class.problems(db)).to include(a_string_matching(/account 3 .*no household/)) + end + + it "checks the backfill's facts only when asked: a name that differs, a count that differs, a role or a kind that differs (AC-6.5)", :aggregate_failures do + renamed = snapshot(users: names) { it.execute("UPDATE households SET name = 'Someone else' WHERE id = 2") } + expect(described_class.problems(renamed)).to be_empty + expect(described_class.problems(renamed, backfill: true)).to include(a_string_matching(/name/)) + extra = snapshot(users: names) { it.execute("INSERT INTO households VALUES (9, 'x'); INSERT INTO memberships VALUES (9, 9, 99, 'manager')") } + expect(described_class.problems(extra, backfill: true)).to include(a_string_matching(/count/)) + member = snapshot(users: names) { it.execute("INSERT INTO memberships VALUES (10, 1, 98, 'member')") } + expect(described_class.problems(member, backfill: true)).to include(a_string_matching(/role/)) + shared = snapshot(users: names) { it.execute("UPDATE accounts SET kind = 'shared' WHERE id = 1") } + expect(described_class.problems(shared, backfill: true)).to include(a_string_matching(/kind/)) + end + + it "lets a joined household and a shared account pass tier 1 (it must outlive #381 and #384)" do + db = snapshot(users: names) do + it.execute("UPDATE memberships SET household_id = 1, role = 'member' WHERE user_id = 2") + it.execute("UPDATE accounts SET household_id = 1 WHERE id = 2") + it.execute("DELETE FROM households WHERE id = 2") # user 2 left its household of one when it joined household 1 + it.execute("INSERT INTO accounts VALUES (10, 1, 'shared')") + end + expect(described_class.problems(db)).to be_empty + end + + it "fails a snapshot with no households table (AC-6.1)" do + path = File.join(upgrade_dir, "old.sqlite3") + build_database(path) { it.execute_batch("CREATE TABLE users (id INTEGER PRIMARY KEY)") } + expect(described_class.problems(open_database(path))).to eq([ "the snapshot has no households or memberships table" ]) + end +end diff --git a/spec/script/upgrade/seeds_spec.rb b/spec/script/upgrade/seeds_spec.rb index eb2cb3dc..334fca74 100644 --- a/spec/script/upgrade/seeds_spec.rb +++ b/spec/script/upgrade/seeds_spec.rb @@ -27,6 +27,12 @@ def summary(output) = JSON.parse(output.lines.grep(/\AUPGRADE-SEED-COMPLETE /).l expect(facts["locations"]).to eq([ "Binder A", "Shelf B" ]) end + it "gives every seeded user a household of one, so households and memberships hold rows (spec 090 AC-6.4)", :aggregate_failures do + run_seed("next.rb") + expect([ Household.count, Membership.count ]).to eq([ User.count, User.count ]) + expect(Upgrade::SeedExceptions.load(seeds.join("../seed_exceptions.yml"), "next").entries).to be_empty + end + it "fills every table db/schema.rb creates, except those listed under next (AC-2.6, FR-7)" do run_seed("next.rb") schema_tables = Rails.root.join("db/schema.rb").read.scan(/^ create_table "([^"]+)"/).flatten From 45bf33b38221a99fd034cb91541eb46c29b81a3e Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:18:28 -0500 Subject: [PATCH 18/41] feat(db): accounts.household_id is NOT NULL, by a copy that loses no row (spec 090 phase 6) Migration (c) of spec 090 decision 5: accounts.household_id becomes NOT NULL through Rails' table copy, run outside Rails' own transaction with foreign keys already off, so the thirteen tables that reference accounts (eleven ON DELETE CASCADE) keep every row. A stragglers step, the copy and the sequence's restore commit together or not at all; counts and new broken keys are checked after the copy. Re-trace of activerecord-8.1.4.1 (sqlite3_adapter.rb, sqlite3/database_statements.rb), step 6.3: 1. disable_referential_integrity runs PRAGMA foreign_keys = OFF before our transaction opens (it is a no-op after BEGIN). 2. alter_table's own disable_referential_integrity (restores to our OFF) and transaction (joins ours) neither re-enable enforcement nor commit early. 3. copy_table copies columns, the foreign key to households, the check constraints, and copy_table_indexes the plain, unique and partial indexes (the spec compares the shapes). 4. move_table creates and drops tables by name; no child is renamed, so the children's keys are untouched. 5. db/schema.rb: lots and users reference accounts with NO ACTION, the other eleven with CASCADE. 6. Rails runs no foreign_key_check after a copy (only fixtures call check_all_foreign_keys_valid!); the migration's own state check does. db/schema.rb differs by the version and `t.integer "household_id", null: false` only. Co-Authored-By: Claude Sonnet 5.5 --- ...3120003_make_account_household_required.rb | 117 +++++++++++ db/schema.rb | 4 +- .../make_account_household_required_spec.rb | 188 ++++++++++++++++++ 3 files changed, 307 insertions(+), 2 deletions(-) create mode 100644 db/migrate/20261113120003_make_account_household_required.rb create mode 100644 spec/migrations/make_account_household_required_spec.rb diff --git a/db/migrate/20261113120003_make_account_household_required.rb b/db/migrate/20261113120003_make_account_household_required.rb new file mode 100644 index 00000000..3f6e947f --- /dev/null +++ b/db/migrate/20261113120003_make_account_household_required.rb @@ -0,0 +1,117 @@ +# Makes accounts.household_id NOT NULL, so "every account has a household" is a fact of the database (spec 090 decision 5 (c)). +# SQLite can't add NOT NULL in place; Rails copies accounts (create the new table, copy the rows, drop the old, rename). accounts is +# referenced by thirteen tables, eleven of them ON DELETE CASCADE (lots and users are NO ACTION), so a copy run with foreign keys +# enforced deletes every collector's decks, places, proxies and working state, silently. Rails turns enforcement off for a copy only +# outside a transaction, so the migration runs without Rails' own (`disable_ddl_transaction!`) and opens one itself with enforcement +# already off: the stragglers step, the copy and the sequence's restore commit together or not at all, so a failed run changes nothing +# and the next boot runs it whole. The stragglers step is spec 049's safety-net UPDATE: BackfillHouseholds is recorded once it has +# run, but a user can be made after it (an older container serving during a deploy), and without this the NOT NULL would fail on +# every boot after. It also removes a household an older container's delete of a user left with no member and no account. An +# account with no user and no household can't be housed by any step: the migration says so and changes nothing. +# @see file:tenancy-and-accounts.md +class MakeAccountHouseholdRequired < ActiveRecord::Migration[8.1] + disable_ddl_transaction! + + BATCH = 500 + + def up + return unless household_nullable? + + connection.disable_referential_integrity do + connection.transaction do + require_enforcement_off + house_stragglers + remove_empty_households + refuse_accounts_nobody_can_house + before = state + keeping_sequence { change_column_null :accounts, :household_id, false } + require_nothing_lost(before) + end + end + end + + def down + return if household_nullable? + + connection.disable_referential_integrity do + connection.transaction do + require_enforcement_off + before = state + keeping_sequence { change_column_null :accounts, :household_id, true } + require_nothing_lost(before) + end + end + end + + private + def household_nullable? = connection.columns(:accounts).find { it.name == "household_id" }&.null + + # The same SQL as BackfillHouseholds (migrations don't share code: a released one is never edited). + def house_stragglers + now = connection.quote(Time.current.utc.strftime("%Y-%m-%d %H:%M:%S.%6N")) + loop do + users = connection.select_rows("SELECT id, account_id FROM users WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.user_id = users.id) ORDER BY id LIMIT #{BATCH.to_i}") + break if users.empty? + + users.each do |user_id, account_id| + household = connection.insert("INSERT INTO households (name, created_at, updated_at) SELECT substr(name, 1, 88) || '''s household', #{now}, #{now} FROM users WHERE id = #{user_id.to_i}") + raise ActiveRecord::MigrationError, "no household was made for user #{user_id}" unless connection.select_value("SELECT changes()").to_i == 1 + connection.execute "INSERT INTO memberships (household_id, user_id, role, created_at, updated_at) VALUES (#{household.to_i}, #{user_id.to_i}, 'manager', #{now}, #{now})" + connection.execute "UPDATE accounts SET household_id = #{household.to_i} WHERE id = #{account_id.to_i}" + end + end + end + + # Rails runs no check after a copy and, inside a transaction, enforcement can't be switched off: if either went wrong the copy would + # have cascaded silently, so a run that finds enforcement on stops and rolls back, and so does one after which a table that + # references accounts holds fewer rows, or a key points at a missing account or household that did not before (a database an + # operator edited with enforcement off may hold dangling rows already; they are not this run's). + def require_enforcement_off + return if connection.select_value("PRAGMA foreign_keys").to_i.zero? + + raise ActiveRecord::MigrationError, "foreign keys are enforced inside the table copy, which would delete the collections that reference accounts; nothing was changed" + end + + # Safe: literal table names. The thirteen tables that reference accounts; accounts itself is counted with them. + REFERENCING = %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users].freeze + + def state + rows = ([ "accounts" ] + REFERENCING).to_h { [ it, connection.select_value("SELECT COUNT(*) FROM #{it}").to_i ] } + broken = connection.select_rows("PRAGMA foreign_key_check").select { |_table, _row, parent, *| %w[accounts households].include?(parent) } + { rows:, broken: } + end + + def require_nothing_lost(before) + after = state + lost = after[:rows].reject { |table, count| count == before[:rows][table] } + new_broken = after[:broken] - before[:broken] + return if lost.empty? && new_broken.empty? + + raise ActiveRecord::MigrationError, "the table copy changed row counts #{lost.inspect} or broke keys #{new_broken.first(3).inspect}; nothing was changed" + end + + def remove_empty_households + connection.execute "DELETE FROM households WHERE NOT EXISTS (SELECT 1 FROM memberships WHERE memberships.household_id = households.id) AND NOT EXISTS (SELECT 1 FROM accounts WHERE accounts.household_id = households.id)" + end + + def refuse_accounts_nobody_can_house + # an account whose user has a membership but which was left with no household (a database edited by hand) is put in it + connection.execute "UPDATE accounts SET household_id = (SELECT memberships.household_id FROM users JOIN memberships ON memberships.user_id = users.id WHERE users.account_id = accounts.id) WHERE household_id IS NULL" + orphans = connection.select_values("SELECT id FROM accounts WHERE household_id IS NULL AND NOT EXISTS (SELECT 1 FROM users WHERE users.account_id = accounts.id)") + return if orphans.empty? + + raise ActiveRecord::MigrationError, "accounts #{orphans.first(5).join(", ")} have no household and no user, so no household can be made for them; " \ + "remove them with `bin/rails runner 'Account.find(ID).destroy'` (which deletes what they own; a bare DELETE with the sqlite3 command leaves rows behind, because it does not enforce keys) and start again; nothing was changed" + end + + # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). + # Always `main.sqlite_sequence`: the copy builds a TEMP table with AUTOINCREMENT, so a bare name resolves to `temp.sqlite_sequence` (found in phase 1). + def keeping_sequence + seq = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") + yield + return unless seq + + connection.execute "UPDATE main.sqlite_sequence SET seq = #{seq.to_i} WHERE name = 'accounts' AND seq < #{seq.to_i}" + connection.execute "INSERT INTO main.sqlite_sequence (name, seq) SELECT 'accounts', #{seq.to_i} WHERE NOT EXISTS (SELECT 1 FROM main.sqlite_sequence WHERE name = 'accounts')" + end +end diff --git a/db/schema.rb b/db/schema.rb index bda15c18..d8303797 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -10,12 +10,12 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema[8.1].define(version: 2026_11_13_120002) do +ActiveRecord::Schema[8.1].define(version: 2026_11_13_120003) do create_table "accounts", force: :cascade do |t| t.datetime "created_at", null: false t.datetime "updated_at", null: false t.string "currency", default: "USD", null: false - t.integer "household_id" + t.integer "household_id", null: false t.string "kind", default: "personal", null: false t.index ["household_id"], name: "index_accounts_on_household_id" t.check_constraint "kind IN ('personal', 'shared')", name: "accounts_kind_known" diff --git a/spec/migrations/make_account_household_required_spec.rb b/spec/migrations/make_account_household_required_spec.rb new file mode 100644 index 00000000..e2cc48ba --- /dev/null +++ b/spec/migrations/make_account_household_required_spec.rb @@ -0,0 +1,188 @@ +require "rails_helper" +require Rails.root.join("db/migrate/20261113120003_make_account_household_required") + +# Spec 090 decision 5 (c), AC-3.4 to AC-3.7: the copy of accounts must lose no row of the thirteen tables that reference it, keep their +# keys, keep the id sequence, give a straggler a household, and change nothing when it fails. It commits its rows and runs through the +# Migrator OUTSIDE any example transaction (inside one, enforcement stays on and the cascades would fire). +RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model do + let(:connection) { ActiveRecord::Base.connection } + let(:tables) { %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users] } + + def context = ActiveRecord::Base.connection_pool.migration_context + + def down = context.run(:down, 2026_11_13_120003) + + def up = context.run(:up, 2026_11_13_120003) + + def nullable? = connection.columns(:accounts).find { it.name == "household_id" }.null + + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + + def counts = (tables + %w[accounts households memberships sessions]).to_h { [ it, count(it) ] } + + def sequence = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") + + def shapes = (%w[accounts] + tables).to_h { [ it, shape(it) ] } + + def shape(table) + [ connection.columns(table).map { [ it.name, it.sql_type, it.null, it.default ] }, + connection.foreign_keys(table).map { [ it.to_table, it.column, it.on_delete ] }.sort_by(&:to_s), + connection.indexes(table).map { [ it.name, it.columns, it.unique, it.where ] }.sort_by(&:to_s), + connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] + end + + # One row in each of the thirteen tables, as the upgrade seed makes them (script/upgrade/seeds/next.rb). + def fill_every_referencing_table(user) + account = user.account + entry = import_printing + Lot.add!(account:, entry:, quantity: 2, finish: "nonfoil") + Scanner::Sitting.add!(account:, printing: entry, finish: "nonfoil", reading_key: SecureRandom.hex(16)) + session = user.sessions.create! + selection = BulkSelection.for(session) + account.lots.limit(3).pluck(:id).each { |lot_id| selection.marks.create!(lot_id:, account:) } + BulkRemoval.create!(session:, account:, copies: 2, + lots_data: [ { "catalog_entry_id" => entry.id, "finish" => "foil", "condition" => "damaged", "price_paid_cents" => nil, "quantity" => 2 } ]) + Location.create!(account:, name: "Binder A") + deck = Deck::Creation.new(account:, name: "Upgrade Deck", format: "mtg/modern").tap(&:save!).deck + Lot.add!(account:, entry:, quantity: 3, finish: "nonfoil", location: deck.zone("main")) + Deck::Proxy.add!(account:, zone: deck.zone("sideboard"), entry:, finish: "foil", quantity: 2) + NumberEntry.add!(account:, entry:, quantity: 2, finish: "foil", line_key: SecureRandom.hex(16)) + ImportUpload.store!(account:, bytes: "{}", plan: ImportHelpers::FakePlan.new("fingerprint")) + end + + around do |example| + ActiveRecord::Migration.verbose = false + example.run + ensure + up if nullable? + Account.reset_column_information + ActiveRecord::Migration.verbose = true + end + + it "turns the migration's transaction off, so the copy runs with foreign keys off, and opens its own" do + expect(described_class.disable_ddl_transaction).to be(true) + end + + it "makes the column NOT NULL and keeps every row of the thirteen tables, going down and up (AC-3.6)", :aggregate_failures do + fill_every_referencing_table(create(:user)) + before = counts + expect(before.values_at(*tables)).to all(be >= 1) + + down + expect(nullable?).to be(true) + expect(counts).to eq(before) + up + + expect(nullable?).to be(false) + expect(counts).to eq(before) + expect(connection.select_rows("PRAGMA foreign_key_check")).to be_empty + expect(connection.select_values("PRAGMA integrity_check")).to eq([ "ok" ]) + end + + it "keeps the foreign keys, checks and indexes of accounts and the thirteen tables, going down and up (AC-3.6)" do + before = shapes + down + up + expect(shapes).to eq(before) + end + + it "keeps the id sequence of accounts, so a deleted account's id is not issued again (AC-3.6)", :aggregate_failures do + create(:user) + gone = create(:user) + connection.execute("DELETE FROM memberships WHERE user_id = #{gone.id}") + gone_account = gone.account_id + connection.execute("DELETE FROM users WHERE id = #{gone.id}") + connection.execute("DELETE FROM accounts WHERE id = #{gone_account}") + seq = sequence + expect(seq).to be >= gone_account + + down + expect(sequence).to eq(seq) + up + expect(sequence).to eq(seq) + expect(connection.insert("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (datetime('now'), datetime('now'), #{create(:household).id})")).to be > seq + end + + it "gives a user made between (b) and (c) a household, removes an emptied household, and succeeds (AC-3.7)", :aggregate_failures do + straggler = create(:user, name: "Late") + down + connection.execute("DELETE FROM memberships WHERE user_id = #{straggler.id}") + connection.execute("UPDATE accounts SET household_id = NULL WHERE id = #{straggler.account_id}") + emptied = connection.insert("INSERT INTO households (name, created_at, updated_at) VALUES ('Left behind', datetime('now'), datetime('now'))") + + up + + expect(connection.select_value("SELECT households.name FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id WHERE users.id = #{straggler.id}")).to eq("Late's household") + expect(connection.select_value("SELECT COUNT(*) FROM households WHERE id = #{emptied}")).to eq(0) + expect { up }.not_to raise_error # a second run changes nothing + end + + it "stops and rolls back when a count of accounts or of any of the thirteen referencing tables differs after the copy (AC-3.6)", :aggregate_failures do + create(:user) + migration = described_class.new + before = migration.send(:state) # the check's own example: each table's count, changed in turn, is a loss it must name + ([ "accounts" ] + described_class::REFERENCING).each do |table| + lost = before.merge(rows: before[:rows].merge(table => before[:rows][table] + 1)) + expect { migration.send(:require_nothing_lost, lost) }.to raise_error(ActiveRecord::MigrationError, /#{table}/), "a changed count of #{table} was not caught" + end + expect { migration.send(:require_nothing_lost, before) }.not_to raise_error + end + + it "refuses to run when foreign keys are enforced, and changes nothing (AC-3.6)", :aggregate_failures do + create(:user) + down + before = counts + + connection.transaction { expect { described_class.new.migrate(:up) }.to raise_error(ActiveRecord::MigrationError, /enforced/) } + + expect([ nullable?, counts ]).to eq([ true, before ]) + end + + it "is not stopped by a dangling row the database already held, and keeps it (AC-3.6)", :aggregate_failures do + create(:user) + down + connection.disable_referential_integrity do # as an operator's sqlite3 session does + connection.execute("INSERT INTO locations (account_id, name, name_key, created_at, updated_at) VALUES (999999, 'Dangling', 'dangling', datetime('now'), datetime('now'))") + end + + expect { up }.not_to raise_error + expect(connection.select_value("SELECT COUNT(*) FROM locations WHERE account_id = 999999")).to eq(1) + end + + it "undoes a failure of the copy itself, after the old table was dropped: nothing is lost (AC-3.6)", :aggregate_failures do + create(:user) + down + orphan = connection.insert("INSERT INTO accounts (created_at, updated_at) VALUES (datetime('now'), datetime('now'))") + before = counts.merge("sequence" => sequence) + # Fault injection: the refusal that would stop the run before any DDL is switched off, so the copy-back's NOT NULL fails. + migration = Class.new(described_class) { private def refuse_accounts_nobody_can_house = nil }.new + + expect { migration.migrate(:up) }.to raise_error(ActiveRecord::StatementInvalid, /NOT NULL/) + + expect([ nullable?, connection.select_value("PRAGMA foreign_keys") ]).to eq([ true, 1 ]) + expect(counts.merge("sequence" => sequence)).to eq(before) + expect(connection.select_value("SELECT COUNT(*) FROM accounts WHERE id = #{orphan}")).to eq(1) + connection.execute("DELETE FROM accounts WHERE id = #{orphan}") + expect { up }.not_to raise_error + end + + it "changes nothing when it refuses, and succeeds on the next run (AC-3.6, the Error Scenarios row)", :aggregate_failures do + create(:user) + straggler = create(:user, name: "Late") + down + connection.execute("DELETE FROM memberships WHERE user_id = #{straggler.id}") + connection.execute("UPDATE accounts SET household_id = NULL WHERE id = #{straggler.account_id}") + orphan = connection.insert("INSERT INTO accounts (created_at, updated_at) VALUES (datetime('now'), datetime('now'))") # no user: no step can house it + before = counts.merge("sequence" => sequence) + + expect { up }.to raise_error(StandardError, /accounts? .*#{orphan}.* no user/m) + + expect(nullable?).to be(true) + expect(connection.select_value("PRAGMA foreign_keys")).to eq(1) + expect(counts.merge("sequence" => sequence)).to eq(before) + expect(connection.select_value("SELECT COUNT(*) FROM memberships WHERE user_id = #{straggler.id}")).to eq(0) # the stragglers step was rolled back + connection.execute("DELETE FROM accounts WHERE id = #{orphan}") + expect { up }.not_to raise_error + expect(nullable?).to be(false) + end +end From 40f3553db48f0fb75769b9895f91433c335b429b Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:25:58 -0500 Subject: [PATCH 19/41] test(db): the earlier households specs follow the final shape (spec 090 phase 6) accounts.household_id is NOT NULL now, so the specs that put a household-less account in the database take migration (c) down first and put it back in an around ensure (backfill_households_spec), the versions list of create_households_and_memberships_spec covers all three migrations, and the raw account inserts of the backup writer, the snapshot spec and the backfill's batch example make a household first. One change beyond the plan: the backfill's batch example runs `described_class.name.constantize` instead of `described_class`. The Migrator reloads a migration's file with `load`, which makes a new class under the same name once any example has taken (c) down; `stub_const("BackfillHouseholds::BATCH", 2)` then stubs the new class while described_class is the old one, so under some seeds (38210) the example saw one batch instead of three. The timing example takes (c) down before inserting its household-less accounts, so its assertion of 10,000 households is unchanged. Co-Authored-By: Claude Sonnet 5.5 --- spec/lib/collector/backup/snapshot_spec.rb | 4 ++- spec/migrations/backfill_households_spec.rb | 25 ++++++++++++++++--- .../create_households_and_memberships_spec.rb | 6 +++-- spec/support/backup_helpers.rb | 3 ++- 4 files changed, 31 insertions(+), 7 deletions(-) diff --git a/spec/lib/collector/backup/snapshot_spec.rb b/spec/lib/collector/backup/snapshot_spec.rb index 5b0c75c3..8d71e62a 100644 --- a/spec/lib/collector/backup/snapshot_spec.rb +++ b/spec/lib/collector/backup/snapshot_spec.rb @@ -9,7 +9,9 @@ def prices_copy(name) = scratch_path(name).tap { described_class.take(source: Collector::Backup::Databases.configured.prices, into: it) } def insert_account_and_user(database, email: "a@example.test") - database.execute("INSERT INTO accounts (created_at, updated_at) VALUES (?, ?)", [ now, now ]) + database.execute("INSERT INTO households (name, created_at, updated_at) VALUES ('h', ?, ?)", [ now, now ]) + household_id = database.last_insert_row_id + database.execute("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (?, ?, ?)", [ now, now, household_id ]) account_id = database.last_insert_row_id database.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (?, 'N', ?, 'd', ?, ?)", [ account_id, email, now, now ]) diff --git a/spec/migrations/backfill_households_spec.rb b/spec/migrations/backfill_households_spec.rb index 604bbdc5..a3a7c336 100644 --- a/spec/migrations/backfill_households_spec.rb +++ b/spec/migrations/backfill_households_spec.rb @@ -7,19 +7,35 @@ RSpec.describe BackfillHouseholds, :backup, :committed, type: :model do let(:connection) { ActiveRecord::Base.connection } + # Migration (c), which makes accounts.household_id NOT NULL, comes after this one: the examples that put a household-less account + # in the database take (c) down first (forget_households) and the around puts it back. around do |example| ActiveRecord::Migration.verbose = false example.run ensure + context.run(:up, 2026_11_13_120003) if household_nullable? + Account.reset_column_information ActiveRecord::Migration.verbose = true end + def context = ActiveRecord::Base.connection_pool.migration_context + + def household_nullable? = connection.columns(:accounts).find { it.name == "household_id" }.null + def run_up = described_class.new.migrate(:up) + # An account in a household of its own, by SQL, as the NOT NULL column needs; answers the account's id. + def insert_account(now = "2026-11-13 12:00:00") + household = connection.insert("INSERT INTO households (name, created_at, updated_at) VALUES ('h', '#{now}', '#{now}')") + connection.insert("INSERT INTO accounts (created_at, updated_at, household_id) VALUES ('#{now}', '#{now}', #{household})") + end + def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec - # Takes every household away, as the previous release leaves users and accounts. + # Takes every household away, as the previous release leaves users and accounts: (c) goes down first, because it forbids an account + # with no household. def forget_households + context.run(:down, 2026_11_13_120003) unless household_nullable? connection.execute("DELETE FROM memberships") connection.execute("UPDATE accounts SET household_id = NULL") connection.execute("DELETE FROM households") @@ -77,18 +93,21 @@ def pairs = connection.select_rows("SELECT users.id, households.name, membership expect(described_class::BATCH).to eq(500) now = "2026-11-13 12:00:00" stub_const("BackfillHouseholds::BATCH", 2) # 5 users make 3 batches - 5.times { |i| connection.execute("INSERT INTO accounts (created_at, updated_at) VALUES ('#{now}', '#{now}')") ; connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (#{connection.select_value("SELECT MAX(id) FROM accounts")}, 'U#{i}', 'u#{i}@example.test', 'x', '#{now}', '#{now}')") } + 5.times { |i| connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (#{insert_account(now)}, 'U#{i}', 'u#{i}@example.test', 'x', '#{now}', '#{now}')") } forget_households begins = 0 counter = ->(*, payload) { begins += 1 if payload[:sql].match?(/\ABEGIN/i) } - ActiveSupport::Notifications.subscribed(counter, "sql.active_record") { run_up } + # The class by its name, not described_class: the Migrator reloads a migration's file (`load`), which makes a new class under the same name once any + # example has taken (c) down, and the constant stubbed above belongs to the new one. + ActiveSupport::Notifications.subscribed(counter, "sql.active_record") { described_class.name.constantize.new.migrate(:up) } expect(count("households")).to eq(5) expect(begins).to eq(3) # one transaction for each of the three batches; the empty read that ends the loop opens none end it "makes 10,000 households in 20 batches, and says how long it took (the NFR)", :timing do + context.run(:down, 2026_11_13_120003) # (c) forbids the household-less accounts of the previous release's database connection.execute("WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < 10000) INSERT INTO accounts (created_at, updated_at) SELECT datetime('now'), datetime('now') FROM n") connection.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) SELECT id, 'U' || id, 'u' || id || '@example.test', 'x', datetime('now'), datetime('now') FROM accounts") started = Process.clock_gettime(Process::CLOCK_MONOTONIC) diff --git a/spec/migrations/create_households_and_memberships_spec.rb b/spec/migrations/create_households_and_memberships_spec.rb index b0ad1dc1..7cdf94dc 100644 --- a/spec/migrations/create_households_and_memberships_spec.rb +++ b/spec/migrations/create_households_and_memberships_spec.rb @@ -1,5 +1,7 @@ require "rails_helper" require Rails.root.join("db/migrate/20261113120001_create_households_and_memberships") +require Rails.root.join("db/migrate/20261113120002_backfill_households") +require Rails.root.join("db/migrate/20261113120003_make_account_household_required") # Spec 090 decision 1, decision 5 (a): the tables, the two columns of accounts, their keys and checks. The migration runs outside a # transaction (disable_ddl_transaction!), so the examples commit their rows (:committed empties the database around them) and go @@ -9,8 +11,8 @@ let(:connection) { ActiveRecord::Base.connection } let(:stamp) { "2026-11-13 12:00:00" } - # Every version of the households migrations that exists, in order. Phase 6 adds the third. - def versions = [ 2026_11_13_120001 ] + # Every version of the households migrations that exists, in order. + def versions = [ 2026_11_13_120001, 2026_11_13_120002, 2026_11_13_120003 ] def context = ActiveRecord::Base.connection_pool.migration_context diff --git a/spec/support/backup_helpers.rb b/spec/support/backup_helpers.rb index a14d8978..c6fb7e62 100644 --- a/spec/support/backup_helpers.rb +++ b/spec/support/backup_helpers.rb @@ -37,7 +37,8 @@ module BackupHelpers created = index database.execute("INSERT INTO price_changes (catalog_entry_id, price_source_id, price_series_id, recorded_on, cents) VALUES (?, 1, 1, '2026-10-21', 1)", [ index ]) elsif kind == "pairs" - database.execute("INSERT INTO accounts (created_at, updated_at) VALUES (?, ?)", [ stamp, stamp ]) + database.execute("INSERT INTO households (name, created_at, updated_at) VALUES ('Pair #{index}', ?, ?)", [ stamp, stamp ]) + database.execute("INSERT INTO accounts (created_at, updated_at, household_id) VALUES (?, ?, ?)", [ stamp, stamp, database.last_insert_row_id ]) created = database.last_insert_row_id database.execute("INSERT INTO users (account_id, name, email_address, password_digest, created_at, updated_at) VALUES (?, ?, ?, 'x', ?, ?)", [ created, "Pair #{index}", "pair#{index}@example.test", stamp, stamp ]) From cc696ca4ab1c612a6b07487c8de38f966325e82c Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:27:47 -0500 Subject: [PATCH 20/41] docs(architecture): tenancy and accounts gains the household (spec 090) Scope, glossary (household, membership, personal and shared account, the second meaning of member), the instance-level tables, the invariants, the cross-account reads, the order of a user's delete, the accepted limit and the sources, each with its spec and ADR. Co-Authored-By: Claude Sonnet 5.5 --- docs/architecture/tenancy-and-accounts.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/docs/architecture/tenancy-and-accounts.md b/docs/architecture/tenancy-and-accounts.md index 4bfd9492..dafc02f4 100644 --- a/docs/architecture/tenancy-and-accounts.md +++ b/docs/architecture/tenancy-and-accounts.md @@ -6,7 +6,7 @@ Who may use an instance and whose data a request sees: the account that owns eve **Covers:** -- The tenant and its holder: `Account`, `User`, `Session`, `InstanceSetting`, `Current` (`Current.session`, `.user`, `.account`) (spec 004 FR-2, FR-3, FR-4; [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)). +- The tenant and its holder: `Account`, `User`, `Household`, `Membership`, `Session`, `InstanceSetting`, `Current` (`Current.session`, `.user`, `.account`) (spec 004 FR-2, FR-3, FR-4; spec 090; [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md); [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md)). - Resolving the tenant and failing closed: `Authentication`, `FirstRun`, `AdminOnly`, `ApplicationController`, and the list of endpoints that skip the sign-in requirement (spec 004 FR-2, AC-3.1, AC-4.6, AC-5.6). - First run and sign-up: `Registration`, `RegistrationsController`, the sign-up setting and `Admin::SignUpSettingsController` (spec 004 Story 3, Story 4, FR-4). - Sign-in, the sign-in rate limit, sessions, their idle expiry and sign-out: `SessionsController` (spec 004 Story 4, AC-4.8, FR-2, NFR Security; spec 049). @@ -26,27 +26,31 @@ Who may use an instance and whose data a request sees: the account that owns eve ## Glossary -- **Account:** the tenant: the owner of all of one collector's data (`Account`). It is created with its user and deleted with it (spec 004 FR-3; ADR 0017). It carries the currency prices are shown in, which is [collection and lots](collection-and-lots.md)'. +- **Account:** the tenant: the owner of all of one collector's data (`Account`). It is created with its user and deleted with it, and is in a household (`accounts.household_id`, required) as a personal account (`accounts.kind`) (spec 004 FR-3; spec 090 decisions 1, 6; ADR 0017; ADR 0030). It carries the currency prices are shown in, which is [collection and lots](collection-and-lots.md)'. - **User:** a person who signs in (`User`): a name (at most 100 characters), an email address, a password kept only as a salted hash (`has_secure_password`), the admin flag, and a saved collection view. Every user has exactly one account and every account exactly one user (`users.account_id` is unique) (spec 004 FR-2, FR-3, AC-4.3; ADR 0017). +- **Household:** a named group of accounts (`Household`, `households`: `name`, at most 100 characters). Today every household is one user's, a **household of one**, made with the user and named for them: the user's first 88 characters and `'s household` (`Household.name_for`). It is stored, so a later rename of the user changes nothing, and no page shows it. A household is the household's, not the collector's: it has no `account_id`, is never scoped through `Current.account` and no request names one (spec 090 decisions 1, 2, 12; ADR 0030). +- **Membership (manager, member):** the row that puts one user in one household with a role (`Membership`, `memberships`: `household_id`, `user_id` unique, `role`). The role is `manager` or `member`, an enum stated by a database `CHECK` too; a household always has a manager. No page shows a role (spec 090 decisions 1, 3; ADR 0030). +- **Personal and shared account:** an account's `kind` is `personal` (a member's own and private: every account today) or `shared` (owned by the household and used by no single user). The schema and the model allow a shared account; nothing makes one, and its rules (one per household, no user) are #384's (spec 090 decision 1, AC-2.5; ADR 0030 decision 1). - **Session:** a signed-in browser (`Session`): a row per sign-in, found by the signed cookie `session_id`. It carries no tenant data itself, and no address or user agent; `Current.user` and `Current.account` come from it (spec 004 FR-2; spec 049 decision 5). Its **stamp** (`last_used_at`) is the time it was created or last used; a session whose stamp is 30 days old or more is **expired**, and an expired session is no session; the nightly `Session::SweepJob` deletes it (spec 049 decisions 1, 2, 3, 4). - **`Current`:** the request's session, user and account (`Current.session`, `.user`, `.account`). `Current.account` is the signed-in user's account, nil without a session (spec 004 FR-2, FR-3; `app/models/current.rb`). - **Collector / visitor:** a collector is a signed-in user (`collector?`, `Current.user.present?`); a visitor has no session. A visitor exists only where an endpoint is open (see the list under Invariants) (spec 023 Terms). -- **Admin / member:** a user with or without the `admin` flag. The first user is an admin; the flag governs the instance's operations and gives no view into another account's collection (spec 004 FR-3; ADR 0017 Consequences). "Member" is spec 015's word (spec 015 Glossary). +- **Admin / member:** a user with or without the `admin` flag. The first user is an admin; the flag governs the instance's operations and gives no view into another account's collection (spec 004 FR-3; ADR 0017 Consequences). "Member" is spec 015's word (spec 015 Glossary). **"Member" also means a household's non-manager** (the `member` role); the two are independent: the admin flag governs the instance, the role governs a household, and no page shows a role yet (spec 090 Terms; ADR 0030 Consequences). - **First run:** an instance with no user. The first visitor becomes its admin (spec 004 Story 3, FR-4). An upgraded instance starts there (spec 004 NFR Reliability, AC-3.5). - **Sign-up setting:** `instance_settings.sign_up_open`, one instance-wide switch in the table's one row (id 1): closed once the first user exists, opened and closed by an admin (spec 004 FR-4, AC-3.3; #229). -- **Tenant-owned table:** a table with `account_id`, a foreign key to `accounts` and an index. Today: `lots`, `locations`, `decks`, `deck_proxies`, `import_uploads`, `number_entries`, `number_entry_lines`, `scanner_sittings`, `scanner_sitting_entries`, `bulk_selections`, `bulk_selection_marks`, `bulk_removals` (spec 004 AC-12.4; `db/schema.rb`). `users` carries `account_id` too, unique. The catalog's tables carry none. +- **Tenant-owned table:** a table with `account_id`, a foreign key to `accounts` and an index. Today: `lots`, `locations`, `decks`, `deck_proxies`, `import_uploads`, `number_entries`, `number_entry_lines`, `scanner_sittings`, `scanner_sitting_entries`, `bulk_selections`, `bulk_selection_marks`, `bulk_removals` (spec 004 AC-12.4; `db/schema.rb`). `users` carries `account_id` too, unique. The catalog's tables carry none. **`households` and `memberships` are not tenant-owned:** `households` carries neither `account_id` nor `user_id`, and `memberships` is user-owned (`user_id`, unique) and instance-level. Neither is scoped through `Current.account`, no request names one, and both are read through the user or the account (spec 090 decisions 1, 12, FR-1; ADR 0030). - **Operator command:** `bin/rails "collector:user[EMAIL]"` (`User::Command`): sets a user's password, or creates an admin (spec 004 AC-5.7, AC-3.6). ## Invariants - **The account is the only tenant key.** Every collector-owned table carries `account_id` with a foreign key and an index; the catalog's tables have none (spec 004 AC-12.4; ADR 0017; `.claude/rules/multi-tenancy.md`). The export document's Coverage tables must list any new table with `account_id` or `user_id` or the suite fails ([data portability](data-portability.md); spec 017 FR-9, AC-3.4, AC-3.5; ADR 0020). -- **One user per account.** `users.account_id` has a unique index and the account is built with the user (`before_validation :build_account`); a user is the only way an account exists (spec 004 FR-3; ADR 0017). Specs shouldn't assume it will always hold where that costs nothing to avoid (ADR 0017 Consequences). +- **One account per user, and a household is a group of accounts.** `users.account_id` has a unique index and the account is built with the user; a user is the only way an account exists, and a personal account has exactly one user. What changed is that an account is in a household, so "an account has exactly one user" is no longer the shape of tenancy (spec 004 FR-3 in part; spec 090 decisions 1, 6; [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md), which supersedes the one-user half of ADR 0017). Specs shouldn't assume one user per account will always hold where that costs nothing to avoid (ADR 0017 Consequences). +- **A household is made with its user, always has a manager, and a personal account is in its user's household.** `User#build_account_and_household` (a `before_validation` on create, when the user has no account) builds the household, the account and the manager membership in the write that saves the user, so first run, sign-up, an admin's form and `collector:user` each get one and a failed save leaves none of the three (spec 090 decision 8, AC-1.1 to AC-1.6). **A household always has a manager** by the model and by what SQLite can state: a membership's role is checked by the database and its user is unique, and both foreign keys into `households` are no action, so a household that holds a member or an account can't be deleted (the model refuses too: `Household` aborts a destroy while it holds either); the model refuses to demote the last manager (a `Membership` validation), to remove a membership on its own (`before_destroy` aborts), and to delete a user who is the last manager of a household that has other members (`User` asks `Membership#last_manager?`). There is no trigger, because `db/schema.rb` doesn't dump one (spec 090 decisions 1, 3, AC-2.1 to AC-2.4, AC-2.9; ADR 0030 decisions 2, 5). **A personal account's household is its user's:** `accounts.household_id` equals the user's membership's `household_id`, which the database can't compare, so the model validates it when both are loaded (a new user, never on a signed-in request's update) and a guard spec and the upgrade proof check it on real rows (spec 090 decision 6, AC-2.6, AC-2.8, AC-6.1; `spec/support/household_invariants.rb`). `household_id`, `role` and `kind` are never permitted from parameters (spec 090 decision 8, AC-1.7). - **A request resolves its tenant once and fails closed.** `ApplicationController` includes `FirstRun` then `Authentication`; `require_authentication` runs before every action and redirects a request with no session to sign-in, so no controller meets a nil `Current.account` unless it asked for a visitor (spec 004 FR-2, AC-4.6; ADR 0017). `Current.session` is set once, from the signed cookie, and `Session.includes(user: :account)` loads the account with it (`Authentication#find_session_by_cookie`). - **A tenant query goes through `Current.account`.** Controllers read collector-owned data from `Current.account.lots`, `.locations`, `.scanner_sitting`, `.number_entry` and the like, or pass `account: Current.account` to the model that does, so an id from another account is not found and answers 404 (spec 004 AC-12.1, AC-12.2, NFR Security; ADR 0017). Nothing in a request names an account: `account_id`, `user_id` and `admin` are never permitted from parameters; sign-up permits only `name`, `email_address`, `password` and `password_confirmation` (`Registration::PERMITTED`), a user form `name`, `email_address` and `password`, and `admin` is set by the admin-only controller from its own checkbox (spec 004 AC-12.3, NFR Security; `.claude/rules/security.md`). The app has no `default_scope` and no `unscoped`. - **Another account's record is a 404, never a 403.** A lookup through `Current.account` raises `ActiveRecord::RecordNotFound`, which answers 404 and leaves the record unchanged; the admin pages answer 404 to a member the same way (spec 004 AC-12.2, AC-5.6; spec 015 AC-7.1). A request spec per tenant-scoped resource proves it (spec 004 AC-12.2; `.claude/rules/multi-tenancy.md`). - **A deck is found by its location's id, through the account, and a place's or a zone's id is 404.** `DecksController`, `Decks::RemovalsController`, `Decks::FormatsController` and `Decks::SelectionsController` read `Current.account.decks.find_by!(location_id: params[:id])` (the last three by `params[:deck_id]`), so the id in every deck URL (`GET /decks/:id`, the deck page, `/decks/:id/edit`, `PATCH` and `DELETE /decks/:id`, `/decks/:id/removal/new`, `/decks/:id/format/edit`, `PATCH /decks/:id/format` and `POST /decks/:id/selection`) is the deck's location id, never `decks.id`: another account's deck, a place's id and a zone's id are all 404, and nothing in a request names an account, a kind, a parent, a zone or a collectible type (the type comes from the checked format pair). The locations routes are the reverse: they look up `Current.account.locations.places`, so a deck's or a zone's id is 404 on every one of them, `POST /locations/:location_id/selection` included, and a zone is never renamed, removed or opened in bulk mode on its own. `DecksController` also answers 404 to an id that is not digits only (`Location::ID_TEXT`), where `find_by!` would have read "12abc" as 12 (spec 066 AC-1.2). A request spec proves each of the two directions and another account's deck (spec 062 decision 4, AC-2.6, AC-3.8, AC-9.7; spec 064 AC-3.2–AC-3.4, AC-3.6; spec 066 AC-1.2; spec 067 decision 2, AC-4.1, AC-4.2, AC-4.6; ADR 0028). A deck removed in another tab is 404 on the Change format page and on the change (spec 064 AC-3.4), and a change of one account's deck writes no row of another account (AC-3.6). **The proxy pages follow the same rule** (spec 068): `Decks::ProxiesController` and `Decks::Proxies::RemovalsController` find the deck through the `FindsDeck` concern (`Current.account.decks.find_by!(location_id: params[:deck_id])`, digits-only guard first), the zone by its key within that deck's zones (`GET`/`POST /decks/:deck_id/zones/:zone/proxies`) and the proxy by its id among the deck's own (`@deck.proxies.find`, the account and the zones' parent by value), so another account's deck or proxy, a place's or a zone's id as the deck, an id that isn't digits only, a zone key the deck lacks, another deck's proxy under this deck and a proxy or deck removed meanwhile are 404, and nothing in a request names an account, a zone id or a kind (a request spec per page and verb from another account's session, `spec/requests/deck_proxy_tenancy_spec.rb`; spec 068 AC-1.7, AC-1.11, AC-3.5, AC-3.6, AC-5.5, AC-8.3). **The two replacement pages follow it too** (spec 070): `Decks::Proxies::ReplacementsController` (Use my copy: `GET /decks/:deck_id/proxies/:proxy_id/replacement/new`, `POST …/replacement`) and `Decks::Proxies::CopiesController` (Add a new copy: `GET …/copy/new`, `POST …/copy`) find the deck and the proxy through `FindsDeck` as above, so another account's deck or proxy, a place's or a zone's id as the deck, an id that isn't digits only, another deck's proxy under this deck and a proxy removed meanwhile are 404 on each verb of each, and so is a proxy whose own printing's collectible type the instance no longer registers (`Deck::Proxy.replaceable?`; spec 070 AC-2.8, AC-2.9, AC-4.8, `spec/requests/deck_proxy_replacement_tenancy_spec.rb`). **A lot id in Use my copy's form is not a lookup that can answer 404:** the lots are read only among `Current.account.lots.free` of the proxy's card, so a lot id that is another account's, another card's, in a zone or removed is "no longer free", the same 422 with "Those copies changed. Check the numbers and try again." under the lots' fieldset for any of them, and nothing tells a collector whether a lot of another account exists; a key that isn't an id of digits only is ignored. This departs from spec 006 AC-8.1, which ignores ids that aren't the account's in a bulk submission, because a number was typed for the lot and ignoring it would move fewer copies than asked (spec 070 decision 9, AC-3.9). Nothing in either request names an account, a zone, a location, a kind or an entry: the zone and the printing are the proxy's (AC-3.13, AC-4.7). No admin page shows a proxy, a deck or a location. - **The Add to deck page reads and writes through the account too** (spec 071): `GET`/`POST /catalog/entries/:entry_external_key/deck_additions` (`DeckAdditionsController`, `Deck::Addition`) read the free copies from `Current.account.lots` and the zone from the account's own zones of the decks of the printing's collectible type, and write through `account.locations.find_placeable!`, so another account's zone, a place's id, a deck's own row, a zone of another type's deck, a removed zone and a printing that isn't searchable are 404; signed out is sent to sign in, and nothing in a request names an account, a user, a kind or a lot (spec 071 AC-1.2, AC-1.3, AC-5.2 to AC-5.4). -- **Cross-account reads are few, and each has a reason.** `Lot.group(:account_id).sum(:quantity)` and `account.copies_count` on the admin users page and its delete confirmation: a count of copies per account, for admins only (spec 004 AC-5.1, AC-5.4; hosted-door rule 2 in the [north star](../north-star.md)); `User.order(:name)` (the list) and `User.find` on the admin pages (users, not collections), and `User.authenticate_by` at sign-in; `User.exists?`, `User.admins` and `User.find_or_initialize_by` in first-run, the last-admin check and the command (spec 004 AC-3.1, AC-5.5, AC-5.7); `ImportUpload.sweep!` and `Session.sweep!`, cross-tenant by nature because the instance runs them (spec 028 decision 9; spec 049 decision 10); the scanner findings tasks' `User.find_by!(email_address: SCANNER_EMAIL).account` (development tools of spec 009 and spec 011, which name the account on purpose); the backup's counts of `accounts`, `users` and `lots`, read with SQLite directly (spec 030). Any other cross-tenant query needs a comment saying why (`.claude/rules/multi-tenancy.md`). +- **Cross-account reads are few, and each has a reason.** `Lot.group(:account_id).sum(:quantity)` and `account.copies_count` on the admin users page and its delete confirmation: a count of copies per account, for admins only (spec 004 AC-5.1, AC-5.4; hosted-door rule 2 in the [north star](../north-star.md)); `User.order(:name)` (the list) and `User.find` on the admin pages (users, not collections), and `User.authenticate_by` at sign-in; `User.exists?`, `User.admins` and `User.find_or_initialize_by` in first-run, the last-admin check and the command (spec 004 AC-3.1, AC-5.5, AC-5.7); `ImportUpload.sweep!` and `Session.sweep!`, cross-tenant by nature because the instance runs them (spec 028 decision 9; spec 049 decision 10); the household tables, read only by the models when a user is made or deleted or a membership changes, by the household migrations and by the upgrade proof's `Upgrade::Households`, and by no request: a request spec counts the queries of the collection, admin users, account menu and catalog search pages and finds none that names `households` or `memberships` (spec 090 decision 9, AC-4.5; `spec/requests/households_invisible_spec.rb`); the scanner findings tasks' `User.find_by!(email_address: SCANNER_EMAIL).account` (development tools of spec 009 and spec 011, which name the account on purpose); the backup's counts of `accounts`, `users` and `lots`, read with SQLite directly (spec 030). Any other cross-tenant query needs a comment saying why (`.claude/rules/multi-tenancy.md`). - **The operator is not the user.** No admin page or task shows or edits the items in a collector's collection: the admin users page shows each account's copy count and may create or delete an account (deleting a user deletes their collection), change a name, email address or password; it never lists or opens what a collection holds, and no admin page shows a deck, a deck name, a zone, a format or a deck count, while the copy count counts the copies in zones as copies (ADR 0017 Consequences; [north star](../north-star.md) hosted-door rule 2; spec 004 AC-5.1, AC-5.4; spec 029 AC-4.5; spec 062 AC-9.8). - **Jobs and broadcasts carrying the tenant is a rule with no instance yet.** ADR 0017 requires an account-scoped job or broadcast to carry and restore the account; no such job or broadcast exists, `ApplicationJob` restores no `Current.account`, and no view subscribes to a Turbo Stream channel. The jobs the app runs are global: the catalog refresh and the art build (catalog data), `ImportUpload::SweepJob` and `Session::SweepJob` (the two that touch tenant data, which they do for every account) (ADR 0017 Decision; spec 028 decision 9; spec 049 decision 8). The first account-scoped job or broadcast makes the rule's code, and updates this doc. - **Cache keys that hold tenant data name the account.** Nothing caches tenant data today; the one thing cached per client is the sign-in rate limit's counters, keyed by address and email (spec 004 AC-4.8; `.claude/rules/multi-tenancy.md`). The art index is served as global data: "never stored under a tenant key, and serving it reads no tenant data" (spec 011 AC-4.4). @@ -119,7 +123,7 @@ Admin only; a member gets 404 for every page and action (spec 004 AC-5.6, FR-5; | Delete another user | A confirmation page (it works without script): "This permanently deletes 's account and all copies in their collection. It can't be undone."; confirming removes the user, their account, their sessions and everything the account owns, and changes no other account (spec 004 AC-5.4, FR-5) | | Delete oneself | Refused with "You can't delete your own account."; the page offers no link for it (spec 004 AC-5.5) | -Deleting a user destroys the account, which deletes its lots first and then its locations, because a lot's place blocks deleting the location (`Account` declares `lots` before `locations` and `decks` and `deck_proxies` after them, so a deck's zones, its `decks` row and their proxies go with its locations and the account by the foreign keys; `spec/models/account_spec.rb`; spec 032 AC-6.6; spec 062 AC-9.6), and destroys its scanner sitting, number entry and stored import upload with their callbacks; the lines and the sitting's entries go by their parents' `delete_all`, and `bulk_selections`, `bulk_removals` and `bulk_selection_marks` by `ON DELETE CASCADE` (the foreign keys of every account-owned table but `lots` cascade too; `lots`' foreign key doesn't cascade, so `Account` deletes them itself) (spec 004 FR-3, AC-5.4; spec 029 AC-4.6). Every table above has a foreign key to its account. +Deleting a user removes, in this order, their membership (deleted without the membership's own refusal, `has_one :membership, dependent: :delete`; the foreign key would cascade it too), their sessions and the user's row, then the account, then the household when no member and no account is left in it (`User#remove_an_empty_household`; `Household` refuses while it holds either); a household with other members keeps them, and the delete is refused when the user is the last manager of one (no page makes such a household yet, so only a model spec reaches it) (spec 090 decision 7, AC-2.3, AC-4.4; ADR 0030). The account's destroy deletes its lots first and then its locations, because a lot's place blocks deleting the location (`Account` declares `lots` before `locations` and `decks` and `deck_proxies` after them, so a deck's zones, its `decks` row and their proxies go with its locations and the account by the foreign keys; `spec/models/account_spec.rb`; spec 032 AC-6.6; spec 062 AC-9.6), and destroys its scanner sitting, number entry and stored import upload with their callbacks; the lines and the sitting's entries go by their parents' `delete_all`, and `bulk_selections`, `bulk_removals` and `bulk_selection_marks` by `ON DELETE CASCADE` (the foreign keys of every account-owned table but `lots` cascade too; `lots`' foreign key doesn't cascade, so `Account` deletes them itself) (spec 004 FR-3, AC-5.4; spec 029 AC-4.6). Every table above has a foreign key to its account. ### The operator command @@ -161,7 +165,7 @@ Deleting a user destroys the account, which deletes its lots first and then its **Accepted limits and open issues:** - A member can't change their own name, email address or password (spec 004 Non-Goals); an admin who forgets theirs uses the command (spec 004 AC-5.7). -- An account has exactly one user. Personal and household collections side by side will supersede that half of ADR 0017, with an export format version bump and migrations (ADR 0017 Consequences; [north star](../north-star.md)). +- Every user has a household of one and nothing uses it: no page, no word, no way to join, leave, rename or share. Joining, leaving, handing the manager role over, the shared account and its collection, transfers and borrowing are the households lane (#381 to #389), and the shared collection's export file raises the format version (#384). The rule that a household is left with no member and no account is removed with its last user is revisited by #381 and #384 (spec 090 decision 7, Non-Goals, Out of Scope; ADR 0030). - No account-scoped job or broadcast exists, so the rule that one carries and restores the account has no code behind it yet (ADR 0017). - The sign-in rate limit is only as lasting as the cache that holds it: a cache clear or a cache-database restore resets the counters, which is accepted ([#218](https://github.com/plainprogrammer/Collector/issues/218) ruling 4; [#230](https://github.com/plainprogrammer/Collector/issues/230)). - A lost phone keeps its access for up to 30 days after its last use, unless an admin resets the password or deletes the user: there is no absolute lifetime and no "sign out everywhere" (spec 049 Non-Goals). @@ -169,6 +173,7 @@ Deleting a user destroys the account, which deletes its lots first and then its ## Sources - [Spec 004](../specs/004-design-system-collection/spec.md): Story 3, Story 4, Story 5, Story 12 (AC-12.4 and the rule behind AC-12.1–AC-12.3, AC-12.5, whose lots are [collection and lots](collection-and-lots.md)'), FR-2 (its supersession of 002 FR-7 is [catalog core](catalog-core.md)'), FR-3, FR-4, FR-5, AC-2.4 and AC-2.6, NFR Security, NFR Reliability's first run, the Error Scenarios rows for sign-in, sign-up, first run, admin actions and another account's lot, and Open Questions for accounts, sign-up, first run, rate limit and the session cookie. Its design-system stories and FRs (Story 1, Story 2 but for AC-2.4 and AC-2.6, FR-1, FR-11–FR-13) are the [design system](../design-system/README.md)'s. +- [Spec 090](../specs/090-every-user-has-a-household/spec.md) (issue [#380](https://github.com/plainprogrammer/Collector/issues/380)): decisions 1 to 3 and 6 to 9 and 12, FR-1, FR-2, Stories 1, 2 and 4 as far as what a household is, how every user gets one, the rules that hold it and that nothing shows it; its migrations and the upgrade proof's check are [release and deploy](release-and-deploy.md)'s and `docs/releasing.md`'s, and the export's coverage is [data portability](data-portability.md)'s. [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md). - [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md); [ADR 0022](../adr/0022-stored-import-upload-retention.md) (the ownership rule); [ADR 0001](../adr/0001-browser-ocr-engine-and-asset-hosting.md) and [ADR 0007](../adr/0007-art-search-in-the-browser.md) (the two public files). - [Spec 015](../specs/015-admin-catalog-and-jobs/spec.md): AC-7.1, AC-7.3 (the admin gate on `/admin/catalog` and `/admin/jobs`, and their More links); the pages are [catalog operations](catalog-operations.md)'. - [Spec 023](../specs/023-signed-out-catalog-browsing/spec.md): FR-1–FR-4, AC-2.3, AC-3.7 as far as who is a visitor and which endpoints skip sign-in; the rest is [catalog core](catalog-core.md)'. From da7aab21bf373c73dff3956aa07b4ccedb85cd96 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:27:57 -0500 Subject: [PATCH 21/41] docs(rules): households and memberships are instance-level (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- .claude/rules/multi-tenancy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.claude/rules/multi-tenancy.md b/.claude/rules/multi-tenancy.md index 2a6a2b7b..8c623e76 100644 --- a/.claude/rules/multi-tenancy.md +++ b/.claude/rules/multi-tenancy.md @@ -5,6 +5,7 @@ - ✓ Scope every query through the tenant (`Current.account.collections.find(params[:id])`); bare `Model.find(params[:id])` on tenant data is a bug. - ✓ Pass the account (or a record owning it) into every job and restore `Current.account` before the job body runs. - ✓ Add a request spec per tenant-scoped resource proving another account's record returns 404. +- ✓ `households` and `memberships` are instance-level, not tenant-owned: read them through the user or the account, never through `Current.account`'s collections, and name none in a request. - ✓ Shared catalog data (e.g. Scryfall cards) is global; user-owned data (copies, conditions, prices paid, notes) is always tenant-scoped. - ✗ Never use `unscoped` or cross-tenant queries without an explanatory comment. - ✗ Don't cache tenant data under keys that omit the account. From bdd8ec1841d1f5f7ae167c9e45a829e9312596e1 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:28:21 -0500 Subject: [PATCH 22/41] docs(architecture): the upgrade proof checks households in two tiers (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- docs/architecture/release-and-deploy.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/architecture/release-and-deploy.md b/docs/architecture/release-and-deploy.md index d6cccc5d..0d0250f5 100644 --- a/docs/architecture/release-and-deploy.md +++ b/docs/architecture/release-and-deploy.md @@ -203,6 +203,8 @@ A new release reaches an instance by `docker compose pull && docker compose up - CI proves one release step of it: the previous release's published image, seeded through its own code, upgraded unattended by the build on the same volume, nothing lost and the app serving the collection as it was; and, once the previous release has `backup:create`, a backup it took restores onto the build and is judged the same way (spec 031 decisions 1, 4, 7). The proof covers the prices database too: the upgraded and the restored volumes must hold `production_prices.sqlite3` with every version of `db/prices_migrate/`, sound, its two large tables `WITHOUT ROWID`; when the previous release's tag has `db/prices_migrate/` (decided from the tag, not from the volume), its prices database is compared with the upgraded and the restored ones by the primary's rule; the next seed fills the prices tables and the coverage rule reaches them. Expected changes and seed exceptions are split between the two databases by the prices tables (`db/prices_structure.sql`), so an entry for no table is still reported; an expected change may name a migration of `db/prices_migrate/`, and the migration guard reads that directory too (spec 081 decision 10, AC-6.4 to AC-6.8; it supersedes in part spec 031 AC-2.1, AC-2.4, AC-2.5, AC-3.2, AC-4.5, AC-6.2 and AC-8.1, whose frozen `contracts/yaml-files.md` still says `db/migrate/` only). Its limits: one step only, from the newest stable tag below the tag being built, never a chain from older releases; the old database is a seeded one, not a real instance's data (spec 031 Non-Goals; see Accepted limits). +The proof checks the households too, in two tiers, on the snapshots the harness takes (`Upgrade::Households`, `script/upgrade/households.rb`, read with SQLite directly like `compare.rb`). **Tier 1** holds on every snapshot of every release from the one that made households on, so it names no count: every user has exactly one membership, every household has a manager, every account has a household, and a personal account's household is its user's. It runs on the upgraded snapshot, on the restored one and on the `next` snapshot of a fresh instance of the code under test, whose seed makes its users through the models so `households` and `memberships` hold rows. **Tier 2**, the backfill's own facts, runs only when the old snapshot has no `households` table (the previous release's seed), on the upgraded and the restored snapshots: the households' count equals the users', every membership is `manager`, every account is `personal`, and each household's name is the user's first 88 characters and `'s household`. A snapshot that breaks one fails the run naming the rule and the first rows; the upgraded and restored legs print `households: ok (backfilled)` (`households: ok` when the old snapshot already had households), and the `next` snapshot prints nothing and fails on a problem. `script/upgrade/expected_changes.yml` stays empty (`changes: []`) for this: `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and the backfill sets `accounts.household_id` only, `updated_at` untouched, so no old column changes (spec 090 decision 10, AC-6.1 to AC-6.5; ADR 0030 decision 7). + The proof stops each instance it boots and compares databases (the previous release's, the upgraded one, the restored one, the next seed's) only after a graceful stop, a status of 0 or 143. It stops an instance only once Solid Queue's dispatcher, worker and scheduler have each logged `Started`, reading the log up to 15 times 2 s apart; an instance whose queue doesn't start that way fails the proof, naming what is missing and quoting its last 20 log lines, as spec 031 FR-5 allows no warning that lets the job pass. Why: Solid Queue's Puma plugin forks its supervisor when Puma has booted and, on a stop, sends it INT and waits for it without a bound. A stop that arrives before the supervisor has trapped signals is swallowed, so Puma waits until the engine kills the container at the end of the grace period (exit 137); a stop that arrives while the supervisor's three processes are still starting has its TERM lost to them and takes the supervisor's 5 s `shutdown_timeout` instead. The proof used to stop an instance at its first `/up` answer, which lands in those windows by chance, more often on a slow runner (#177: exit 137 on `Image (linux/amd64)` in runs 42131733342, 42124366255 and 42658829910). A stop that is not graceful fails with how long it took and the container's last 20 log lines (#177). Only the proof stops an instance gracefully: `bin/image-smoke` and `bin/backup-smoke` remove their containers, so they need no wait. ## Named constants @@ -297,6 +299,7 @@ The proof stops each instance it boots and compares databases (the previous rele - [Spec 013](../specs/013-ghcr-untagged-cleanup/spec.md): all. - [Spec 020](../specs/020-readme-introduction/spec.md): all, as far as it sets rules (the README is true; the suite checks no prose document). The record of what the sweep found is its `verification.md`. - [Spec 025](../specs/025-import-preview/spec.md): decision 6 and AC-3.4, the request body limit only. +- [Spec 090](../specs/090-every-user-has-a-household/spec.md): decision 10 and AC-6.1 to AC-6.5, the upgrade proof's household check, as far as the harness and `expected_changes.yml` go; the household rules are [tenancy and accounts](tenancy-and-accounts.md)'; [#380](https://github.com/plainprogrammer/Collector/issues/380). - [Spec 028](../specs/028-apply-import/spec.md): decisions 8 and 9 (the stored upload, its retention, the sweep, the gated Active Storage routes) and their criteria (AC-1.11, AC-1.12, AC-5.1–AC-5.6); the apply itself is data portability's. - [Spec 030](../specs/030-backup-restore/spec.md): all but decision 11 (the export versus the backup), which is data portability's (its decision 14 and AC-5.7 are superseded in part by spec 031 decision 7). - [Spec 031](../specs/031-upgrade-from-previous-release/spec.md): all (AC-2.1, AC-2.4, AC-2.5, AC-3.2, AC-4.5, AC-6.2 and AC-8.1 as spec 081 extends them to the prices database). From 0f00e86900efa9d13b8dd94f1da5f96310e4546b Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:28:50 -0500 Subject: [PATCH 23/41] docs(releasing): the upgrade note for the household migrations (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- docs/releasing.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/releasing.md b/docs/releasing.md index 4af26786..ebb2b4b0 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -64,6 +64,11 @@ is safe and is how `latest` is restored if something else overwrote it. Notes an operator needs when upgrading, newest first. `bin/docker-entrypoint` runs `bin/rails db:prepare` on boot, so every migration listed here runs unattended. +- **Every user gets a household (spec 090, ADR 0030, #380).** Three migrations run on boot, each without a transaction: `CreateHouseholdsAndMemberships` (two tables and two columns on `accounts`), `BackfillHouseholds` (a household of one for every user, named "'s household", with the user as its manager) and `MakeAccountHouseholdRequired` (`accounts.household_id` becomes `NOT NULL`, which SQLite does by copying the `accounts` table with foreign keys switched off, inside one transaction of its own). Nothing visible changes: no page, no word, nobody is signed out, and no row of any old table is changed (the backfill sets `accounts.household_id` and nothing else). Nothing is required of a healthy instance. **Take a backup before this upgrade, as always, because going back is not an undo:** + - **An older image can't create a user once the third migration has run,** because it doesn't know the new required column. Going back means restoring the backup taken before the upgrade onto a new volume under the older image and losing what was written since; Collector documents no other downgrade. + - **A failed third migration changes nothing and stops the boot.** Its backfill of stragglers, the copy and the restore of the id sequence are one transaction, so the next start runs it whole. With Compose the app stays down until it is fixed; with Kamal the old container keeps serving. + - **During a Kamal deploy, the old container's sign-up or add-a-user fails (a 500) for the seconds between the third migration and the switch to the new container.** A user an old container made between the second and the third migration is given a household by the third. + - **An instance that holds an account with no user will not start** until it is removed, and the third migration names the ids (the first five) and changes nothing. Such an account exists only if the database was edited by hand, since an account is made with its user and deleted with it. Remove it with `bin/rails runner 'Account.find(ID).destroy'`, which deletes what it owns; run it in a one-off container of the new release, such as `docker compose run --rm web bin/rails runner 'Account.find(ID).destroy'` (`bin/docker-entrypoint` runs `db:prepare` only for the server command, so it doesn't meet the failing migration first). Don't delete the row with the `sqlite3` command: it doesn't enforce foreign keys and leaves what the account owned behind. Then start again. - **A start refuses a volume with a prices database and no primary (spec 086, #375).** No migration. `bin/docker-entrypoint` now runs `bin/rails start:check` before `db:prepare` on a server start. A volume that holds `production_prices.sqlite3` (or one of its `-wal`, `-shm` or `-journal` files) and no primary database, which a `backup:restore` killed between its two placings leaves, no longer starts a blank instance on stale prices: the container exits with status 1 and says what it found, why it stops and what to do. Every other volume starts as before. If you see it, remove the prices file and its three sidecar files, then restore again onto a volume that holds no other Collector database, or start blank. Nothing is required of a healthy instance. - **A prices database, held by the backup (spec 081).** The start creates a fifth SQLite file, `production_prices.sqlite3`, on the same volume, and its one migration, `CreatePriceTables`, runs on boot; it holds no row yet and the primary database is unchanged. Backups taken by this release are backup format version 2 (`manifest.json`, `primary.sqlite3`, `prices.sqlite3`), which the release before can't restore: restore them on this release or a newer one. A version 1 backup from an earlier release restores here, and the first start creates an empty prices database. `backup:create` refuses an instance with no prices database, so take backups after the upgraded instance has started once. Nothing is required of the operator. - **Proxies (spec 068).** One migration, `CreateDeckProxies`, runs on boot; it creates the `deck_proxies` table and changes nothing else. A collector records the cards a deck's zone needs and they don't own as proxies; an export now carries them as `proxies` in format version 4, which no release had shipped before, so a version 4 file written by a development build before this change (an `edge` image) has no `proxies` and is refused as not well-formed. Nothing is required of the operator; the backup includes the table with the primary database. From a1d0e7dd35a333a95709593c94ef7a704e80c103 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:29:03 -0500 Subject: [PATCH 24/41] docs(claude): every user belongs to a household (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- CLAUDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8e634bb4..60ad4a8c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Status -Collector is a self-hostable, multi-tenant web app for tracking collectibles, starting with Magic: The Gathering cards. The Rails app skeleton exists (stack, RSpec, `bin/ci`, Compose and Kamal deployment). Feature 002 added a collectible-agnostic catalog core (`app/models/catalog/`) with an MTG extension (`app/models/mtg/`), Scryfall bulk-data ingestion via a background refresh, and a card search at `/catalog/entries`, which has needed sign-in since accounts (spec 004) unless the operator opens it with `COLLECTOR_PUBLIC_CATALOG` (spec 023). Accounts (row-level tenancy through `Current.account`) and collections of lots (spec 004, list view spec 006) exist, and the card scanner (specs 005, 007–009) adds scanned cards to the collection through its confirm flow. Admins start and watch catalog refreshes and art index builds at `/admin/catalog`, and see, retry and discard background jobs at `/admin/jobs` (spec 015). A collector's data has a documented export format, and a collector downloads, previews and applies an export file from the account menu or More (`CollectionExport`, `CollectionImport`, `CollectionImport::Apply`; [`docs/architecture/data-portability.md`](docs/architecture/data-portability.md)). Every page's footer ends with Collector's licence line, AGPL-3.0 with a link to the source of the running build, and shows the running version and short SHA, which the image's build passes as `COLLECTOR_VERSION`, `COLLECTOR_REVISION` and `COLLECTOR_SOURCE_URL` and `Collector::BuildFacts` reads once at boot; `bin/image-smoke` fails an image without them (spec 084). The line ends with a link to `/licences`, which anyone can open and which lists the licences of the third-party software and assets in the image: the gems from the running bundle, the OCR engine, the importmap JavaScript, the fonts and the icons from `config/third_party_licences.yml`, kept honest by a guard spec (`Collector::ThirdPartyLicences`, `LicencesController`, spec 087). A collector adds cards from a set by collector number at `/collection/number_entry` ("Add by number", spec 026): one line per number, quantity and finish, each added at once to a per-account list with Undo (`NumberEntry`, `NumberEntry::Line`), which Done clears; the list is working state, left out of the export. A collector manages the places their collection is kept in at `/locations` (`Location`, spec 029) and records which location a lot is in when adding or editing it, or moves some or all of a lot's copies with Move copies (`Lot::Move`, spec 032; a location that holds copies can't be removed; export format version 3) and moves every copy of many lots at once from bulk mode with Move to… (`Lot::BulkMove`, spec 041), opens a location's lots in bulk mode from its removal refusal page with one button (`Locations::SelectionsController`, spec 059), and the collection's grid tiles and table rows show it (spec 033) and filter by it (spec 037). A collector creates, renames and removes decks at `/decks` (`Deck`, `Deck::Creation`, `Catalog::Formats`, spec 062, [ADR 0028](docs/adr/0028-decks-are-locations-with-zones.md)): a deck is a location of kind `deck` whose zones are child locations that hold its copies, the collectible extension registers the formats, a lot in a zone is shown as "Deck › Zone", the export format (version 4) carries decks, a collector changes a deck's format at `/decks/:id/format/edit` (`Deck::FormatChange`, spec 064), opens a deck's lots in bulk mode from its removal refusal page (`Decks::SelectionsController`, spec 067), and opens a deck at `/decks/:id` to read the cards they own in it, zone by zone, with their cost and legality (`Deck::Page`, the seam `Catalog::DeckCards`, spec 066), and the deck's shape, its mana curve, card types and coloured mana symbols, in a panel beside the zones (`Catalog::DeckShape`, `MTG::DeckShape`, spec 069), and records the cards a deck's zone needs and they don't own as proxies, shown beside the owned copies and counted in a deck's cards but never as owned (`Deck::Proxy`, spec 068): a change of format moves them into Considering and the export carries them (format version 4 has `proxies`). A collector swaps a proxy for a free copy they own (Use my copy) or a new copy (Add a new copy) from the row's menu, and the proxy shrinks or goes in the same write (`Deck::Proxy::UseCopies`, `Deck::Proxy::AddCopy`, `Lot.free`, `Lot::MoveIntoZone`, spec 070). A collector adds a card to a deck from its card page, their free copies or a proxy (`Deck::Addition`, `Lot::FreeCopies`, spec 071). The card page shows how many copies are free to use and which decks hold the rest, in its stats and an "In decks" table (`Catalog::CardOverview`, `Deck::CardUses`, spec 073). A collector checks a deck on demand from its page, its size, copy limits and the cards with owned copies and proxies in one zone, and nothing about the check is stored (`Catalog::DeckChecks`, `MTG::Collecting.deck_rules`, `Decks::ChecksController`, spec 079). An operator backs up a running instance and restores it onto a fresh volume with two rake tasks, `backup:create[DIRECTORY]` and `backup:restore[FILE]` (`Collector::Backup`, `lib/collector/backup/`, spec 030, ADR 0024). Market prices will live in a fifth SQLite database, `prices` (`PricesRecord`, `db/prices_migrate/`, `db/prices_structure.sql`; spec 081, ADR 0029; [prices](docs/architecture/prices.md)), whose five tables exist, with a price source contract and Scryfall's prices through it (`Prices::Sources`, `MTG::Scryfall::PriceQuotes`, `PriceSource.sync!`; spec 085, #364) and no refresh or reader yet, and the backup holds it beside the primary (backup format version 2); a server start refuses a volume that holds the prices database and no primary (`Collector::StartCheck`, `start:check`, spec 086). CI proves an upgrade from the previous release's published image (`bin/upgrade-smoke`, `script/upgrade/`, spec 031, #42; [release and deploy](docs/architecture/release-and-deploy.md)). License: AGPL-3.0. +Collector is a self-hostable, multi-tenant web app for tracking collectibles, starting with Magic: The Gathering cards. The Rails app skeleton exists (stack, RSpec, `bin/ci`, Compose and Kamal deployment). Feature 002 added a collectible-agnostic catalog core (`app/models/catalog/`) with an MTG extension (`app/models/mtg/`), Scryfall bulk-data ingestion via a background refresh, and a card search at `/catalog/entries`, which has needed sign-in since accounts (spec 004) unless the operator opens it with `COLLECTOR_PUBLIC_CATALOG` (spec 023). Accounts (row-level tenancy through `Current.account`) and collections of lots (spec 004, list view spec 006) exist, every user belongs to a household (spec 090, ADR 0030): a household of one is made with the user and an existing instance gains one on upgrade, nothing visible yet; the shared collection and joining are the households lane (#381 to #389), and the card scanner (specs 005, 007–009) adds scanned cards to the collection through its confirm flow. Admins start and watch catalog refreshes and art index builds at `/admin/catalog`, and see, retry and discard background jobs at `/admin/jobs` (spec 015). A collector's data has a documented export format, and a collector downloads, previews and applies an export file from the account menu or More (`CollectionExport`, `CollectionImport`, `CollectionImport::Apply`; [`docs/architecture/data-portability.md`](docs/architecture/data-portability.md)). Every page's footer ends with Collector's licence line, AGPL-3.0 with a link to the source of the running build, and shows the running version and short SHA, which the image's build passes as `COLLECTOR_VERSION`, `COLLECTOR_REVISION` and `COLLECTOR_SOURCE_URL` and `Collector::BuildFacts` reads once at boot; `bin/image-smoke` fails an image without them (spec 084). The line ends with a link to `/licences`, which anyone can open and which lists the licences of the third-party software and assets in the image: the gems from the running bundle, the OCR engine, the importmap JavaScript, the fonts and the icons from `config/third_party_licences.yml`, kept honest by a guard spec (`Collector::ThirdPartyLicences`, `LicencesController`, spec 087). A collector adds cards from a set by collector number at `/collection/number_entry` ("Add by number", spec 026): one line per number, quantity and finish, each added at once to a per-account list with Undo (`NumberEntry`, `NumberEntry::Line`), which Done clears; the list is working state, left out of the export. A collector manages the places their collection is kept in at `/locations` (`Location`, spec 029) and records which location a lot is in when adding or editing it, or moves some or all of a lot's copies with Move copies (`Lot::Move`, spec 032; a location that holds copies can't be removed; export format version 3) and moves every copy of many lots at once from bulk mode with Move to… (`Lot::BulkMove`, spec 041), opens a location's lots in bulk mode from its removal refusal page with one button (`Locations::SelectionsController`, spec 059), and the collection's grid tiles and table rows show it (spec 033) and filter by it (spec 037). A collector creates, renames and removes decks at `/decks` (`Deck`, `Deck::Creation`, `Catalog::Formats`, spec 062, [ADR 0028](docs/adr/0028-decks-are-locations-with-zones.md)): a deck is a location of kind `deck` whose zones are child locations that hold its copies, the collectible extension registers the formats, a lot in a zone is shown as "Deck › Zone", the export format (version 4) carries decks, a collector changes a deck's format at `/decks/:id/format/edit` (`Deck::FormatChange`, spec 064), opens a deck's lots in bulk mode from its removal refusal page (`Decks::SelectionsController`, spec 067), and opens a deck at `/decks/:id` to read the cards they own in it, zone by zone, with their cost and legality (`Deck::Page`, the seam `Catalog::DeckCards`, spec 066), and the deck's shape, its mana curve, card types and coloured mana symbols, in a panel beside the zones (`Catalog::DeckShape`, `MTG::DeckShape`, spec 069), and records the cards a deck's zone needs and they don't own as proxies, shown beside the owned copies and counted in a deck's cards but never as owned (`Deck::Proxy`, spec 068): a change of format moves them into Considering and the export carries them (format version 4 has `proxies`). A collector swaps a proxy for a free copy they own (Use my copy) or a new copy (Add a new copy) from the row's menu, and the proxy shrinks or goes in the same write (`Deck::Proxy::UseCopies`, `Deck::Proxy::AddCopy`, `Lot.free`, `Lot::MoveIntoZone`, spec 070). A collector adds a card to a deck from its card page, their free copies or a proxy (`Deck::Addition`, `Lot::FreeCopies`, spec 071). The card page shows how many copies are free to use and which decks hold the rest, in its stats and an "In decks" table (`Catalog::CardOverview`, `Deck::CardUses`, spec 073). A collector checks a deck on demand from its page, its size, copy limits and the cards with owned copies and proxies in one zone, and nothing about the check is stored (`Catalog::DeckChecks`, `MTG::Collecting.deck_rules`, `Decks::ChecksController`, spec 079). An operator backs up a running instance and restores it onto a fresh volume with two rake tasks, `backup:create[DIRECTORY]` and `backup:restore[FILE]` (`Collector::Backup`, `lib/collector/backup/`, spec 030, ADR 0024). Market prices will live in a fifth SQLite database, `prices` (`PricesRecord`, `db/prices_migrate/`, `db/prices_structure.sql`; spec 081, ADR 0029; [prices](docs/architecture/prices.md)), whose five tables exist, with a price source contract and Scryfall's prices through it (`Prices::Sources`, `MTG::Scryfall::PriceQuotes`, `PriceSource.sync!`; spec 085, #364) and no refresh or reader yet, and the backup holds it beside the primary (backup format version 2); a server start refuses a volume that holds the prices database and no primary (`Collector::StartCheck`, `start:check`, spec 086). CI proves an upgrade from the previous release's published image (`bin/upgrade-smoke`, `script/upgrade/`, spec 031, #42; [release and deploy](docs/architecture/release-and-deploy.md)). License: AGPL-3.0. Mission and principles: `.claude/memory/foundation.md`. Goal and spec rules: `docs/north-star.md`. Order of work: `docs/roadmap.md`. Decisions: `docs/adr/` (index in its README). Current rules per subsystem: `docs/architecture/` (area docs). Detailed coding rules: `.claude/rules/`. Feature specs: `docs/specs/` (index: `docs/specs/README.md`). From fc684603c98f90e6201b2e9771f7d01a5f05d339 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:29:13 -0500 Subject: [PATCH 25/41] docs(adr): ADR 0030 is accepted with spec 090 Co-Authored-By: Claude Sonnet 5.5 --- docs/adr/0030-a-household-is-a-group-of-accounts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index 8682280c..4f0eb36d 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -2,7 +2,7 @@ ## Status -Proposed (2026-11-13), written with spec 090; it becomes Accepted with the pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) was ruled by the maintainer the same day: the table copy, not the nullable fallback, on two conditions (the plan review traces it against Rails 8.1.4.1's `alter_table`, and the upgrade proof runs from the previous release built from its tag). +Accepted (2026-11-13), written with spec 090 and accepted with its pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) was ruled by the maintainer the same day: the table copy, not the nullable fallback, on two conditions (the plan review traces it against Rails 8.1.4.1's `alter_table`, and the upgrade proof runs from the previous release built from its tag). **Date:** 2026-11-13 **Feature:** 090-every-user-has-a-household From dd4e551447e30e4dee31d429c2a417b7d264b0e6 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:29:24 -0500 Subject: [PATCH 26/41] docs(specs): spec 090 is approved and supersedes spec 004 FR-3 in part (index) Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/specs/README.md b/docs/specs/README.md index 58bf1093..1c9ffce9 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -24,7 +24,7 @@ A plan is deleted from the tree in a commit of its own after its feature's pull | [001 Rails Application Skeleton](001-rails-app-skeleton/spec.md) | Frozen, by [release and deploy](../architecture/release-and-deploy.md) | Release and deploy (container, Compose, Kamal); CLAUDE.md (R5): stack, CI, hooks | retired in `96d5e651ea8a2696f6372ecc41907a2d20a2efe1`; read it with `git show 96d5e651ea8a2696f6372ecc41907a2d20a2efe1^:docs/specs/001-rails-app-skeleton/plan.md` | | The Compose file that builds the image from a checkout and Kamal's placeholder registry (FR-5, FR-6, AC-3.1, AC-4.1): 012 (FR-4, FR-5; AC-1.5, AC-5.1). AC-2.5's single check job, and Story 2's "CI passes" meaning the same in the pre-push hook for documentation-only branches: 035 (decisions 3, 5) | | [002 Scryfall Catalog Ingestion and Card Search](002-scryfall-catalog-search/spec.md) | Frozen, by [catalog operations](../architecture/catalog-operations.md) (FR-3–FR-6 and their Error Scenarios rows) and [catalog core](../architecture/catalog-core.md) (FR-1, FR-2, FR-7–FR-10; FR-9's convention stays in the rules) | [Catalog operations](../architecture/catalog-operations.md) (FR-3–FR-6); [Catalog core](../architecture/catalog-core.md) (FR-1, FR-2, FR-7–FR-10) | retired in `7876bfd8c0126b232043cc6968693b4184c86256`; read it with `git show 7876bfd8c0126b232043cc6968693b4184c86256^:docs/specs/002-scryfall-catalog-search/plan.md` | | Non-Goals and FR-7, public search: 004 (sign-in), 023 (an operator setting). AC-1.5, AC-1.7, AC-1.8 and AC-1.11's messages, AC-10.1, FR-8's search link: 004 (AC-6.5, AC-8.2). AC-3.2, the card page's attribution: 018 (the footer). AC-1.11, the not-loaded notice: 015 (per catalog type, AC-1.1, AC-1.2). FR-4, the 6-hour "already running" rule: 015 (15-minute stall, job rules, AC-3.3, AC-3.4). FR-4 and its two 429 Error Scenarios rows, the wait for another job's retry: 024 (amended in place, v1.4.0). FR-3, FR-4 and Error Scenarios, rate-limit waits: amended in place for #74 and #81 (v1.2.0, v1.3.0). 043: FR-4's "more than 3 such retries of one job" and the Error Scenarios row "rate limits the same job a fourth time" (the fourth rate limit in a row with no progress between them) | | [003 Worktree Environment Setup](003-worktree-environment-setup/spec.md) | Frozen, by [release and deploy](../architecture/release-and-deploy.md) | CLAUDE.md (R5) | retired in `8498f1b0e3a07b8ec0e933f7c16df1e6339e335a`; read it with `git show 8498f1b0e3a07b8ec0e933f7c16df1e6339e335a^:docs/specs/003-worktree-environment-setup/plan.md` | | | -| [004 Design System, Accounts, Adding Cards and the Collection Grid](004-design-system-collection/spec.md) | Frozen, by [collection and lots](../architecture/collection-and-lots.md) (the collection and adding cards), [catalog core](../architecture/catalog-core.md) (search, the card page's catalog content and the vocabulary), [tenancy and accounts](../architecture/tenancy-and-accounts.md) (accounts, users, admin users) and the [design system](../design-system/README.md) (its design-system parts) | [Collection and lots](../architecture/collection-and-lots.md) (collection, adding cards); [Catalog core](../architecture/catalog-core.md) (search, the card page's catalog content, the collecting vocabulary); [Tenancy and accounts](../architecture/tenancy-and-accounts.md) (accounts, users, admin users); [design system](../design-system/README.md) | retired in `528c0c22c08486e63c3452148f7a5440f6c71755`; read it with `git show 528c0c22c08486e63c3452148f7a5440f6c71755^:docs/specs/004-design-system-collection/plan.md` | 002 Non-Goals and FR-7, public search: search needs sign-in (FR-2). 002 AC-1.5, AC-1.7, AC-1.8 and AC-1.11's messages, AC-10.1, FR-8's search link (AC-6.5, AC-8.2) | FR-2, sign-in for the search, card and printings pages: 023 (not needed when the operator opens the catalog). AC-6.5, the not-loaded message: 015 (AC-1.1, AC-1.2). AC-6.3, unowned tiles shown faded: 021 (a marker, the image unfaded). AC-8.2, the card page's attribution: 018 (the footer). AC-11.7, FR-10's filter bar, the return target of Edit copy and Remove (AC-2.7, AC-10.1, AC-10.4), AC-10.4 and bulk Undo: 006 (AC-1.7). AC-11.6, the empty collection's sentence, and the collection page head: 026 (a link and an "Add by number" action, its Supersedes list). The header's navigation for a collector (Collection, Search, Scan): 029 (gains Locations after Scan, decision 8). FR-6 and the merge criteria (AC-7.4, AC-9.3, AC-10.2), the add and edit forms, the card page's Your copies table: 032 (a lot's identity includes its location; Location field, column and Move copies). The tile gains a place line (no location line, AC-11.2, the Non-Goal): 033; Story 11's filter bar having only the name search: 037 (the Location select). FR-6's price paid in one instance-wide currency and its "Currency: an instance setting" line, AC-9.1's "in the instance currency": 038 (each account's own; `COLLECTOR_CURRENCY` is the default a new account starts with). FR-1's `CLAUDE.md` guidance that the repo copy of the design system is updated by re-exporting: 057 (decision 12; the design system is the project's own, in code); 071 (AC-9.5 in part: Add a copy stays visible and is the flexible control of the head row beside Add to deck and the menu, as ItemPagePhone documents); 073 (FR-9's single stat, its Non-Goal and Out of Scope entries, AC-8.1's order in part) | +| [004 Design System, Accounts, Adding Cards and the Collection Grid](004-design-system-collection/spec.md) | Frozen, by [collection and lots](../architecture/collection-and-lots.md) (the collection and adding cards), [catalog core](../architecture/catalog-core.md) (search, the card page's catalog content and the vocabulary), [tenancy and accounts](../architecture/tenancy-and-accounts.md) (accounts, users, admin users) and the [design system](../design-system/README.md) (its design-system parts) | [Collection and lots](../architecture/collection-and-lots.md) (collection, adding cards); [Catalog core](../architecture/catalog-core.md) (search, the card page's catalog content, the collecting vocabulary); [Tenancy and accounts](../architecture/tenancy-and-accounts.md) (accounts, users, admin users); [design system](../design-system/README.md) | retired in `528c0c22c08486e63c3452148f7a5440f6c71755`; read it with `git show 528c0c22c08486e63c3452148f7a5440f6c71755^:docs/specs/004-design-system-collection/plan.md` | 002 Non-Goals and FR-7, public search: search needs sign-in (FR-2). 002 AC-1.5, AC-1.7, AC-1.8 and AC-1.11's messages, AC-10.1, FR-8's search link (AC-6.5, AC-8.2) | FR-2, sign-in for the search, card and printings pages: 023 (not needed when the operator opens the catalog). AC-6.5, the not-loaded message: 015 (AC-1.1, AC-1.2). AC-6.3, unowned tiles shown faded: 021 (a marker, the image unfaded). AC-8.2, the card page's attribution: 018 (the footer). AC-11.7, FR-10's filter bar, the return target of Edit copy and Remove (AC-2.7, AC-10.1, AC-10.4), AC-10.4 and bulk Undo: 006 (AC-1.7). AC-11.6, the empty collection's sentence, and the collection page head: 026 (a link and an "Add by number" action, its Supersedes list). The header's navigation for a collector (Collection, Search, Scan): 029 (gains Locations after Scan, decision 8). FR-6 and the merge criteria (AC-7.4, AC-9.3, AC-10.2), the add and edit forms, the card page's Your copies table: 032 (a lot's identity includes its location; Location field, column and Move copies). The tile gains a place line (no location line, AC-11.2, the Non-Goal): 033; Story 11's filter bar having only the name search: 037 (the Location select). FR-6's price paid in one instance-wide currency and its "Currency: an instance setting" line, AC-9.1's "in the instance currency": 038 (each account's own; `COLLECTOR_CURRENCY` is the default a new account starts with). FR-1's `CLAUDE.md` guidance that the repo copy of the design system is updated by re-exporting: 057 (decision 12; the design system is the project's own, in code); 071 (AC-9.5 in part: Add a copy stays visible and is the flexible control of the head row beside Add to deck and the menu, as ItemPagePhone documents); 073 (FR-9's single stat, its Non-Goal and Out of Scope entries, AC-8.1's order in part); 090 (FR-3's "an account has exactly one user", in part: an account is now in a household and `users.account_id` stays unique; [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md)) | | [005 Card Scanner Phase 0 — Feasibility Spikes](005-card-scanner-phase-0/spec.md) | Frozen, by [scanner](../architecture/scanner.md) | [Scanner](../architecture/scanner.md) | retired in `a5e252f9273889df360dd759e5bffb5380963cc1`; read it with `git show a5e252f9273889df360dd759e5bffb5380963cc1^:docs/specs/005-card-scanner-phase-0/plan.md` | | | | [006 Collection Table View and Bulk Editing](006-collection-list-view/spec.md) | Frozen, by [collection and lots](../architecture/collection-and-lots.md) | [Collection and lots](../architecture/collection-and-lots.md) | retired in `b6cc3b6d8d2c5f8dcde27d05ba0aee554182733f`; read it with `git show b6cc3b6d8d2c5f8dcde27d05ba0aee554182733f^:docs/specs/006-collection-list-view/plan.md` | 004 AC-11.7, FR-10's filter bar, the return target of Edit copy and Remove (AC-2.7, AC-10.1, AC-10.4), AC-10.4 and bulk Undo (AC-1.7) | The deferred Move to… stays deferred (#48); bulk Set condition and the bulk removal's Undo keep each lot's place: 032 (Story 7). The table gains a Location column and the phone line the place (AC-2.1, AC-2.8): 033; FR-4's selection context (name filter and sort): 037 (decision 5, adds the location). The table's price paid "in the instance currency": 038 (the account's). FR-5's Must not "Offer or perform Move to…" and its Must "Provide Set condition and Remove", AC-4.4's bulk bar actions, its Non-Goal and Out of Scope entries, for Move to… only: 041. FR-4 (the submissions that start, change or clear a selection) and Story 4 (Edit many as the only way in): 059 | | [007 Card Scanner Phase 1 — Live Capture and Re-measure](007-card-scanner-live-capture/spec.md) | Frozen, by [scanner](../architecture/scanner.md) and [catalog core](../architecture/catalog-core.md) (FR-4's name index and name matching, FR-5, AC-3.4, AC-3.5 and AC-3.9) | [Scanner](../architecture/scanner.md); [Catalog core](../architecture/catalog-core.md) (the name index) | retired in `d73733273b6cf710be0a19bda06bc6a94d2a0d64`; read it with `git show d73733273b6cf710be0a19bda06bc6a94d2a0d64^:docs/specs/007-card-scanner-live-capture/plan.md` | | AC-1.7 and FR-1's "reachable by URL only", AC-3.2, AC-3.3, AC-3.10, AC-4.2, FR-1's "no tenant data", FR-3's "only recognised text", FR-4's "no finish": 009 (its Supersedes list). FR-3, match results sent: 011 (amended in place, v2.2.0). FR-6, measurement mode records art: 011. AC-1.1's stage (the 3:4 stage and its guide shares) for live capture: 082 (decisions 2, 3; the photo path keeps them) | @@ -103,4 +103,4 @@ A plan is deleted from the tree in a commit of its own after its feature's pull | [085 A Price Source Contract, and Scryfall's Prices Through It](085-price-source-contract/spec.md) | Frozen, by [prices](../architecture/prices.md) | [Prices](../architecture/prices.md) (the source contract, Scryfall's source, the source and series rows) | retired in `e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4`; `git show e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4^:docs/specs/085-price-source-contract/plan.md` | | | | [086 A Start Refuses a Volume That Holds a Prices Database and No Primary](086-start-refuses-orphan-prices/spec.md) | Frozen, by [release and deploy](../architecture/release-and-deploy.md) | [Release and deploy](../architecture/release-and-deploy.md) (the start check before `db:prepare`, its message, the restore's advice and the smoke leg); [prices](../architecture/prices.md) (a pointer only) | retired in `2c5939d4604cc1221534677c3e4bcc4aa5bdc916`; `git show 2c5939d4604cc1221534677c3e4bcc4aa5bdc916^:docs/specs/086-start-refuses-orphan-prices/plan.md` | In part, spec 081 decision 8's "Collector must never be started on such a volume" and AC-5.9's advice (the start now refuses it; the restore's advice says so) ([#375](https://github.com/plainprogrammer/Collector/issues/375)) | | | [087 The Licences of the Third-Party Software and Assets in the Image, Readable from the App](087-third-party-licences-in-app/spec.md) | Frozen, by [catalog core](../architecture/catalog-core.md) (the licence line's third sentence, the page, the reader, the manifest, the guard), [release and deploy](../architecture/release-and-deploy.md) (the smoke check) and [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint); the `Licences` pattern is in the [design system](../design-system/README.md) | [Catalog core](../architecture/catalog-core.md) (the footer and the third-party licences page); [release and deploy](../architecture/release-and-deploy.md) (the smoke check); [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint that skips sign-in); [design system](../design-system/README.md) (`Licences`, `Footer`) | retired in `11de80918d734d3b6e001d16b013f8a77c820ffc`; `git show 11de80918d734d3b6e001d16b013f8a77c820ffc^:docs/specs/087-third-party-licences-in-app/plan.md` | 084: AC-1.4's "no other part of the line is a link", the full wording of AC-2.1, AC-2.3, AC-2.4 and AC-2.14 and decision 2's wording table, AC-2.13's two links, FR-1's "with two links", the Accessibility NFR's and Users and Context's two links, decision 12's no-script notice (it gains the link), the Non-Goals "a licence page in the app" and "Listing third-party licences" ([#376](https://github.com/plainprogrammer/Collector/issues/376)) | | -| [090 Every User Has a Household](090-every-user-has-a-household/spec.md) | Draft (Gate A pending) | [Tenancy and accounts](../architecture/tenancy-and-accounts.md); [data portability](../architecture/data-portability.md) (coverage); [release and deploy](../architecture/release-and-deploy.md) (the upgrade proof) | present | [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md); spec 004 FR-3's "one user per account" in part (an account is now in a household) ([#380](https://github.com/plainprogrammer/Collector/issues/380)) | | +| [090 Every User Has a Household](090-every-user-has-a-household/spec.md) | Approved | [Tenancy and accounts](../architecture/tenancy-and-accounts.md); [data portability](../architecture/data-portability.md) (coverage); [release and deploy](../architecture/release-and-deploy.md) (the upgrade proof) | present | [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md); spec 004 FR-3's "one user per account" in part (an account is now in a household) ([#380](https://github.com/plainprogrammer/Collector/issues/380)) | | From 7187f2f132c8b7b8c0fb46459ef0e6010e1d8f88 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:38:37 -0500 Subject: [PATCH 27/41] fix(migrations): guard the households migration's down like the third, and prove it with rows in every referencing table (spec 090) The first migration's down copies accounts twice in one transaction. It now refuses when foreign keys are enforced and rolls back when a copy changed a row count, as the third migration does. Its spec fills a row in all thirteen tables that reference accounts (the helper moves to spec/support), goes down and up through all three migrations, and runs down inside an outer transaction. The AC-3.4 example compares the full shape of accounts, households and memberships before and after, and the AC-3.7 re-run now runs the migration itself. Co-Authored-By: Claude Sonnet 5.5 --- ...20001_create_households_and_memberships.rb | 32 ++++++++++++++- .../create_households_and_memberships_spec.rb | 39 +++++++++++++++++++ .../make_account_household_required_spec.rb | 26 +++---------- spec/support/referencing_table_helpers.rb | 26 +++++++++++++ 4 files changed, 102 insertions(+), 21 deletions(-) create mode 100644 spec/support/referencing_table_helpers.rb diff --git a/db/migrate/20261113120001_create_households_and_memberships.rb b/db/migrate/20261113120001_create_households_and_memberships.rb index 83d6d470..db6de8d5 100644 --- a/db/migrate/20261113120001_create_households_and_memberships.rb +++ b/db/migrate/20261113120001_create_households_and_memberships.rb @@ -2,7 +2,8 @@ # Without a transaction, as AddDecks explains: dropping the two columns on the way down copies accounts, and with foreign keys # enforced inside a transaction the copy's DROP TABLE would run the delete rules of the thirteen tables that reference it. Every # step is guarded, so a run that stopped part way is finished by the next boot. Going down refuses while a household has two -# members or a shared account exists (an older release can't hold either), and keeps accounts' id sequence across each copy. +# members or a shared account exists (an older release can't hold either), keeps accounts' id sequence across each copy, and +# stops and rolls back, as the third migration does, when enforcement is on or a copy changed a row count. class CreateHouseholdsAndMemberships < ActiveRecord::Migration[8.1] disable_ddl_transaction! @@ -35,6 +36,8 @@ def down refuse_what_the_older_release_cannot_hold connection.disable_referential_integrity do connection.transaction do + require_enforcement_off + before = state keeping_sequence do remove_check_constraint :accounts, name: "accounts_kind_known" if check_constraint_exists?(:accounts, name: "accounts_kind_known") remove_column :accounts, :kind if column_exists?(:accounts, :kind) @@ -43,6 +46,7 @@ def down remove_index :accounts, :household_id if index_exists?(:accounts, :household_id) remove_column :accounts, :household_id if column_exists?(:accounts, :household_id) end + require_nothing_lost(before) end end drop_table :memberships, if_exists: true @@ -50,6 +54,32 @@ def down end private + # The same two guards as MakeAccountHouseholdRequired (migrations don't share code: a released one is never edited). Its + # comments say why: the copy of accounts cascades silently when foreign keys are enforced, and Rails runs no check after it. + def require_enforcement_off + return if connection.select_value("PRAGMA foreign_keys").to_i.zero? + + raise ActiveRecord::MigrationError, "foreign keys are enforced inside the table copy, which would delete the collections that reference accounts; nothing was changed" + end + + # Safe: literal table names. The thirteen tables that reference accounts; accounts itself is counted with them. + REFERENCING = %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users].freeze + + def state + rows = ([ "accounts" ] + REFERENCING).to_h { [ it, connection.select_value("SELECT COUNT(*) FROM #{it}").to_i ] } + broken = connection.select_rows("PRAGMA foreign_key_check").select { |_table, _row, parent, *| parent == "accounts" } + { rows:, broken: } + end + + def require_nothing_lost(before) + after = state + lost = after[:rows].reject { |table, count| count == before[:rows][table] } + new_broken = after[:broken] - before[:broken] + return if lost.empty? && new_broken.empty? + + raise ActiveRecord::MigrationError, "the table copy changed row counts #{lost.inspect} or broke keys #{new_broken.first(3).inspect}; nothing was changed" + end + def refuse_what_the_older_release_cannot_hold shared_households = table_exists?(:memberships) ? connection.select_value("SELECT COUNT(*) FROM (SELECT 1 FROM memberships GROUP BY household_id HAVING COUNT(*) > 1)").to_i : 0 shared_accounts = column_exists?(:accounts, :kind) ? connection.select_value("SELECT COUNT(*) FROM accounts WHERE kind = 'shared'").to_i : 0 diff --git a/spec/migrations/create_households_and_memberships_spec.rb b/spec/migrations/create_households_and_memberships_spec.rb index 7cdf94dc..6c570a72 100644 --- a/spec/migrations/create_households_and_memberships_spec.rb +++ b/spec/migrations/create_households_and_memberships_spec.rb @@ -8,6 +8,8 @@ # down and up through the Migrator, as db:rollback does (the precedent is spec/migrations/add_decks_spec.rb). Rows are made with # SQL, so the examples don't depend on the models. RSpec.describe CreateHouseholdsAndMemberships, :backup, :committed, type: :model do + include ReferencingTableHelpers + let(:connection) { ActiveRecord::Base.connection } let(:stamp) { "2026-11-13 12:00:00" } @@ -22,6 +24,13 @@ def up_all = versions.each { |version| context.run(:up, version) } def count(table) = connection.select_value("SELECT COUNT(*) FROM #{table}") # Safe: a literal table name from this spec + # The thirteen tables that reference accounts, and accounts itself. + def referencing_tables = %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users] + + def counts = ([ "accounts" ] + referencing_tables).to_h { [ it, count(it) ] } + + def account_rows = connection.select_rows("SELECT * FROM accounts ORDER BY id") + def sequence = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") # What PRAGMA says about a table: columns, foreign keys, indexes, checks (not the table's SQL text). @@ -130,8 +139,10 @@ def insert_user(number = 1) end it "leaves accounts, households and memberships, after down and up, as the schema states them (AC-3.4)", :aggregate_failures do + before = %i[accounts households memberships].to_h { [ it, shape(it) ] } # the test database is loaded from db/schema.rb down_all up_all + expect(%i[accounts households memberships].to_h { [ it, shape(it) ] }).to eq(before) # The literals are what db/schema.rb dumps for the migration; if a check's expression text differs, correct the literal to the # dumped text and say so, don't loosen the example. @@ -144,6 +155,34 @@ def insert_user(number = 1) [ [ "memberships_role_known", "role IN ('manager', 'member')" ] ] ]) end + it "keeps every row of the thirteen tables that reference accounts, the accounts and their id sequence, going down through all three migrations and up (AC-3.5)", :aggregate_failures do + # Down and up through all three versions, as the file's other examples do: (c) is undone before (a) can drop household_id. + fill_every_referencing_table(create(:user)) + gone = create(:user) + connection.execute("DELETE FROM users WHERE id = #{gone.id}") + connection.execute("DELETE FROM accounts WHERE id = #{gone.account_id}") + before = [ counts, account_rows.map { it.first(4) }, sequence ] + expect(before.first.values_at(*referencing_tables)).to all(be >= 1) + expect(before.last).to be > before[1].last.first + + [ -> { down_all }, -> { up_all } ].each do |step| + step.call + expect([ counts, account_rows.map { it.first(4) }, sequence ]).to eq(before) + expect(connection.select_rows("PRAGMA foreign_key_check")).to be_empty + end + end + + it "refuses to go down when foreign keys are enforced, and changes nothing (AC-3.5)", :aggregate_failures do + fill_every_referencing_table(create(:user)) + context.run(:down, 2026_11_13_120003) + context.run(:down, 2026_11_13_120002) + before = { counts:, sequence:, columns: connection.columns(:accounts).map(&:name) } + + connection.transaction { expect { described_class.new.migrate(:down) }.to raise_error(ActiveRecord::MigrationError, /enforced/) } + + expect({ counts:, sequence:, columns: connection.columns(:accounts).map(&:name) }).to eq(before) + end + it "refuses to go down while a household has two members or a shared account exists, and changes nothing (AC-3.5)", :aggregate_failures do one, _a, first = insert_user(1) two, second_account, second = insert_user(2) diff --git a/spec/migrations/make_account_household_required_spec.rb b/spec/migrations/make_account_household_required_spec.rb index e2cc48ba..c01838c4 100644 --- a/spec/migrations/make_account_household_required_spec.rb +++ b/spec/migrations/make_account_household_required_spec.rb @@ -5,6 +5,8 @@ # keys, keep the id sequence, give a straggler a household, and change nothing when it fails. It commits its rows and runs through the # Migrator OUTSIDE any example transaction (inside one, enforcement stays on and the cascades would fire). RSpec.describe MakeAccountHouseholdRequired, :backup, :committed, type: :model do + include ReferencingTableHelpers + let(:connection) { ActiveRecord::Base.connection } let(:tables) { %w[bulk_removals bulk_selection_marks bulk_selections deck_proxies decks import_uploads locations lots number_entries number_entry_lines scanner_sitting_entries scanner_sittings users] } @@ -31,25 +33,6 @@ def shape(table) connection.check_constraints(table).map { [ it.name, it.expression ] }.sort_by(&:to_s) ] end - # One row in each of the thirteen tables, as the upgrade seed makes them (script/upgrade/seeds/next.rb). - def fill_every_referencing_table(user) - account = user.account - entry = import_printing - Lot.add!(account:, entry:, quantity: 2, finish: "nonfoil") - Scanner::Sitting.add!(account:, printing: entry, finish: "nonfoil", reading_key: SecureRandom.hex(16)) - session = user.sessions.create! - selection = BulkSelection.for(session) - account.lots.limit(3).pluck(:id).each { |lot_id| selection.marks.create!(lot_id:, account:) } - BulkRemoval.create!(session:, account:, copies: 2, - lots_data: [ { "catalog_entry_id" => entry.id, "finish" => "foil", "condition" => "damaged", "price_paid_cents" => nil, "quantity" => 2 } ]) - Location.create!(account:, name: "Binder A") - deck = Deck::Creation.new(account:, name: "Upgrade Deck", format: "mtg/modern").tap(&:save!).deck - Lot.add!(account:, entry:, quantity: 3, finish: "nonfoil", location: deck.zone("main")) - Deck::Proxy.add!(account:, zone: deck.zone("sideboard"), entry:, finish: "foil", quantity: 2) - NumberEntry.add!(account:, entry:, quantity: 2, finish: "foil", line_key: SecureRandom.hex(16)) - ImportUpload.store!(account:, bytes: "{}", plan: ImportHelpers::FakePlan.new("fingerprint")) - end - around do |example| ActiveRecord::Migration.verbose = false example.run @@ -114,7 +97,10 @@ def fill_every_referencing_table(user) expect(connection.select_value("SELECT households.name FROM users JOIN memberships ON memberships.user_id = users.id JOIN households ON households.id = memberships.household_id WHERE users.id = #{straggler.id}")).to eq("Late's household") expect(connection.select_value("SELECT COUNT(*) FROM households WHERE id = #{emptied}")).to eq(0) - expect { up }.not_to raise_error # a second run changes nothing + + settled = counts.merge("sequence" => sequence) + described_class.new.migrate(:up) # the Migrator skips a recorded version, so the migration itself is run again + expect(counts.merge("sequence" => sequence)).to eq(settled) end it "stops and rolls back when a count of accounts or of any of the thirteen referencing tables differs after the copy (AC-3.6)", :aggregate_failures do diff --git a/spec/support/referencing_table_helpers.rb b/spec/support/referencing_table_helpers.rb new file mode 100644 index 00000000..cf010ec7 --- /dev/null +++ b/spec/support/referencing_table_helpers.rb @@ -0,0 +1,26 @@ +# Fills the tables that reference accounts, for the specs that copy accounts (spec 090 decision 5 (a) and (c)). +module ReferencingTableHelpers + # One row in each of the thirteen tables that reference accounts, as the upgrade seed makes them (`script/upgrade/seeds/next.rb`). + # + # Writes through the models, so the example needs a database that holds the household columns; it does not need `:committed`. + # + # @param user [User] the user whose account owns the rows + # @return [void] + def fill_every_referencing_table(user) + account = user.account + entry = import_printing + Lot.add!(account:, entry:, quantity: 2, finish: "nonfoil") + Scanner::Sitting.add!(account:, printing: entry, finish: "nonfoil", reading_key: SecureRandom.hex(16)) + session = user.sessions.create! + selection = BulkSelection.for(session) + account.lots.limit(3).pluck(:id).each { |lot_id| selection.marks.create!(lot_id:, account:) } + BulkRemoval.create!(session:, account:, copies: 2, + lots_data: [ { "catalog_entry_id" => entry.id, "finish" => "foil", "condition" => "damaged", "price_paid_cents" => nil, "quantity" => 2 } ]) + Location.create!(account:, name: "Binder A") + deck = Deck::Creation.new(account:, name: "Upgrade Deck", format: "mtg/modern").tap(&:save!).deck + Lot.add!(account:, entry:, quantity: 3, finish: "nonfoil", location: deck.zone("main")) + Deck::Proxy.add!(account:, zone: deck.zone("sideboard"), entry:, finish: "foil", quantity: 2) + NumberEntry.add!(account:, entry:, quantity: 2, finish: "foil", line_key: SecureRandom.hex(16)) + ImportUpload.store!(account:, bytes: "{}", plan: ImportHelpers::FakePlan.new("fingerprint")) + end +end From 051d55ecec576bc3de24e384c5e27caeaa360370 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:38:46 -0500 Subject: [PATCH 28/41] test(households): check the invariants after an apply and the new tables' queries on every invisible page (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- .../household_untouched_spec.rb | 1 + spec/requests/households_invisible_spec.rb | 18 ++++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/spec/models/collection_import/household_untouched_spec.rb b/spec/models/collection_import/household_untouched_spec.rb index 997af10b..1445e154 100644 --- a/spec/models/collection_import/household_untouched_spec.rb +++ b/spec/models/collection_import/household_untouched_spec.rb @@ -29,5 +29,6 @@ def household_facts(user) expect([ household_facts(source), household_facts(target) ]).to eq(before) expect(target.account.lots.sum(:quantity)).to eq(3) + expect_household_invariants! end end diff --git a/spec/requests/households_invisible_spec.rb b/spec/requests/households_invisible_spec.rb index 0073b6c8..141cd89c 100644 --- a/spec/requests/households_invisible_spec.rb +++ b/spec/requests/households_invisible_spec.rb @@ -25,17 +25,23 @@ def statements_naming(pattern, &) it "shows a collector who is not an admin no page with the word (AC-4.1)", :aggregate_failures do sign_in_as(create(:user)) - [ collection_path, collection_path(view: "table"), more_path, catalog_entries_path ].each do |path| - get path - expect(response.body.downcase).not_to include("household"), "#{path} says household" + seen = statements_naming(/\b(households|memberships)\b/i) do + [ collection_path, collection_path(view: "table"), more_path, catalog_entries_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end end + expect(seen).to be_empty end it "shows no signed-out page the word (AC-4.1)", :aggregate_failures do - [ new_session_path, new_registration_path ].each do |path| - get path - expect(response.body.downcase).not_to include("household"), "#{path} says household" + seen = statements_naming(/\b(households|memberships)\b/i) do + [ new_session_path, new_registration_path ].each do |path| + get path + expect(response.body.downcase).not_to include("household"), "#{path} says household" + end end + expect(seen).to be_empty end it "lists users as before: a row, its copies and the same buttons (AC-4.5)", :aggregate_failures do From 26f210a7b0f587f91def77107c0b3cac2a6ed72c Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:38:55 -0500 Subject: [PATCH 29/41] docs(households): say which specs hold the household name equal, and drop a history phrase (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- app/models/household.rb | 4 +++- db/migrate/20261113120003_make_account_household_required.rb | 2 +- spec/script/upgrade/households_spec.rb | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/app/models/household.rb b/app/models/household.rb index ca1ab1e4..9ffa9c0d 100644 --- a/app/models/household.rb +++ b/app/models/household.rb @@ -14,7 +14,9 @@ class Household < ApplicationRecord before_destroy :refuse_while_held, prepend: true # The name a household of one gets (spec 090 decision 2): the user's first 88 characters and "'s household", at most 100. - # `script/upgrade/households.rb` and the backfill migration build the same name in SQL with `substr`; a spec holds the three equal. + # `script/upgrade/households.rb` and the backfill migration build the same name in SQL with `substr`: + # `spec/migrations/backfill_households_spec.rb` holds the backfill's SQL equal to this method, and + # `spec/script/upgrade/households_spec.rb` builds its fixture names with it, against the check's SQL. # # @param user_name [String] the user's name # @return [String] diff --git a/db/migrate/20261113120003_make_account_household_required.rb b/db/migrate/20261113120003_make_account_household_required.rb index 3f6e947f..c33857b0 100644 --- a/db/migrate/20261113120003_make_account_household_required.rb +++ b/db/migrate/20261113120003_make_account_household_required.rb @@ -105,7 +105,7 @@ def refuse_accounts_nobody_can_house end # As CreateHouseholdsAndMemberships: the copy resets accounts' row of sqlite_sequence, which the proof can't see (compare.rb skips sqlite_ tables). - # Always `main.sqlite_sequence`: the copy builds a TEMP table with AUTOINCREMENT, so a bare name resolves to `temp.sqlite_sequence` (found in phase 1). + # Always `main.sqlite_sequence`: the copy builds a TEMP table with AUTOINCREMENT, so a bare name resolves to `temp.sqlite_sequence`. def keeping_sequence seq = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") yield diff --git a/spec/script/upgrade/households_spec.rb b/spec/script/upgrade/households_spec.rb index 7a72185c..3559f51e 100644 --- a/spec/script/upgrade/households_spec.rb +++ b/spec/script/upgrade/households_spec.rb @@ -17,7 +17,7 @@ def snapshot(users:, memberships: nil, households: nil, accounts: nil) SQL users.each_with_index do |name, i| id = i + 1 - db.execute("INSERT INTO households VALUES (?, ?)", [ id, "#{name[0, 88]}'s household" ]) unless households == :none + db.execute("INSERT INTO households VALUES (?, ?)", [ id, Household.name_for(name) ]) unless households == :none db.execute("INSERT INTO accounts VALUES (?, ?, 'personal')", [ id, id ]) db.execute("INSERT INTO users VALUES (?, ?, ?)", [ id, name, id ]) db.execute("INSERT INTO memberships VALUES (?, ?, ?, 'manager')", [ id, id, id ]) unless memberships == :none From 2c251ca4c592ebc3cb8f2807c18294713f55eac3 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:39:04 -0500 Subject: [PATCH 30/41] docs(households): the upgrade note, the export format and the ADR say exactly what holds (spec 090) An old container's delete of a user after the third migration leaves an empty household, the export format gives the reason households are not in the file, and the ADR says foreign keys are switched off before the copy's own transaction opens. Co-Authored-By: Claude Sonnet 5.5 --- docs/adr/0030-a-household-is-a-group-of-accounts.md | 2 +- docs/export-format/README.md | 2 +- docs/releasing.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index 4f0eb36d..5a287bf7 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -45,7 +45,7 @@ What ADR 0017 keeps, unchanged: the `account_id` rule, `Current.account` resolve - **Good:** a household of one is the case every user is in on upgrade, so each later feature starts from a state that already exists on every instance, not from a migration of its own. - **Cost:** one fact, a personal account's household, is held twice (the account's column and its user's membership). A later issue that moves a user between households changes both in one transaction; until then a guard and the upgrade proof check them. - **Cost:** a copy is in one account at a time, so a household card in a personal deck means a transfer or a borrow (design decision 15, #385, #388). -- **Cost:** the third migration copies `accounts`, the most referenced table in the schema (eleven cascading children), with foreign keys switched off outside a transaction; inside one the copy would silently delete those children. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost; the copy also restores the table's id sequence. Had the fallback been chosen, `accounts.household_id` would be nullable in the database and the model alone would say it is present. +- **Cost:** the third migration copies `accounts`, the most referenced table in the schema (eleven cascading children), with foreign keys switched off before it opens its own transaction; with them on inside it the copy would silently delete those children. The upgrade proof, which seeds a row in every table that references `accounts`, is what shows no row is lost; the copy also restores the table's id sequence. Had the fallback been chosen, `accounts.household_id` would be nullable in the database and the model alone would say it is present. - **Cost:** "member" now means two things: spec 015's user without the admin flag, and a household's non-manager. They are independent, and no page shows either yet; #389 and the pages of #381 onward choose their words with both in view. - **Cost:** an older release can't hold a household with two members or a shared account, so rolling back the first migration refuses when either exists. - **Follow-up:** ADR 0017's status line records that this supersedes its one-user half; its text isn't rewritten. The tenancy area doc, the format document's Coverage and `CLAUDE.md`'s status paragraph change with spec 090. diff --git a/docs/export-format/README.md b/docs/export-format/README.md index 0eee8a63..25013836 100644 --- a/docs/export-format/README.md +++ b/docs/export-format/README.md @@ -527,7 +527,7 @@ Every table in Collector's database that belongs to an account or to a user is l `currency` is `accounts.currency`. -`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4). +`households` has neither `account_id` nor `user_id`, so the guard above does not ask for it; it is not in the file either, because a household is a fact about a group of accounts, not about the collection, and the question of what the shared collection's file carries about its household is decided with the shared collection (issue #384, the design's open question 4). ## Examples diff --git a/docs/releasing.md b/docs/releasing.md index ebb2b4b0..39ad52c1 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -67,7 +67,7 @@ Notes an operator needs when upgrading, newest first. `bin/docker-entrypoint` ru - **Every user gets a household (spec 090, ADR 0030, #380).** Three migrations run on boot, each without a transaction: `CreateHouseholdsAndMemberships` (two tables and two columns on `accounts`), `BackfillHouseholds` (a household of one for every user, named "'s household", with the user as its manager) and `MakeAccountHouseholdRequired` (`accounts.household_id` becomes `NOT NULL`, which SQLite does by copying the `accounts` table with foreign keys switched off, inside one transaction of its own). Nothing visible changes: no page, no word, nobody is signed out, and no row of any old table is changed (the backfill sets `accounts.household_id` and nothing else). Nothing is required of a healthy instance. **Take a backup before this upgrade, as always, because going back is not an undo:** - **An older image can't create a user once the third migration has run,** because it doesn't know the new required column. Going back means restoring the backup taken before the upgrade onto a new volume under the older image and losing what was written since; Collector documents no other downgrade. - **A failed third migration changes nothing and stops the boot.** Its backfill of stragglers, the copy and the restore of the id sequence are one transaction, so the next start runs it whole. With Compose the app stays down until it is fixed; with Kamal the old container keeps serving. - - **During a Kamal deploy, the old container's sign-up or add-a-user fails (a 500) for the seconds between the third migration and the switch to the new container.** A user an old container made between the second and the third migration is given a household by the third. + - **During a Kamal deploy, the old container's sign-up or add-a-user fails (a 500) for the seconds between the third migration and the switch to the new container.** A user an old container made between the second and the third migration is given a household by the third. An old container's delete of a user after the third migration has run leaves a household with no member, and nothing removes it (only the third migration removes such households, and only once); the households issue #381 or an operator removes it. - **An instance that holds an account with no user will not start** until it is removed, and the third migration names the ids (the first five) and changes nothing. Such an account exists only if the database was edited by hand, since an account is made with its user and deleted with it. Remove it with `bin/rails runner 'Account.find(ID).destroy'`, which deletes what it owns; run it in a one-off container of the new release, such as `docker compose run --rm web bin/rails runner 'Account.find(ID).destroy'` (`bin/docker-entrypoint` runs `db:prepare` only for the server command, so it doesn't meet the failing migration first). Don't delete the row with the `sqlite3` command: it doesn't enforce foreign keys and leaves what the account owned behind. Then start again. - **A start refuses a volume with a prices database and no primary (spec 086, #375).** No migration. `bin/docker-entrypoint` now runs `bin/rails start:check` before `db:prepare` on a server start. A volume that holds `production_prices.sqlite3` (or one of its `-wal`, `-shm` or `-journal` files) and no primary database, which a `backup:restore` killed between its two placings leaves, no longer starts a blank instance on stale prices: the container exits with status 1 and says what it found, why it stops and what to do. Every other volume starts as before. If you see it, remove the prices file and its three sidecar files, then restore again onto a volume that holds no other Collector database, or start blank. Nothing is required of a healthy instance. - **A prices database, held by the backup (spec 081).** The start creates a fifth SQLite file, `production_prices.sqlite3`, on the same volume, and its one migration, `CreatePriceTables`, runs on boot; it holds no row yet and the primary database is unchanged. Backups taken by this release are backup format version 2 (`manifest.json`, `primary.sqlite3`, `prices.sqlite3`), which the release before can't restore: restore them on this release or a newer one. A version 1 backup from an earlier release restores here, and the first start creates an empty prices database. `backup:create` refuses an instance with no prices database, so take backups after the upgraded instance has started once. Nothing is required of the operator. From 30c19498696a6cfeb5580f67fb9f582e5b083737 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:39:55 -0500 Subject: [PATCH 31/41] test(migrations): compare accounts by named columns, as a schema-loaded table orders them differently (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- spec/migrations/create_households_and_memberships_spec.rb | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/spec/migrations/create_households_and_memberships_spec.rb b/spec/migrations/create_households_and_memberships_spec.rb index 6c570a72..b8f9bd40 100644 --- a/spec/migrations/create_households_and_memberships_spec.rb +++ b/spec/migrations/create_households_and_memberships_spec.rb @@ -29,7 +29,8 @@ def referencing_tables = %w[bulk_removals bulk_selection_marks bulk_selections d def counts = ([ "accounts" ] + referencing_tables).to_h { [ it, count(it) ] } - def account_rows = connection.select_rows("SELECT * FROM accounts ORDER BY id") + # Named columns: a table loaded from db/schema.rb and one that went down and up order the new columns differently. + def account_rows = connection.select_rows("SELECT id, created_at, updated_at, currency FROM accounts ORDER BY id") def sequence = connection.select_value("SELECT seq FROM main.sqlite_sequence WHERE name = 'accounts'") @@ -161,13 +162,13 @@ def insert_user(number = 1) gone = create(:user) connection.execute("DELETE FROM users WHERE id = #{gone.id}") connection.execute("DELETE FROM accounts WHERE id = #{gone.account_id}") - before = [ counts, account_rows.map { it.first(4) }, sequence ] + before = [ counts, account_rows, sequence ] expect(before.first.values_at(*referencing_tables)).to all(be >= 1) expect(before.last).to be > before[1].last.first [ -> { down_all }, -> { up_all } ].each do |step| step.call - expect([ counts, account_rows.map { it.first(4) }, sequence ]).to eq(before) + expect([ counts, account_rows, sequence ]).to eq(before) expect(connection.select_rows("PRAGMA foreign_key_check")).to be_empty end end From 62a437c6b8b1cc42e5f5bcebc9a82d1fd7a41e56 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:43:28 -0500 Subject: [PATCH 32/41] docs(specs): verification of spec 090 with the implementation review (#380) Verdict SPEC-ALIGNED WITH FINDINGS; every finding fixed or recorded. The smoke run and the full CI are to be recorded when they run. Co-Authored-By: Claude Sonnet 5.5 --- .../verification.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 docs/specs/090-every-user-has-a-household/verification.md diff --git a/docs/specs/090-every-user-has-a-household/verification.md b/docs/specs/090-every-user-has-a-household/verification.md new file mode 100644 index 00000000..77d9de18 --- /dev/null +++ b/docs/specs/090-every-user-has-a-household/verification.md @@ -0,0 +1,64 @@ +# Verification: spec 090, every user has a household + +Spec v0.3.2, plan, ADR 0030. Issue #380. Branch `090-every-user-has-a-household`. Models: the spec, ADR and plan were written by the main session (Sonnet 5.5 until the plan, Opus 5.5 and Sonnet 5.5 after, as the session reminders gave them); every implementer batch was Sonnet 5.5; the spec reviews (two rounds), the plan review and the extra phase 6 review were read-only Fable 5.2; the implementation review was read-only Opus 5.5. + +## Implementation review + +SPEC-ALIGNED WITH FINDINGS + +Reviewed 2026-11-13 (sdd-review Mode B, read-only, Opus 5.5) against spec 090 v0.3.2, plan 090 and ADR 0030, over `git diff 71212946..HEAD` at the documentation commits (code 3c01b6a3 to 40f3553d). Nothing blocking; one important finding and nine minor; all fixed or recorded below. `bin/upgrade-smoke` from the v0.3.0 tag (AC-6.6) and the one full `bin/ci` had not run at the time of the review. + +### Coverage (AC to test, from the review) + +| AC | Test | Note | +|----|------|------| +| 1.1 to 1.4, 1.6, 1.7 | `spec/requests/households_creation_spec.rb`; `spec/models/user_spec.rb` | the first-user race is approximated by an admin created between the GET and the POST | +| 1.5 | `household_spec.rb` `.name_for`; `user_spec.rb`; `backfill_households_spec.rb` | the same three names in the harness fixture | +| 2.1 to 2.7, 2.9 | `membership_spec.rb`, `account_spec.rb`, `household_spec.rb`, `create_households_and_memberships_spec.rb` (model and database halves) | | +| 2.8 | `expect_household_invariants!` after the four create paths, the delete and an apply | the apply was added after the review (m1) | +| 3.1 to 3.3, 3.8 | `spec/migrations/backfill_households_spec.rb` | | +| 3.4 | `create_households_and_memberships_spec.rb` (literals and the full shape of the three tables before and after) | m3 fixed | +| 3.5 | `create_households_and_memberships_spec.rb` (all thirteen referencing tables filled, down and up), `make_account_household_required_spec.rb` | I1 fixed | +| 3.6, 3.7 | `make_account_household_required_spec.rb` (ten examples: counts, sequence, PRAGMA checks, shapes, the enforcement refusal, a dangling row kept, the fault after the DROP, the orphan refusal, the stragglers step, a count of each of the thirteen tables changing) | m2 fixed | +| 4.1, 4.5 | `spec/requests/households_invisible_spec.rb` (admin, collector, signed out; queries naming the new tables counted on all) | m4 fixed | +| 4.2 | `bin/rails routes` check, recorded below | | +| 4.3 | the existing tenancy request specs, unedited | 1403 examples green on their implementer's run; the full `bin/ci` is the final proof | +| 4.4 | `households_creation_spec.rb`, `user_spec.rb` | | +| 5.1 to 5.4 | `spec/export_format_document_spec.rb`; `collection_import/household_untouched_spec.rb`; the backup guards; the format README | | +| 6.1 to 6.5 | `spec/script/upgrade/households_spec.rb`, `harness_spec.rb` ("the households"), `seeds_spec.rb` | the restore leg's household check is proved by the smoke run (m5) | +| 6.6 | `bin/upgrade-smoke`, recorded below | | + +### Findings and their disposition + +- **I1 (important), AC-3.5: migration (a)'s `down` was untested with rows in the eleven cascading tables and lacked (c)'s guards.** Fixed in `7187f2f1`: `fill_every_referencing_table` is a spec support module used by both migration specs; (a)'s `down` has `require_enforcement_off` and the before and after state check; two examples (rows in all thirteen referencing tables going down through all three migrations and back; an outer-transaction refusal) were added. The refusal example failed first for the right reason (the cascades deleted rows and the columns were dropped); the rows example is a characterisation of the working path and passed first. +- **m1** invariants after an apply: fixed (`051d55ec`). **m2** "run again changes nothing" could not fail: fixed to capture the counts and sequence and run the migration instance (`7187f2f1`). **m3** AC-3.4 literals only: the full shape before and after added (`7187f2f1`, `30c19498` for the column order a schema-loaded table has). **m4** queries counted on the collector's and signed-out requests: fixed (`051d55ec`). +- **m5** the harness spec has no restore-leg household example: accepted; the smoke run below is the evidence that the restored snapshot passes both tiers. +- **m6** an older container's delete of a user after the third migration leaves a household with no member that nothing removes: added to `docs/releasing.md`'s Upgrade note (`2c251ca4`); #381 or an operator removes it. +- **m7** comment accuracy: `Household.name_for`'s comment names the two specs, the harness fixture uses `Household.name_for`, the history phrase is gone (`26f210a7`). **m8** the format README's sentence carries its reason (`2c251ca4`). **m9** ADR 0030's wording (`2c251ca4`). +- **m10** (c)'s orphan-account refusal and the re-homing `UPDATE` before it go beyond decision 5: deliberate, sound, and inside the one transaction; recorded here. The plan's note on why they are right is in `plan.md` phase 6. + +### Migration (c) review trail + +Traced line by line against activerecord 8.1.4.1's `sqlite3_adapter.rb` and `database_statements.rb` by a plan review (Fable 5.2) before the batch, and again as changed by a second review (APPROVE WITH CHANGES; the dangling-row boot loop, the unproven guards and the lambda's association instance were fixed), then by this review. One bug neither trace found was found by the phase 1 implementer: a table copy builds a TEMP table with AUTOINCREMENT, so a bare `sqlite_sequence` resolves to `temp.sqlite_sequence`; every migration now reads and writes `main.sqlite_sequence` (lesson recorded in the wrap). + +## Evidence + +All commands from the worktree. Targeted runs until the final one; one full `bin/ci` is owed after the smoke. + +| Check | Result | +|---|---| +| `bin/ci checks` on the code commit before the review fixes | green (RuboCop, YARD with no warning, the comment ratchet on 6 objects, Brakeman, bundler-audit, importmap audit) | +| `bin/ci checks` on the merge `b13fb164` (origin/main at f7df86ce, with the new licences step) | green, 30.4 s | +| Targeted specs on the merge `b13fb164`: `spec/migrations spec/script/upgrade`, the household model, request and import specs, the export guard, the backup guards | 471 examples, 0 failures (seed 15539) | +| Phase runs by the implementers | phase 1: 11 + 9 examples; phase 2: 39; phase 3: 7 (+ the 10,000-user timing example, 3.0 s and 4.1 s); phases 4 and 5: `spec/requests` 1403, `spec/script/upgrade` 171; phase 6: `spec/migrations` 80 under three seeds; review fixes: 95 | +| One-off search (decision 9 (iv)): `git grep -n -i 'household\|membership' -- app/views app/helpers app/controllers app/javascript config/locales ':!.kamal/secrets'` | nothing | +| Routes (AC-4.2, decision 9 (iii)) | `git diff main -- config/routes.rb` empty; no controller, route or view file changed in the diff; `bin/rails routes | wc -l` is 145 on the branch | +| `households_invisible_spec` and `household_untouched_spec` on their first run | passed (nothing visible was added; an apply never wrote a household), as the plan expected | + +### `bin/upgrade-smoke` + +To be recorded when it runs on the commit that is pushed (the previous release v0.3.0 built from its tag with `git archive`, the code under test built beside it, `--previous-version 0.3.0`). **Expected to print** for the households: `households: ok (backfilled)` after the upgraded snapshot's comparison and again after the restored snapshot's, and nothing for the `next` snapshot unless it fails (it raises `AC-6.1: the next snapshot: …`); the comparison lines show `accounts`, `users`, `lots` and every old table `same`; the run ends `OK: v0.3.0`. + +### The full `bin/ci` + +To be recorded with the CI run of the push. From 90379d991af33baf04b604d83931b77f3f2202ae Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:44:45 -0500 Subject: [PATCH 33/41] docs(adr): ADR 0030 stays Proposed; the maintainer accepts it (#380) The maintainer ruled its decisions on 2026-11-13; accepting the ADR is theirs, recorded in the plan-retirement pull request as for ADR 0029. Spec 090 v0.3.3 says so. Co-Authored-By: Claude Sonnet 5.5 --- docs/adr/0030-a-household-is-a-group-of-accounts.md | 2 +- docs/specs/090-every-user-has-a-household/spec.md | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/adr/0030-a-household-is-a-group-of-accounts.md b/docs/adr/0030-a-household-is-a-group-of-accounts.md index 5a287bf7..4d0de753 100644 --- a/docs/adr/0030-a-household-is-a-group-of-accounts.md +++ b/docs/adr/0030-a-household-is-a-group-of-accounts.md @@ -2,7 +2,7 @@ ## Status -Accepted (2026-11-13), written with spec 090 and accepted with its pull request. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) was ruled by the maintainer the same day: the table copy, not the nullable fallback, on two conditions (the plan review traces it against Rails 8.1.4.1's `alter_table`, and the upgrade proof runs from the previous release built from its tag). +Proposed (2026-11-13), written with spec 090. The maintainer ruled its decisions on 2026-11-13, and accepts the ADR itself with the pull request; the acceptance is recorded in the plan-retirement pull request, as for ADR 0029. It supersedes the one-user half of [ADR 0017](0017-row-level-tenancy-one-user-per-account.md): an account no longer has to be the only account of its user's group. It keeps the rest of ADR 0017. The ownership model it records was ruled by the maintainer on 2026-11-13 (design decision 9; roadmap rulings 43 and 44), and so were what the export carries (decision 6 here, spec 090 decision 4) and the household's name and manager rule. The shape of the third migration (decision 7) was ruled by the maintainer the same day: the table copy, not the nullable fallback, on two conditions (the plan review traces it against Rails 8.1.4.1's `alter_table`, and the upgrade proof runs from the previous release built from its tag). **Date:** 2026-11-13 **Feature:** 090-every-user-has-a-household diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index 0a9e4aaa..f5793279 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -1,7 +1,7 @@ # Feature 090: Every User Has a Household **Status:** Approved (2026-11-13, Gate A: the coordinator under the maintainer's delegation, v0.3.1; the maintainer ruled decision 5's step (c) on 2026-11-13, v0.3.2) -**Version:** 0.3.2 +**Version:** 0.3.3 **Created:** 2026-11-13 **Last Updated:** 2026-11-13 **Depends on:** [spec 004](../004-design-system-collection/spec.md) (accounts, FR-2 to FR-5; frozen, by [tenancy and accounts](../../architecture/tenancy-and-accounts.md)), [spec 017](../017-export-format/spec.md) FR-9 (the coverage guard), [spec 031](../031-upgrade-from-previous-release/spec.md) (the upgrade proof), [spec 049](../049-session-idle-expiry/spec.md) (three migrations without a transaction, the precedent for decision 5) @@ -17,6 +17,7 @@ | Version | Date | Change | |---------|------|--------| +| 0.3.3 | 2026-11-13 | ADR 0030 stays Proposed in the pull request (the maintainer ruled its decisions; accepting an ADR is theirs, recorded in the plan-retirement pull request); decision 11 and FR-5 say so. No requirement changed | | 0.3.2 | 2026-11-13 | **The maintainer ruled decision 5's step (c): take the table copy, not the nullable fallback**, on the two conditions the decision records. No requirement changed; the open question is closed | | 0.3.1 | 2026-11-13 | **Gate A: approved** by the coordinator under the maintainer's delegation (2026-11-13): two Fable rounds, nothing blocking after round 2, every finding fixed; decision 5's step (c) and the nullable fallback stay with the maintainer, whose ruling the coordinator sends. The plan makes (c) and everything that exists only for it one phase of its own, last among the migrations, with (a) and (b) complete and green without it. No requirement changed | | 0.3.0 | 2026-11-13 | Spec review round 2 (Fable 5.2, APPROVE WITH CHANGES, 0 blocking, 5 important, 8 minor) fixed: (c) is one transaction around its stragglers step, the copy and the sequence restore (upsert), so a failed (c) changes nothing; the README claim is corrected (no downgrade is documented) and `docs/releasing.md`'s Upgrade notes join the docs; the fallback's stragglers hole is stated and what it would need; four facts added to the comparison (a failed (c) stops the boot, the deploy window, fresh volumes and backups, three copies on a full rollback); the suite's bare accounts are named allowed edits; the name's length `CHECK`; an empty household left by an older container is removed by (c); AC-3.4 by `PRAGMA`, AC-3.5's sequence, AC-3.6's `:committed` example and its fault case; #381 and #384 revisit decision 7 | @@ -80,7 +81,7 @@ This feature adds the structure and nothing that uses it: a household with a nam 8. **A household is made by the user model, so every path gets one.** The one place an account is made from a user today (`before_validation :build_account`) also makes the household and the manager membership; the first-run sign-up, the open sign-up, the admin's add-a-user form and `bin/rails collector:user` all go through it, and a spec per path says so. Nothing in a request names a household, a role or a kind (`household_id`, `role` and `kind` are not permitted from parameters). 9. **"Nothing looks different" is checked by five things, beyond the existing specs passing unchanged.** (i) A request spec renders the collection, the admin users page, the account menu, sign-in, sign-up and the catalog search as a collector and as an admin and asserts that none of the bodies contains "household" (case-insensitive), and counts the queries of those requests that name `households` or `memberships` (none). (ii) The existing tenancy request specs and system specs are not edited, except where they build a user by hand (the factory makes the household), and the run proves they pass. **The allowed edits to the suite's support code:** `factory :account` (used by `spec/models/account_spec.rb`, `spec/helpers/scanners_helper_spec.rb` and `spec/lib/collector/scanner_findings/art_sitting_report_spec.rb`) gains a household, a bare one with no member (those examples have no user, and the helper of AC-2.8 isn't called after them); and the two places that insert an account by SQL (`spec/support/backup_helpers.rb`, `spec/lib/collector/backup/snapshot_spec.rb`) insert a household first. Nothing else in `spec/` changes for the tenancy checks. (iii) `bin/rails routes` on the branch and on `main` before it are compared once, at the implementation review, and the empty diff is recorded in `verification.md` (no permanent route-list spec: it would conflict with every branch that adds a route). (iv) A one-off search of `app/views`, `app/helpers`, `app/controllers`, `app/javascript` and `config/locales` for "household" and "membership" finds nothing outside the models, recorded beside the routes diff. (v) The upgrade proof's web leg (spec 031 Story 5) signs in as each seeded user on the upgraded instance and loads the collection, locations and export pages, as it does today, and that leg runs after the backfill. 10. **The upgrade proof gains a household check in two tiers, and `expected_changes.yml` stays empty.** *(Ruled by the maintainer 2026-11-13, as recommended.)* `compare.rb` reads only the old database's tables and the columns both databases have, so the new tables and columns are invisible to it and no entry would match; the harness gains a check on the snapshots it takes. **Tier 1, always** (it must stay true when #381 and #384 add joined households and shared accounts, so it names no count): every user has exactly one membership, every household has at least one manager, every account has a household, and a personal account's household is its user's. **Tier 2, only when the old snapshot has no `households` table** (the backfill's own facts): the households' count equals the users' (the seed's two users make two), every membership is `manager`, every account is `personal`, and every household's name is its user's name's form of decision 2. Tier 1 runs on the upgraded snapshot, on the restore leg's, and on the `next` snapshot of a fresh instance of the code under test (`next_release`), the one place the production image's creation path is proved. The seed for the next release (`script/upgrade/seeds/next.rb`) creates its users through the models, so the new tables hold rows, as the seed rule needs. -11. **The ADR is 0030, accepted with the pull request, and the area docs change with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 has a status line saying it is superseded in part; its one-user text isn't rewritten; ADR 0030 is Accepted in the same pull request (its status line already cites it). The docs that change: [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) (glossary, invariants, the delete, accepted limits, and a line that `households` and `memberships` are instance-level tables never scoped through `Current.account`), [`.claude/rules/multi-tenancy.md`](../../../.claude/rules/multi-tenancy.md) (the same line), [`data-portability.md`](../../architecture/data-portability.md) and the [export format](../../export-format/README.md) (coverage, decision 4, and the sentence at its version history), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) (the harness's check), `docs/releasing.md`'s Upgrade notes (the migration's entry and warning), the [specs index](../README.md) (this spec's row and spec 004's, both directions), and `CLAUDE.md`'s Project Status paragraph. +11. **The ADR is 0030, proposed with the pull request (the maintainer accepts it), and the area docs change with it.** [ADR 0030](../../adr/0030-a-household-is-a-group-of-accounts.md) supersedes the one-user half of ADR 0017 (an account may now be one of several in a household, `users.account_id` stays unique, an account stays the unit of ownership and privacy) and keeps the rest of ADR 0017 (the `account_id` rule, `Current.account`, 404 for another account's record). ADR 0017 has a status line saying it is superseded in part; its one-user text isn't rewritten; ADR 0030 stays Proposed in this pull request: the maintainer's ruling on its decisions is recorded in it, and accepting the ADR itself is theirs, recorded in the plan-retirement pull request as for ADR 0029. The docs that change: [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) (glossary, invariants, the delete, accepted limits, and a line that `households` and `memberships` are instance-level tables never scoped through `Current.account`), [`.claude/rules/multi-tenancy.md`](../../../.claude/rules/multi-tenancy.md) (the same line), [`data-portability.md`](../../architecture/data-portability.md) and the [export format](../../export-format/README.md) (coverage, decision 4, and the sentence at its version history), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) (the harness's check), `docs/releasing.md`'s Upgrade notes (the migration's entry and warning), the [specs index](../README.md) (this spec's row and spec 004's, both directions), and `CLAUDE.md`'s Project Status paragraph. 12. **Hosted-door rules.** *Cost per household:* one household row, one membership row per member, no row per lot or per request. *Stored per account or user:* the household's name and members' roles are the household's, nothing else is stored about a collector's choices. *The operator is not the user:* no admin page changes, so none shows a collection; the household's fields are not on any page. ## User Stories and Acceptance Criteria @@ -186,7 +187,7 @@ The harness checks the upgraded and the restored snapshots (decision 10). ### FR-5: Documentation -The ADR (accepted with the pull request), the tenancy area doc, `.claude/rules/multi-tenancy.md`, the format document's Coverage and sentences, the release-and-deploy area doc's upgrade-proof paragraph, `docs/releasing.md`'s Upgrade notes (this migration's entry and its warning), `CLAUDE.md`'s Project Status line for accounts, the specs index in both directions (decision 11). +The ADR (proposed, with the maintainer's rulings on its decisions), the tenancy area doc, `.claude/rules/multi-tenancy.md`, the format document's Coverage and sentences, the release-and-deploy area doc's upgrade-proof paragraph, `docs/releasing.md`'s Upgrade notes (this migration's entry and its warning), `CLAUDE.md`'s Project Status line for accounts, the specs index in both directions (decision 11). **Must:** record the supersession in this spec, ADR 0030 and the index in both directions (`.claude/rules/documentation.md`). **Must not:** rewrite ADR 0017's text, or restate an area's rules in `CLAUDE.md`. From 77bb2dc29653fedfd9d4f7eac3bd3f730222bd73 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:44:54 -0500 Subject: [PATCH 34/41] chore(memory): capture session learnings (spec 090) Two lessons (a table copy of a referenced table; association(:name) creates the association) and one feedback memory (re-review a changed data-loss step). Co-Authored-By: Claude Sonnet 5.5 --- .claude/memory/MEMORY.md | 1 + .../rereview-changed-data-loss-steps.md | 12 ++++++++ ...1-13-a-table-copy-of-a-referenced-table.md | 29 +++++++++++++++++++ ...ssociation-name-creates-the-association.md | 25 ++++++++++++++++ 4 files changed, 67 insertions(+) create mode 100644 .claude/memory/rereview-changed-data-loss-steps.md create mode 100644 docs/lessons/2026-11-13-a-table-copy-of-a-referenced-table.md create mode 100644 docs/lessons/2026-11-13-association-name-creates-the-association.md diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md index 66aab3b5..cb1be3e0 100644 --- a/.claude/memory/MEMORY.md +++ b/.claude/memory/MEMORY.md @@ -90,3 +90,4 @@ - [No git stash or sqlite3 in briefs](no-git-stash-in-briefs.md) — forbid git stash/reset/checkout and the sqlite3 command (it prompts the maintainer) in every subagent brief (except restoring a mutation-tested file); a stash/pop on a clean tree applied an unrelated old stash in spec 080 - [Unset `config.x` answers nil](rails-config-x-unset.md) — an unset `Rails.configuration.x` key is an empty OrderedOptions; assert a boot reading's class, not only nil attributes - [Mutation proof in place of failing-first](mutation-proof-when-behaviour-already-built.md) — only where an earlier task already built the behaviour; each mutation and its failure go in the commit body (coordinator, spec 085 Gate B) +- [Re-review a changed data-loss step](rereview-changed-data-loss-steps.md) — after a plan changes a step that can lose data (a table copy, a delete), run the read-only review again on that step before its batch (spec 090 phase 6) diff --git a/.claude/memory/rereview-changed-data-loss-steps.md b/.claude/memory/rereview-changed-data-loss-steps.md new file mode 100644 index 00000000..a13c34ef --- /dev/null +++ b/.claude/memory/rereview-changed-data-loss-steps.md @@ -0,0 +1,12 @@ +--- +name: rereview-changed-data-loss-steps +description: when a plan step that can lose a collector's data changes after its review, run the read-only review again on the changed step before its batch (spec 090's phase 6) +metadata: + type: feedback +--- + +A plan review traced migration (c) as first written; the plan then changed it (guards, a fault test, the corrected `down`). The coordinator asked for a second read-only Fable review limited to the changed step before its batch ran, and it found a boot loop on dangling rows that the first review of the unchanged step could not have. + +**Why:** a trace is of the code it read; a fix to that code is new code. On a step that deletes data on a failure, the cost of one more read-only review is small. + +**How to apply:** after any change to a data-loss step (a table copy, a delete, a backfill) that follows its review, dispatch the review again, limited to the step and what changed, and send its verdict to the coordinator before the implementer batch starts. See [[plan-reviews-run-the-code]] for how plan reviews trace. diff --git a/docs/lessons/2026-11-13-a-table-copy-of-a-referenced-table.md b/docs/lessons/2026-11-13-a-table-copy-of-a-referenced-table.md new file mode 100644 index 00000000..aa11b81f --- /dev/null +++ b/docs/lessons/2026-11-13-a-table-copy-of-a-referenced-table.md @@ -0,0 +1,29 @@ +--- +date: 2026-11-13 +spec: 090 (#380) +tags: [migrations, sqlite, upgrade, tenancy] +--- + +# Lesson: copying a table that others reference needs enforcement off before the transaction, one transaction, and a before-and-after check + +## Context + +Spec 090's third migration makes `accounts.household_id` `NOT NULL`. SQLite can't do that in place, so Rails copies `accounts`, which thirteen tables reference (eleven with `ON DELETE CASCADE`). + +## What happened + +- Inside a transaction, `PRAGMA foreign_keys = OFF` does nothing, so the copy's `DROP TABLE` runs the cascades and deletes every collector's decks, places and working state with no error. The migration therefore turns enforcement off first and opens its own transaction around the stragglers step, the copy and the sequence restore, so a failure rolls all three back. +- The copy builds a TEMP table with `AUTOINCREMENT`, which gives the connection a `temp.sqlite_sequence`; a bare `sqlite_sequence` then resolves to it. Restoring the id sequence there restores nothing. Two spec reviews and two plan reviews traced the Rails code and missed it; the first implementer found it by running the spec on a fresh test database. Read and write `main.sqlite_sequence`. +- The first post-check reported every `PRAGMA foreign_key_check` row. A database an operator edited with the `sqlite3` command (which doesn't enforce keys) holds dangling rows already, so the check would have stopped every boot after that. The check now compares the state before and after the copy (row counts of the thirteen tables, and the key violations pointing at `accounts` or `households`) and raises only on a difference. +- Rails runs no check after a copy, so the migration checks enforcement is off before it starts and the counts after. + +## What to do next time + +- A migration that copies a referenced table: enforcement off before the transaction, one transaction for everything that must commit together, the id sequence restored in it (as `main.sqlite_sequence`), and a post-check that compares with the state before, never with a clean database. +- Prove it with a committed spec that fills a row in every referencing table and counts them before and after, and one that makes the copy itself fail after the `DROP`. +- The refusals a migration makes on its own (an account with no user) must say how to clear them and be written in `docs/releasing.md`'s Upgrade note. + +## Signals to watch for + +- A migration that calls `change_column_null`, `remove_column` or `add_foreign_key` on a table with cascading children and runs inside Rails' migration transaction. +- A reading of `sqlite_sequence` with no schema prefix. diff --git a/docs/lessons/2026-11-13-association-name-creates-the-association.md b/docs/lessons/2026-11-13-association-name-creates-the-association.md new file mode 100644 index 00000000..c6b9e820 --- /dev/null +++ b/docs/lessons/2026-11-13-association-name-creates-the-association.md @@ -0,0 +1,25 @@ +--- +date: 2026-11-13 +spec: 090 (#380) +tags: [rails, validations, tenancy, performance] +--- + +# Lesson: `association(:name)` creates the association, so a guard that reads it can cause the read it avoids + +## Context + +`User` validates that its membership is in its account's household. The validation must not query on an ordinary update of a signed-in user, whose account the session has already loaded (spec 090 AC-4.5), so it runs only when both associations are loaded. + +## What happened + +The first draft guarded with `association(:membership).loaded? && association(:account).loaded?`. Calling `association(:membership)` creates the association instance even when nothing was loaded. On the next validation of the same object, the autosave check of a has-one association finds that instance and reads it, which runs a `SELECT` on `memberships` and then on `households`. No ordinary path validates a user twice, so no spec failed; the second Fable review found it by reading the Rails source (`validate_has_one_association`, `association_cached?`). + +## What to do next time + +- Put `association_cached?(:name)` before `association(:name).loaded?` in any condition that must not load. +- Prove it with an example that loads the record the way the request does (here `Session.includes(user: :account)`), validates it twice and updates it under a subscription to `sql.active_record`, and expects no query naming the table. + +## Signals to watch for + +- A validation or callback condition that calls `association(:x)` on a record that callers load partially. +- A requirement of the form "an ordinary request reads no X" with only a one-validation example. From 3dc5765c08c7f36b15fbf7c4986b9669ab61a59b Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:45:15 -0500 Subject: [PATCH 35/41] docs(architecture): distil spec 090 into tenancy and accounts, release and deploy and data portability The area docs were written with the feature (phase 7); this adds their distillation log rows and freezes the spec's index row with its plan marked retirement owed. Co-Authored-By: Claude Sonnet 5.5 --- docs/architecture/data-portability.md | 1 + docs/architecture/release-and-deploy.md | 1 + docs/architecture/tenancy-and-accounts.md | 1 + docs/specs/README.md | 2 +- 4 files changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/architecture/data-portability.md b/docs/architecture/data-portability.md index 835ad905..5fff8559 100644 --- a/docs/architecture/data-portability.md +++ b/docs/architecture/data-portability.md @@ -297,3 +297,4 @@ Measured bars, not constants (recorded in each spec's `verification.md`): the ba | 2026-11-10 | Spec 064 (an import that changes a deck's format moves no card into Considering, the app's change of format does; no format change, no field, no version) | [#303](https://github.com/plainprogrammer/Collector/pull/303) | | 2026-11-11 | Spec 068 (export format version 4 gains the required `proxies` list: the export, `duplicate_proxy` and the proxy rows, the plan's proxy outcomes, the fingerprint, the preview's Proxies sections, the apply's order and upsert, the confirmation and notice; an older file leaves proxies alone) | [#313](https://github.com/plainprogrammer/Collector/pull/313) | | 2026-11-13 | Spec 081 (decision 4, AC-3.2): an export holds no price; a backup holds the primary and prices databases | [#405](https://github.com/plainprogrammer/Collector/pull/405) | +| 2026-11-13 | Spec 090 (the household's tables and columns are left out of the export, `households` is outside the guard, the format stays version 4; #380) | spec 090's pull request | diff --git a/docs/architecture/release-and-deploy.md b/docs/architecture/release-and-deploy.md index 00da356d..51039915 100644 --- a/docs/architecture/release-and-deploy.md +++ b/docs/architecture/release-and-deploy.md @@ -333,3 +333,4 @@ The proof stops each instance it boots and compares databases (the previous rele | 2026-11-13 | Spec 087 (`bin/image-smoke` reads the third-party licences page: 200 before any user exists, the `tesseract.js` and `turbo-rails` rows, a gem's licence file under its own name, no `yard` row, and the sign-in page's link; no `Dockerfile` or workflow change; #376). Written during the build (T6.1) and checked here | [#412](https://github.com/plainprogrammer/Collector/pull/412) | | 2026-11-13 | Spec 086 (decisions 1, 2, 4, 5, 6 and 7, FR-1 to FR-4: the start check before `db:prepare` on a server start, what makes a primary "there", the message and exit status, the restore's changed advice, the smoke leg that proves it on the image; #375) | [#415](https://github.com/plainprogrammer/Collector/pull/415) | | 2026-11-13 | Spec 091 (the licence check is a `bin/ci checks` step after the importmap audit, with no workflow change; #402) | spec 091's pull request | +| 2026-11-13 | Spec 090 (the upgrade proof's two-tier households check on the upgraded, restored and next snapshots; `expected_changes.yml` stays empty; the third migration's table copy and its upgrade note; #380) | spec 090's pull request | diff --git a/docs/architecture/tenancy-and-accounts.md b/docs/architecture/tenancy-and-accounts.md index dafc02f4..e6cf7dbf 100644 --- a/docs/architecture/tenancy-and-accounts.md +++ b/docs/architecture/tenancy-and-accounts.md @@ -205,3 +205,4 @@ Deleting a user removes, in this order, their membership (deleted without the me | 2026-11-11 | Spec 070 (the replacement pages' 404 rule through `FindsDeck`, and that a lot id in Use my copy's form that is not the account's free lot of the card is "no longer free", never a distinct answer) | [#323](https://github.com/plainprogrammer/Collector/pull/323) | | 2026-11-11 | Spec 071 (the Add to deck routes' 404 rule: another account's zone, a place's id, a deck's own row, a zone of another type's deck, a removed zone and a printing that can't be added to a deck; signed out goes to sign in) | [#325](https://github.com/plainprogrammer/Collector/pull/325) | | 2026-11-13 | Spec 087 (`/licences` joins the endpoints that answer without a sign-in, also before the first user: no tenant data, the same page for everyone, the versions it shows are public by decision; #376). Written during the build (T6.1) and checked here | [#412](https://github.com/plainprogrammer/Collector/pull/412) | +| 2026-11-13 | Spec 090 (every user has a household: `households` and `memberships`, an account's `household_id` and `kind`, the rules of a household, a household made with every user by `User`, the delete order, the instance-level tables; ADR 0030 supersedes ADR 0017's one-user half; #380) | spec 090's pull request | diff --git a/docs/specs/README.md b/docs/specs/README.md index b1c97ccc..72aef8ca 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -103,5 +103,5 @@ A plan is deleted from the tree in a commit of its own after its feature's pull | [085 A Price Source Contract, and Scryfall's Prices Through It](085-price-source-contract/spec.md) | Frozen, by [prices](../architecture/prices.md) | [Prices](../architecture/prices.md) (the source contract, Scryfall's source, the source and series rows) | retired in `e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4`; `git show e78c902bc0f9f6158bfddc1fbfffdf1edf9da3e4^:docs/specs/085-price-source-contract/plan.md` | | | | [086 A Start Refuses a Volume That Holds a Prices Database and No Primary](086-start-refuses-orphan-prices/spec.md) | Frozen, by [release and deploy](../architecture/release-and-deploy.md) | [Release and deploy](../architecture/release-and-deploy.md) (the start check before `db:prepare`, its message, the restore's advice and the smoke leg); [prices](../architecture/prices.md) (a pointer only) | retired in `2c5939d4604cc1221534677c3e4bcc4aa5bdc916`; `git show 2c5939d4604cc1221534677c3e4bcc4aa5bdc916^:docs/specs/086-start-refuses-orphan-prices/plan.md` | In part, spec 081 decision 8's "Collector must never be started on such a volume" and AC-5.9's advice (the start now refuses it; the restore's advice says so) ([#375](https://github.com/plainprogrammer/Collector/issues/375)) | | | [087 The Licences of the Third-Party Software and Assets in the Image, Readable from the App](087-third-party-licences-in-app/spec.md) | Frozen, by [catalog core](../architecture/catalog-core.md) (the licence line's third sentence, the page, the reader, the manifest, the guard), [release and deploy](../architecture/release-and-deploy.md) (the smoke check) and [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint); the `Licences` pattern is in the [design system](../design-system/README.md) | [Catalog core](../architecture/catalog-core.md) (the footer and the third-party licences page); [release and deploy](../architecture/release-and-deploy.md) (the smoke check); [tenancy and accounts](../architecture/tenancy-and-accounts.md) (the endpoint that skips sign-in); [design system](../design-system/README.md) (`Licences`, `Footer`) | retired in `11de80918d734d3b6e001d16b013f8a77c820ffc`; `git show 11de80918d734d3b6e001d16b013f8a77c820ffc^:docs/specs/087-third-party-licences-in-app/plan.md` | 084: AC-1.4's "no other part of the line is a link", the full wording of AC-2.1, AC-2.3, AC-2.4 and AC-2.14 and decision 2's wording table, AC-2.13's two links, FR-1's "with two links", the Accessibility NFR's and Users and Context's two links, decision 12's no-script notice (it gains the link), the Non-Goals "a licence page in the app" and "Listing third-party licences" ([#376](https://github.com/plainprogrammer/Collector/issues/376)) | | -| [090 Every User Has a Household](090-every-user-has-a-household/spec.md) | Approved | [Tenancy and accounts](../architecture/tenancy-and-accounts.md); [data portability](../architecture/data-portability.md) (coverage); [release and deploy](../architecture/release-and-deploy.md) (the upgrade proof) | present | [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md); spec 004 FR-3's "one user per account" in part (an account is now in a household) ([#380](https://github.com/plainprogrammer/Collector/issues/380)) | | +| [090 Every User Has a Household](090-every-user-has-a-household/spec.md) | Frozen, by [tenancy and accounts](../architecture/tenancy-and-accounts.md), [release and deploy](../architecture/release-and-deploy.md) and [data portability](../architecture/data-portability.md) | [Tenancy and accounts](../architecture/tenancy-and-accounts.md); [data portability](../architecture/data-portability.md) (coverage); [release and deploy](../architecture/release-and-deploy.md) (the upgrade proof) | retirement owed | [ADR 0017](../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by [ADR 0030](../adr/0030-a-household-is-a-group-of-accounts.md); spec 004 FR-3's "one user per account" in part (an account is now in a household) ([#380](https://github.com/plainprogrammer/Collector/issues/380)) | | | [091 CI Fails on a Dependency Whose Licence Isn't Allowed](091-licence-check-in-ci/spec.md) | Frozen, by [catalog core](../architecture/catalog-core.md) (the policy file, the check and its rules) and [release and deploy](../architecture/release-and-deploy.md) (the `bin/ci checks` step); [ADR 0031](../adr/0031-a-dependency-licence-is-a-decision.md) | [Catalog core](../architecture/catalog-core.md) (the third-party licences page, its guard, the policy and the check); [release and deploy](../architecture/release-and-deploy.md) (the step) | retirement owed | | | From 469449dab79da847d603901ec44303fd4ac780df Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:45:29 -0500 Subject: [PATCH 36/41] docs(specs): mark spec 090 frozen by the tenancy, release and data portability area docs (spec 027 D4) Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/spec.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/specs/090-every-user-has-a-household/spec.md b/docs/specs/090-every-user-has-a-household/spec.md index f5793279..f3e3917f 100644 --- a/docs/specs/090-every-user-has-a-household/spec.md +++ b/docs/specs/090-every-user-has-a-household/spec.md @@ -10,6 +10,7 @@ **ADR:** [0030](../../adr/0030-a-household-is-a-group-of-accounts.md), which supersedes the one-user half of [ADR 0017](../../adr/0017-row-level-tenancy-one-user-per-account.md) **Supersedes:** spec 004 FR-3 in part ("an account has exactly one user": an account is now in a household, and `users.account_id` stays unique) and [ADR 0017](../../adr/0017-row-level-tenancy-one-user-per-account.md)'s one-user half, by ADR 0030; the area doc [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md) records it **Design:** [households and the hosted service](../../superpowers/specs/2026-11-13-households-and-hosted-service-design.md), decisions 9 to 11 (rulings 43 and 44 of the [roadmap](../../roadmap.md)) +**Frozen:** the current rules are in [`tenancy-and-accounts.md`](../../architecture/tenancy-and-accounts.md), [`release-and-deploy.md`](../../architecture/release-and-deploy.md) and [`data-portability.md`](../../architecture/data-portability.md) (whole spec); see the [specs index](../README.md) --- From 1363ae29128a5f8604202b1db5c9ec39cf3263f3 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 21:45:42 -0500 Subject: [PATCH 37/41] docs(specs): record the distillation of spec 090 in its verification Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/verification.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/specs/090-every-user-has-a-household/verification.md b/docs/specs/090-every-user-has-a-household/verification.md index 77d9de18..258042b5 100644 --- a/docs/specs/090-every-user-has-a-household/verification.md +++ b/docs/specs/090-every-user-has-a-household/verification.md @@ -62,3 +62,7 @@ To be recorded when it runs on the commit that is pushed (the previous release v ### The full `bin/ci` To be recorded with the CI run of the push. + +## Distillation + +Written during phase 7 and checked by the implementation review against the code: `tenancy-and-accounts.md` (the household, membership, personal and shared account, the invariants, the instance-level tables, the delete order, the accepted limits), `release-and-deploy.md` (the upgrade proof's two-tier households check), `data-portability.md` (the household's tables and columns left out, `households` outside the guard), `.claude/rules/multi-tenancy.md`, `docs/releasing.md`'s Upgrade note and `CLAUDE.md`. At the wrap (maintainer-approved digest): one Distillation log row in each of the three area docs with the placeholder "spec 090's pull request" (replaced by the pull request's link once it is open), the index row Frozen, by those three docs, Plan "retirement owed", and the spec's frozen line (spec 027 D4). No new ADR: ADR 0030 stays Proposed in the pull request; the maintainer accepts it, recorded in the plan-retirement pull request as for ADR 0029. Lessons: two (`docs/lessons/2026-11-13-a-table-copy-of-a-referenced-table.md`, `...-association-name-creates-the-association.md`); one feedback memory (`rereview-changed-data-loss-steps`). From 25c6e94c1aac97050e1eab42af23d04b9676cedf Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 22:26:56 -0500 Subject: [PATCH 38/41] docs(specs): record the upgrade smoke from v0.3.0 for spec 090 (#380) households: ok (backfilled) after the upgraded and the restored snapshots; 28 tables compared, OK: v0.3.0 upgraded to 0a098089957c. Co-Authored-By: Claude Sonnet 5.5 --- .../verification.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/specs/090-every-user-has-a-household/verification.md b/docs/specs/090-every-user-has-a-household/verification.md index 258042b5..61518089 100644 --- a/docs/specs/090-every-user-has-a-household/verification.md +++ b/docs/specs/090-every-user-has-a-household/verification.md @@ -57,7 +57,20 @@ All commands from the worktree. Targeted runs until the final one; one full `bin ### `bin/upgrade-smoke` -To be recorded when it runs on the commit that is pushed (the previous release v0.3.0 built from its tag with `git archive`, the code under test built beside it, `--previous-version 0.3.0`). **Expected to print** for the households: `households: ok (backfilled)` after the upgraded snapshot's comparison and again after the restored snapshot's, and nothing for the `next` snapshot unless it fails (it raises `AC-6.1: the next snapshot: …`); the comparison lines show `accounts`, `users`, `lots` and every old table `same`; the run ends `OK: v0.3.0`. +Run 2026-11-13 on the merge `0a098089957c19835529fbee6ce0d28b5c58cfb1` (origin/main at b9da192e merged; the commits after it change only `verification.md`), docker through colima: the previous release v0.3.0 built from its tag (`git archive v0.3.0` into `tmp/smoke090/v030`, image `collector:v030local`, digest `sha256:61834f698f57…`), the code under test built from the worktree (`collector:smoke090`, `--build-arg` COLLECTOR_VERSION, REVISION and SOURCE_URL), `TMPDIR` under the worktree, `/opt/homebrew/bin` appended to the PATH: + +``` +TMPDIR="$PWD/tmp/smoke090/run" bin/upgrade-smoke collector:v030local collector:smoke090 --previous-version 0.3.0 +``` + +It passed, as expected. The output that reads against the expectation: + +- After the upgraded snapshot's comparison: all 28 old tables `same` (accounts 2, users 2, lots 2003, locations 6, decks 1, deck_proxies 1, bulk_selections 1, bulk_selection_marks 3, bulk_removals 1, import_uploads 1, number_entries 1, scanner_sittings 1 and the rest, each with its digest), then **`households: ok (backfilled)`**, then the prices line. +- After the backup leg's comparison: `sessions`, `bulk_removals`, `bulk_selections` and `bulk_selection_marks` `differs` and listed as `expected: … rows_removed (backup:restore ends every session …)` (spec 030, not this feature), every other table `same`, then **`households: ok (backfilled)`** again. +- The seed of the next release printed nothing for the households (its check raises only on a problem). +- The last line: `OK: v0.3.0 (collector@sha256:61834f698f57…) upgraded to 0a098089957c; 28 tables compared, behaviour verified, 83 s`. + +So the two tiers held on the real database the v0.3.0 seed leaves (two users, two accounts, 2,003 lots, a row in each of the thirteen tables that reference `accounts`): every user has one manager membership in a household named for them, every account is in its user's household and personal, nothing old changed (`expected_changes.yml` is empty), and the third migration's table copy lost no row. This closes AC-6.1, AC-6.2 (restore leg), AC-6.6 and the m5 finding. ### The full `bin/ci` From aaf84e6dd8c352b4a94ed1c1f23abca8dc190ab6 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 22:43:41 -0500 Subject: [PATCH 39/41] docs(architecture): name the pull request of spec 090 in the distillation logs Co-Authored-By: Claude Sonnet 5.5 --- docs/architecture/data-portability.md | 2 +- docs/architecture/release-and-deploy.md | 2 +- docs/architecture/tenancy-and-accounts.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/architecture/data-portability.md b/docs/architecture/data-portability.md index 5fff8559..d3d866ac 100644 --- a/docs/architecture/data-portability.md +++ b/docs/architecture/data-portability.md @@ -297,4 +297,4 @@ Measured bars, not constants (recorded in each spec's `verification.md`): the ba | 2026-11-10 | Spec 064 (an import that changes a deck's format moves no card into Considering, the app's change of format does; no format change, no field, no version) | [#303](https://github.com/plainprogrammer/Collector/pull/303) | | 2026-11-11 | Spec 068 (export format version 4 gains the required `proxies` list: the export, `duplicate_proxy` and the proxy rows, the plan's proxy outcomes, the fingerprint, the preview's Proxies sections, the apply's order and upsert, the confirmation and notice; an older file leaves proxies alone) | [#313](https://github.com/plainprogrammer/Collector/pull/313) | | 2026-11-13 | Spec 081 (decision 4, AC-3.2): an export holds no price; a backup holds the primary and prices databases | [#405](https://github.com/plainprogrammer/Collector/pull/405) | -| 2026-11-13 | Spec 090 (the household's tables and columns are left out of the export, `households` is outside the guard, the format stays version 4; #380) | spec 090's pull request | +| 2026-11-13 | Spec 090 (the household's tables and columns are left out of the export, `households` is outside the guard, the format stays version 4; #380) | [#426](https://github.com/plainprogrammer/Collector/pull/426) | diff --git a/docs/architecture/release-and-deploy.md b/docs/architecture/release-and-deploy.md index 2240310c..cbbc99a3 100644 --- a/docs/architecture/release-and-deploy.md +++ b/docs/architecture/release-and-deploy.md @@ -333,4 +333,4 @@ The proof stops each instance it boots and compares databases (the previous rele | 2026-11-13 | Spec 087 (`bin/image-smoke` reads the third-party licences page: 200 before any user exists, the `tesseract.js` and `turbo-rails` rows, a gem's licence file under its own name, no `yard` row, and the sign-in page's link; no `Dockerfile` or workflow change; #376). Written during the build (T6.1) and checked here | [#412](https://github.com/plainprogrammer/Collector/pull/412) | | 2026-11-13 | Spec 086 (decisions 1, 2, 4, 5, 6 and 7, FR-1 to FR-4: the start check before `db:prepare` on a server start, what makes a primary "there", the message and exit status, the restore's changed advice, the smoke leg that proves it on the image; #375) | [#415](https://github.com/plainprogrammer/Collector/pull/415) | | 2026-11-13 | Spec 091 (the licence check is a `bin/ci checks` step after the importmap audit, with no workflow change; #402) | [#422](https://github.com/plainprogrammer/Collector/pull/422) | -| 2026-11-13 | Spec 090 (the upgrade proof's two-tier households check on the upgraded, restored and next snapshots; `expected_changes.yml` stays empty; the third migration's table copy and its upgrade note; #380) | spec 090's pull request | +| 2026-11-13 | Spec 090 (the upgrade proof's two-tier households check on the upgraded, restored and next snapshots; `expected_changes.yml` stays empty; the third migration's table copy and its upgrade note; #380) | [#426](https://github.com/plainprogrammer/Collector/pull/426) | diff --git a/docs/architecture/tenancy-and-accounts.md b/docs/architecture/tenancy-and-accounts.md index e6cf7dbf..dcf4ebd2 100644 --- a/docs/architecture/tenancy-and-accounts.md +++ b/docs/architecture/tenancy-and-accounts.md @@ -205,4 +205,4 @@ Deleting a user removes, in this order, their membership (deleted without the me | 2026-11-11 | Spec 070 (the replacement pages' 404 rule through `FindsDeck`, and that a lot id in Use my copy's form that is not the account's free lot of the card is "no longer free", never a distinct answer) | [#323](https://github.com/plainprogrammer/Collector/pull/323) | | 2026-11-11 | Spec 071 (the Add to deck routes' 404 rule: another account's zone, a place's id, a deck's own row, a zone of another type's deck, a removed zone and a printing that can't be added to a deck; signed out goes to sign in) | [#325](https://github.com/plainprogrammer/Collector/pull/325) | | 2026-11-13 | Spec 087 (`/licences` joins the endpoints that answer without a sign-in, also before the first user: no tenant data, the same page for everyone, the versions it shows are public by decision; #376). Written during the build (T6.1) and checked here | [#412](https://github.com/plainprogrammer/Collector/pull/412) | -| 2026-11-13 | Spec 090 (every user has a household: `households` and `memberships`, an account's `household_id` and `kind`, the rules of a household, a household made with every user by `User`, the delete order, the instance-level tables; ADR 0030 supersedes ADR 0017's one-user half; #380) | spec 090's pull request | +| 2026-11-13 | Spec 090 (every user has a household: `households` and `memberships`, an account's `household_id` and `kind`, the rules of a household, a household made with every user by `User`, the delete order, the instance-level tables; ADR 0030 supersedes ADR 0017's one-user half; #380) | [#426](https://github.com/plainprogrammer/Collector/pull/426) | From 2ad3f428742dd959c860e466de051a82c30f7d28 Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 22:43:52 -0500 Subject: [PATCH 40/41] docs(specs): record the second upgrade smoke on the merged commit (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/verification.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/specs/090-every-user-has-a-household/verification.md b/docs/specs/090-every-user-has-a-household/verification.md index 61518089..f3859a79 100644 --- a/docs/specs/090-every-user-has-a-household/verification.md +++ b/docs/specs/090-every-user-has-a-household/verification.md @@ -72,6 +72,8 @@ It passed, as expected. The output that reads against the expectation: So the two tiers held on the real database the v0.3.0 seed leaves (two users, two accounts, 2,003 lots, a row in each of the thirteen tables that reference `accounts`): every user has one manager membership in a household named for them, every account is in its user's household and personal, nothing old changed (`expected_changes.yml` is empty), and the third migration's table copy lost no row. This closes AC-6.1, AC-6.2 (restore leg), AC-6.6 and the m5 finding. +**Second run, on the merged commit.** After spec 089's merge (two prices-database migrations, `db/prices_structure.sql`) the same run on `60951eec3404c0bbefee752ecaced6cd68c579ec` (the commit that is pushed), with the current image rebuilt from it: all 28 old tables `same` after the upgrade; **`households: ok (backfilled)`** after the upgraded and again after the restored snapshot; the prices line reads `schema 20261113190002`; the backup leg's `expected:` rows are the same four (`sessions` and the three bulk tables); `OK: v0.3.0 (collector@sha256:61834f698f57…) upgraded to 60951eec3404; 28 tables compared, behaviour verified, 62 s`. On the same tree `bin/rails db:prepare` and 325 targeted examples passed (`spec/script/upgrade`, the backup guards and upgrade, `spec/models/prices_record_spec.rb`, `spec/prices_core_spec.rb`, `spec/migrations`, the household model and request specs, the export guard): 0 failures, seed 6927. + ### The full `bin/ci` To be recorded with the CI run of the push. From 64c75d1cc46231f507eb3cec24e30960e9d5fe0e Mon Sep 17 00:00:00 2001 From: James Thompson Date: Fri, 13 Nov 2026 23:18:48 -0500 Subject: [PATCH 41/41] docs(specs): record that the third merge was docs only, so no smoke re-run (spec 090) Co-Authored-By: Claude Sonnet 5.5 --- docs/specs/090-every-user-has-a-household/verification.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/specs/090-every-user-has-a-household/verification.md b/docs/specs/090-every-user-has-a-household/verification.md index f3859a79..ed0b898e 100644 --- a/docs/specs/090-every-user-has-a-household/verification.md +++ b/docs/specs/090-every-user-has-a-household/verification.md @@ -74,6 +74,8 @@ So the two tiers held on the real database the v0.3.0 seed leaves (two users, tw **Second run, on the merged commit.** After spec 089's merge (two prices-database migrations, `db/prices_structure.sql`) the same run on `60951eec3404c0bbefee752ecaced6cd68c579ec` (the commit that is pushed), with the current image rebuilt from it: all 28 old tables `same` after the upgrade; **`households: ok (backfilled)`** after the upgraded and again after the restored snapshot; the prices line reads `schema 20261113190002`; the backup leg's `expected:` rows are the same four (`sessions` and the three bulk tables); `OK: v0.3.0 (collector@sha256:61834f698f57…) upgraded to 60951eec3404; 28 tables compared, behaviour verified, 62 s`. On the same tree `bin/rails db:prepare` and 325 targeted examples passed (`spec/script/upgrade`, the backup guards and upgrade, `spec/models/prices_record_spec.rb`, `spec/prices_core_spec.rb`, `spec/migrations`, the household model and request specs, the export guard): 0 failures, seed 6927. +**Third merge (docs only).** `origin/main` at ad9c76cd (the retirement of spec 089's plan, PR #427) touched only the specs index and two area docs' distillation logs; the merge `386b9942` conflicted only on the index (both rows kept). No code, migration, schema or `script/upgrade/` file changed, so the smoke was not run again; the second run above, on `60951eec`, is the proof for the code that ships. + ### The full `bin/ci` To be recorded with the CI run of the push.