diff --git a/CHANGELOG.md b/CHANGELOG.md index 1558b15e1..8d6ced9f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -73,6 +73,18 @@ All notable changes to this project are documented in this file. The format is b A single-tenant install with nothing bound keeps running the last two. See `docs/framework/multi-tenancy.md`. +- Alembic autogenerate now emits newly modeled foreign keys on existing tables + while preserving migration-only reflected foreign keys (#812). A foreign key + with the same columns and target but changed options (`ondelete`, `deferrable`) + is emitted as a drop plus a create; on SQLite wrap those ops in `op.batch_alter_table(...)` by hand. +- **Behaviour change:** StrEnum migration rendering preserves configured labels and enum storage options, + keeping runtime reads/writes and generated constraints consistent; a plain + `StrEnum` field now renders SQLAlchemy's member-name labels, matching what the + model binds (#813). +- Metadata discovery propagates missing dependencies inside models modules instead + of silently producing an incomplete schema (#814). +- **Behaviour change:** doctor data compatibility checks use framework DB sessions with soft-delete and + strict tenant filters; installation-wide checks must opt into `all_tenants()` (#815). - **Behaviour change:** `LocaleMiddleware` never picks a language the browser sends with `q=0` ("not acceptable"), including through a regional tag such as `es-MX;q=0` and when `q` follows another parameter or is spelled `Q`; diff --git a/docs/database/migrations.md b/docs/database/migrations.md index 2d58bf673..8eb81d8e6 100644 --- a/docs/database/migrations.md +++ b/docs/database/migrations.md @@ -2,6 +2,23 @@ All migrations live in `host/migrations/versions/` — **not** in module packages. Alembic runs from the repo root via `host/alembic.ini` and shares the app's `.env` / `SM_DATABASE_URL`. +Autogenerate's `make_include_object()` filter preserves foreign keys declared +only in migrations by ignoring unmatched **reflected** constraints. Newly modeled +foreign keys on existing tables still generate `CreateForeignKeyOp`; set +`ignore_unmodeled_fks=False` to allow dropping database-only constraints (#812). +A foreign key with the same columns and target but different options (`ondelete`, +`deferrable`) is emitted as a drop plus a create; one with other columns or another +target is a different constraint and stays preserved. SQLite cannot `ALTER` constraints: wrap such ops in +`op.batch_alter_table(...)` by hand before applying the revision there. + +For `sa.Enum` columns backed by Python `StrEnum`, generated revisions freeze the +column's actual configured labels and preserve `name`, `native_enum`, +`create_constraint`, `length` and configured `values_callable` behavior without +importing application enums/callables. Plain enum fields store member names +(e.g. `READY`); to store values (e.g. `ready`), configure the **runtime** type with +`values_callable=lambda e: [m.value for m in e]`. Custom callable labels are frozen +in the revision. Existing revisions are not rewritten (#813). + ## Why centralized? - **Dependency ordering is global.** If `invoices` depends on `orders_order.id`, their migrations must order correctly. One shared Alembic history, with `down_revision` links between modules, enforces this. @@ -31,6 +48,14 @@ first runs defer checks until the wizard applies migrations. No automatic credential deletion happens at startup. See the [scaffolded host upgrade guide](../guide/upgrading.md) for the operator workflow and token sign-out risk. +Doctor uses the DB session factory and listeners used at boot: soft-deleted +rows are hidden and tenant queries are scoped. Because doctor has no host settings, +it always enforces strict tenant isolation, which is stricter than a single-tenant +boot: a check that passes doctor also passes boot. A check must use `tenant_context(id)` +for one tenant or explicitly use `all_tenants()` / `execution_options(all_tenants=True)` +for an installation-wide check. Add `execution_options(include_deleted=True)` +only when the check also needs trashed rows. These opt-ins work at boot too (#815). + ## The real graph The history is **not** a single linear chain. It has several roots and several heads, and `make migrate` runs `upgrade heads` for that reason. On `main` today: @@ -174,7 +199,7 @@ context.configure( - `target_metadata` — union of every module's `MetaData`. - `include_object` — accepts only tables present in `target_metadata`, so autogenerate never diffs system tables (`alembic_version`) or any host-owned tables outside the module system. -- `render_item` — collapses SQLModel's `AutoString` to `sa.String` and renders `StrEnum` columns with `values_callable` so generated migrations are importable. +- `render_item` — collapses SQLModel's `AutoString` to `sa.String` and renders `StrEnum` columns with their configured labels and storage options (no application imports in revisions, #813). - `process_revision_directives` — re-emits expression-based (functional) indexes that autogenerate silently drops under SQLite. ## Boot-time migration check diff --git a/docs/database/per-module-base.md b/docs/database/per-module-base.md index 4242f7108..6fd7c052f 100644 --- a/docs/database/per-module-base.md +++ b/docs/database/per-module-base.md @@ -34,7 +34,7 @@ target_metadata = build_module_metadata() This function iterates every discovered module, imports its `models` submodule (if one exists), and unions all the per-module `MetaData`s into one. Autogenerate then diffs the DB against that union. -If a module has no `models.py`, it contributes nothing — fine. If a module has a `models.py` that doesn't import, the union fails — fix the import. +If a module has no `models.py` (or its parent package is absent), it contributes nothing — fine. If an existing `models.py` fails to import a dependency, the union fails with that import error — fix the import. Only `ModuleNotFoundError` naming the models module itself or one of its parents is skipped; missing sibling modules and dependencies are errors (#814). ## `make_include_object()` @@ -42,7 +42,7 @@ Alembic's `include_object` callback filters which tables `autogenerate` consider - **Include** only tables present in the combined module `MetaData` (the allowlist is `{t.name for t in metadata.tables.values()}`). - **Exclude** everything else — tables owned by the Alembic runtime (`alembic_version`) and any host-owned tables that aren't part of an installed module. -- Optionally skip unmodeled foreign-key constraints (`ignore_unmodeled_fks=True` by default) so migration-level cross-module FKs aren't dropped on every autogen run. +- Skip unmatched reflected foreign-key constraints (`ignore_unmodeled_fks=True` by default) so migration-level cross-module FKs aren't dropped on every autogen run. Newly modeled FKs on existing tables are still included (#812). If you write a one-off host-level table that autogenerate shouldn't track, keep it out of the module metadata — `make_include_object` already excludes anything not in the allowlist. diff --git a/docs/framework/lifecycle.md b/docs/framework/lifecycle.md index 8b1aff777..438109491 100644 --- a/docs/framework/lifecycle.md +++ b/docs/framework/lifecycle.md @@ -368,8 +368,9 @@ Use stable semantic ids and inspect actual schema/data state so hosts with independently ported revisions pass. Checks run before module DB-settings hydration and `on_startup`; errors refuse boot outside development. A never- migrated first run defers checks until migrations have been applied. Doctor -reports the same issues as `SM033`. Checks must never clear credentials or apply -migrations; see [Migrations](../database/migrations.md). +reports the same issues as `SM033`, through the same filtered sessions +(tenant and soft-delete rules in [Migrations](../database/migrations.md)). +Checks must never clear credentials or apply migrations. ## `on_startup()` / `on_shutdown()` diff --git a/docs/module-authoring.md b/docs/module-authoring.md index 2147c79db..6676d85b8 100644 --- a/docs/module-authoring.md +++ b/docs/module-authoring.md @@ -376,8 +376,9 @@ generated file against this checklist before treating it as portable `sa.Enum(Status)` the stored label is the member *name* (`server_default="PUBLISH"`), not its value (`"publish"`). SQLite accepts either; PostgreSQL rejects the value with `invalid input value for enum`. - (A `StrEnum` rendered with `values_callable`, which is what `render_item` - emits, stores the *value* — match the default to whichever the column uses.) + (A `StrEnum` column whose runtime type sets `values_callable` stores the + *value*; `render_item` freezes whichever labels the runtime type uses, so + match the default to the column's labels.) Prove it on PostgreSQL before merging. Create a disposable database once (`createdb sm_migrations_check` on the shared dev Postgres), then: diff --git a/framework/db/simple_module_db/_migration_render.py b/framework/db/simple_module_db/_migration_render.py index 448d3c6ff..dcb9830bd 100644 --- a/framework/db/simple_module_db/_migration_render.py +++ b/framework/db/simple_module_db/_migration_render.py @@ -18,10 +18,10 @@ def render_item(type_, obj, autogen_context): to ``sa.DateTime(timezone=True)`` so migrations don't need to ``import sqlmodel`` (GH #440). Any other ``sqlmodel`` type still gets the import added, so the migration stays importable. - * Renders Python ``StrEnum`` columns with ``values_callable`` so the - Postgres enum labels match the lowercase ``StrEnum`` values rather than - SQLAlchemy's default of using uppercase attribute names. This means raw - SQL like ``WHERE status = 'ready'`` actually works against the live DB. + * Renders Python ``StrEnum`` columns using their configured labels and + storage options, without importing application enums into revisions. + Value-based labels require ``values_callable`` on the runtime type too; + plain enum columns keep SQLAlchemy's member-name labels. * Renders ``true()``/``false()`` server defaults as ``sa.true()``/``sa.false()`` instead of the dialect-specific ``sa.text('1')``/``sa.text('0')`` (GH #342). * Renders ``LTree``/``TSVector`` through ``simple_module_db.types`` (with @@ -63,10 +63,19 @@ def render_item(type_, obj, autogen_context): if isinstance(obj, sa.Enum): python_type = getattr(obj, "enum_class", None) if isinstance(python_type, type) and issubclass(python_type, StrEnum): - name = getattr(obj, "name", None) or python_type.__name__.lower() - members = ", ".join(repr(m.value) for m in python_type) - return ( - f"sa.Enum({members}, name={name!r}, values_callable=lambda e: [m.value for m in e])" - ) + members = ", ".join(repr(label) for label in obj.enums) + options = [f"name={obj.name!r}"] + if not obj.native_enum: + options.append("native_enum=False") + if obj.create_constraint: + options.append("create_constraint=True") + if obj.length != max((len(label) for label in obj.enums), default=0): + options.append(f"length={obj.length!r}") + if obj.values_callable is not None: + # Freeze custom labels; revisions must not import application callables. + values = [member.value for member in python_type] + callable_body = "[m.value for m in e]" if obj.enums == values else repr(obj.enums) + options.append(f"values_callable=lambda e: {callable_body}") + return f"sa.Enum({members}, {', '.join(options)})" return False # let alembic use its default rendering diff --git a/framework/db/simple_module_db/data_compatibility.py b/framework/db/simple_module_db/data_compatibility.py index dcd293fd4..170837ac0 100644 --- a/framework/db/simple_module_db/data_compatibility.py +++ b/framework/db/simple_module_db/data_compatibility.py @@ -7,7 +7,9 @@ from simple_module_core.data_compatibility import collect_data_compatibility from simple_module_core.diagnostics import Diagnostic, DiagnosticLevel from simple_module_core.module import ModuleBase -from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from simple_module_db.listeners import register_listeners +from simple_module_db.session import init_db DATA_COMPATIBILITY_CODE = "SM033" @@ -15,11 +17,19 @@ async def database_compatibility_diagnostics( modules: Sequence[ModuleBase], database_url: str ) -> list[Diagnostic]: + state = None try: if not _database_exists(database_url): return [] # Nothing migrated yet; opening it would create an empty SQLite file. - engine = create_async_engine(database_url) + # Read-only diagnostics: never switch a legacy SQLite file to WAL. + state = init_db(database_url, sqlite_wal=False) + # Doctor has no host settings: checks must explicitly opt into all tenants. + state.tenant_strict = True + register_listeners(state) + engine = state.engine except Exception: + if state is not None: + await state.engine.dispose() # A malformed or non-async URL must be a finding, not a traceback; the URL can # carry credentials, so it is not echoed. return [_problem(DiagnosticLevel.WARNING, "Could not open the database URL.")] @@ -37,7 +47,7 @@ async def database_compatibility_diagnostics( if not migrated: return [] try: - async with async_sessionmaker(engine)() as db: + async with state.session_factory() as db: issues = await collect_data_compatibility(modules, db) except Exception as exc: # A check that crashes is a bug to fix, not a pass: doctor must fail. diff --git a/framework/db/simple_module_db/migrations.py b/framework/db/simple_module_db/migrations.py index 51c3fdfca..8a97d1d6e 100644 --- a/framework/db/simple_module_db/migrations.py +++ b/framework/db/simple_module_db/migrations.py @@ -17,13 +17,14 @@ import importlib import logging from collections.abc import Callable, Sequence -from typing import Literal +from typing import Any, Literal from alembic.operations.ops import CreateIndexOp, CreateTableOp, DropIndexOp, DropTableOp from simple_module_core import ModuleBase from simple_module_core.diagnostics._expression_index import index_is_expression_based from simple_module_core.discovery import discover_modules, get_module_package_name from sqlalchemy import Index, MetaData +from sqlalchemy.exc import NoReferenceError from sqlalchemy.schema import SchemaItem from simple_module_db._migration_render import render_item # noqa: F401 (re-exported) @@ -70,9 +71,14 @@ def build_module_metadata(modules: Sequence[ModuleBase] | None = None) -> MetaDa for mod in modules: pkg = get_module_package_name(mod) + models_module = f"{pkg}.models" try: - importlib.import_module(f"{pkg}.models") - except ModuleNotFoundError: + importlib.import_module(models_module) + except ModuleNotFoundError as exc: + if exc.name is None or not ( + exc.name == models_module or models_module.startswith(f"{exc.name}.") + ): + raise logger.debug("No models submodule for module '%s' (pkg=%s)", mod.meta.name, pkg) combined = MetaData() @@ -82,6 +88,32 @@ def build_module_metadata(modules: Sequence[ModuleBase] | None = None) -> MetaDa return combined +def _fk_signature(fk: Any) -> tuple[str, frozenset[tuple[str, str]]] | None: + """Referred table and (local column, referred column) pairs of a foreign key.""" + try: + elements = list(fk.elements) + referred = elements[0].column.table.fullname + except (AttributeError, IndexError, LookupError, NoReferenceError): + return None # unresolvable target: never treat it as a match + return referred, frozenset((e.parent.name, e.column.name) for e in elements) + + +def _fk_replaced_in_model(metadata: MetaData, reflected_fk: SchemaItem) -> bool: + """Whether the model declares the same foreign key with different options. + + A changed option (``ondelete``, ``deferrable``...) makes Alembic report the old + constraint as removed and the new one as added. Hiding only the removal would + leave the database with two foreign keys on the same columns. A key that differs + in its columns or its target is a different constraint and stays preserved. + """ + table = getattr(reflected_fk, "table", None) + modeled = metadata.tables.get(table.fullname) if table is not None else None + signature = _fk_signature(reflected_fk) + if modeled is None or signature is None: + return False + return any(_fk_signature(fk) == signature for fk in modeled.foreign_key_constraints) + + def make_include_object( metadata: MetaData, *, @@ -96,8 +128,8 @@ def make_include_object( table added by the host developer outside the module system). :param ignore_unmodeled_fks: When ``True`` (default), foreign-key - constraints that exist in the live DB but are absent from the SQLModel - metadata are skipped instead of being emitted as ``op.drop_constraint`` + constraints that exist in the live DB (reflected) but are absent from the + SQLModel metadata are skipped instead of being emitted as ``op.drop_constraint`` in the autogenerated migration. The framework's recommended pattern declares cross-module FKs at the migration level only (no Python-level relationship between modules), so those constraints look "unmodeled" to @@ -115,7 +147,13 @@ def include_object( ) -> bool: if type_ == "table": return name in allowlist - if ignore_unmodeled_fks and type_ == "foreign_key_constraint" and compare_to is None: + if ( + ignore_unmodeled_fks + and type_ == "foreign_key_constraint" + and reflected + and compare_to is None + and not _fk_replaced_in_model(metadata, object) + ): return False parent_table = getattr(object, "table", None) if parent_table is not None: diff --git a/framework/db/tests/test_db_doctor_session_filters.py b/framework/db/tests/test_db_doctor_session_filters.py new file mode 100644 index 000000000..bcc50bd44 --- /dev/null +++ b/framework/db/tests/test_db_doctor_session_filters.py @@ -0,0 +1,91 @@ +"""Doctor compatibility hooks use boot's scoped, filtered DB sessions (#815).""" + +import pytest +import sqlalchemy as sa +from simple_module_core.data_compatibility import collect_data_compatibility +from simple_module_core.module import ModuleBase, ModuleMeta +from simple_module_db.base import create_module_base +from simple_module_db.data_compatibility import database_compatibility_diagnostics +from simple_module_db.listeners import register_listeners +from simple_module_db.mixins import MultiTenantMixin, SoftDeleteMixin +from simple_module_db.session import init_db +from simple_module_db.tenancy import all_tenants, tenant_context +from sqlmodel import Field + +Base = create_module_base("db_doctor_session_review") + + +class DoctorRow(Base, MultiTenantMixin, SoftDeleteMixin, table=True): # ty: ignore[unsupported-base] + __tablename__ = "db_doctor_session_review_row" + id: int = Field(primary_key=True) + + +@pytest.fixture +async def doctor_database(tmp_path): + url = f"sqlite+aiosqlite:///{tmp_path / 'doctor.db'}" + state = init_db(url) + state.tenant_strict = True + register_listeners(state) + try: + async with state.engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + await conn.execute(sa.text("CREATE TABLE alembic_version (version_num VARCHAR(32))")) + await conn.execute(sa.text("INSERT INTO alembic_version VALUES ('review')")) + await conn.execute( + sa.insert(DoctorRow), + [ + {"id": 1, "tenant_id": "a", "is_deleted": False}, + {"id": 2, "tenant_id": "a", "is_deleted": True}, + {"id": 3, "tenant_id": "b", "is_deleted": False}, + ], + ) + yield state, url + finally: + await state.engine.dispose() + + +@pytest.mark.parametrize( + "scope,include_deleted,expected", + [("a", False, [1]), ("a", True, [1, 2]), ("all", False, [1, 3]), ("all", True, [1, 2, 3])], +) +async def test_db_doctor_and_boot_apply_same_filters( + doctor_database, scope, include_deleted, expected +): + state, url = doctor_database + seen = [] + + class Check(ModuleBase): + meta = ModuleMeta(name="DoctorSessionReview") + + async def check_data_compatibility(self, db): + context = all_tenants() if scope == "all" else tenant_context(scope) + with context: + result = await db.execute( + sa.select(DoctorRow.id) + .order_by(DoctorRow.id) + .execution_options(include_deleted=include_deleted) + ) + seen.append(list(result.scalars())) + return [] + + module = Check() + async with state.session_factory() as db: + await collect_data_compatibility([module], db) + assert await database_compatibility_diagnostics([module], url) == [] + assert seen == [expected, expected] + + +async def test_db_doctor_fails_closed_without_tenant_context(doctor_database): + _, url = doctor_database + + class Check(ModuleBase): + meta = ModuleMeta(name="DoctorTenantReview") + + async def check_data_compatibility(self, db): + await db.execute(sa.select(DoctorRow)) + return [] + + findings = await database_compatibility_diagnostics([Check()], url) + assert len(findings) == 1 + assert findings[0].level == "error" + assert "MissingTenantError" in findings[0].message diff --git a/framework/db/tests/test_db_migration_review_regressions.py b/framework/db/tests/test_db_migration_review_regressions.py new file mode 100644 index 000000000..22fbf367e --- /dev/null +++ b/framework/db/tests/test_db_migration_review_regressions.py @@ -0,0 +1,186 @@ +"""Autogenerate must preserve model constraints and fail on broken imports.""" + +import sys +from enum import StrEnum +from types import SimpleNamespace + +import pytest +import sqlalchemy as sa +from alembic.autogenerate import produce_migrations +from alembic.operations.ops import CreateForeignKeyOp +from alembic.runtime.migration import MigrationContext +from simple_module_db import migrations +from simple_module_db.migration_portability import iter_ops_recursive +from simple_module_db.migrations import make_include_object, render_item +from sqlalchemy.exc import IntegrityError + + +class Status(StrEnum): + READY = "ready" + DONE = "done" + + +def test_db_autogenerate_adds_fk_to_existing_column(): + old = sa.MetaData() + sa.Table("parent", old, sa.Column("id", sa.Integer, primary_key=True)) + sa.Table( + "child", + old, + sa.Column("id", sa.Integer, primary_key=True), + sa.Column("parent_id", sa.Integer), + ) + target = sa.MetaData() + sa.Table("parent", target, sa.Column("id", sa.Integer, primary_key=True)) + sa.Table( + "child", + target, + sa.Column("id", sa.Integer, primary_key=True), + sa.Column("parent_id", sa.Integer, sa.ForeignKey("parent.id")), + ) + engine = sa.create_engine("sqlite://") + try: + old.create_all(engine) + with engine.connect() as conn: + ctx = MigrationContext.configure( + conn, opts={"include_object": make_include_object(target)} + ) + script = produce_migrations(ctx, target) + assert any( + isinstance(op, CreateForeignKeyOp) for op in iter_ops_recursive(script.upgrade_ops) + ) + finally: + engine.dispose() + + +@pytest.mark.parametrize("labels", ["names", "values", "custom"]) +@pytest.mark.parametrize( + "native,constraint,length", [(True, False, "default"), (False, True, 20), (False, True, None)] +) +@pytest.mark.parametrize("name", ["status", None]) +def test_db_enum_rendering_preserves_storage_and_constraints( + labels, native, constraint, length, name +): + options = {"name": name, "native_enum": native, "create_constraint": constraint} + if length != "default": + options["length"] = length + if labels == "values": + options["values_callable"] = lambda e: [m.value for m in e] + elif labels == "custom": + options["values_callable"] = lambda e: [m.value.upper() + "_label" for m in e] + runtime = sa.Enum(Status, **options) + rendered = eval(render_item("type", runtime, None), {"sa": sa}) + expected = { + "names": ["READY", "DONE"], + "values": ["ready", "done"], + "custom": ["READY_label", "DONE_label"], + }[labels] + assert rendered.enums == expected + assert rendered.name == name + assert rendered.native_enum is native + assert rendered.create_constraint is constraint + assert rendered.length == (max(map(len, expected)) if length == "default" else length) + assert (rendered.values_callable is not None) == (runtime.values_callable is not None) + if runtime.values_callable: + assert rendered.values_callable(Status) == expected + engine = sa.create_engine("sqlite://") + try: + metadata = sa.MetaData() + table = sa.Table("row", metadata, sa.Column("status", rendered)) + metadata.create_all(engine) + with engine.begin() as conn: + conn.execute(table.insert().values(status=expected[0])) + assert ( + conn.execute(sa.select(sa.type_coerce(table.c.status, runtime))).scalar_one() + == Status.READY + ) + if constraint: + with pytest.raises(IntegrityError): + conn.execute(sa.text("INSERT INTO row (status) VALUES ('INVALID')")) + finally: + engine.dispose() + + +def test_db_value_enum_rendering_keeps_existing_output(): + declared = sa.Enum(Status, values_callable=lambda e: [m.value for m in e]) + assert render_item("type", declared, None) == ( + "sa.Enum('ready', 'done', name='status', values_callable=lambda e: [m.value for m in e])" + ) + + +@pytest.mark.parametrize( + "missing", + [ + "db_review_models.models", + "db_review_models", + "uninstalled_dep", + "db_review_models.helper", + None, + ], +) +def test_db_metadata_only_ignores_missing_models_module(tmp_path, monkeypatch, missing): + package = tmp_path / "db_review_models" + package.mkdir() + (package / "__init__.py").write_text("") + (package / "models.py").write_text(f'raise ModuleNotFoundError("missing", name={missing!r})') + monkeypatch.delitem(sys.modules, "db_review_models", raising=False) + monkeypatch.delitem(sys.modules, "db_review_models.models", raising=False) + monkeypatch.syspath_prepend(str(tmp_path)) + monkeypatch.setattr(migrations, "get_module_package_name", lambda m: "db_review_models") + module = SimpleNamespace(meta=SimpleNamespace(name="ReviewModels")) + if missing in ("db_review_models", "db_review_models.models"): + migrations.build_module_metadata([module]) + else: + with pytest.raises(ModuleNotFoundError) as exc: + migrations.build_module_metadata([module]) + assert exc.value.name == missing + + +def _autogenerate_ops(live, target): + """Ops autogenerate emits to bring a database shaped like ``live`` to ``target``.""" + engine = sa.create_engine("sqlite://") + try: + live.create_all(engine) + with engine.connect() as conn: + ctx = MigrationContext.configure( + conn, opts={"include_object": make_include_object(target)} + ) + return list(iter_ops_recursive(produce_migrations(ctx, target).upgrade_ops)) + finally: + engine.dispose() + + +def _parent_child(target="parent.id", **fk_options): + metadata = sa.MetaData() + sa.Table("parent", metadata, sa.Column("id", sa.Integer, primary_key=True)) + sa.Table("other", metadata, sa.Column("id", sa.Integer, primary_key=True)) + sa.Table( + "child", + metadata, + sa.Column("id", sa.Integer, primary_key=True), + sa.Column("parent_id", sa.Integer, sa.ForeignKey(target, **fk_options)), + ) + return metadata + + +def test_db_autogenerate_replaces_fk_with_changed_options(): + ops = _autogenerate_ops(_parent_child(), _parent_child(ondelete="CASCADE")) + assert any(isinstance(op, CreateForeignKeyOp) for op in ops) + assert any(type(op).__name__ == "DropConstraintOp" for op in ops) + + +def test_db_autogenerate_keeps_migration_only_fk_to_another_table(): + # The database carries a migration-only FK to ``other``; the model points at ``parent``. + ops = _autogenerate_ops(_parent_child("other.id"), _parent_child("parent.id")) + assert any(isinstance(op, CreateForeignKeyOp) for op in ops) + assert not any(type(op).__name__ == "DropConstraintOp" for op in ops) + + +def test_db_fk_signature_pairs_local_and_referred_columns(): + metadata = sa.MetaData() + sa.Table("p", metadata, sa.Column("x", sa.Integer), sa.Column("y", sa.Integer)) + child = sa.Table("c", metadata, sa.Column("a", sa.Integer), sa.Column("b", sa.Integer)) + straight = sa.ForeignKeyConstraint(["a", "b"], ["p.x", "p.y"]) + swapped = sa.ForeignKeyConstraint(["a", "b"], ["p.y", "p.x"]) + child.append_constraint(straight) + child.append_constraint(swapped) + assert migrations._fk_signature(straight) != migrations._fk_signature(swapped) diff --git a/skills/simple-module-migrations/SKILL.md b/skills/simple-module-migrations/SKILL.md index 60169fce4..dc5ea7dae 100644 --- a/skills/simple-module-migrations/SKILL.md +++ b/skills/simple-module-migrations/SKILL.md @@ -43,9 +43,9 @@ process_revision_directives = make_process_revision_directives(target_metadata) # context.configure(..., render_item=render_item) ``` -`build_module_metadata()` walks the `simple_module` entry-point group, imports each module's `models` submodule, and merges every module's per-module `MetaData` into one unified `MetaData`. `make_include_object(metadata)` allowlists only tables owned by installed modules — any host-owned tables (and `alembic_version`) outside the module system are preserved untouched. So one autogenerate call covers every installed module in a single pass; editable installs and wheels behave identically. +`build_module_metadata()` walks the `simple_module` entry-point group, imports each module's `models` submodule (a missing `models` module is skipped; a missing dependency inside it raises), and merges every module's per-module `MetaData` into one unified `MetaData`. `make_include_object(metadata)` allowlists only tables owned by installed modules — any host-owned tables (and `alembic_version`) outside the module system are preserved untouched. So one autogenerate call covers every installed module in a single pass; editable installs and wheels behave identically. -Two extras matter for correctness: `render_item` collapses SQLModel's `AutoString` to `sa.String` and renders `StrEnum` columns so the generated file imports cleanly, and `make_process_revision_directives` re-emits expression-based (functional) indexes — e.g. `lower(email)` — that autogenerate silently drops under SQLite. +Two extras matter for correctness: `render_item` collapses SQLModel's `AutoString` to `sa.String` and renders `StrEnum` columns with the runtime type's labels so the generated file imports cleanly, and `make_process_revision_directives` re-emits expression-based (functional) indexes — e.g. `lower(email)` — that autogenerate silently drops under SQLite. All module tables live in the **host's single shared schema** on both Postgres and SQLite (each module still owns its own `MetaData` purely so autogenerate can attribute a table to its module). There is no schema-per-module; `__tablename__` is prefixed with the module name (`orders_order`) to avoid collisions, and a cross-module FK is an ordinary same-schema reference.