Skip to content

Add cookbook recipe for adding a non-nullable column - #1838

Open
ArockiaRajamanickam wants to merge 1 commit into
sqlalchemy:mainfrom
ArockiaRajamanickam:cookbook-non-nullable-column
Open

Add cookbook recipe for adding a non-nullable column#1838
ArockiaRajamanickam wants to merge 1 commit into
sqlalchemy:mainfrom
ArockiaRajamanickam:cookbook-non-nullable-column

Conversation

@ArockiaRajamanickam

Copy link
Copy Markdown

Adds the cookbook recipe requested in the second bullet of #681: how to add a non-nullable column to a table that already has rows, without a server default.

The recipe follows what was agreed in the thread rather than inventing an approach:

  • The main recipe is CaselIT's option 2 (add as nullable, backfill with an UPDATE, then alter_column(nullable=False)), which was the one both of you settled on.
  • A .. warning:: covers zzzeek's two points: the UPDATE/ALTER can lock the table and take a live site down on a large dataset, and adding a non-nullable column is hard to make work against an application that keeps running during the migration.
  • The server_default alternative is mentioned second, with the note that dropping a server default behaves differently across backends.
  • Per CaselIT's comment, the create-new-table-and-copy approach is deliberately not suggested; the section links to :ref:batch_migrations`` instead.

The backfill uses the lightweight sa.table() / sa.column() constructs rather than the application model, so the migration keeps working as the model changes.

Scope: this PR only covers the cookbook half of #681. The env.py logging half was declined by zzzeek in the same thread, so it is left alone, and I have not used Fixes: since the issue bundles both requests.

Verification:

  • Ran both recipes through a real MigrationContext against SQLite 3.53 and PostgreSQL 16: confirmed the naive one-step add_column(nullable=False) fails on a populated table, that the three-step recipe leaves an enforced NOT NULL column, that a subsequent NULL insert is rejected, that the downgrade drops the column, and that the server_default variant leaves no residual default.
  • sphinx -b html builds with no new warnings; the :ref: and :meth: targets and the two SQLAlchemy intersphinx links all resolve.
  • Code blocks are black-clean at the project's 79 character line length.

One note on the SQLite cross-reference: the seealso says the nullable=False step may need batch mode on backends with limited ALTER support, rather than stating flatly that SQLite cannot do it. While testing, SQLite 3.53 accepted ALTER TABLE ... ALTER COLUMN ... SET NOT NULL directly, so the absolute phrasing would not have been accurate on current versions. Happy to reword if you would rather it match the framing in batch.rst.

I used an AI assistant while drafting this; the wording and the testing above are mine and I have checked the content against the thread and the docs build.

Documents the add-nullable, backfill, then set NOT NULL pattern
agreed on in issue sqlalchemy#681, along with the server_default alternative
and a warning about table locking on a running application.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant