Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/fork-ocr-model-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Weekly check for a newer Claude Opus on Bedrock; opens/updates one issue here
# when OCR_BEDROCK_MODEL is behind. Notify-only (GITHUB_TOKEN cannot write
# Actions variables). Logic lives in gburd/ci-workflows.

name: OCR model self-check

on:
schedule:
- cron: '0 12 * * 1' # Mondays 12:00 UTC
workflow_dispatch:

jobs:
check:
uses: gburd/ci-workflows/.github/workflows/ocr-model-check.yml@v1
with:
aws_role_arn: ${{ vars.AWS_ROLE_ARN }}
aws_region: ${{ vars.AWS_REGION }}
bedrock_model: ${{ vars.OCR_BEDROCK_MODEL }}
43 changes: 43 additions & 0 deletions .github/workflows/fork-ocr-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# AI PR review (Claude Opus on Bedrock). Logic and review rules live in
# gburd/ci-workflows so this fork's master carries a stub instead of ~1000
# lines that have to be rebased onto upstream forever.
#
# Needs repo variables AWS_ROLE_ARN, AWS_REGION, OCR_BEDROCK_MODEL. Auth is
# GitHub OIDC; no static AWS keys. See the ci-workflows README.

name: OCR AI Review

on:
pull_request:
# Note: no draft filter — drafts are reviewed too.
types: [opened, synchronize, reopened, ready_for_review]
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to review'
required: true
type: number

# One review per PR; cancel superseded runs to save Bedrock spend.
concurrency:
group: ocr-review-${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }}
cancel-in-progress: true

jobs:
review:
# Comment events only when the comment is on a PR and starts with the
# trigger keyword; PR events and manual dispatch always.
if: |
github.event_name == 'pull_request' ||
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' && github.event.issue.pull_request &&
(startsWith(github.event.comment.body, '/open-code-review') ||
startsWith(github.event.comment.body, '@open-code-review') ||
startsWith(github.event.comment.body, '/pg-history')))
uses: gburd/ci-workflows/.github/workflows/ocr-review.yml@v1
with:
aws_role_arn: ${{ vars.AWS_ROLE_ARN }}
aws_region: ${{ vars.AWS_REGION }}
bedrock_model: ${{ vars.OCR_BEDROCK_MODEL }}
48 changes: 48 additions & 0 deletions .github/workflows/fork-sync-upstream.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Keep this fork's master rebased on postgres/postgres master, carrying only
# the fork-owned CI stubs (.github/workflows/fork-*.yml). The workflows they
# call, and the OCR review rules, live in gburd/ci-workflows.
#
# Requires the SYNC_TOKEN secret: a PAT with repo+workflow scope. The default
# GITHUB_TOKEN is not allowed to push commits that touch .github/workflows/,
# and every commit we carry does.

name: Sync from upstream

on:
schedule:
- cron: '17 */3 * * *'
workflow_dispatch:

concurrency:
group: fork-sync-upstream

permissions:
contents: read

jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
token: ${{ secrets.SYNC_TOKEN }}

- name: Rebase master onto upstream
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git remote add upstream https://github.com/postgres/postgres.git
git fetch --no-tags upstream master

# Master carries fork CI only. Anything else means work landed here
# that belongs on a branch; force-rebasing it on a timer would be wrong.
if git diff --name-only upstream/master...HEAD |
grep -v '^\.github/workflows/fork-'; then
echo "::error::master carries non-CI changes (listed above); move them to a branch"
exit 1
fi

git rebase upstream/master
git push --force-with-lease
1 change: 1 addition & 0 deletions doc/src/sgml/filelist.sgml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
<!ENTITY runtime SYSTEM "runtime.sgml">
<!ENTITY config SYSTEM "config.sgml">
<!ENTITY user-manag SYSTEM "user-manag.sgml">
<!ENTITY undo SYSTEM "undo.sgml">
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
Expand Down
1 change: 1 addition & 0 deletions doc/src/sgml/postgres.sgml
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ break is not needed in a wider output rendering.
&high-availability;
&monitoring;
&wal;
&undo;
&logical-replication;
&jit;
&regress;
Expand Down
137 changes: 137 additions & 0 deletions doc/src/sgml/undo.sgml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
<!-- doc/src/sgml/undo.sgml -->

<chapter id="undo" xreflabel="UNDO Logging">
<title>UNDO Logging</title>

<indexterm zone="undo">
<primary>UNDO logging</primary>
</indexterm>

<para>
<productname>PostgreSQL</productname> provides UNDO logging infrastructure
that records the inverse of a change, so that the change can be reversed if
the transaction that made it rolls back or is still in progress when the
server crashes. UNDO records are written into the WAL stream, so they are
durable, replicated and archived like every other WAL record.
</para>

<para>
UNDO is not a cluster-wide switch. It is always-on infrastructure that is
used only by the subsystems that write UNDO records. The default
<literal>heap</literal> table access method does not, and ordinary heap and
index changes carry no UNDO overhead. The one user of UNDO in
<productname>PostgreSQL</productname> today is transactional filesystem
operations (FILEOPS), which make the directory and file changes done by
<command>CREATE DATABASE</command>, <command>ALTER DATABASE SET
TABLESPACE</command>, <command>CREATE TABLESPACE</command> and
<command>DROP TABLESPACE</command> roll back with the transaction.
</para>

<sect1 id="undo-fileops">
<title>Transactional Filesystem Operations</title>

<para>
When a transaction creates, renames or writes a file, creates a directory,
changes a file's mode or owner, truncates a file, or creates a link,
FILEOPS makes the change and then writes a WAL record describing it and an
UNDO record describing how to reverse it. A change that fails is an
ordinary <literal>ERROR</literal> and leaves nothing in the WAL. If the
transaction commits, the UNDO record is never used. If it rolls back, the
abort record carries the inverses, and the backend applies them, newest
first, before <command>ROLLBACK</command> returns: a created file is
removed, a changed mode is restored, a new database directory is removed
with everything in it, and so on. Removing a file or directory is not
done immediately; it is deferred to commit and simply dropped on rollback.
</para>

<para>
Between the change and its WAL record there is a short window in which a
crash leaves the change in place with nothing to reverse it, as it would
without FILEOPS. Doing it the other way round, logging first, would turn
every ordinary filesystem error into a <literal>PANIC</literal>.
</para>

<para>
FILEOPS writes WAL and UNDO only when <xref linkend="guc-wal-level"/> is
<literal>replica</literal> or higher. At <literal>minimal</literal> it
writes neither, and a crash in the middle of a transaction can leave a
partly done filesystem change behind, as without FILEOPS.
</para>
</sect1>

<sect1 id="undo-crash-recovery">
<title>Crash Recovery and Replication</title>

<para>
At the end of crash recovery, after redo, the startup process finds every
transaction that wrote UNDO records but neither committed nor aborted
before the crash, applies its UNDO records, and writes a WAL record of the
rollback, before the server accepts connections. Each checkpoint lists
the UNDO chains of transactions in progress, so this includes a
transaction whose UNDO records all precede the checkpoint redo starts
from. This step takes time proportional to
the number of UNDO records those transactions wrote. A prepared
transaction is left prepared; <command>ROLLBACK PREPARED</command>
applies its UNDO records later, and <command>COMMIT PREPARED</command>
discards them.
</para>

<para>
An UNDO record that cannot be applied, for example because the server no
longer has permission to change the file it describes, is reported with a
<literal>WARNING</literal> and is not retried, and the change it describes
stays in place. The end-of-recovery summary then also reports how many
records could not be applied.
</para>

<para>
A physical standby replays the WAL record of each filesystem change, and
the inverses carried by each abort record, so it ends up with the same
files as the primary. Crash recovery replays them the same way. A
transaction lost in a crash of the primary writes no abort record; the
primary's crash recovery writes the rollback record instead, and a
standby replays it.
</para>
</sect1>

<sect1 id="undo-space-management">
<title>UNDO Space Management</title>

<para>
UNDO records live in the WAL stream as <literal>XLOG_UNDO_BATCH</literal>
records; there are no separate UNDO files. A checkpoint does not remove
WAL that still holds the UNDO records of a transaction that is in progress
or prepared, so a long-running or long-prepared transaction that wrote UNDO
holds back WAL removal until it ends, much as a replication slot does.
</para>
</sect1>

<sect1 id="undo-monitoring">
<title>Monitoring</title>

<para>
The server log shows UNDO being applied during crash recovery:
</para>

<programlisting>
LOG: UNDO recovery: rolling back transaction 742
LOG: UNDO recovery complete: 1 transactions rolled back, 3 records applied
</programlisting>

<para>
and, if some records could not be applied:
</para>

<programlisting>
WARNING: UNDO recovery complete: 1 transactions processed, 0 records applied, 1 records could not be applied
DETAIL: The changes those records describe were not rolled back.
</programlisting>

<para>
<application>pg_waldump</application> shows UNDO records with
<option>--rmgr=Undo</option> and the filesystem changes themselves with
<option>--rmgr=FileOps</option>.
</para>
</sect1>

</chapter>
3 changes: 2 additions & 1 deletion src/backend/access/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ SUBDIRS = \
sequence \
table \
tablesample \
transam
transam \
undo \
Comment on lines +25 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last SUBDIRS entry here carries a trailing \ followed by a blank line. The tree convention is that the final element of a SUBDIRS/list has no line-continuation backslash (e.g. src/backend/Makefile ends its list with jit and no \; this file previously ended with transam and no \). GNU Make tolerates the dangling \ + empty line, but it is a needless deviation from the established style and leaves a stray empty continuation element. Drop the trailing backslash after undo.

Suggested change
transam \
undo \
transam \
undo

Comment on lines +25 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The final SUBDIRS entry carries a dangling line-continuation backslash (undo \) followed by a blank line. The tree's convention is that the last element of a \-continued list has no trailing backslash (e.g. sync in src/backend/storage/Makefile, and transam here previously). Drop the backslash on the last entry to match the convention and avoid an empty trailing continuation line.

Suggested change
transam \
undo \
transam \
undo

Comment on lines +25 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing backslash on the last list element is inconsistent with the tree's Makefile style (the previous last entry transam had none, and the sibling rmgrdesc/Makefile in this same patch correctly ends xlogdesc.o with no backslash). This adds a spurious empty trailing word to SUBDIRS and a stray continuation into the blank line. Drop the backslash.

Suggested change
transam \
undo \
transam \
undo

Comment on lines +25 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: the last SUBDIRS entry now has a dangling trailing backslash followed by a blank line. The PostgreSQL convention (see src/backend/Makefile, src/backend/storage/Makefile) is that the final SUBDIRS entry carries no continuation backslash. GNU Make tolerates this, but it deviates from the tree convention and leaves the diff reading non-idiomatically. Drop the trailing backslash after undo.

Confidence: high.

Suggested change
transam \
undo \
transam \
undo


include $(top_srcdir)/src/backend/common.mk
1 change: 1 addition & 0 deletions src/backend/access/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ subdir('spgist')
subdir('table')
subdir('tablesample')
subdir('transam')
subdir('undo')
2 changes: 2 additions & 0 deletions src/backend/access/rmgrdesc/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ OBJS = \
clogdesc.o \
committsdesc.o \
dbasedesc.o \
fileopsdesc.o \
genericdesc.o \
gindesc.o \
gistdesc.o \
Expand All @@ -29,6 +30,7 @@ OBJS = \
spgdesc.o \
standbydesc.o \
tblspcdesc.o \
undodesc.o \
xactdesc.o \
xlogdesc.o

Expand Down
Loading
Loading