Skip to content

PYTHON-6266 Make the synchro pre-commit hook atomic and write only real changes - #3230

Draft
blink1073 wants to merge 14 commits into
mongodb:mainfrom
blink1073:PYTHON-6266
Draft

blink1073 wants to merge 14 commits into
mongodb:mainfrom
blink1073:PYTHON-6266

Conversation

@blink1073

Copy link
Copy Markdown
Member

PYTHON-6266

Changes in this PR

The synchro hook regenerated the synchronous driver directly in the working tree: a failed run left spurious modifications across untouched files, and ruff failures on generated output were silently ignored locally.

  • Generate the synchronous files in a staging area, copying them into the working tree only when their content changes: failed runs leave the tree untouched, no-op runs stage nothing, and in CI the hook still fails on out-of-date sync files, copying the drift first so the diff shows it.
  • Fail the hook when ruff reports problems in the generated output, naming the async source to fix, and update the contributor docs.
  • Mirror async test files only when their sync counterpart already exists in test/.

Test Plan

New unit and integration tests in tools/test_synchro.py cover the staging, copy-on-change, manifest, and failure behavior:

uv run --extra test --with unasync pytest tools/test_synchro.py -v --override-ini="addopts="

pre-commit run --all-files passes on the real repo with no generated-file changes.

Checklist

Checklist for Author

  • Did you update the changelog (if necessary)? (No — internal tooling change, nothing user-facing.)
  • Is there test coverage?
  • Is any followup work tracked in a JIRA ticket? If so, add link(s). (No followup work.)

Checklist for Reviewer

  • Does the title of the PR reference a JIRA Ticket?
  • Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
  • Is all relevant documentation (README or docstring) updated?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A failed write can leave a destination corrupted because rollback skips it, and generated-output Ruff failures suppress the diagnostics contributors need to locate and fix the source.

2 open issues
What changed in this PR

This PR makes PyMongo’s async-to-sync generation hook safer by staging generated output before updating the working tree and copying only files whose content changes.

Changes:

  • Stage generation and formatting, publish changed files, and preserve CI drift reporting.
  • Make generated-output Ruff failures fatal and mirror async tests only when a synchronous counterpart exists.
  • Add regression tests and update contributor guidance.
File Description
tools/​test_synchro.py Adds coverage for staging, copy-on-change, rollback, failures, and test mirroring.
tools/​synchro.py Implements staged generation, changed-file publication, and fatal Ruff checks.
CONTRIBUTING.md Documents staged generation and handling generated-output errors.
.gitignore Ignores the temporary staging directory.

Review effort: Balanced

Comment thread tools/synchro.py Resolved
Comment thread tools/synchro.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The forward rollback in tools/synchro.py can restore larger files before releasing later allocations, causing disk-full failures that corrupt a file and leave others unrestored.

1 open issue
2 resolved since last review

Review effort: Balanced

Comment thread tools/synchro.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The sync manifest can omit untracked generated modules from commits, and manifest-write failures can leave generated files modified without rollback.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

High severity Include untracked generated outputs in the staging manifest

tools/​synchro.py:525

If just synchro creates a new sync module and only its async source is staged, the commit hook sees identical generated content and clears this manifest. The sync module remains untracked, so synchro-stage-files omits it from the commit. Include untracked generated outputs in the manifest while continuing to skip unchanged tracked files.

Medium severity Roll back generated files when manifest publication fails

tools/​synchro.py:525

If this manifest write fails, the generated files have already been replaced and their rollback data discarded, so the failed hook leaves the working tree modified. Publish the manifest within the same rollback transaction as the generated files, preserving the intentional CI drift behavior. Add a regression test that makes manifest publication fail and verifies that the original destinations are restored.

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A partial manifest write under ENOSPC can truncate .synchro-modified and exhaust space needed for rollback, risking further file truncation; atomic publication and a partial-write failure test are needed.

1 open issue

Review effort: Balanced

Comment thread tools/synchro.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed for an unpublished concern that interrupting publication can bypass rollback and leave destination files inconsistently updated.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because an interrupt immediately after destination replacement can leave regenerated content outside rollback tracking when the hook fails.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Rollback in tools/synchro.py can truncate and corrupt an intact original file during persistent disk-full or quota failures, even when its replacement was never published.

1 open issue

Review effort: Balanced

Comment thread tools/synchro.py Resolved
@blink1073
blink1073 requested a balanced review from Copilot November 17, 2026 17:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A critical rollback path can truncate an untouched original file under persistent ENOSPC errors when both hard-link backup creation and the temporary write fail.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because interruption and rollback failures can leave the manifest inconsistent with generated files or cause loss of original on-disk content.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because failure paths can stage unintended edits from a stale manifest and delete the only recoverable copy of original files during rollback.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unconditional backup deletion in tools/synchro.py can destroy the only recoverable copy of the original contents after a rollback failure.

1 open issue

Review effort: Balanced

Comment thread tools/synchro.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Backup cleanup occurs outside rollback protection, so a cleanup failure can report completed publication as failed and leave stale backups that block subsequent regeneration.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread tools/synchro.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the fallback used when hard links are unavailable can irreversibly truncate the original if publication and rollback both fail.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to evaluate a critical risk of losing original files when fallback publication and rollback both fail.

No open issues 🎉

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants