Isolate the test suite from real application data and the OS keyring - #62
Merged
Merged
Conversation
Several suites clear tables outright (DELETE FROM trackers, shipments and others), and app/config.py resolves APP_DATA_DIR at import from EASYPOST_DESKTOP_DATA_DIR or else the real per-user data directory. Nothing set the variable for pytest, so running the suite on a machine that also runs the app wiped its local records in both modes. Importing app.core.client also built ClientManager against the real OS keyring. tests/conftest.py now, before any app module is imported, points the data directory at a per-session temporary directory (unconditionally, so an exported override is not inherited) and installs an in-memory keyring backend (at the backend, since names bound by `from x import f` are not reached by patching x.f). If app.config has already resolved anywhere else, the session refuses to start. tests/test_suite_isolation.py observes the file SQLite opened, the settings file written and the backend a credential lands in, each with a positive control in a clean interpreter, plus the refusal and its control. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The pytest suite had no isolation from the developer's own application data.
app/config.pyresolvesAPP_DATA_DIRonce, at import, fromEASYPOST_DESKTOP_DATA_DIRor elseplatformdirs.user_data_dir, and nothing set the variable for pytest (noconftest.py,pytest.inior pyproject setting). Several suites clear tables outright —tests/test_tracking_states.pyrunsDELETE FROM trackers, others clear shipments and similar — sopython -m pytest tests/on a machine that also runs the app wiped its local tracker and shipment records for both test and production modes. Importingapp.core.clientalso builtClientManagerat module level against the real OS keyring.Fix
tests/conftest.py, which pytest imports before collecting any test module beside it:EASYPOST_DESKTOP_DATA_DIRat a per-session temporary directory, set unconditionally so a developer's exported override is not inherited;_ScreenshotKeyringpattern frompackaging/make_screenshots.py). It stubs the backend, not module attributes, because a name bound byfrom x import fis not reached by patchingx.f;app.configand refuses to start the session if its data directory resolved anywhere else — a path computed before the conftest ran cannot be moved, so a warning would not protect anything;Proof by observation
tests/test_suite_isolation.pychecks what the app itself does, not the variable conftest set:PRAGMA database_listafterinit_db()names a file inside the session directory, not the real per-user oneapp.config.DATABASE_PATHis the realplatformdirspath, outside the session directorysave_settings()writessettings.jsoninside the session directorykeyring.get_keyring()is the session backend, not akeyring.backends.*class; a credential saved throughcredential_storelands in it and is read back throughapp.core.client's by-name bindingkeyring.get_keyring()is akeyring.backends.*platform backendapp.configfirst exits non-zero with "Refusing to run the tests"Results
mainCI run (bc53957) was 595 passed, 6 skipped on both legs; the difference is exactly the 7 new tests, and--collect-onlygives 608 with and without the conftest, so no existing test was dropped.%LOCALAPPDATA%\EasyPostDesktopand%APPDATA%\EasyPostDesktopwere recorded before the run and are identical after it.easypost-tests-*directory is left behind.No application code changes.
🤖 Generated with Claude Code