|
1 | 1 | # Pre-Release Validation Checklist |
2 | 2 |
|
3 | | -This checklist ensures SDK releases meet quality standards. **Complete ALL items before publishing to PyPI.** |
| 3 | +The authoritative release design is documented in |
| 4 | +[`docs/RELEASE_PROCESS.md`](../docs/RELEASE_PROCESS.md). Complete every gate |
| 5 | +below before publishing a non-prerelease GitHub Release. |
4 | 6 |
|
5 | | -## Automated Checks (Run Script) |
| 7 | +## Automated Gate |
6 | 8 |
|
7 | | -```bash |
8 | | -./scripts/pre-release-validation.sh |
9 | | -``` |
10 | | - |
11 | | -This script runs all automated validations and reports pass/fail status. |
12 | | - |
13 | | -## Manual Checklist |
14 | | - |
15 | | -### 1. Version Management ✅ |
16 | | -- [ ] Version bumped in `pyproject.toml` |
17 | | -- [ ] Version bumped in `oilpriceapi/__init__.py` |
18 | | -- [ ] Version updated in `CHANGELOG.md` |
19 | | -- [ ] CHANGELOG has comprehensive release notes |
20 | | -- [ ] No `UNRELEASED` sections in CHANGELOG |
21 | | - |
22 | | -### 2. Code Quality ✅ |
23 | | -- [ ] All unit tests pass (`pytest tests/unit -v`) |
24 | | -- [ ] All integration tests pass (`pytest tests/integration -v`) |
25 | | -- [ ] Test coverage ≥ 80% (`pytest --cov`) |
26 | | -- [ ] No linting errors (`ruff check .`) |
27 | | -- [ ] No type errors (`mypy oilpriceapi`) |
28 | | -- [ ] Code formatted (`black --check .`) |
29 | | - |
30 | | -### 3. Integration Validation ✅ |
31 | | -- [ ] Historical endpoint tests pass (catches timeout bug) |
32 | | -- [ ] Performance baselines met: |
33 | | - - 1-week queries: <30s |
34 | | - - 1-month queries: <60s |
35 | | - - 1-year queries: <120s |
36 | | -- [ ] All commodities tested |
37 | | -- [ ] Error handling verified |
38 | | - |
39 | | -### 4. Documentation ✅ |
40 | | -- [ ] README.md updated with new features |
41 | | -- [ ] API documentation current |
42 | | -- [ ] Code examples work |
43 | | -- [ ] Migration guide included (if breaking changes) |
44 | | -- [ ] Docstrings updated for new/changed functions |
45 | | - |
46 | | -### 5. Build & Package ✅ |
47 | | -- [ ] Clean build: `rm -rf dist/ build/ *.egg-info` |
48 | | -- [ ] Build succeeds: `python -m build` |
49 | | -- [ ] Wheel created: `ls dist/*.whl` |
50 | | -- [ ] Source distribution created: `ls dist/*.tar.gz` |
51 | | -- [ ] Package installs locally: `pip install dist/*.whl` |
52 | | -- [ ] Imports work: `python -c "import oilpriceapi; print(oilpriceapi.__version__)"` |
53 | | - |
54 | | -### 6. Backwards Compatibility ✅ |
55 | | -- [ ] No breaking changes (or documented in CHANGELOG) |
56 | | -- [ ] Existing code samples still work |
57 | | -- [ ] Deprecations properly warned |
58 | | -- [ ] Migration guide provided (if needed) |
59 | | - |
60 | | -### 7. Security ✅ |
61 | | -- [ ] No hardcoded credentials |
62 | | -- [ ] No secrets in code or tests |
63 | | -- [ ] Dependencies scanned: `pip-audit` |
64 | | -- [ ] SECURITY.md reviewed and current |
65 | | - |
66 | | -### 8. Git & GitHub ✅ |
67 | | -- [ ] All changes committed |
68 | | -- [ ] Commit message follows convention |
69 | | -- [ ] Git tag created: `git tag v1.X.Y` |
70 | | -- [ ] Tag pushed: `git push --tags` |
71 | | -- [ ] No uncommitted changes |
72 | | - |
73 | | -### 9. PyPI Publishing ✅ |
74 | | -- [ ] Test PyPI upload works: `twine upload --repository testpypi dist/*` |
75 | | -- [ ] Test installation from TestPyPI |
76 | | -- [ ] Production PyPI upload: `twine upload dist/*` |
77 | | -- [ ] Verify on PyPI: https://pypi.org/project/oilpriceapi/ |
78 | | -- [ ] Installation works: `pip install --upgrade oilpriceapi` |
79 | | - |
80 | | -### 10. Post-Release ✅ |
81 | | -- [ ] GitHub release created with notes |
82 | | -- [ ] Documentation site updated |
83 | | -- [ ] Announcement prepared (if major release) |
84 | | -- [ ] Monitor error tracking for 24 hours |
85 | | -- [ ] Check PyPI download stats |
86 | | - |
87 | | -## What Would Have Caught the v1.4.1 Bug? |
88 | | - |
89 | | -The historical timeout bug (reported by idan@comity.ai) would have been caught by: |
90 | | - |
91 | | -1. ✅ **Integration Tests** (`tests/integration/test_historical_endpoints.py`) |
92 | | - - `test_7_day_query_uses_past_week_endpoint` - Would fail (67s timeout) |
93 | | - - `test_365_day_query_uses_past_year_endpoint` - Would fail (30s timeout) |
94 | | - |
95 | | -2. ✅ **Performance Baselines** (`TestHistoricalPerformanceBaselines`) |
96 | | - - All tests would fail with timeouts |
97 | | - |
98 | | -3. ✅ **Pre-Release Script** (`scripts/pre-release-validation.sh`) |
99 | | - - Integration tests would fail |
100 | | - - Script would prevent release |
101 | | - |
102 | | -## Automation Script |
103 | | - |
104 | | -The `pre-release-validation.sh` script automates items 1-7: |
| 9 | +Run from a clean checkout of the release commit: |
105 | 10 |
|
106 | 11 | ```bash |
107 | | -# Run full validation |
108 | | -./scripts/pre-release-validation.sh |
109 | | - |
110 | | -# Run with verbose output |
111 | | -./scripts/pre-release-validation.sh --verbose |
112 | | - |
113 | | -# Skip slow tests (for quick checks) |
114 | | -./scripts/pre-release-validation.sh --skip-slow |
| 12 | +python -m pip install --upgrade pip |
| 13 | +python -m pip install -e '.[dev]' build pip-audit |
| 14 | +ruff check oilpriceapi/ |
| 15 | +mypy oilpriceapi/ --ignore-missing-imports |
| 16 | +pytest tests/ --ignore=tests/integration --ignore=tests/contract -m 'not slow' |
| 17 | +python scripts/validate_storefront_claims.py |
| 18 | +python scripts/generate_snippet_manifest.py --source-commit "$(git rev-parse HEAD)" --output artifacts/snippets/oilpriceapi-python-snippets-v1.json |
| 19 | +python -m build |
| 20 | +./scripts/clean-wheel-smoke.sh |
| 21 | +pip-audit |
115 | 22 | ``` |
116 | 23 |
|
117 | | -**Exit Codes:** |
118 | | -- `0` - All checks passed, ready to release |
119 | | -- `1` - One or more checks failed, DO NOT release |
120 | | - |
121 | | -## Emergency Release Procedure |
| 24 | +The hosted Python 3.8-3.12 matrix, keyless and keyed live tests, canonical |
| 25 | +production snippets, and `Scheduled SDK Synthetic` must all be green at the |
| 26 | +same release commit. The repository test gate enforces at least 50% aggregate |
| 27 | +coverage; increases to that threshold require a reviewed test-coverage change. |
122 | 28 |
|
123 | | -If critical bug requires immediate release: |
124 | | - |
125 | | -1. Run minimum validation: |
126 | | - ```bash |
127 | | - pytest tests/unit -v --tb=short |
128 | | - pytest tests/integration/test_historical_endpoints.py -v |
129 | | - ``` |
130 | | - |
131 | | -2. Verify the specific fix works |
132 | | - |
133 | | -3. Document in CHANGELOG as emergency release |
134 | | - |
135 | | -4. **Still run full validation after emergency release** |
136 | | - |
137 | | -## Failed Validation - What to Do |
138 | | - |
139 | | -### Tests Failed |
140 | | -1. Fix failing tests |
141 | | -2. Re-run full validation |
142 | | -3. Update CHANGELOG if fixes required code changes |
143 | | - |
144 | | -### Performance Regression |
145 | | -1. Investigate using profiling |
146 | | -2. Fix performance issue |
147 | | -3. Re-establish baseline |
148 | | - |
149 | | -### Documentation Missing |
150 | | -1. Update documentation |
151 | | -2. Add code examples |
152 | | -3. Test examples actually work |
153 | | - |
154 | | -### Build Failed |
155 | | -1. Check `pyproject.toml` for errors |
156 | | -2. Verify all files included in manifest |
157 | | -3. Test clean build: `rm -rf dist/ && python -m build` |
158 | | - |
159 | | -## Version History |
160 | | - |
161 | | -| Version | Date | Validator | Result | Notes | |
162 | | -|---------|------|-----------|--------|-------| |
163 | | -| v1.4.2 | 2025-12-16 | Manual | ✅ Pass | Fixed historical timeout bug | |
164 | | -| v1.4.1 | 2025-12-15 | None | ❌ Fail | Historical timeout bug shipped | |
165 | | - |
166 | | -*Note: v1.4.1 did not use this checklist, which is why the bug reached production.* |
167 | | - |
168 | | -## Integration with CI/CD |
169 | | - |
170 | | -### GitHub Actions (Recommended) |
171 | | - |
172 | | -```yaml |
173 | | -name: Pre-Release Validation |
174 | | - |
175 | | -on: |
176 | | - push: |
177 | | - tags: |
178 | | - - 'v*' |
179 | | - |
180 | | -jobs: |
181 | | - validate: |
182 | | - runs-on: ubuntu-latest |
183 | | - steps: |
184 | | - - uses: actions/checkout@v3 |
185 | | - |
186 | | - - name: Set up Python |
187 | | - uses: actions/setup-python@v4 |
188 | | - with: |
189 | | - python-version: '3.11' |
190 | | - |
191 | | - - name: Install dependencies |
192 | | - run: | |
193 | | - pip install -e ".[dev]" |
194 | | - pip install twine pip-audit |
195 | | -
|
196 | | - - name: Run pre-release validation |
197 | | - env: |
198 | | - OILPRICEAPI_KEY: ${{ secrets.OILPRICEAPI_KEY }} |
199 | | - run: ./scripts/pre-release-validation.sh |
200 | | - |
201 | | - - name: Build package |
202 | | - if: success() |
203 | | - run: python -m build |
204 | | - |
205 | | - - name: Publish to PyPI |
206 | | - if: success() |
207 | | - env: |
208 | | - TWINE_USERNAME: __token__ |
209 | | - TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} |
210 | | - run: twine upload dist/* |
211 | | -``` |
| 29 | +## Release Metadata |
212 | 30 |
|
213 | | -## Contact |
| 31 | +- `pyproject.toml` and `oilpriceapi/version.py` contain the same version. |
| 32 | +- `CHANGELOG.md` has one release section for that version with customer-visible |
| 33 | + behavior and recovery guidance. |
| 34 | +- The version is absent from PyPI and from existing GitHub releases. |
| 35 | +- The release tag is exactly `v<package-version>`. |
| 36 | +- The worktree is clean and the tag resolves to the reviewed main commit. |
214 | 37 |
|
215 | | -Questions about the validation process: |
216 | | -- GitHub Issues: https://github.com/OilpriceAPI/python-sdk/issues |
217 | | -- Email: support@oilpriceapi.com |
| 38 | +## Publication And Recovery |
218 | 39 |
|
219 | | -## Related Issues |
| 40 | +Publish through a non-prerelease GitHub Release only. The `Publish to PyPI` |
| 41 | +workflow verifies the tag, repeats the tests and dependency audit, builds and |
| 42 | +installs the wheel in a clean environment, attaches the snippet manifest, and |
| 43 | +uses PyPI trusted publishing. Do not upload with Twine or a local API token. |
220 | 44 |
|
221 | | -- [#20](https://github.com/OilpriceAPI/python-sdk/issues/20) - Integration tests |
222 | | -- [#21](https://github.com/OilpriceAPI/python-sdk/issues/21) - Performance baselines |
223 | | -- [#22](https://github.com/OilpriceAPI/python-sdk/issues/22) - Pre-release validation (this document) |
| 45 | +PyPI artifacts are immutable. If a production defect appears, stop promotion, |
| 46 | +yank the affected version, add a failing regression test, and publish a new |
| 47 | +patch version through the same gate. |
0 commit comments