Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,15 @@ jobs:
run: flake8 src

- name: Test with pytest
run: pytest --cov=src --cov-report=lcov
run: pytest --cov=src --cov-report=lcov --cov-report=json

- name: Upload code coverage
uses: actions/upload-artifact@v4
with:
name: coverage-${{ matrix.python }}
path: ./coverage.lcov
path: |
./coverage.lcov
./coverage.json

- name: Analyse with MyPy
run: mypy
Expand Down Expand Up @@ -136,6 +138,24 @@ jobs:
with:
name: coverage-3.12

- name: Parse coverage percentage
id: get-coverage
run: |
COVERAGE=$(jq -r '.totals.percent_covered_display' coverage.json)
echo "coverage=$COVERAGE" >> $GITHUB_OUTPUT

- name: Update Coverage Badge
# Only run this on the main branch so PRs don't overwrite the main badge
if: github.ref == 'refs/heads/main'
uses: Schneegans/dynamic-badges-action@v1.7.0
with:
auth: ${{ secrets.GIST_SECRET }}
gistID: 936bcec8e1815a49d1e7f947924ffa3f
filename: labthings-fastapi-coverage.json
label: coverage
message: ${{ steps.get-coverage.outputs.coverage }}%
color: green

- name: Download code coverage report for base branch
id: download-base-coverage
continue-on-error: true
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[![codecov](https://codecov.io/gh/rwb27/labthings-fastapi/branch/main/graph/badge.svg?token=IR4QNA8X6M)](https://codecov.io/gh/rwb27/labthings-fastapi)
![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/bprobert97/936bcec8e1815a49d1e7f947924ffa3f/raw/labthings-fastapi-coverage.json)
[![Documentation Status](https://readthedocs.org/projects/labthings-fastapi/badge/?version=latest)](https://labthings-fastapi.readthedocs.io/en/latest/?badge=latest)

# labthings-fastapi
Expand Down
85 changes: 85 additions & 0 deletions docs/source/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Testing `labthings-fastapi`

Our test suite ensures the framework functions correctly, maintains code quality, and integrates seamlessly with static type checkers.

## Continuous Integration (CI) Pipeline

When you submit a Pull Request (PR), our GitHub Actions CI pipeline automatically runs a comprehensive suite of checks. Your PR must pass these checks before it can be merged.

Here is what the CI pipeline tests:

* **Matrix Testing:** We run the core test suite (`pytest`) and static type checks (`mypy`) across Python versions 3.10, 3.11, 3.12, and 3.13.
* **Code Quality & Security:** The pipeline runs `ruff` (formatting and linting), `flake8` (docstrings), `codespell` (spelling), and a `pip-audit` to check for dependency vulnerabilities.
* **Coverage:** We track code coverage. A report is generated on your PR to show if your changes increased or decreased overall test coverage.
* **Dependency Checks:** We run the tests twice: once with pinned dependencies (`dev-requirements.txt`) for reproducibility, and once with unpinned dependencies (`.[dev]`) to catch upstream breakages early.
* **Downstream Integration:** The pipeline installs your version of `labthings-fastapi` alongside the `v3` branch of the OpenFlexure Microscope Server. It then runs the entire OFM test suite (unit, integration, and lifecycle) to guarantee backwards compatibility. *(Note: You can target a specific OFM branch by adding `OFM-Feature-Branch: branch-name` to your PR description).*

---

## Running Core Tests Locally

To ensure your code will pass CI, you should run these checks locally before pushing your commits.

### 1. Local Environment Setup

We recommend running the test suite using the pinned development dependencies to mirror the primary CI environment. Ensure you have cloned the repository, then install the package and dependencies:

```bash
git clone https://github.com/labthings/labthings-fastapi.git
cd labthings-fastapi
pip install -e . -r dev-requirements.txt
```

### 2. Linting, Formatting, and Spelling

Check that your code adheres to the project's formatting and style guidelines from the root of the repository:

* **Linting:** `ruff check .`
* **Formatting:** `ruff format --check .`
* **Spelling:** `codespell .`
* **Docstrings:** `flake8 src`

### 3. Static Type Checking

`labthings-fastapi` is designed to be fully type-hinted. We explicitly test that `mypy` can infer the correct types for `Thing` attributes. Run static type checking across the source code and our dedicated typing tests folder:

```bash
mypy src typing_tests
```

### 4. Unit Tests & Coverage

We use `pytest` for our core test suite. Execute the unit tests and generate a coverage report using:

```bash
pytest --cov=src
```

---

## Downstream Integration Testing Locally

Because `labthings-fastapi` underpins the [OpenFlexure Microscope software], major architectural changes should be tested against the downstream server locally. This matches the `test-against-ofm-v3` job in our CI pipeline.

Assuming you have both repositories cloned in the same parent directory:

```bash
# 1. Setup the OpenFlexure Microscope Server
cd ../openflexure-microscope-server
git checkout v3
pip install -e .[dev]

# 2. Install your local version of labthings-fastapi
pip install -e ../labthings-fastapi

# 3. Pull the OFM web app (required for integration)
python ./pull_webapp.py

# 4. Run the OFM Test Suite
pytest # Run OFM unit tests
pytest tests/integration_tests # Run OFM integration tests
python tests/lifecycle_test/testfile.py # Run OFM lifecycle tests
mypy src # Run OFM static type checks
```

[OpenFlexure Microscope software]: https://gitlab.com/openflexure/openflexure-microscope-server/
Loading