Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 40 additions & 103 deletions .github/workflows/build-deb.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
name: Build Debian Packages

# Builds the package from debian/ with dpkg-buildpackage in a container of each release.
# Pull requests build amd64 on trixie and bookworm. Main also builds amd64 on noble and jammy
# and arm64 on trixie and bookworm. The release (called by release.yml) and the manual runs with
# full build all of them. arm64 and armhf are built natively on arm runners, which run 32 bit
# arm containers.
on:
push:
branches: [ main ]
Expand All @@ -8,10 +13,19 @@ on:
paths:
- '.github/workflows/build-deb.yml'
- 'debian/**'
- 'docker/**'
- 'Dockerfile.linux'
- 'bin/build_monero_cpp.sh'
workflow_dispatch:
inputs:
full:
description: Build every release and architecture
type: boolean
default: false
workflow_call:
inputs:
full:
description: Build every release and architecture
type: boolean
default: false

# A new push to a pull request cancels the previous run. Other events get their own
# group so they are never cancelled or replaced.
Expand All @@ -25,61 +39,18 @@ permissions:

jobs:
build:
name: ${{ matrix.distro_name }} (${{ matrix.name }} / ${{ matrix.arch }})
runs-on: ubuntu-latest
name: ${{ contains('noble jammy', matrix.name) && 'ubuntu' || 'debian' }} (${{ matrix.name }} / ${{ matrix.arch }})
runs-on: ${{ matrix.arch == 'amd64' && 'ubuntu-latest' || 'ubuntu-24.04-arm' }}
timeout-minutes: 360

strategy:
fail-fast: false
matrix:
include:
- name: noble
distro_name: ubuntu
base_image: ubuntu:24.04
arch: amd64
- name: noble
distro_name: ubuntu
base_image: ubuntu:24.04
arch: arm64
- name: noble
distro_name: ubuntu
base_image: ubuntu:24.04
arch: armhf
- name: jammy
distro_name: ubuntu
base_image: ubuntu:22.04
arch: amd64
- name: jammy
distro_name: ubuntu
base_image: ubuntu:22.04
arch: arm64
- name: jammy
distro_name: ubuntu
base_image: ubuntu:22.04
arch: armhf
- name: trixie
distro_name: debian
base_image: debian:trixie
arch: amd64
- name: trixie
distro_name: debian
base_image: debian:trixie
arch: arm64
- name: trixie
distro_name: debian
base_image: debian:trixie
arch: armhf
- name: bookworm
distro_name: debian
base_image: debian:bookworm
arch: amd64
- name: bookworm
distro_name: debian
base_image: debian:bookworm
arch: arm64
- name: bookworm
distro_name: debian
base_image: debian:bookworm
arch: armhf
name: [noble, jammy, trixie, bookworm]
arch: [amd64, arm64, armhf]
# pull requests build amd64 on trixie and bookworm, main adds amd64 on the Ubuntu releases and
# arm64 on the Debian ones, the tags and the manual runs with full build everything
exclude: ${{ fromJSON((github.ref_type == 'tag' || inputs.full) && '[]' || (github.event_name == 'pull_request' && '[{"arch":"armhf"},{"arch":"arm64"},{"name":"noble"},{"name":"jammy"}]' || '[{"arch":"armhf"},{"arch":"arm64","name":"noble"},{"arch":"arm64","name":"jammy"}]')) }}

steps:
- name: Checkout repository
Expand All @@ -90,43 +61,14 @@ jobs:
- name: Determine package version
id: version
run: |
VERSION=$(grep -m1 '^version' pyproject.toml | sed -E 's/version = "(.*)"/\1/')
VERSION=$(sed -n 's/^version = "\(.*\)"$/\1/p' pyproject.toml | head -n 1)
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"

- name: Setup qemu
if: matrix.arch == 'arm64' || matrix.arch == 'armhf'
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0

- name: Setup docker buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0

- name: Free up disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc || true
docker system prune -af || true

- name: Prepare build environment
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v4.0.0
with:
timeout_minutes: 60
max_attempts: 3
retry_wait_seconds: 15
command: |
PLATFORM=${{ matrix.arch == 'amd64' && 'linux/amd64' ||
matrix.arch == 'armhf' && 'linux/arm/v7' ||
'linux/arm64' }}

docker buildx build \
--platform $PLATFORM \
--build-arg BASE_IMAGE=${{ matrix.base_image }} \
--build-arg THREADS=3 \
-t monero-python:${{ matrix.name }}-${{ matrix.arch }} \
-f Dockerfile.linux \
--cache-from type=gha,scope=${{ matrix.name }}-${{ matrix.arch }} \
--cache-to type=gha,mode=max,scope=${{ matrix.name }}-${{ matrix.arch }} \
--load \
.

- name: Restore ccache
uses: actions/cache/restore@v4
with:
Expand All @@ -135,26 +77,21 @@ jobs:
restore-keys: |
ccache-deb-${{ matrix.name }}-${{ matrix.arch }}_

- name: Build monero-python
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v4.0.0
# Installs the Build-Depends of debian/control in the container, sets the version of
# debian/changelog from pyproject.toml and runs dpkg-buildpackage. The workspace is
# mounted at /github/workspace in the container.
- name: Build package
uses: jtdor/build-deb-action@473136f14a42a231124c1f7cefaedfe941203c38 # v1.9.1
env:
CCACHE_DIR: /github/workspace/.ccache
CCACHE_MAXSIZE: 500M
with:
timeout_minutes: 240
max_attempts: 3
retry_wait_seconds: 15
command: |
PLATFORM=${{ matrix.arch == 'amd64' && 'linux/amd64' ||
matrix.arch == 'armhf' && 'linux/arm/v7' ||
'linux/arm64' }}
PACKAGE_NAME="python3-monero_${{ steps.version.outputs.version }}-1${{ matrix.name }}1_${{ matrix.arch }}"

docker run --rm \
--platform $PLATFORM \
-v $PWD:/monero-python -w /monero-python \
-e PACKAGE_NAME="${PACKAGE_NAME}" \
-e CCACHE_DIR=/monero-python/.ccache \
-e CCACHE_MAXSIZE=500M \
monero-python:${{ matrix.name }}-${{ matrix.arch }} \
bash -euxc "./docker/build.sh"
# the image is not in the matrix: an include entry of a release left out by exclude
# would be a job of its own
docker-image: ${{ fromJSON('{"noble":"ubuntu:24.04","jammy":"ubuntu:22.04","trixie":"debian:trixie-slim","bookworm":"debian:bookworm-slim"}')[matrix.name] }}
extra-docker-args: --platform ${{ matrix.arch == 'armhf' && 'linux/arm/v7' || format('linux/{0}', matrix.arch) }}
before-build-hook: sh debian/set-version.sh ${{ matrix.name }}
buildpackage-opts: --build=binary --no-sign --jobs=auto

- name: Save ccache
# Pull requests only read the cache, pushes to main and manual runs replace it.
Expand All @@ -167,4 +104,4 @@ jobs:
uses: actions/upload-artifact@v4
with:
name: python3-monero_${{ steps.version.outputs.version }}-1${{ matrix.name }}1_${{ matrix.arch }}
path: build/*.deb
path: debian/artifacts/python3-monero_*.deb
42 changes: 31 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@ permissions:
actions: read

jobs:
release:
name: Publish draft release
# the macos and windows builds and the tests are those of main, the deb ones are built here
check:
name: Check the tag and the builds of main
runs-on: ubuntu-latest
permissions:
contents: write
actions: read
outputs:
build_macos_run_id: ${{ steps.find_runs.outputs.build_macos_run_id }}
build_windows_run_id: ${{ steps.find_runs.outputs.build_windows_run_id }}
test_run_id: ${{ steps.find_runs.outputs.test_run_id }}

steps:
- name: Check the tag against the version in pyproject.toml
Expand All @@ -40,7 +42,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
for wf in build-deb.yml build-macos.yml build-windows.yml test.yml; do
for wf in build-macos.yml build-windows.yml test.yml; do
key=$(echo "${wf%.*}" | tr '-' '_')
run_id=$(gh run list --repo "${{ github.repository }}" --workflow "$wf" \
--branch main --status success --json databaseId,headSha \
Expand All @@ -52,11 +54,29 @@ jobs:
echo "${key}_run_id=${run_id}" >> "$GITHUB_OUTPUT"
done

# every release and architecture, the builds of main only have some of them
deb:
name: Build the deb packages
needs: check
uses: ./.github/workflows/build-deb.yml
permissions:
contents: read
actions: write # build-deb.yml replaces its ccache entries
with:
full: true

release:
name: Publish draft release
needs: [check, deb]
runs-on: ubuntu-latest
permissions:
contents: write
actions: read

steps:
- name: Download deb packages
uses: actions/download-artifact@v4
with:
github-token: ${{ github.token }}
run-id: ${{ steps.find_runs.outputs.build_deb_run_id }}
pattern: python3-monero_*
path: release-assets
merge-multiple: true
Expand All @@ -65,7 +85,7 @@ jobs:
uses: actions/download-artifact@v4
with:
github-token: ${{ github.token }}
run-id: ${{ steps.find_runs.outputs.build_macos_run_id }}
run-id: ${{ needs.check.outputs.build_macos_run_id }}
pattern: monero-macos-*
path: release-assets
merge-multiple: true
Expand All @@ -74,7 +94,7 @@ jobs:
uses: actions/download-artifact@v4
with:
github-token: ${{ github.token }}
run-id: ${{ steps.find_runs.outputs.build_windows_run_id }}
run-id: ${{ needs.check.outputs.build_windows_run_id }}
name: monero-python-win-amd64
path: release-assets
merge-multiple: true
Expand All @@ -83,7 +103,7 @@ jobs:
uses: actions/download-artifact@v4
with:
github-token: ${{ github.token }}
run-id: ${{ steps.find_runs.outputs.test_run_id }}
run-id: ${{ needs.check.outputs.test_run_id }}
name: coverage-reports-linux
path: coverage-report

Expand Down
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,11 @@ coverage*
monero.log
monero_tests_*.log
.ccache
# left by dpkg-buildpackage
debian/artifacts
debian/python3-monero
debian/.debhelper
debian/debhelper-build-stamp
debian/files
debian/*.substvars
debian/*.log
46 changes: 0 additions & 46 deletions Dockerfile.linux

This file was deleted.

28 changes: 13 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,32 +146,30 @@ On anything else, build it from source.

> **Faster rebuilds:** if [ccache](https://ccache.dev/) is installed it is used automatically for monero and monero-python (pass `-DUSE_CCACHE=OFF` to cmake to disable it). To use it for monero-cpp as well, build that with `-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache`.

### Linux Docker Build
### Debian package

The package is built from [debian/](debian) with `dpkg-buildpackage`, the way CI does it. It installs the build dependencies and some static libraries system wide, so build it in a container of the release you want it for (`ubuntu:24.04`, `ubuntu:22.04`, `debian:trixie` or `debian:bookworm`).

1. Install [Docker](https://docs.docker.com/engine/install/)
2. Clone the repository
```sh
git clone --recursive https://github.com/libmonero/monero-python.git
```
3. Prepare build environment
```sh
cd monero-python
docker build --tag monero-python:build-linux --file Dockerfile.linux .
```
4. (Optional) Specify build image
```sh
cd monero-python
docker build --tag monero-python:build-linux --build-arg BASE_IMAGE="ubuntu:22.04" --file Dockerfile.linux .
```

5. Build
3. Build, replacing `ubuntu:24.04` with the release you want
```sh
docker run --rm -it -v <MONERO_PYTHON_DIR_FULL_PATH>:/monero-python -w /monero-python monero-python:build-linux bash ./docker/build.sh
docker run --rm -it -v <MONERO_PYTHON_DIR_FULL_PATH>:/monero-python -w /monero-python ubuntu:24.04 bash -c '
apt-get update && apt-get install -y dpkg-dev &&
apt-get build-dep -y ./ &&
debian/set-version.sh &&
dpkg-buildpackage --build=binary --no-sign &&
mkdir -p debian/artifacts && mv ../python3-monero_* debian/artifacts/'
```

* `MONERO_PYTHON_DIR_FULL_PATH` - full path to `monero-python` directory
* `debian/set-version.sh` adds an entry for the version of `pyproject.toml` to `debian/changelog`, there is no need to commit it
* add `--jobs=4` to `dpkg-buildpackage` to build faster, each compiler needs 1 to 2 GB of memory

6. Library build will be placed in `monero-python/build` directory
4. The package is placed in `monero-python/debian/artifacts`

### Windows

Expand Down
Loading
Loading