PEP 440 When Installing cloudstack-marvin breaks building wheels

[loth]

ISSUE TYPE

• Bug Report

COMPONENT NAME

cloudstack-marvin

CLOUDSTACK VERSION

4.17, 4.18

CONFIGURATION

N/A

OS / ENVIRONMENT

Ubuntu 20.04

SUMMARY

When installing cloudstack-marvin from built debian packages, we are getting a PEP 440 warning and an error building wheels

STEPS TO REPRODUCE

Build, install package

EXPECTED RESULTS

cloudstack-marvin builds/installs correctly

ACTUAL RESULTS

Marvin errors with a wheel build issue

# apt install cloudstack-marvin
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libffi-dev
The following NEW packages will be installed:
  cloudstack-marvin libffi-dev
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 605 kB of archives.
After this operation, 878 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirror.leaseweb.com/ubuntu focal/main amd64 libffi-dev amd64 3.3-4 [57.0 kB]
Get:2 https://artifactory.devleaseweb.com/artifactory/cloudstack-apt focal/4-18-0-leaseweb amd64 cloudstack-marvin all 4.18.1.0-20230712T075151~focal [549 kB]
Fetched 605 kB in 0s (5891 kB/s)
Selecting previously unselected package libffi-dev:amd64.
(Reading database ... 71916 files and directories currently installed.)
Preparing to unpack .../libffi-dev_3.3-4_amd64.deb ...
Unpacking libffi-dev:amd64 (3.3-4) ...
Selecting previously unselected package cloudstack-marvin.
Preparing to unpack .../cloudstack-marvin_4.18.1.0-20230712T075151~focal_all.deb ...
Unpacking cloudstack-marvin (4.18.1.0-20230712T075151~focal) ...
Setting up libffi-dev:amd64 (3.3-4) ...
Setting up cloudstack-marvin (4.18.1.0-20230712T075151~focal) ...
Collecting pip
  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
     |████████████████████████████████| 2.1 MB 13.9 MB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pip'. No files were found to uninstall.
  WARNING: The scripts pip, pip3, pip3.11 and pip3.8 are installed in '/usr/local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.2.1
Processing /usr/share/cloudstack-marvin/Marvin-4.18.1.0-20230712T075151.tar.gz
  Preparing metadata (setup.py) ... done
Collecting PyCrypt (from Marvin===4.18.1.0-20230712T075151)
  Downloading pycrypt-0.7.2.tar.gz (47 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.0/48.0 kB 6.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting ddt>=0.4.0 (from Marvin===4.18.1.0-20230712T075151)
  Downloading ddt-1.6.0-py2.py3-none-any.whl (7.1 kB)
Requirement already satisfied: dnspython in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (1.16.0)
Collecting ipmisim>=0.7 (from Marvin===4.18.1.0-20230712T075151)
  Downloading ipmisim-0.10.tar.gz (10 kB)
  Preparing metadata (setup.py) ... done
Collecting kubernetes (from Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for kubernetes from https://files.pythonhosted.org/packages/99/89/3ab0cb3069f49ae2eaf73f884c82164f18f70fcc598e0312edea71614ad7/kubernetes-27.2.0-py2.py3-none-any.whl.metadata
  Downloading kubernetes-27.2.0-py2.py3-none-any.whl.metadata (1.5 kB)
Requirement already satisfied: mysql-connector-python<=8.0.30 in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (8.0.15)
Requirement already satisfied: netaddr>=0.7.14 in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (0.7.19)
Collecting nose>=1.3.3 (from Marvin===4.18.1.0-20230712T075151)
  Downloading nose-1.3.7-py3-none-any.whl (154 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.7/154.7 kB 16.1 MB/s eta 0:00:00
Collecting paramiko>=1.13.0 (from Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for paramiko>=1.13.0 from https://files.pythonhosted.org/packages/bb/8f/3cef65d3fe76e59f320405027d594a0332e44852fef722f0ee4e81e2e7e3/paramiko-3.3.1-py3-none-any.whl.metadata
  Downloading paramiko-3.3.1-py3-none-any.whl.metadata (4.4 kB)
Collecting pytz (from Marvin===4.18.1.0-20230712T075151)
  Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.3/502.3 kB 35.4 MB/s eta 0:00:00
Collecting pyvmomi>=5.5.0 (from Marvin===4.18.1.0-20230712T075151)
  Downloading pyvmomi-8.0.1.0.2.tar.gz (909 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 909.6/909.6 kB 38.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests>=2.2.1 in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (2.22.0)
Collecting retries (from Marvin===4.18.1.0-20230712T075151)
  Downloading retries-1.0.tar.gz (2.5 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: setuptools>=40.3.0 in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (45.2.0)
Requirement already satisfied: urllib3 in /usr/lib/python3/dist-packages (from Marvin===4.18.1.0-20230712T075151) (1.25.8)
Collecting future==0.18.2 (from ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Downloading future-0.18.2.tar.gz (829 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 829.2/829.2 kB 32.8 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pycrypto==2.6.1 (from ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Downloading pycrypto-2.6.1.tar.gz (446 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.2/446.2 kB 36.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pyghmi==1.2.16 (from ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Downloading pyghmi-1.2.16.tar.gz (152 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.4/152.4 kB 17.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: cryptography>=2.1 in /usr/lib/python3/dist-packages (from pyghmi==1.2.16->ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151) (2.8)
Collecting bcrypt>=3.2 (from paramiko>=1.13.0->Marvin===4.18.1.0-20230712T075151)
  Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (593 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 593.7/593.7 kB 26.7 MB/s eta 0:00:00
Collecting cryptography>=2.1 (from pyghmi==1.2.16->ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for cryptography>=2.1 from https://files.pythonhosted.org/packages/46/74/f9eba8c947f57991b5dd5e45797fdc68cc70e444c32e6b952b512d42aba5/cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl.metadata
  Downloading cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl.metadata (5.2 kB)
Collecting pynacl>=1.5 (from paramiko>=1.13.0->Marvin===4.18.1.0-20230712T075151)
  Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 19.4 MB/s eta 0:00:00
Requirement already satisfied: six>=1.7.3 in /usr/lib/python3/dist-packages (from pyvmomi>=5.5.0->Marvin===4.18.1.0-20230712T075151) (1.14.0)
Requirement already satisfied: certifi>=14.05.14 in /usr/lib/python3/dist-packages (from kubernetes->Marvin===4.18.1.0-20230712T075151) (2019.11.28)
Collecting python-dateutil>=2.5.3 (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 16.9 MB/s eta 0:00:00
Collecting pyyaml>=5.4.1 (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for pyyaml>=5.4.1 from https://files.pythonhosted.org/packages/c8/6b/6600ac24725c7388255b2f5add93f91e58a5d7efaf4af244fdbcc11a541b/PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting google-auth>=1.0.1 (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for google-auth>=1.0.1 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
  Downloading websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting requests-oauthlib (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting oauthlib>=3.2.2 (from kubernetes->Marvin===4.18.1.0-20230712T075151)
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 23.3 MB/s eta 0:00:00
Collecting cffi>=1.12 (from cryptography>=2.1->pyghmi==1.2.16->ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Downloading cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 442.7/442.7 kB 35.6 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0 (from google-auth>=1.0.1->kubernetes->Marvin===4.18.1.0-20230712T075151)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth>=1.0.1->kubernetes->Marvin===4.18.1.0-20230712T075151) (0.2.1)
Collecting rsa<5,>=3.1.4 (from google-auth>=1.0.1->kubernetes->Marvin===4.18.1.0-20230712T075151)
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=2.1->pyghmi==1.2.16->ipmisim>=0.7->Marvin===4.18.1.0-20230712T075151)
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 10.3 MB/s eta 0:00:00
Requirement already satisfied: pyasn1>=0.1.3 in /usr/lib/python3/dist-packages (from rsa<5,>=3.1.4->google-auth>=1.0.1->kubernetes->Marvin===4.18.1.0-20230712T075151) (0.4.2)
Downloading paramiko-3.3.1-py3-none-any.whl (224 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 224.8/224.8 kB 26.4 MB/s eta 0:00:00
Downloading kubernetes-27.2.0-py2.py3-none-any.whl (1.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 22.1 MB/s eta 0:00:00
Downloading cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 26.9 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 15.4 MB/s eta 0:00:00
Downloading PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 736.6/736.6 kB 35.2 MB/s eta 0:00:00
Downloading websocket_client-1.6.2-py3-none-any.whl (57 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 4.8 MB/s eta 0:00:00
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Building wheels for collected packages: Marvin, ipmisim, future, pycrypto, pyghmi, pyvmomi, PyCrypt, retries
  Building wheel for Marvin (setup.py) ... done
  Created wheel for Marvin: filename=Marvin-4.18.1.0_20230712T075151-py3-none-any.whl size=1213189 sha256=85228baee4e8d4d2f9ee8415cc4d6aa01c81899e1a64ecc976790204102fb715
  Stored in directory: /root/.cache/pip/wheels/8d/d1/63/f4d49bbdcd419269fb529bd4b84b3fbdbaa66acddb0b5c9078
  WARNING: Built wheel for Marvin is invalid: Metadata 1.2 mandates PEP 440 version, but '4.18.1.0-20230712T075151' is not
  Building wheel for ipmisim (setup.py) ... done
  Created wheel for ipmisim: filename=ipmisim-0.10-py3-none-any.whl size=12071 sha256=6fba4b771a8c0d9de3dae93f0cf6f34ea50e3342f5d4bf1b2154fe1d320724ee
  Stored in directory: /root/.cache/pip/wheels/d6/f7/5c/e2357d607173355771a2d1d9fd50d084b9296cc2fc37ed1736
  Building wheel for future (setup.py) ... done
  Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=cbda37fcd33fc88ca9f12b3d0cd1d1a2e365dce1f4d8b4f6d03ff023bfbe91e4
  Stored in directory: /root/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4
  Building wheel for pycrypto (setup.py) ... done
  Created wheel for pycrypto: filename=pycrypto-2.6.1-cp38-cp38-linux_x86_64.whl size=498474 sha256=cf04d14426005a64494fe03b5baa312a3b46af10944c6c110c9360ce1ef8e67f
  Stored in directory: /root/.cache/pip/wheels/d0/99/d0/0298ea019d63f1d63a0965b9944b719e875f9bd6ffc6dcf293
  Building wheel for pyghmi (setup.py) ... done
  Created wheel for pyghmi: filename=pyghmi-1.2.16-py3-none-any.whl size=163889 sha256=2789fab84ba71c80da5d48205bf474110e00dcb4d6b4d8acf9dd58e105110bc6
  Stored in directory: /root/.cache/pip/wheels/09/5b/17/14c0c21ad744625707d87ff363bc810662b87715d74ee0bb20
  Building wheel for pyvmomi (setup.py) ... done
  Created wheel for pyvmomi: filename=pyvmomi-8.0.1.0.2-py2.py3-none-any.whl size=382040 sha256=32c27f332a91d58f5c5a8b5523f2d16ece68d21e14fbe4caf8c2533416ff54cc
  Stored in directory: /root/.cache/pip/wheels/3c/b4/4e/1054fa7c65aae9064584ca6675557510dc7f52a29629154076
  Building wheel for PyCrypt (setup.py) ... done
  Created wheel for PyCrypt: filename=pycrypt-0.7.2-cp38-cp38-linux_x86_64.whl size=58397 sha256=f8000b8df88da62c11cabe4d8fb02386c6f7c37b780b4a92317a3a6e29be88bc
  Stored in directory: /root/.cache/pip/wheels/94/cb/76/317fbe7e76f2cfc1ed11d70fc814c2462d96769dcd9acb7863
  Building wheel for retries (setup.py) ... done
  Created wheel for retries: filename=retries-1.0-py3-none-any.whl size=2760 sha256=b56d51fb5a64965b325164034d33f05794db24df5527159ea7c9ea2a9f5eab66
  Stored in directory: /root/.cache/pip/wheels/49/6b/26/13f1b7215ad75bf20b9f88ffe7cd5acfe4e5ed10cb3834f0c1
Successfully built ipmisim future pycrypto pyghmi pyvmomi PyCrypt retries
Failed to build Marvin
ERROR: Could not build wheels for Marvin, which is required to install pyproject.toml-based projects
dpkg: error processing package cloudstack-marvin (--configure):
 installed cloudstack-marvin package post-installation script subprocess returned error exit status 1
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for install-info (6.7.0.dfsg.2-5) ...
Errors were encountered while processing:
 cloudstack-marvin

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[weizhouapache]

@loth
can you try with the package for 4.18.1.0-RC1 ?
https://download.cloudstack.org/testing/4.18.1.0-RC20230828T2026/debian/4.18/

[weizhouapache]

@loth
I can reproduce the issue when install a self-built package. It would be related to the version (4.18.1.0-xxxxxx)
official package can be installed successfully.

[soreana]

@weizhouapache It seems that the problem is the - in the package name. Python's latest naming convention no longer allows the use of dashes in package names.

Final release should only contain dot and numeric values.

https://peps.python.org/pep-0440/#final-releases

[weizhouapache]

@weizhouapache It seems that the problem is the - in the package name. Python's latest naming convention no longer allows the use of dashes in package names.

Final release should only contain dot and numeric values.

https://peps.python.org/pep-0440/#final-releases

yes, please try with the official packages
we use centos7 as marvin node, it works fine.

please feel free to create a fix

[soreana]

@weizhouapache I've managed to install Marvin on CentOS 8, but I've had additional challenges such as the absence of the Rust compiler and some missing dependencies that were not listed in the 'required.txt' file of pip. It requires some work to be rock solid.

[shwstppr]

@DaanHoogland will you be working on this?

[DaanHoogland]

@shwstppr , not sure if work is really needed, but I'll investigate

[DaanHoogland]

@soreana @loth , I use fedora (38?) as my dev box and have no issues installing marvin 4.18.2 or 4.19 over each other. I am trying a ubuntu 22 lab box now to see if I encounter your issues. I think at most it will be a matter of prerequisites though.