manylinux wheels with Github Actions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

manylinux wheels with Github Actions

Robert McLeod
Everyone,

I'm looking to move `numexpr` into GitHub Actions for the combination of testing but also building wheels for deployment to pypi. I've never really been satisfied with the Appveyor/Travis combo and the documentation around Actions seems to be a lot stronger. I thought I might ask for advice on the mailing list before I jump in.

Does anyone here know of good, working examples of such for scientific packages that I could crib as a guide? I would want/need to make use of manylinux1 Docker images for building Linux wheels.

https://github.com/pypa/manylinux

This is an example recipe that builds Cython using a custom Docker image. It's a nice starting point but it would be preferable to use the official pypa images.

https://github.com/marketplace/actions/python-wheels-manylinux-build

Kivy is a working example with a pretty complete test and build setup (although it's a bit over complicated for my purposes):

https://github.com/kivy/kivy/tree/master/.github/workflows

Anyone have any experiences to share with test and deploy via Actions?

Robert


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: manylinux wheels with Github Actions

Eric Larson
I have had good experiences for about a year now using CIBUILDWHEEL on Azure for VisPy and on python-rtmixer to deploy Linux, Windows, and macOS wheels to PyPi automatically when a release is tagged. It wasn't difficult to set up rtmixer after David Hoese did the heavy lifting sorting out everything with VisPy.

I haven't used other methods so I can't really offer any comparison, but the overhead for those two fairly simple (from a build-and-deploy perspective) projects at least has seemed pretty low.

My 2c,
Eric

On Mon, Nov 16, 2020 at 12:27 PM Robert McLeod <[hidden email]> wrote:
Everyone,

I'm looking to move `numexpr` into GitHub Actions for the combination of testing but also building wheels for deployment to pypi. I've never really been satisfied with the Appveyor/Travis combo and the documentation around Actions seems to be a lot stronger. I thought I might ask for advice on the mailing list before I jump in.

Does anyone here know of good, working examples of such for scientific packages that I could crib as a guide? I would want/need to make use of manylinux1 Docker images for building Linux wheels.

https://github.com/pypa/manylinux

This is an example recipe that builds Cython using a custom Docker image. It's a nice starting point but it would be preferable to use the official pypa images.

https://github.com/marketplace/actions/python-wheels-manylinux-build

Kivy is a working example with a pretty complete test and build setup (although it's a bit over complicated for my purposes):

https://github.com/kivy/kivy/tree/master/.github/workflows

Anyone have any experiences to share with test and deploy via Actions?

Robert

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: manylinux wheels with Github Actions

Andrew Nelson-6
In reply to this post by Robert McLeod
For my project (refnx) I solely use GH Actions to test and make wheels. In my workflow (https://github.com/refnx/refnx/blob/master/.github/workflows/pythonpackage.yml) I make a 3.7/3.8/3.9 matrix across Linux/macOS/Windows. First of all I make the wheels for all those combos, then install the wheels, then run tests on the installed wheel (thereby checking that the libraries are all delocated nicely). The final step in the workflow is uploading the wheel artefacts (stored somewhere on GH).
To make the Linux 2010/2014 wheels I use PyPA docker images (https://github.com/refnx/refnx/blob/master/.github/workflows/pythonpackage.yml#L70). I specify what platform I want to make wheels for, as an environment variable that is used to select the correct docker image ["manylinux2010_x86_64", "manylinux2014_x86_64"]. There are other images available. The docker image runs a script to make all the wheels (https://github.com/refnx/refnx/blob/master/tools/build_manylinux_wheels.sh) and put them in a wheelhouse, that is then uploaded as an artefact along with the macOS and windows wheels.

I decided to make the deployment step to PyPI a manual process, just so I can check that everything went ok. When I've just made a release tag I download the artefacts corresponding to the release commit, then upload them from my personal PC.

It all works really well.

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: manylinux wheels with Github Actions

Thomas Caswell
Matplotlib has also migrated to building wheels via github actions and it has been working well.

Tom

On Mon, Nov 16, 2020, 17:48 Andrew Nelson <[hidden email]> wrote:
For my project (refnx) I solely use GH Actions to test and make wheels. In my workflow (https://github.com/refnx/refnx/blob/master/.github/workflows/pythonpackage.yml) I make a 3.7/3.8/3.9 matrix across Linux/macOS/Windows. First of all I make the wheels for all those combos, then install the wheels, then run tests on the installed wheel (thereby checking that the libraries are all delocated nicely). The final step in the workflow is uploading the wheel artefacts (stored somewhere on GH).
To make the Linux 2010/2014 wheels I use PyPA docker images (https://github.com/refnx/refnx/blob/master/.github/workflows/pythonpackage.yml#L70). I specify what platform I want to make wheels for, as an environment variable that is used to select the correct docker image ["manylinux2010_x86_64", "manylinux2014_x86_64"]. There are other images available. The docker image runs a script to make all the wheels (https://github.com/refnx/refnx/blob/master/tools/build_manylinux_wheels.sh) and put them in a wheelhouse, that is then uploaded as an artefact along with the macOS and windows wheels.

I decided to make the deployment step to PyPI a manual process, just so I can check that everything went ok. When I've just made a release tag I download the artefacts corresponding to the release commit, then upload them from my personal PC.

It all works really well.
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion