Ship COMPAS C++ through PyPI with bundled runtime (pip install compas-popsynth) on Linux + Macs#1473
Conversation
add CI runner
|
One thing to note: right now we have two CI jobs that are doing the same thing (building compas) This is in my newly created I think itd be better to have one signle linux ci builder. The CI would then do:
I can either do this now, or in a future PR. |
ilyamandel
left a comment
There was a problem hiding this comment.
Looks amazing, @avivajpeyi !
@jeffriley , would you like to take a look as well?
|
@avivajpeyi and I discussed this as Avi was building it. I like it. I think there are some caveats - esp wrt what OS versions we can support, delivering the correct library versions etc. (i.e. if COMPAS requires a specific version of (say) boost, and not the latest version), but as long as those are known and accepted by the user then all good. It adds an alternative to docker that will be useful for some users. Make it clear that this only delivers the executable and whatever dependencies required to run the executable - no source. So fine for users that just want to run COMPAS, not so much if they want to make speculative changes to the physics. EDIT: architectures too - the executable delivered will contain architecture-specific machine code, so won't be compatible with all systems. I'm happy for this to be merged - it adds functionality and doesn't subtract what's already available. |
|
so shall we merge this in? Are there any other things that we need before merging this in? (one future TODO: we need to make a |
|
Merge away...
Dr Jeff Riley
Monash University, Australia
Ph: +61 (0)407 811 282
***@***.***
…On Thu, 9 Apr 2026, 4:16 am Avi Vajpeyi, ***@***.***> wrote:
*avivajpeyi* left a comment (TeamCOMPAS/COMPAS#1473)
<#1473 (comment)>
so shall we merge this in? Are there any other things that we need before
merging this in?
—
Reply to this email directly, view it on GitHub
<#1473 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGM5XNQV5HKCUTY27EKLA634U2JRVAVCNFSM6AAAAACXMAIE4OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DEMBYGQ3TAOBRG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|

This PR adds a PyPI install path for COMPAS that bundles the native C++ executable with the Python package, so users can install and run COMPAS via
pip(something mike, reinhold and I have been dreaming of since '22!)See demo on google colab:

And demo on OzStar:


And on my macbook:

What does this work on?
Works on x86_64 glibc-based Linux systems supported by the wheel’s manylinux tag (ie normal pip installs on Ubuntu/Debian/RHEL/Fedora-class systems).
Also should work on macOS 14+ for both Apple Silicon and Intel Macs.
Because the wheel bundles the COMPAS binary plus Boost/GSL/HDF5, users should not need to separately install those libraries. That is handled by the bundle/launcher logic in package-linux-bundle.sh and run_compas.sh
ATM It is not targeting: arm64 Linux, Alpine / musl Linux, Windows
Those will need more work...
I wonder if the current platforms will suffice for now.
Main changes:
compas_runas the packaged launcher entrypoint (had to change setup.py for this)Some details on changes:
1. CPP ?= g++ in Makefile:
So CI or callers can override the compiler without editing the file.
for example:
Linux bundle build uses CPP="ccache g++" or CPP=g++
mac scaffolding uses CPP=clang++
2. Setup.py and init
I had to add a CLI entrypoint for the COMPAS exe so once someone installs the compas-python, one can actually run it.
I also had to add an email in init as that is required for python packages.
Finally, we needed some extra lines in the setup.py to link the built Cpp with python.
3. cicd scripts
I had to update the builder for mac and linux, initially made by ADACS teams for the web-compas tool.
I also had to add some additional scripts to actuall "bundle" the COMPAS exes.
How to deploy new python build?
Right now, its still a manual process.
Everytime there is a commit to dev, the linux/mac wheels will be built
But to "ship" the python release associated with new compas CPP, a user will have to bump the python version and click on the pypi-release action.
This is temporary while we are still developing/deciding if we want this feature.
Later, we can "ship" the new python release jointly with every new COMPAS release tag, so the packaged Python release can track the underlying COMPAS version instead of using a separate manually bumped Python version.
Eg when the new tag is made, we will make a new release for the python version as well.