To generate all of the st2 wheels, run this from the root of the StackStorm/st2 repo:
Here, :: is like a recursive glob so that pants will package everything it knows about (based on BUILD metadata).
To build a subset of the wheels, replace :: with the path where it is defined. To build just st2common and the orquesta_runner, do:
pants package st2common contrib/runners/orquesta_runner
Pants puts all packaged artifacts under the dist/ directory.
The package goal
The package goal (where goals refer to the thing(s) you want to do with pants: pants <goal(s)> <path(s)>) documentation is here:
https://www.pantsbuild.org/2.18/docs/python/goals/package
The package goal makes various kinds of installable or distributable artifact. The only packageable artifacts in st2 are python_distribution artifacts, which includes sdists and wheels. In particular, it will generate wheels for all the components (defined in st2*/ like st2commom and st2api) and the runners (defined in contrib/runners/*_runner) when given the :: recursive target glob.
python_distribution
We use python_distribution in BUILD metadata to tell pants about each wheel we want it to generate. Documentation about building python distributions is here:
https://www.pantsbuild.org/2.18/docs/python/overview/building-distributions
Defining this BUILD metadata for st2 was the focus of these PRs:
Dependencies defined in other PRs also have an affect on what gets included in the wheel.
Testing the wheels
Open or install the sdists and/or wheels. Possible things to check:
- Is everything in the wheel or sdist that should be there? Python? conf files?
- Are there any files in the wrong place?
- Is the metadata sane?
- Pants generates setup.py: does it look sane?
- Does it install?
- Anything else you can think of to inspect.
As you're looking at it, what kinds of tests would be good for the wheels? Maybe even write some tests for pants to run via pytest (make the python_test target depend on the relevant python_distribution target to get the wheel in the test sandbox.
What to do when the wheel is missing something
When something is missing from a wheel, look for something else that is included in the wheel that might need a manual dependency on whatever is missing. If there is nothing obvious inside the wheel that needs the dependency (eg because it is a public API in st2common) then add the dependency to the python_distribution metadata.
What to do when files are in the wrong place in the wheel
I suspect this happens when the files are owned by a resources or files target when they should probably be the other one. We might also need to use relocated_files to move them similar to how I added LICENSE to all the wheels.
To generate all of the st2 wheels, run this from the root of the StackStorm/st2 repo:
Here,
::is like a recursive glob so that pants will package everything it knows about (based on BUILD metadata).To build a subset of the wheels, replace
::with the path where it is defined. To build just st2common and theorquesta_runner, do:Pants puts all packaged artifacts under the
dist/directory.The
packagegoalThe package goal (where goals refer to the thing(s) you want to do with pants:
pants <goal(s)> <path(s)>) documentation is here:https://www.pantsbuild.org/2.18/docs/python/goals/package
The package goal makes various kinds of installable or distributable artifact. The only packageable artifacts in st2 are
python_distributionartifacts, which includes sdists and wheels. In particular, it will generate wheels for all the components (defined inst2*/likest2commomandst2api) and the runners (defined incontrib/runners/*_runner) when given the::recursive target glob.python_distributionWe use
python_distributionin BUILD metadata to tell pants about each wheel we want it to generate. Documentation about building python distributions is here:https://www.pantsbuild.org/2.18/docs/python/overview/building-distributions
Defining this BUILD metadata for st2 was the focus of these PRs:
st2_*_python_distribution()macros topants-plugins/macros.pyst2#5901python_distributionBUILD metadata for runners st2#5907python_distributionBUILD metadata for st2* components st2#5909python_distributions(except st2common/st2tests) st2#5928python_distributionsst2#5929Dependencies defined in other PRs also have an affect on what gets included in the wheel.
Testing the wheels
Open or install the sdists and/or wheels. Possible things to check:
As you're looking at it, what kinds of tests would be good for the wheels? Maybe even write some tests for pants to run via pytest (make the
python_testtarget depend on the relevantpython_distributiontarget to get the wheel in the test sandbox.What to do when the wheel is missing something
When something is missing from a wheel, look for something else that is included in the wheel that might need a manual dependency on whatever is missing. If there is nothing obvious inside the wheel that needs the dependency (eg because it is a public API in
st2common) then add the dependency to thepython_distributionmetadata.What to do when files are in the wrong place in the wheel
I suspect this happens when the files are owned by a
resourcesorfilestarget when they should probably be the other one. We might also need to userelocated_filesto move them similar to how I addedLICENSEto all the wheels.