ci: automate releases with semantic-release#171
Open
nielspardon wants to merge 5 commits into
Open
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
Member
Author
|
I did create a new |
nielspardon
requested review from
andrew-coleman,
benbellick,
bestbeforetoday,
tokoko and
vbarua
Adopt the semantic-release model used by other Substrait repos (e.g. substrait-java) to drive releases from Conventional Commits. A new scheduled (weekly) + manual `Semantic Release` workflow runs semantic-release under a GitHub App token: it computes the next version, updates CHANGELOG.md, creates the vX.Y.Z tag, and publishes a GitHub Release with generated notes. Because substrait-python derives its version from the git tag via setuptools_scm and publishes to PyPI via Trusted Publishing (OIDC), the publish step stays in the existing release.yml (filename preserved for the PyPI trusted publisher). It is repurposed to trigger on the tag pushed by semantic-release, dropping the now-duplicate GitHub Release job. The App token ensures the tag push triggers this workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nielspardon
force-pushed
the
ci/automated-semantic-release
branch
from
0cb4daf to
bdef034
Compare
bvolpato
reviewed
Jul 13, 2026
bvolpato
reviewed
Jul 13, 2026
Address review feedback on the semantic-release automation: - release.yml: restore the job that attaches the wheel/sdist to the GitHub release. semantic-release now creates the release, so the job waits for it to appear and uploads onto it (omitting the body so the generated notes are preserved), and PyPI publish depends on it. - Pin the semantic-release toolchain in run.sh -- a floating conventional-changelog preset silently dropped the Features/Bug Fixes sections from the generated notes. - Move the release config to .releaserc.mjs so the side-effecting github/git plugins are omitted unless RELEASE_DRY_RUN=false, enabling a credential-free dry run. - Add ci/release/dry_run.sh and a release-notes-check workflow that run the pinned toolchain against synthetic history and assert the notes contain the expected sections.
The Release Notes Check failed in CI: under GitHub Actions, semantic-release's env-ci resolved GITHUB_REF=refs/pull/N/merge to a PR build, decided the branch was not "main", and computed no release -- so the notes were empty and the section assertion failed. Clear the Actions env markers in dry_run.sh so env-ci falls back to the synthetic repo's own main branch, as in a local run.
Rework dry_run.sh to mirror substrait-java: run the pinned semantic-release in dry-run mode against the actual repository history in a throwaway worktree, instead of seeding a temp repo with synthetic feat/fix commits. When a release is due it still asserts the generated notes carry sections (guarding the PR substrait-io#171 regression); when nothing is releasable it passes without asserting.
Fold in the release-notes trailer stripping from the substrait spec repo (also in the substrait-java PR). The conventional-commits parser ends a BREAKING CHANGE note only at a reference or another note keyword, so a trailing Signed-off-by / Co-authored-by trailer gets absorbed into the note text. Add a release-notes-generator writerOpts.transform that drops trailing git trailer lines, which requires importing the conventionalcommits preset -- hence the loadPreset resolver for the npx-installed package.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Automate releases with
semantic-release, matching the
model used by other Substrait repos such as substrait-java.
How it fits together
A new scheduled (weekly) + manual
Semantic Releaseworkflow runssemantic-release under a GitHub App token. It analyzes Conventional
Commits, computes the next
vX.Y.Z, updatesCHANGELOG.md, pushes thetag, and creates a GitHub Release with generated notes.
The existing
release.ymlkeeps its filename (the PyPI trusted-publisherbinding depends on it) and is repurposed to trigger on that tag: it
builds the package, attaches the wheel/sdist to the release that
semantic-release created, and publishes to PyPI via Trusted Publishing.
This is a hybrid of the java model rather than a literal copy, because
substrait-python derives its version from the git tag via setuptools_scm
(no version file to bump) and publishes via Trusted Publishing / OIDC,
which must run in
pypa/gh-action-pypi-publishrather than a shellpublishCmd.Changes
ci/release/run.sh— runssemantic-release --civia npx with apinned toolchain (a floating conventional-changelog preset silently
dropped the Features/Bug Fixes sections from the notes).
.releaserc.mjs— conventionalcommits preset on main, breaking tominor (pre-1.0, matching java), changelog + github + git plugins. An
ESM config so the side-effecting github/git plugins can be gated behind
RELEASE_DRY_RUN=false(allowing a credential-free dry run) and so therelease-notes-generator can strip git trailers (
Signed-off-by, etc.)the parser would otherwise fold into BREAKING CHANGE notes.
ci/release/dry_run.sh+.github/workflows/release-notes-check.yml— a credential-free dry run against the real history (in a throwaway
worktree) that fails if a release is due but its notes come out with no
sections, guarding against a toolchain regression.
.github/workflows/semantic-release.yml— scheduled + manual,runs under a GitHub App token so the pushed tag triggers release.yml.
.github/workflows/release.yml— tags-only trigger; agithub-releasejob waits for the semantic-release release to appearand attaches the wheel/sdist to it (leaving the generated notes
intact), and PyPI publish depends on that job.
RELEASING.md— documents the automated flow and therelease-notes guard.
Required before this works
installed on this repo with push and branch-protection-bypass rights.
and environment pypi (preserved by this PR).
Verification done
ci/release/run.shandci/release/dry_run.sh— cleanci/release/dry_run.shagainst real history (last tagv0.29.0→0.30.0): the notes render the expected Features / Bug Fixes sectionswith
Signed-off-by/Co-authored-bytrailers produces clean notesCloses #147