Skip to content

Add binary integration testing support to GitHub Actions workflow#8693

Merged
roger-zhangg merged 24 commits intoaws:developfrom
roger-zhangg:binary-test
Feb 26, 2026
Merged

Add binary integration testing support to GitHub Actions workflow#8693
roger-zhangg merged 24 commits intoaws:developfrom
roger-zhangg:binary-test

Conversation

@roger-zhangg
Copy link
Member

Changes

  • Add a second schedule trigger (4am PST / 12pm UTC daily) that tests against the SAM CLI nightly binary instead of source
  • Add workflow_dispatch input install_mode with options: code (default), binary (nightly), latest-release
  • Add Makefile targets: init-nightly, init-latest-release, setup-pytest
  • Add scripts/install-sam-cli-binary.sh for cross-platform (Linux/macOS) SAM CLI binary installation
  • Use gh release download instead of raw curl for GitHub release assets
  • Unset SAM_CLI_DEV in binary flows so tests use sam instead of samdev
  • Create /usr/local/bin/zig wrapper for SAM CLI's cargo-lambda Rust builds
  • Use pytest venv python for setup/reset scripts when available

Testing

Tested via workflow_dispatch with install_mode=binary on the binary-test branch.

@roger-zhangg roger-zhangg requested a review from a team as a code owner February 25, 2026 22:22
@roger-zhangg
Copy link
Member Author

valerena
valerena previously approved these changes Feb 26, 2026
Copy link
Contributor

@valerena valerena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor comments, but it looks good

Comment on lines 57 to 59
# Nightly installs as sam-nightly; rename to sam
if [ -f /usr/local/bin/sam-nightly ]; then
sudo mv /usr/local/bin/sam-nightly /usr/local/bin/sam
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but you could move this if to outside of the case

Copy link
Member Author

@roger-zhangg roger-zhangg Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the first comment, I think it make more sense to actually install sam cli in a venv for normal integ tests. So we get a clearer env and consistency accross the tests. I will make that change

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yes, the goal is to avoid "making a test wrongly succeed"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it's not super straightforward to do, the main blocker as "make pr" if we install sam cli in a venv, we need to use that venv to run all the make PR steps and eventually we need to run pytest in the venv as well. It's basically the same as running in system python directly. Unless we create a new workflow "make init-inte-test" but this makes test confusing. Probably better to stick to just make init & install in the system.

@roger-zhangg
Copy link
Member Author

Updated a minor case: our nightly release will not publish, it keeps editing an old release so need to listen on edit event.

Copy link
Contributor

@valerena valerena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new changes look good!

@roger-zhangg roger-zhangg added this pull request to the merge queue Feb 26, 2026
Merged via the queue into aws:develop with commit caf6ac9 Feb 26, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants