feat: Python version matrix for Docker images + error-masking fix#76
feat: Python version matrix for Docker images + error-masking fix#76
Conversation
Parameterize all 4 Dockerfiles with PYTHON_VERSION build arg. GPU images also accept PYTORCH_BASE to select the correct PyTorch base image per Python version. Add versioned Makefile targets: - build-all-versioned: builds 10 images (GPU 3.11/3.12, CPU 3.10-3.12) - build-wip-versioned: multi-platform push with latest alias - smoketest-versioned: verify Python version in each image GPU base image mapping: - Python 3.11: pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime - Python 3.12: pytorch/pytorch:2.10.0-cuda12.8-cudnn9-runtime
Previously, _load_generated_handler() silently returned None on any failure (missing file, import error, syntax error), causing deployed endpoints to fall back to the FunctionRequest/Live Serverless handler. This masked real deployment issues like Python version mismatches. Now deployed mode (FLASH_RESOURCE_NAME set) treats handler loading failures as fatal RuntimeError. Live Serverless mode skips the generated handler entirely since it only uses FunctionRequest protocol.
Tests expected None returns but handler.py now raises RuntimeError in deployed mode. Updated all 8 TestLoadGeneratedHandler tests to use pytest.raises(RuntimeError). Also synced uv.lock to pick up latest runpod-flash version.
There was a problem hiding this comment.
Pull request overview
Adds a Python-versioned Docker image matrix (CPU/GPU, QB/LB) to prevent runtime Python mismatches with binary deps, and makes deployed endpoints fail fast when the generated handler can’t be loaded (instead of silently falling back to the Live Serverless protocol).
Changes:
- Parameterizes Docker builds via
PYTHON_VERSION/PYTORCH_BASEbuild args and introduces versioned image tags (e.g.,:py3.11-...). - Adds Makefile targets to build/push/smoketest the multi-version image matrix.
- Updates generated-handler loading behavior to raise in deployed mode, with corresponding unit test updates.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
Makefile |
Adds Python version matrix variables and versioned build/push/smoketest targets. |
Dockerfile |
Switches GPU base image selection to PYTORCH_BASE arg (and adds PYTHON_VERSION arg). |
Dockerfile-lb |
Same as GPU QB Dockerfile, but for LB image. |
Dockerfile-cpu |
Parameterizes CPU base image via python:${PYTHON_VERSION}-slim. |
Dockerfile-lb-cpu |
Parameterizes CPU LB base image via python:${PYTHON_VERSION}-slim. |
src/handler.py |
Makes deployed-mode handler loading failures fatal; keeps Live Serverless path separate. |
tests/unit/test_handler.py |
Updates _load_generated_handler tests to expect RuntimeError instead of None. |
uv.lock |
Updates lockfile with multiple dependency version bumps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Include platform.python_version() in the worker boot banner for runtime version visibility during E2E testing.
- Add build-time Python version validation to GPU Dockerfiles - Restructure build-all-versioned to run setup once via internal targets - Add version assertion to smoketest-versioned (fail on mismatch)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
tests/unit/test_handler.py:179
test_loads_generated_handler_from_filepatcheshandler.Pathto return atmp_pathfile, but_load_generated_handler()enforceshandler_file.resolve().is_relative_to(Path('/app').resolve()). Sincetmp_pathis not under/app, this test will raise the "resolves outside /app" RuntimeError instead of returning a loaded handler. Adjust the test to bypass the/appsandbox check (e.g., mockresolve()/is_relative_toor patch the/approot used by the function) or refactor_load_generated_handlerto use a patchable constant for the app root.
def test_loads_generated_handler_from_file(self, tmp_path):
"""With valid generated handler file, loads and returns handler function."""
handler_file = tmp_path / "handler_gpu_config.py"
handler_file.write_text(
"async def handler(event):\n"
" return {'result': event.get('input', {}).get('prompt', 'default')}\n"
)
with patch.dict("os.environ", {"FLASH_RESOURCE_NAME": "gpu_config"}):
with patch("handler.Path", return_value=handler_file):
result = _load_generated_handler()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- Add default case to pytorch_base() shell function in all Makefile targets - Guard test_handler.py import against FLASH_RESOURCE_NAME env var
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
…ng running interpreter GPU Dockerfiles symlinked /usr/local/bin/python to /usr/bin/python3.X, switching to the system Python which lacks torch and other base image packages. Removed the symlink to preserve the pytorch base image's environment. Changed dependency_installer to use sys.executable instead of --system so runtime package installs go into the same site-packages as torch.
- CI matrix builds py3.10/3.11/3.12 for all image types - Removed per-version pytorch base mapping (single runpod/pytorch image) - GPU/LB builds amd64-only (pytorch base has no arm64 manifest) - Added --break-system-packages to CPU Dockerfiles - Auto-detect local Python version in Makefile for build-wip - Bumped runpod-flash dependency to >=1.7.0 - Removed .python-version (version comes from build args)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
GPU workers are pinned to Python 3.12 (torch/CUDA only installed for 3.12 in base image). CPU workers support 3.10-3.12. Added base image Python layout details to architecture doc.
- GPU Dockerfiles: remove PYTHON_VERSION ARG (base image is 3.12), add numpy install (excluded from tarballs by flash build) - CPU Dockerfiles: default PYTHON_VERSION from 3.11 to 3.12 - Makefile: GPU_PYTHON_VERSIONS reduced to 3.12 only, remove --build-arg PYTHON_VERSION from GPU/LB build targets
PYTHON_VERSION was hardcoded to 3.12 default, ignoring the user's local Python. Restore auto-detection so `make build-wip` with Python 3.10 produces py3.10-wip CPU images as expected. GPU images remain fixed at 3.12.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 16 changed files in this pull request and generated 9 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
…afety - Fix CI matrix duplication: use include-only for GPU jobs (3.12 only) - Remove duplicate python-version list from CPU job matrices - Add Python version validation to GPU Dockerfiles - Use python -m pip instead of bare pip in Dockerfiles - Use sys.executable for pip fallback in dependency_installer.py - Update docs to match uv pip --python sys.executable implementation - Correct image count comment in Makefile (8 not 10)
Summary
runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2204ships 3.12 only -- not parameterizable)PYTHON_VERSIONbuild arg for 3.10, 3.11, 3.12build-*-versioned,build-wip-versioned,smoketest-versionedtargetspython -m pipinstead of barepipin Dockerfiles for interpreter safetysys.executable -m pipin dependency_installer.py fallback for correct interpreter targetingContext
AE-2391: Python version mismatch between user build environment and worker runtime causes silent failures with binary packages (numpy, etc). This PR adds versioned Docker images so the SDK can select the correct runtime for the user's Python version.
GPU images are 3.12 only because the PyTorch base image ships a single Python version. CPU images use
python:X.Y-slimbases which are version-parameterizable.Companion PR: runpod/flash#261 (SDK-side version detection and image selection)
Test plan
make build-all-versionedbuilds all 8 images locallymake smoketest-versionedverifies correct Python version in each imagemake testpasses unchanged