Skip to content

Fix SageMaker processing stopped state handling#67291

Open
Codingaditya17 wants to merge 1 commit into
apache:mainfrom
Codingaditya17:fix-sagemaker-processing-stopped-state
Open

Fix SageMaker processing stopped state handling#67291
Codingaditya17 wants to merge 1 commit into
apache:mainfrom
Codingaditya17:fix-sagemaker-processing-stopped-state

Conversation

@Codingaditya17
Copy link
Copy Markdown

This PR fixes SageMakerProcessingOperator so that Processing jobs ending in the Stopped state are treated as failed terminal states when deferrable=True and wait_for_completion=True.

The operator was checking the generic SageMaker failed_states, which does not include Stopped. This updates it to use processing_job_failed_states and adds a unit test for the Stopped state before deferral.

Closes #67281

Tests:

  • Not run locally: uv test environment failed while building jpype1 because Java runtime is not installed on my machine.

@Codingaditya17 Codingaditya17 requested a review from o-nikolas as a code owner May 21, 2026 16:35
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels May 21, 2026
@Codingaditya17 Codingaditya17 force-pushed the fix-sagemaker-processing-stopped-state branch from 84eff90 to ca962aa Compare May 21, 2026 16:39
@eladkal eladkal requested a review from vincbeck May 21, 2026 17:10
@Codingaditya17
Copy link
Copy Markdown
Author

Thanks for the review. The only failing check appears to be Integration core otel, and the visible log only shows Process completed with exit code 1 without a traceback or failure related to the SageMaker provider changes in this PR.

This PR only touches SageMakerProcessingOperator and its unit test. Could a maintainer please re-run the failed integration job if possible?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SageMakerProcessingOperator would not terminate if Sagemaker processing job ends in the Stopped state

2 participants