[SDK Sentinel] Remove flaky Gemini replay workflow deadline - #1816
Merged
Conversation
brianstrauch
approved these changes
Sep 4, 2026
brianstrauch
enabled auto-merge (squash)
September 4, 2026 20:29
This was referenced Sep 10, 2026
DABH
added a commit
that referenced
this pull request
Sep 10, 2026
Several openai_agents tests started workflows with a 5-10s execution timeout nested inside pytest-timeout's 60s bound. On a loaded macOS CI runner the first workflow task was occasionally delayed past the default 10s workflow task timeout; the retried task then ran after the workflow's own execution timeout had already fired, so test_run_config_models failed with 'TimeoutError: Workflow timed out' and test_exception_handling found no activity in history ('assert found'). Drop those deadlines, as #1816 did for test_replay_tool_loop; pytest-timeout still bounds real hangs. Also assert the failure cause in assert_status_retry_behavior so an unexpected failure mode is reported directly.
tconley1428
pushed a commit
that referenced
this pull request
Sep 10, 2026
…ts tests (#1835) * Remove redundant workflow deadlines from google_genai tests The google_genai integration tests nested 10-30s workflow execution timeouts inside pytest-timeout's 60s bound. On a stalled macOS CI runner the deadline expired before the worker processed the next workflow task, so healthy workflows failed with "Workflow timed out" (test_mcp_side_effects, test_mcp_full_schema_propagation, test_file_upload_str_path, test_interaction_lifecycle, test_agents_crud). Drop the deadlines, as #1816 did for test_replay_tool_loop; pytest-timeout still bounds real hangs. test_register_files_without_credentials_fails relied on the 10s deadline to cut off activity retries (the server caps schedule-to-close at the run timeout), so its workflow now disables retries for the permanent missing-credentials error instead of depending on deadline arithmetic. * Remove short workflow deadlines from openai_agents tests Several openai_agents tests started workflows with a 5-10s execution timeout nested inside pytest-timeout's 60s bound. On a loaded macOS CI runner the first workflow task was occasionally delayed past the default 10s workflow task timeout; the retried task then ran after the workflow's own execution timeout had already fired, so test_run_config_models failed with 'TimeoutError: Workflow timed out' and test_exception_handling found no activity in history ('assert found'). Drop those deadlines, as #1816 did for test_replay_tool_loop; pytest-timeout still bounds real hangs. Also assert the failure cause in assert_status_retry_behavior so an unexpected failure mode is reported directly. * Keep the plugin's 60s activity timeout in RegisterFilesWorkflow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Caution
This PR contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.
Summary
Prevents the Gemini replay test from failing when a loaded runner exceeds its internal 15-second workflow deadline.
Root cause
The macOS ARM suite [passed before the onset](https://github.com/temporalio/sdk-python/actions/runs/33667580176/job/100373011391), then [timed out](https://github.com/temporalio/sdk-python/actions/runs/33676556946/job/100402620014), with a later [target-branch recurrence](https://github.com/temporalio/sdk-python/actions/runs/33777018495/job/100721154678). The test nested a 15-second workflow execution timeout inside pytest's independent 60-second timeout, so runner load could time out a healthy three-activity workflow before replay began.
Fix
Remove the redundant workflow deadline. Pytest still bounds genuine hangs at 60 seconds, while ordinary load no longer changes workflow outcome.
Validation
Five focused runs passed before and after the edit on Linux Python 3.14; the natural failure did not reproduce. The unchanged baseline passed canonical validation. Candidate lint and type checks passed, but bridge lint could not start because this environment lacks a configured Rust toolchain. macOS ARM CI and full canonical validation remain required.
Validation status:
validation-incompletehighmediummediumstandard— 1 changed file; 652 Bnot-reproduced— Ten focused Linux executions passed across pre-edit and post-edit states; macOS ARM was unavailable.pass-after-only— The existing test passed five times after the candidate, but no deterministic fail-before reproduction was obtained.Poe lint, bridge lint, and editable build) — passedPoe lint, bridge lint, and editable build) — passedInvestigator-run checks
poe test -s -k test_replay_tool_loop --flake-finder --flake-runs=5— passed; 10 attempt(s). Five pre-edit and five post-edit Linux Python 3.14 executions passed; the natural failure did not reproduce.poe test -s -k test_worker_with_worker_deployment_config— passed; 1 attempt(s). The separate known deployment cancellation did not reproduce locally..ci-flake-runtime/input/validate.sh— passed; 1 attempt(s). Trusted unchanged baseline validation passed..ci-flake-runtime/input/validate.sh— failed; 1 attempt(s). Candidate formatting, import, and type checks passed; bridge lint could not start because rustup had no configured toolchain.Required target CI
Residual risks
Automation provenance
f937aed1d4a1b5f3ed5b369636f9608d876898eeIf this finding should not be fixed, apply the
sdk-sentinel:false-positivelabel and close the PR. Sentinel will suppress the finding until its affected source changes.SDK Sentinel recurrence history
2026-09-03 16:29 UTC2026-09-03 16:29 UTC