Skip to content

Fail viable/strict update when no green commit found #16650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mergennachin merged 6 commits into from
Jan 20, 2026
Merged

Fail viable/strict update when no green commit found #16650

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ad5f97d
Fail viable/strict update when no green commit found
mergennachin Jan 16, 2026
3853086
Also include cancelled jobs in viable/strict failure summary
mergennachin Jan 16, 2026
2b6260c
Improve viable/strict failure summary to show commits between branches
mergennachin Jan 16, 2026
377f503
Address code review feedback for viable/strict summary
mergennachin Jan 16, 2026
768cfa3
Use ASCII unit separator for robust field parsing
mergennachin Jan 16, 2026
427c297
Fix jq syntax and simplify workflow
mergennachin Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .github/workflows/update-viablestrict.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ jobs:
environment: ${{ (github.event_name == 'schedule') && 'update-viable-strict' || '' }}
steps:
- name: Update viable/strict
id: update
uses: pytorch/test-infra/.github/actions/update-viablestrict@main
with:
repository: pytorch/executorch
Expand All @@ -25,3 +26,93 @@ jobs:
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-username: ${{ secrets.CLICKHOUSE_VIABLESTRICT_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_VIABLESTRICT_PASSWORD }}

- name: Summarize CI failures
if: steps.update.outputs.latest_viable_sha == 'None'
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The string comparison == 'None' suggests the output is the literal string 'None' rather than an empty string or null. This is an unusual pattern for GitHub Actions outputs. Consider verifying this matches the actual output format from the upstream action, or use a more conventional empty check like == '' if appropriate.

Suggested change
if: steps.update.outputs.latest_viable_sha == 'None'
if: steps.update.outputs.latest_viable_sha == ''

Copilot uses AI. Check for mistakes.
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "## ❌ No green commit found" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "viable/strict branch was not updated because no commit passed all required checks." >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY

echo "### Failures by commit (recent)" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY

# Get commits between viable/strict and main using GitHub API
VIABLE_SHA=$(gh api repos/pytorch/executorch/git/ref/heads/viable/strict --jq '.object.sha' 2>/dev/null || echo "")
MAIN_SHA=$(gh api repos/pytorch/executorch/git/ref/heads/main --jq '.object.sha' 2>/dev/null || echo "")

found_failures=false

if [ -n "$VIABLE_SHA" ] && [ -n "$MAIN_SHA" ]; then
# Get commits between viable/strict and main (up to 20)
# Use ASCII unit separator (0x1f) to avoid issues with special chars in names
COMMITS=$(gh api "repos/pytorch/executorch/compare/${VIABLE_SHA}...${MAIN_SHA}" \
--jq '.commits | reverse | .[:20] | .[] | "\(.sha)\u001f\(.commit.message | split("\n")[0] | .[:50])"' 2>/dev/null || echo "")

if [ -n "$COMMITS" ]; then
echo "| Commit | Title | Failed Job |" >> $GITHUB_STEP_SUMMARY
echo "|--------|-------|------------|" >> $GITHUB_STEP_SUMMARY

while IFS=$'\x1f' read -r sha title; do
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using $'\x1f' (hexadecimal escape) is inconsistent with the \u001f (Unicode escape) used in jq queries. While both represent the ASCII unit separator (0x1F), mixing notation styles reduces readability. Consider standardizing on one approach throughout the script.

Copilot uses AI. Check for mistakes.
[ -z "$sha" ] && continue
short_sha="${sha:0:7}"
# Sanitize title for markdown table
title=$(echo "$title" | tr '|' '-')
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sanitization using tr '|' '-' only replaces pipe characters. Other markdown special characters (backticks, brackets, asterisks, underscores) in commit messages could break the table formatting or create unintended markdown rendering. Consider more comprehensive escaping or using a safer delimiter pattern.

Copilot uses AI. Check for mistakes.

# Get workflow runs for this commit (use unit separator)
failed_run=$(gh api "repos/pytorch/executorch/actions/runs?head_sha=${sha}&per_page=100" \
--jq '.workflow_runs[] | select(.conclusion == "failure") | "\(.id)\u001f\(.name)"' 2>/dev/null | head -1)

if [ -n "$failed_run" ]; then
IFS=$'\x1f' read -r run_id workflow_name <<< "$failed_run"
# Sanitize workflow name for markdown table
workflow_name=$(echo "$workflow_name" | tr '|' '-')
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sanitization using tr '|' '-' only replaces pipe characters. Other markdown special characters (backticks, brackets, asterisks, underscores) in workflow names could break the table formatting or create unintended markdown rendering. Consider more comprehensive escaping or using a safer delimiter pattern.

Copilot uses AI. Check for mistakes.

# Get failed job name
failed_job=$(gh api "repos/pytorch/executorch/actions/runs/${run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | .name' 2>/dev/null | head -1 | tr '|' '-')
Comment on lines +75 to +76
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API call is made inside a loop for each failed run, potentially causing many sequential API requests. Consider batching or caching job information, or at minimum adding rate limit awareness to prevent hitting GitHub API limits when many failures exist.

Copilot uses AI. Check for mistakes.

echo "| ${short_sha} | ${title} | ${workflow_name} / ${failed_job:-unknown} |" >> $GITHUB_STEP_SUMMARY
found_failures=true
fi
done <<< "$COMMITS"
fi
fi

if [ "$found_failures" = false ]; then
# Fallback: show recent failed runs from main branch only
echo "> Note: Could not determine commits between branches. Showing recent failures from main branch." >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Commit | Title | Failed Job |" >> $GITHUB_STEP_SUMMARY
echo "|--------|-------|------------|" >> $GITHUB_STEP_SUMMARY

# Filter by branch=main and use unit separator
gh api "repos/pytorch/executorch/actions/runs?branch=main&per_page=20" \
--jq '.workflow_runs[] | select(.conclusion == "failure") | "\(.id)\u001f\(.head_sha | .[:7])\u001f\(.display_title | .[:50])\u001f\(.name)"' 2>/dev/null | \
while IFS=$'\x1f' read -r run_id sha title workflow; do
# Sanitize for markdown table
title=$(echo "$title" | tr '|' '-')
workflow=$(echo "$workflow" | tr '|' '-')
failed_job=$(gh api "repos/pytorch/executorch/actions/runs/${run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | .name' 2>/dev/null | head -1 | tr '|' '-')
Comment on lines +95 to +100
Copy link

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sanitization using tr '|' '-' only replaces pipe characters. Other markdown special characters (backticks, brackets, asterisks, underscores) could break the table formatting or create unintended markdown rendering. Consider more comprehensive escaping or using a safer delimiter pattern.

Suggested change
while IFS=$'\x1f' read -r run_id sha title workflow; do
# Sanitize for markdown table
title=$(echo "$title" | tr '|' '-')
workflow=$(echo "$workflow" | tr '|' '-')
failed_job=$(gh api "repos/pytorch/executorch/actions/runs/${run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | .name' 2>/dev/null | head -1 | tr '|' '-')
sanitize_md() {
local input="$1"
# Replace table separators and escape other markdown-special characters
input=${input//|/-}
input=${input//\`/\'}
input=${input//\[/\\[}
input=${input//]/\\]}
input=${input//*/\\*}
input=${input//_/\\_}
printf '%s' "$input"
}
while IFS=$'\x1f' read -r run_id sha title workflow; do
# Sanitize for markdown table
title=$(sanitize_md "$title")
workflow=$(sanitize_md "$workflow")
failed_job=$(gh api "repos/pytorch/executorch/actions/runs/${run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | .name' 2>/dev/null | head -1)
failed_job=$(sanitize_md "$failed_job")

Copilot uses AI. Check for mistakes.
if [ -n "$failed_job" ]; then
echo "| ${sha} | ${title} | ${workflow} / ${failed_job} |" >> $GITHUB_STEP_SUMMARY
fi
done
fi

echo "" >> $GITHUB_STEP_SUMMARY
echo "### Summary by job (main branch, last 24h)" >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
# Filter by main branch and created in last 24 hours (GNU date syntax for Ubuntu runner)
yesterday=$(date -u -d '24 hours ago' '+%Y-%m-%dT%H:%M:%SZ')
gh api "repos/pytorch/executorch/actions/runs?branch=main&created=>${yesterday}&per_page=100" \
--jq '.workflow_runs[] | select(.conclusion == "failure") | .name' 2>/dev/null \
| sort | uniq -c | sort -rn | head -10 >> $GITHUB_STEP_SUMMARY || echo "Failed to fetch data" >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY

echo "::error::No green commit found - viable/strict was not updated"
exit 1
Loading