Skip to content

Commit 26e4c30

Browse files
waleedlatif1waleedlatif1
authored
fix(ci): unblock the deploy chain and remove the promotion-dedup gate (#5881)
* fix(ci): unblock the deploy chain — explicit need results on jobs downstream of test-build test-build's needs chain now contains dedup-promotion, which is skipped on every push event. A skipped transitive ancestor fails the implicit success() on downstream jobs, so migrate, promote-images, create-ghcr-manifests, process-docs, and create-release all cascade-skipped on push runs — blocking staging and main deploys (no ECR tag push, no CodeDeploy). Each of those jobs now uses !cancelled() plus explicit needs.<job>.result == 'success' checks, preserving their original semantics while ignoring the skipped ancestor. * chore(ci): remove the promotion-dedup gate — savings don't justify deploy-graph complexity The bespoke dedup job hand-rolled what content-addressed caching solves idiomatically, saved only ~$50-90/mo, and its needs edge just caused the deploy-chain skip incident. test-build returns to its original shape; the explicit need-result conditions on the deploy chain stay as hygiene.
1 parent 227cd65 commit 26e4c30

1 file changed

Lines changed: 30 additions & 130 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 30 additions & 130 deletions
Original file line numberDiff line numberDiff line change
@@ -34,135 +34,9 @@ permissions:
3434
contents: read
3535

3636
jobs:
37-
# Promotion PRs (staging→main etc.) double-run the test suite: every commit
38-
# on staging/main already gets a push-event run of the exact same test-build,
39-
# and the pull_request "synchronize" run for the open release PR re-runs it on
40-
# the same sha seconds later (~39 duplicate runs / 5 days measured Jul 2026).
41-
# This gate skips the PR run's test-build ONLY when it can prove the identical
42-
# work already passed elsewhere:
43-
# 1. the PR base adds no file changes over the merge base with the head sha
44-
# (compare head...base has an empty diff), so the merge result's tree is
45-
# identical to the head tree the push run tested. Plain ancestry is not
46-
# enough of a check here: main's merge-only ruleset leaves merge commits
47-
# on main that staging lacks, so main...staging is permanently
48-
# "diverged" — but those merge commits carry no tree delta. A real
49-
# hotfix landed directly on the base makes the diff non-empty and we
50-
# run tests here;
51-
# 2. the push-event CI run at the same head sha finished its test jobs with
52-
# conclusion success (polled, since push + PR runs start simultaneously).
53-
# Fail-open by construction: any API error, timeout, missing run, or push-run
54-
# failure leaves covered=false and the PR run tests normally, so the PR check
55-
# is green only if tests passed either here or on the identical tree. Not a
56-
# workflow-level filter on purpose — a job-level skip still reports a
57-
# (successful) check context. NOTE: when test-build is skipped, its nested
58-
# "Test and Build / ..." contexts are not created; verified 2026-07-22 that
59-
# no required status checks are configured on main/staging (rulesets contain
60-
# only pull_request/deletion/non_fast_forward). If required checks are ever
61-
# added, require the caller "Test and Build" context, not the nested ones.
62-
# dev is excluded: push runs on dev skip test-build, so dev-headed PRs have
63-
# no push-run coverage to reuse.
64-
dedup-promotion:
65-
name: Dedup Promotion PR
66-
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
67-
timeout-minutes: 15
68-
if: >-
69-
github.event_name == 'pull_request' &&
70-
github.event.pull_request.head.repo.full_name == github.repository &&
71-
contains(fromJSON('["main", "staging"]'), github.event.pull_request.head.ref)
72-
permissions:
73-
contents: read
74-
actions: read
75-
outputs:
76-
covered: ${{ steps.probe.outputs.covered }}
77-
steps:
78-
- name: Probe for a passing push run at the same sha
79-
id: probe
80-
env:
81-
GH_TOKEN: ${{ github.token }}
82-
REPO: ${{ github.repository }}
83-
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
84-
BASE_SHA: ${{ github.event.pull_request.base.sha }}
85-
BASE_REF: ${{ github.event.pull_request.base.ref }}
86-
run: |
87-
COVERED=false
88-
89-
# (1) Merge-tree equivalence: compare head...base diffs the merge
90-
# base against the base tip. An empty diff means the base contributes
91-
# nothing beyond what head already contains (merge commits only), so
92-
# the PR merge tree equals the head tree the push run tested. Any
93-
# error yields "unknown" and we run the tests.
94-
BASE_DELTA="$(gh api "repos/${REPO}/compare/${HEAD_SHA}...${BASE_SHA}" --jq '.files | length' 2>/dev/null || echo unknown)"
95-
if [ "$BASE_DELTA" != "0" ]; then
96-
echo "Base tip changes ${BASE_DELTA} file(s) over the merge base; merge tree differs from head — running tests in this PR run."
97-
echo "covered=false" >> "$GITHUB_OUTPUT"
98-
exit 0
99-
fi
100-
101-
# (2) Poll the push-event CI run's test jobs (they start seconds after
102-
# this run and take ~4 min). Any conclusion other than success, or
103-
# deadline expiry, falls through to covered=false.
104-
DEADLINE=$((SECONDS + 600))
105-
while [ "$SECONDS" -lt "$DEADLINE" ]; do
106-
RUN_JSON="$(gh api "repos/${REPO}/actions/workflows/ci.yml/runs?event=push&head_sha=${HEAD_SHA}&per_page=5" 2>/dev/null || echo '')"
107-
RUN_ID="$(printf '%s' "$RUN_JSON" | jq -r '.workflow_runs[0].id // empty' 2>/dev/null || echo '')"
108-
if [ -n "$RUN_ID" ]; then
109-
# Jobs from the reusable test-build workflow are prefixed
110-
# "Test and Build /". If that name ever changes, fall back to the
111-
# overall run conclusion (stricter, still correct).
112-
STATE="$(gh api "repos/${REPO}/actions/runs/${RUN_ID}/jobs?per_page=100" --jq '
113-
[.jobs[] | select(.name | startswith("Test and Build /"))] as $t |
114-
if ($t | length) == 0 then "nojobs"
115-
elif all($t[]; .conclusion == "success") then "success"
116-
elif any($t[]; .conclusion != null and .conclusion != "success") then "failed"
117-
else "pending" end' 2>/dev/null || echo pending)"
118-
if [ "$STATE" = "nojobs" ]; then
119-
# Nested reusable-workflow jobs appear only after the caller
120-
# starts, so an in-progress run with no "Test and Build /"
121-
# jobs yet just needs another poll. Once the run has
122-
# COMPLETED without them, fail closed: the job was renamed or
123-
# tests were skipped — never infer coverage from the overall
124-
# run conclusion. Update the prefix here on a rename.
125-
RUN_STATUS="$(printf '%s' "$RUN_JSON" | jq -r '.workflow_runs[0].status // "unknown"' 2>/dev/null || echo unknown)"
126-
if [ "$RUN_STATUS" = "completed" ]; then
127-
echo "Push run ${RUN_ID} completed with no 'Test and Build /' jobs — running tests in this PR run (update the prefix if the job was renamed)."
128-
break
129-
fi
130-
STATE="pending"
131-
fi
132-
if [ "$STATE" = "success" ]; then
133-
# (3) Re-verify merge-tree equivalence against the LIVE base
134-
# tip at decision time: the base branch may have gained real
135-
# commits during the poll, in which case the frozen BASE_SHA
136-
# check from step (1) is stale and the skip would be unsound.
137-
# Any error yields "unknown" and we run the tests.
138-
LIVE_DELTA="$(gh api "repos/${REPO}/compare/${HEAD_SHA}...heads/${BASE_REF}" --jq '.files | length' 2>/dev/null || echo unknown)"
139-
if [ "$LIVE_DELTA" = "0" ]; then
140-
COVERED=true
141-
echo "Push run ${RUN_ID} passed its test jobs for ${HEAD_SHA} and the live base tip still adds no file changes — skipping duplicate test-build."
142-
else
143-
echo "Base branch moved during the poll (live delta: ${LIVE_DELTA}) — running tests in this PR run."
144-
fi
145-
break
146-
elif [ "$STATE" = "failed" ]; then
147-
echo "Push run ${RUN_ID} did not pass (state: ${STATE}) — running tests in this PR run."
148-
break
149-
fi
150-
fi
151-
sleep 20
152-
done
153-
154-
echo "covered=${COVERED}" >> "$GITHUB_OUTPUT"
155-
15637
test-build:
15738
name: Test and Build
158-
needs: [dedup-promotion]
159-
# !cancelled(): dedup-promotion is skipped on every non-promotion event and
160-
# a skipped need would otherwise skip this job too. covered != 'true' is
161-
# fail-open — empty (skipped/failed probe) means run the tests.
162-
if: >-
163-
!cancelled() &&
164-
(github.ref != 'refs/heads/dev' || github.event_name == 'pull_request') &&
165-
needs.dedup-promotion.outputs.covered != 'true'
39+
if: github.ref != 'refs/heads/dev' || github.event_name == 'pull_request'
16640
uses: ./.github/workflows/test-build.yml
16741
secrets: inherit
16842

@@ -202,7 +76,13 @@ jobs:
20276
migrate:
20377
name: Migrate DB
20478
needs: [test-build]
79+
# Explicit need results instead of the implicit success(): a skipped job
80+
# anywhere in the transitive needs chain silently fails implicit success()
81+
# and cascade-skips the deploy chain (migrate -> promote-images ->
82+
# CodeDeploy) — this bit us on 2026-07-23. State requirements explicitly.
20583
if: >-
84+
!cancelled() &&
85+
needs.test-build.result == 'success' &&
20686
github.event_name == 'push' &&
20787
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging')
20888
uses: ./.github/workflows/migrations.yml
@@ -430,7 +310,11 @@ jobs:
430310
promote-images:
431311
name: Promote Images
432312
needs: [migrate, build-amd64]
313+
# Explicit results: see migrate's comment.
433314
if: >-
315+
!cancelled() &&
316+
needs.migrate.result == 'success' &&
317+
needs.build-amd64.result == 'success' &&
434318
github.event_name == 'push' &&
435319
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging')
436320
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
@@ -553,7 +437,13 @@ jobs:
553437
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
554438
timeout-minutes: 10
555439
needs: [promote-images, build-ghcr-arm64, detect-version]
556-
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
440+
# Explicit results: see migrate's comment.
441+
if: >-
442+
!cancelled() &&
443+
needs.promote-images.result == 'success' &&
444+
needs.build-ghcr-arm64.result == 'success' &&
445+
needs.detect-version.result == 'success' &&
446+
github.event_name == 'push' && github.ref == 'refs/heads/main'
557447
permissions:
558448
contents: read
559449
packages: write
@@ -640,7 +530,12 @@ jobs:
640530
process-docs:
641531
name: Process Docs
642532
needs: [promote-images, check-docs-changes]
643-
if: needs.check-docs-changes.outputs.docs_changed == 'true'
533+
# Explicit results: see migrate's comment.
534+
if: >-
535+
!cancelled() &&
536+
needs.promote-images.result == 'success' &&
537+
needs.check-docs-changes.result == 'success' &&
538+
needs.check-docs-changes.outputs.docs_changed == 'true'
644539
uses: ./.github/workflows/docs-embeddings.yml
645540
secrets: inherit
646541

@@ -650,7 +545,12 @@ jobs:
650545
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
651546
timeout-minutes: 10
652547
needs: [create-ghcr-manifests, detect-version]
653-
if: needs.detect-version.outputs.is_release == 'true'
548+
# Explicit results: see migrate's comment.
549+
if: >-
550+
!cancelled() &&
551+
needs.create-ghcr-manifests.result == 'success' &&
552+
needs.detect-version.result == 'success' &&
553+
needs.detect-version.outputs.is_release == 'true'
654554
permissions:
655555
contents: write
656556
steps:

0 commit comments

Comments
 (0)