Skip to content

test(js): fix flaky load fail-fast tests on slow CI runners#9171

Merged
manovotny merged 1 commit into
mainfrom
manovotny/intelligent-einstein-7736ac
Jul 15, 2026
Merged

test(js): fix flaky load fail-fast tests on slow CI runners#9171
manovotny merged 1 commit into
mainfrom
manovotny/intelligent-einstein-7736ac

Conversation

@manovotny

@manovotny manovotny commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Description

The fake-timer tests added in #9065 (describe: "when the client fetch fails or hangs at load") are flaking across unrelated PRs — two 5s timeouts on run 29305425991 and run 29413333718.

The race: load() awaits native crypto for the cookie suffix before timeLimit() schedules its initialization timer. pumpUntilSettled advanced fake time on a fixed 20-iteration budget, but crypto.subtle.digest resolves on the real event loop — fake timers can't move it. On a slow runner the digest lands after all 20 pumps have drained, the 7s fake timer sits unadvanced forever, and vitest's 5s testTimeout kills the test. That's why CI failures always report ~5015ms and local runs finish in ~17ms.

The fix teaches pumpUntilSettled to yield real event-loop turns until a fake timer is actually scheduled (or the promise settles) before spending its advance budget. No runtime code changes.

To verify, I reproduced the race deterministically by delaying crypto.subtle.digest by 200ms: exactly the two CI-flaky tests fail with the old helper, and all pass with the new one. The full file then passed 5 consecutive runs, plus 3 more with all cores saturated by busy loops.

Checklist

  • pnpm test runs as expected.
  • pnpm build runs as expected.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other:

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Improved reliability of load-failure tests across slower environments and different timer states.
  • Chores
    • Added release metadata for the changes included in this update.

…timers

The pumpUntilSettled helper advanced fake time on a fixed budget, but
load() awaits native crypto (cookie suffix) before scheduling its
initialization timeout. On slow CI runners the crypto work resolves
after all pump iterations have drained, leaving the fake timer
unadvanced and the test hung until vitest's 5s testTimeout.

Yield real event-loop turns until a fake timer is actually scheduled
(or load settles) before spending the advance budget. Reproduced
deterministically by delaying crypto.subtle.digest by 200ms: the two
CI-flaky tests fail with the old helper and pass with the new one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
clerk-js-sandbox Ready Ready Preview, Comment Jul 15, 2026 4:47pm
swingset Ready Ready Preview, Comment Jul 15, 2026 4:47pm

Request Review

@changeset-bot

changeset-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 50398eb

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: af6c4727-9b6e-429e-8046-f3d9b1eccd33

📥 Commits

Reviewing files that changed from the base of the PR and between 59774c6 and 50398eb.

📒 Files selected for processing (2)
  • .changeset/warm-shrimps-fall.md
  • packages/clerk-js/src/core/__tests__/clerk.test.ts

📝 Walkthrough

Walkthrough

The PR updates pumpUntilSettled to accommodate asynchronous crypto initialization before fake timers exist, while preserving repeated 5-second timer advancement. It also adds a two-line changeset entry.

Changes

Load test timer handling

Layer / File(s) Summary
Timer settlement helper and release metadata
packages/clerk-js/src/core/__tests__/clerk.test.ts, .changeset/warm-shrimps-fall.md
The test helper yields to real event-loop turns when no timers are registered, then advances fake timers as needed; a changeset records the update.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: nikosdouvlis, brkalow

Poem

I’m a rabbit who waits for the timers to hop,
Letting crypto finish before clocks start to pop.
Five seconds at a time, the tests gently flow,
With a fresh changeset planted below.
Wiggle your nose—settled code is aglow!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing flaky JavaScript load fail-fast tests on slow CI runners.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Jul 15, 2026

Copy link
Copy Markdown

Open in StackBlitz

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@9171

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@9171

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@9171

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@9171

@clerk/electron

npm i https://pkg.pr.new/@clerk/electron@9171

@clerk/electron-passkeys

npm i https://pkg.pr.new/@clerk/electron-passkeys@9171

@clerk/eslint-plugin

npm i https://pkg.pr.new/@clerk/eslint-plugin@9171

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@9171

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@9171

@clerk/express

npm i https://pkg.pr.new/@clerk/express@9171

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@9171

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@9171

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@9171

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@9171

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@9171

@clerk/react

npm i https://pkg.pr.new/@clerk/react@9171

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@9171

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@9171

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@9171

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@9171

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@9171

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@9171

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@9171

commit: 50398eb

@github-actions

Copy link
Copy Markdown
Contributor

API Changes Report

Generated by Break Check on 2026-07-15T16:48:48.323Z

Summary

Metric Count
Packages analyzed 19
Packages with changes 0
🔴 Breaking changes 0
🟡 Non-breaking changes 0
🟢 Additions 0

No API Changes Detected

All packages have stable APIs with no detected changes.


Report generated by Break Check

Last ran on 50398eb.

@manovotny manovotny merged commit 2c7b1b1 into main Jul 15, 2026
53 checks passed
@manovotny manovotny deleted the manovotny/intelligent-einstein-7736ac branch July 15, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants