feat: the visual matchers can wait until the image matches (wait, interval)

[plum117]

Fixes #690

Why

A page or an element is not always in its final state when the test checks it, for example during an animation, and another condition to wait for is not always available. The visual matchers checked once, so the test failed. The reporter of #690 proposed to run the check again until it matches, as the other WebdriverIO matchers do.

What

toMatchScreenSnapshot(), toMatchFullPageSnapshot(), toMatchElementSnapshot() and toMatchTabbablePageSnapshot() accept 2 new options, with the same names and meaning as in the expect-webdriverio matchers (but wait is 0 by default, so it is opt-in):

• wait (milliseconds, default 0): check again until the image matches, or with .not until it does not match;
• interval (milliseconds, default 100): the time between 2 checks.

await expect($('#chart')).toMatchElementSnapshot('chart', { wait: 3000, interval: 250 })

• Opt-in: without wait, a matcher checks once, as before (a failing check does not take longer, and the screenshot is not repeated).
• The matcher always checks at least once. When it checked more than once, the failure message tells how often and how long.
• .not uses isNot of the matcher context. The Jasmine adapter now gives isNot too (compare and negativeCompare).
• wait and interval are not passed to the check commands; an invalid value (negative, not a number) throws a clear error.
• Each attempt is a full check. The actual and diff images and the JSON report of a check have the same file names, so the last attempt stays (no extra report entries). When a later attempt matches, the images that the earlier failed attempts wrote are removed (a file with an older write time, for example of an earlier check, stays).
• toMatchElementSnapshot(): an element that the page renders again (stale) is found again, and an element that is not in the page yet is waited for, at most for the rest of the wait time (the instances of a multiremote element share that time) (before, it failed at once with Unsupported type: function). At the end of the wait time, the error tells which element is missing.
• Types: WdioMatcherWaitOptions, added to the options of the 4 matchers. A changeset (minor). The feature ships after 11.0.0, so it is not in the v11 migration guide; the options need a line in the WebdriverIO docs (matchers).

Test

• Unit tests (tests/matcher.test.ts, with fake timers): one check without wait, check again until it matches, stop at the wait time (4 checks at 0, 100, 200 and 300 ms, with the message), options not passed to the check, .not, element and multiremote (2 instances that match at different attempts), a stale element, an element that is not in the page yet (found, not found in time, no wait), the images of failed attempts with real files, full page and tabbable, invalid values; tests/jasmine.test.ts: isNot for compare and negativeCompare.
• e2e with a local page (tests/fixtures/v10/delayed-change.html): tests/specs/v10.browsingContexts.spec.ts (Mocha: the box gets its final color later, the page renders the box again, the box is not in the page yet) and tests/specs/v10.jasmine.spec.ts (Jasmine, also .not with wait). Each new e2e test fails without its fix.
• Matrix through the real expect of the test runner (local Chrome, with alwaysSaveActualImage on and off, 16 cells): the 4 matchers with a page or element that changes after 1.5 s; never matches (message and duration); no wait (one check); .not with wait in Mocha (a match, then no match; always matches); an asymmetric matcher; a number as the expected result; an element that comes after 1.5 s and one that never comes; an invalid value; an interval longer than the wait. All pass, and after a match no diff (and with alwaysSaveActualImage: false no actual image) of a failed attempt stays; the JSON report has the result of the last attempt.
• All core and visual-service unit tests, lint and types pass. The local Chrome suites (test.local.chrome.v10, .jasmine, .emulation, test.local.desktop.multi) pass.

Known and not changed here: without wait, toMatchElementSnapshot() on an element that is not in the page fails with Unsupported type: function (also on main), which does not tell that the element is missing.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 885cb8e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@wdio/visual-service	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[plum117]

Keeping draft until @wswebcreation approval

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; both previous findings are addressed and no new blocking issue remains.

Summary

The PR adds opt-in retries through wait and interval to all four visual matchers, including .not through Jasmine.

• Visual snapshot matchers repeat checks until the expected image result appears.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start visual matcher] --> B[Run visual check]
    B --> C{Expected result reached?}
    C -->|Yes| D[Clean up earlier retry images]
    D --> E[Return result]
    C -->|No| F{Wait time used up?}
    F -->|Yes| E
    F -->|No| G[Wait for next attempt]
    G --> B

Loading

_{Reviews (3) · Last reviewed commit: "fix: share the matcher wait time between..." · Reviewed by Greptile}