Skip to content

Improve the ConsoleAssert.AssertExpectation output #69

@BenjaminMichaelis

Description

@BenjaminMichaelis

Goal: Enhance the ConsoleAssert.AssertExpectation method to provide a detailed character-by-character diff when wildcard matching fails, showing exactly where wildcards matched and where mismatches occurred.
Context: The current test output only shows the full expected and actual strings, making it hard to identify where wildcard patterns (*) failed to match. This is especially problematic in multiline outputs like ping results on Linux.
Source: You have access to the full IntelliTect.TestTools.Console codebase, including ConsoleAssert.cs.

Expectations:

  • When a mismatch occurs, print:
    • Each expected/actual line pair
    • A match status (✅ or ❌)
    • A list of what each * matched in the actual string
  • If a line is present in actual but not expected, mark it as an unexpected extra line
  • If a line is present in expected but not actual, mark it as missing
  • Keep the output console-friendly (no colors or side-by-side views)
  • Add a flag or overload to enable this verbose diff mode only when needed

For example, a test like this:

        string expected = $@"*
Pinging * ?::1? with 32 bytes of data:
Reply from ::1: time*";
        string pingArgs = "-c 4 localhost";
        if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
        {
            pingArgs = pingArgs.Replace("-c ", "-n ");
        }
        else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
        {
            expected = $@"PING *(* (::1)) 56 data bytes
64 bytes from * (::1): icmp_seq=1 ttl=64 time=* ms*";
        }
        else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
        {
            expected = $@"PING *(*): 56 data bytes
64 bytes from *: icmp_seq=? ttl=64 time=* ms*";
        }

        ConsoleAssert.ExecuteProcess(
            expected,
            "ping", pingArgs, out string _, out _);

would currently give an output lets say like this if it failed:

Expected: ("PING *(* (::1)) 56 data bytes\n64 bytes from * (::1): icmp_seq=1 ttl=64 time=* ms*\n*--- localhost ping statistics ---*\n* packets transmitted, * received, *% packet loss, time *ms\nrtt min/avg/max/mdev = */*/*/* ms*")
Actual  : @"PING localhost (::1) 56 data bytes
64 bytes from localhost (::1): icmp_seq=1 ttl=64 time=0.018 ms
64 bytes from localhost (::1): icmp_seq=2 ttl=64 time=0.031 ms
64 bytes from localhost (::1): icmp_seq=3 ttl=64 time=0.033 ms
64 bytes from localhost (::1): icmp_seq=4 ttl=64 time=0.032 ms

--- localhost ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3048ms
rtt min/avg/max/mdev = 0.018/0.028/0.033/0.006 ms
"
-----------------------------------
The expected length of 212 does not match the output length of [43](https://github.com/IntelliTect/TestTools.Console/actions/runs/15918145657/job/44899524168#step:6:44)5.

Instead, we could show something that better points out exactly where the output doesn't match the expected pattern and we somehow can highlight the differences between the strings.

You have some creative freedom with how we do this, but we are constricted by the output being shown in the console.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions