Skip to content

Improve test diagnostics when expected_results.txt contains syntax errors#6002

Merged
StephanTLavavej merged 5 commits intomicrosoft:mainfrom
vmichal:line-number-expected-results
Feb 17, 2026
Merged

Improve test diagnostics when expected_results.txt contains syntax errors#6002
StephanTLavavej merged 5 commits intomicrosoft:mainfrom
vmichal:line-number-expected-results

Conversation

@vmichal
Copy link
Contributor

@vmichal vmichal commented Jan 9, 2026

When a formatting error is encountered while parsing expected_results.txt, report the line number as well.

The situation notably improved in #5948, but there is still a tiny improvement to be made.

@vmichal vmichal requested a review from a team as a code owner January 9, 2026 13:24
@github-project-automation github-project-automation bot moved this to Initial Review in STL Code Reviews Jan 9, 2026
@StephanTLavavej StephanTLavavej self-assigned this Jan 9, 2026
@StephanTLavavej StephanTLavavej added enhancement Something can be improved test Related to test code labels Jan 9, 2026
@github-project-automation github-project-automation bot moved this from Initial Review to Work In Progress in STL Code Reviews Jan 9, 2026
@StephanTLavavej StephanTLavavej removed their assignment Jan 9, 2026
@vmichal vmichal marked this pull request as draft January 11, 2026 23:36
@StephanTLavavej StephanTLavavej marked this pull request as ready for review February 14, 2026 11:36
@StephanTLavavej StephanTLavavej changed the title Test tooling: Report line number when expected_results.txt contains format errors Improve test diagnostics when expected_results.txt contains syntax errors Feb 14, 2026
@StephanTLavavej
Copy link
Member

Thanks! 😻 I hulked out check_libcxx_paths.py to more intelligently complain about damaged lines (those whose syntax appears to be very malformed), unknown results (the common mistake of saying SKIP instead of SKIPPED), and nonexistent tests only if the syntax is otherwise valid. This is a vast improvement over the earlier behavior, thanks for prompting me to do this.

check_libcxx_paths.py doesn't give line numbers but I believe its output is easy enough to track down. This is a nice belt-and-suspenders approach.

@StephanTLavavej StephanTLavavej moved this from Work In Progress to Ready To Merge in STL Code Reviews Feb 14, 2026
@StephanTLavavej StephanTLavavej moved this from Ready To Merge to Merging in STL Code Reviews Feb 17, 2026
@StephanTLavavej
Copy link
Member

I'm mirroring this to the MSVC-internal repo. Please notify me if any further changes are pushed, otherwise no action is required.

@StephanTLavavej StephanTLavavej merged commit b77a97c into microsoft:main Feb 17, 2026
45 checks passed
@github-project-automation github-project-automation bot moved this from Merging to Done in STL Code Reviews Feb 17, 2026
@StephanTLavavej
Copy link
Member

Thanks for making it easier to understand test harness failures! 🧠 💡 😸

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Something can be improved test Related to test code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants