Mutation testing for the agentic era

[carlospolop]

🤖 Automated Content Update

This PR was automatically generated by the HackTricks News Bot based on a technical blog post.

📝 Source Information

• Blog URL: https://blog.trailofbits.com/2026/04/01/mutation-testing-for-the-agentic-era/
• Blog Title: Mutation testing for the agentic era
• Suggested Section: Blockchain & Crypto -> (add new page) Mutation Testing for Smart Contracts (slither-mutate, MuTON/mewt) OR Generic Methodologies & Resources -> Fuzzing Methodology (subsection: Mutation testing / mutant prioritization / result persistence & SARIF)

🎯 Content Summary

Trail of Bits argues that code coverage is a dangerous quality metric because it measures only whether code was executed, not whether its behavior was verified. As a result, a test suite can show “high coverage” while still missing assertions for security-critical behavior that changes over time. They cite a real outcome from an audit: mutation testing uncovered a high-severity Arkis protocol vulnerability that coverage metrics missed, and the issue could ha...

🔧 Technical Details

Use mutation testing to reveal security-relevant testing blind spots: Systematically introduce controlled bugs (“mutants”) into code and rerun the test suite. A killed mutant (tests fail) indicates the behavior is asserted; a surviving mutant (tests still pass) indicates a blind spot—either the path wasn’t executed or the behavior wasn’t actually verified. Surviving mutants often map to missing requirements/assertions around authorization, invariants, revert conditions, and other security-critical logic.

Reduce campaign runtime with mutant prioritization: Run high-signal mutations first and skip redundant variants once a line is already shown to be weak. Example prioritization: (1) high severity—replace statements with revert to detect unexecuted paths; (2) medium severity—comment out logic to detect missing assertions on side effects; (3) low severity—subtle operator swaps. If a high-severity mutant survives for a given line, skip lower-severity mut...

🤖 Agent Actions

I’m mapping the repository first and reading candidate HackTricks pages around smart contract testing, fuzzing, and Slither so the new material lands in the right place and doesn’t duplicate existing content.
There is already a dedicated blockchain page for mutation testing, so I’m checking that first alongside adjacent smart-contract and generic fuzzing docs to see whether the blog adds new technique-level content or just new tooling details.
The existing page already covers the core Slither workflow. I’m now checking the upstream tool docs so any new additions about mewt/MuTON, persistence, and campaign design are technically accurate rather than inferred from the blog alone.
I’ve narrowed it to a single-file update: broaden the existing smart-contract mutation-testing page so it keeps the Slither workflow but adds the genuinely new material from the blog, mainly Tree-sitter-backed mutation, persistent campaign storage, runtime tuning, and triage discipline.
Updated mutation-testing-with-slither.md as the single best target. I kept it in the blockchain smart-contract-security section instead of creating a new page, because HackTricks already had a dedicated mutation-testing entry and the blog mainly adds new technique-level depth rather than a separate topic.

The page now covers the non-duplicated material from the April 1, 2026 Trail of Bits post: syntax-aware mutation versus regex mutation, mewt/MuTON as Tree-sitter-based evolutions, SQLite-backed persistent campaigns and SARIF-oriented triage, runtime optimization with mutant prioritization and two-phase/per-target campaigns, and the warning against blindly generating tests that merely kill survivors. I also updated the references at the end with the new blog and relevant upstream docs. No additional files were changed.

---

This PR was automatically created by the HackTricks Feed Bot. Please review the changes carefully before merging.

[carlospolop]

🔗 Additional Context

Original Blog Post: https://blog.trailofbits.com/2026/04/01/mutation-testing-for-the-agentic-era/

Content Categories: Based on the analysis, this content was categorized under "Blockchain & Crypto -> (add new page) Mutation Testing for Smart Contracts (slither-mutate, MuTON/mewt) OR Generic Methodologies & Resources -> Fuzzing Methodology (subsection: Mutation testing / mutant prioritization / result persistence & SARIF)".

Repository Maintenance:

• MD Files Formatting: 964 files processed

Review Notes:

• This content was automatically processed and may require human review for accuracy
• Check that the placement within the repository structure is appropriate
• Verify that all technical details are correct and up-to-date
• All .md files have been checked for proper formatting (headers, includes, etc.)

Bot Version: HackTricks News Bot v1.0