Beyond test flakiness: A manifesto for a holistic approach to test suite health
flaky tests
literature review
mutation testing
Proceedings of the 2nd International Flaky Tests Workshop
Abstract
Large numbers of flaky tests are a sure sign of a dysfunctional, or “unhealthy”, test suite. In this paper, we identify several further indicators of unhealthy test suites, arguing that trade-offs exist among some indicators, with others complementing one another. We encourage researchers and practitioners not to stop at test flakiness — or any individual metric of “good” or “bad” tests — and instead focus on developing and adopting techniques and tools that holistically address test suite health. For more information, see https://philmcminn.com/test-suite-health.Details
Reference
@inproceedings{McMinn2025,
author = {Phil McMinn and Muhammad Firhard Roslan and Gregory M. Kapfhammer},
booktitle = {Proceedings of the 2nd International Flaky Tests Workshop},
title = {Beyond test flakiness: A manifesto for a holistic approach to test suite health},
year = {2025}
}