I’ll go first: about 40% of ours is effectively dead weight.
We have ~1,800 automated tests in a Django + React app. A solid chunk of them:
- Haven’t been touched in 18+ months
- Fail intermittently and get ignored rather than fixed
- Test implementation details instead of behavior
I’m a QA lead in Indonesia and this came up in a retrospective last week. The devs were actually the ones who raised it — said they don’t trust the test suite anymore, which is a bad sign.
We’re planning a test audit sprint next quarter. Has anyone done one of these formally? What does your process look like for deciding what to delete vs fix vs rewrite?
Our current flaky test count
We have 73 tests marked as @flaky in our suite that have been in that state for over 6 months. Yes, it’s embarrassing.
73 flaky tests for 6 months… I felt that in my soul. We had a similar situation.
The fix that worked for us was treating flaky tests exactly like production bugs — they go into the backlog with severity and priority, get assigned to someone, and have a deadline. No more “we’ll fix it later” energy.
Once the team felt the pain of being paged because of a false failure at 2am, suddenly people cared a lot more about test reliability. 😅