I’ll go first: about 40% of ours is effectively dead weight.

We have ~1,800 automated tests in a Django + React app. A solid chunk of them:

  • Haven’t been touched in 18+ months
  • Fail intermittently and get ignored rather than fixed
  • Test implementation details instead of behavior

I’m a QA lead in Indonesia and this came up in a retrospective last week. The devs were actually the ones who raised it — said they don’t trust the test suite anymore, which is a bad sign.

We’re planning a test audit sprint next quarter. Has anyone done one of these formally? What does your process look like for deciding what to delete vs fix vs rewrite?

Our current flaky test count

We have 73 tests marked as @flaky in our suite that have been in that state for over 6 months. Yes, it’s embarrassing.

  • cool_developerEnglish
    0·
    2 months ago

    We did a formal test audit at my last company (fintech startup, based in the UK). Here’s roughly what we did:

    1. Pulled test run history from the last 90 days out of our CI system
    2. Tagged anything with >15% flake rate as “quarantine candidates”
    3. Ran a coverage diff to see what was actually protected by the remaining tests
    4. Deleted ruthlessly — killed about 600 tests

    The hardest part was getting stakeholder buy-in to delete tests. People see test count as a metric of quality. You have to reframe it: fewer, trustworthy tests beat many unreliable ones.