Glossary

What Is Test Flakiness?

Definition

Test flakiness describes inconsistent results that do not correspond to a relevant change in the behavior under test. A flake rate is meaningful only when its measurement method is defined.

Why it matters

A suite that frequently needs unexplained reruns creates review work and undermines the merge gate. A passing retry should remain visible so the team can distinguish a clean pass from recovery.

How teams handle it today

Teams track the original failure, repeatability, fixture state and investigation time. They distinguish assertion failures from infrastructure problems and keep flaky tests assigned to an owner.

How Zerocheck approaches it

Zerocheck labels a test that passes after its supported transient retry as flaky. Use --fail-on-flaky to make a flaky blocking test fail the CLI workflow. Run details retain the step evidence rather than hiding the earlier error.

Related terms