How to investigate flaky browser tests

Start with the failing step, test data and observed outcome. Keep recovery visible instead of treating a retry as proof that nothing was wrong.

Why this is hard to test

  • •Timing, shared state and external services can produce similar symptoms.
  • •A rerun can change the evidence you needed to investigate.
  • •A locator failure and a product assertion failure require different fixes.

Plan the test around an observable outcome

  1. 1.Keep the original failure’s artifacts.
  2. 2.Check whether the expected account and fixture state existed.
  3. 3.Wait for meaningful app state instead of arbitrary time.
  4. 4.Assign an owner to the root cause before restoring a trusted merge gate.

Where Zerocheck fits

  1. 1.Inspect the exact test revision, failing step and screenshots in the run detail.
  2. 2.Read console errors, failure type and all recorded attempts.
  3. 3.Review healing history separately from retry results.
  4. 4.Use --fail-on-flaky when the CI policy should reject a flaky blocking test.
Flaky Test Triage →

What a retry means in Zerocheck

A transient AI-service error can be retried once in the same browser. If the test then passes it is labeled flaky. Assertion failures are not retried, and the shipped CLI and hosted workflow do not automatically replay a failed test after its browser actions have run.

Review the expected outcome first

If a test verifies “success” without saying which record or value should exist, a green result may be difficult to interpret. Tighten the expectation and ensure the fixture is stable before investigating target resolution or timing.

Common pitfalls

  • —Do not retry assertion failures until they pass.
  • —Do not label every infrastructure error a product bug.
  • —Do not confuse a healed target with a retried test.

FAQ

Does Zerocheck hide a pass after retry?

No. It remains labeled flaky in the result, CLI and PR reporting.

Can a harmless layout change heal?

A moved target can be resolved again. A heal is saved only after the full unchanged test passes with independent evidence.

How to investigate flaky browser tests

Describe a customer flow or paste your release checklist, review the browser result, and save the tests you want to rerun.

Get a demo