> ## Documentation Index
> Fetch the complete documentation index at: https://docs.testorim.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reliability

> Making a suite you trust: stable targets, the right auth mode, and the signals that tell you a test is drifting.

## Write targets that survive a redesign

In rough order of durability:

1. **A test id**: `[data-testid="checkout-submit"]`. Nothing in a redesign
   moves it.
2. **An accessible name**: the button's visible label, the field's `<label>`.
   Durable and readable.
3. **A row-scoped control**: *"Delete in the Blue Top row"*. Survives
   reordering.
4. **An ordinal**: *"the first product"*. Fine for listings, fragile if the
   order is not stable.
5. **Loose prose**: resolved by the grounding model. It works, and the
   warning tells you what it picked, but it is the one most affected by a
   redesign.

## Set the auth mode

The single most common cause of a test that passes alone and fails in a batch.
Sign-in, sign-up and sign-out tests need `fresh_logged_out`; dashboard
workflows need `reuse_login_state`. See
[Reusable flows](/test-creation/reusable-flows#login-and-auth-mode).

## Flake tracking

The project overview carries a flakiness section: pass rate per saved test over
7, 30 or 90 days, sorted worst-first by a composite score built from three
signals.

| Signal                                                       | Strength                                                      |
| ------------------------------------------------------------ | ------------------------------------------------------------- |
| **Divergent batches**, where the legs of one batch disagreed | Strongest. The same test, the same moment, two answers.       |
| **Pass-rate distance from 100%**                             | A test that mostly passes is a test you will learn to ignore. |
| **Passed-after-retry counts**                                | A step that needed a second attempt is telling you something. |

Fix the top of that list before adding tests.

## Retry on fail

Use it for genuine flake such as an animation or a lazy-loaded element, and not as a
band-aid for a real bug. A step that passed only on retry is marked, and those
marks are a flake signal. `press` steps are never retried, because re-sending a
key after the page has already acted on it produces a false pass.

## Visual baselines

Set a baseline per saved test, step and viewport. Later runs diff against it
with a pixel comparison and show baseline, actual and difference side by side.

Best on pages whose layout is meant to be stable: a pricing table, a receipt,
an empty state. Worst on anything with live data in it.

## Performance budgets

Per saved test, cap LCP, FCP and TTFB with a tolerance percentage. Metrics are
collected per page load and checked after the run; a breach marks the run's
performance status `degraded` and alerts like a failure.

Set the caps from what the site does today, not from an aspiration. A budget
that is always red is a budget nobody reads.

## Adversarial fuzzing

For a saved test with `type` steps, fuzzing generates a payload per field
across six categories (empty, boundary, XSS, SQL injection, unicode, special
characters) and fires them as a batch of variants alongside the parent run.

A confirmation shows the variant count and estimated time before anything runs,
and the whole batch can be cancelled from the active-runs toast.

## Role boundary probe

Give it a list of URLs and it walks each one as a privileged session and as a
low-privilege session. A URL both can reach is reported as a possible IDOR. A
redirect to a login page counts as blocked; only a `2xx` counts as reachable.

## Run hygiene

* Give slow flows a real time limit instead of letting them hit the default.
* Cancel runs you do not need. It frees a browser immediately and gives the
  allowance back.
* Keep pull-request test selections small. A slow PR comment gets ignored.
* Split long tests. Three signals beat one, and a 40-step plan is a ceiling you
  do not want to be near.


## Related topics

- [Debugging a failed run](/test-execution/debugging.md)
- [Support](/resources/support.md)