> ## Documentation Index
> Fetch the complete documentation index at: https://docs.testorim.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Python package and pytest

> Run Testorim tests from Python code and from pytest with the testorim package.

The [`testorim`](https://pypi.org/project/testorim/) package runs Testorim tests from Python code and adds a pytest plugin, so a real-browser check can sit next to your other tests and fail the build the same way. It has no dependencies and needs Python 3.9 or newer.

```bash theme={null}
pip install testorim
```

Create an API key in Testorim under **Settings → API keys** and put it in `TESTORIM_API_KEY`. See [Authentication](/developers/authentication) for what a key may change.

## From code

```python theme={null}
from testorim import Testorim

client = Testorim()  # reads TESTORIM_API_KEY

run = client.run_test(
    "Shop",  # a project's name, address or id
    "Add the Blue Top to the cart, open the cart and check that it shows 1 item.",
)
print(run.summary())
run.assert_passed()  # raises TestorimRunFailed unless the verdict is passed
```

`run_test` starts the run and waits for the verdict (up to 10 minutes by default). Other calls:

```python theme={null}
client.projects()                                    # the workspace's projects
client.tests("Shop")                                 # a project's saved tests
client.run_saved("Checkout", project="Shop")         # replay a saved test by name or id
client.run_test("Shop", "...", base_url="https://pr-42.preview.example.com")
client.run_test("Shop", "Sign in with a wrong password and check that an error is shown.", expect_failure=True)
run = client.run_test("Shop", "...", wait=False)     # start without waiting
client.wait_for(run.id)                              # then wait
client.cancel(run.id)
```

A run has a `verdict` (`passed`, `failed` or `needs_review`; `None` while it runs or when cancelled), the step counts, the written `report`, `failed_steps` (each with the reason and `blame`: `app`, `test`, `unsupported` or `internal`), the run's `url`, and links to the video, Playwright trace and final screenshot, which expire an hour after they were read. `summary()` leaves those links out because it often lands in CI logs.

## In pytest

Installing the package adds a `testorim` fixture and a `testorim` marker:

```python theme={null}
import pytest

@pytest.mark.testorim
def test_checkout(testorim):
    testorim.run_saved("Checkout", project="Shop").assert_passed()
```

* Run only these tests with `pytest -m testorim`, or leave them out with `pytest -m "not testorim"`.
* `--testorim-base-url URL` points every run made through the fixture at another address, such as a pull request's preview.
* Without an API key the test fails with a message saying how to create one; set `TESTORIM_SKIP_WITHOUT_KEY=1` to skip instead (useful for pull requests from forks, which get no secrets).

A failed run fails the test with its summary:

```text theme={null}
TestorimRunFailed: Verdict: FAILED (1 passed, 1 failed, 1 skipped, 41s)
Run: https://app.testorim.com/runs/...
Failed step 2: click "Create account": The page showed Something went wrong [the app did not behave as described]
```

## In GitHub Actions

```yaml theme={null}
name: Browser tests
on: pull_request

jobs:
  testorim:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install testorim pytest
      - name: Run the Testorim tests against the preview
        env:
          TESTORIM_API_KEY: ${{ secrets.TESTORIM_API_KEY }}
          TESTORIM_SKIP_WITHOUT_KEY: "1"
        run: pytest -m testorim --testorim-base-url "https://pr-${{ github.event.number }}.preview.example.com"
```

Put your preview's real address in `--testorim-base-url`, and make sure the preview is deployed before the job runs. A saved test that types a saved password replays only on its project's own host or one of the project's [environments](/projects/environments).

## Errors

Every exception derives from `TestorimError`. The ones you will meet:

| Exception             | When                                                                                                                                                     |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AuthenticationError` | 401: the key is missing, wrong or revoked.                                                                                                               |
| `RefusedError`        | 402, 429 or 503: Testorim will not start the run now. `code` and `retryable` say why and whether waiting helps; see [API errors](/api-reference/errors). |
| `NotFoundError`       | No project or saved test matches, or it is in another workspace.                                                                                         |
| `TestorimUnreachable` | The API could not be reached.                                                                                                                            |
| `TestorimRunFailed`   | `assert_passed()` on a run that did not pass.                                                                                                            |

The client never retries a trigger by itself, because a retried trigger is a second run. If your workspace lives on another host, set `TESTORIM_API_URL`. The full reference is on the [package's PyPI page](https://pypi.org/project/testorim/).

## Limits

* The site must be reachable from the internet: Testorim's browsers refuse `localhost` and private addresses.
* Every run counts against your plan, whichever tool started it.
* For coding agents, use the [MCP server](/developers/mcp) instead; for JavaScript or TypeScript test suites, the [library](/developers/javascript); for shell scripts and other CI, the [CLI](/developers/cli).


## Related topics

- [JavaScript and TypeScript library](/developers/javascript.md)
- [CLI](/developers/cli.md)
- [MCP server for coding agents](/developers/mcp.md)