CLI - Testorim

npm install -g @testorim/cli

Requires Node 20 or later.

Signing in

testorim login

It asks for a key, checks it against the server, and writes it to ~/.testorim/config.json with 0600 permissions. In CI, set TESTORIM_API_KEY in the environment instead. It takes precedence over the stored file. Mint a key under Settings → API keys. A key runs tests in the workspace it was created in.

Commands

testorim run <procedure-id>

testorim run <procedure-id> --base-url https://preview-42.staging.example.com

testorim run \
  --project <project-id> \
  --description "Sign in with test@example.com and verify the dashboard loads"

testorim run \
  --project <project-id> \
  --description "Try to log in with the wrong password and verify the error" \
  --expectation fail

testorim list                    # projects
testorim list --project <id>     # that project's saved procedures

testorim init                    # writes .testorim.json in this directory

testorim mcp                     # for coding agents, see the MCP server page

testorim init creates a .testorim.json so you can give saved procedures short aliases and run them by name: testorim run checkout. It will not overwrite an existing file unless you pass --force. run prints the run’s page address, a status line as the run moves, the report when it ends, and a summary line with the verdict and the step counts. testorim mcp (version 0.3.0 and later) starts an MCP server so a coding agent such as Claude Code, Codex or Cursor can run tests and read verdicts. Setup for each agent is on the MCP server page. The same package is a JavaScript and TypeScript library (import { Testorim } from "@testorim/cli") for Playwright Test, Vitest and Jest. Working in Python? The testorim package runs the same tests from code and from pytest.

--expectation applies to ad-hoc tests. A saved procedure keeps its own intent: a procedure saved from a negative test replays as one.

Flags

Flag	Meaning
`--api-key <key>`	Override the stored key
`--api-url <url>`	Override the API base URL (default `https://app.testorim.com`)
`--project <id>`	Project id, required for ad-hoc tests
`--description "…"`	The plain-English instruction for an ad-hoc test
`--base-url <url>`	Override the project’s address for this run only
`--expectation <pass\|fail\|unknown>`	Test intent for an ad-hoc test (default `pass`)
`--json`	Machine-readable output
`--quiet`	Suppress progress lines
`--help`	Usage

The CLI does not pick an environment. A run it starts uses the project’s default environment when there is one and you did not pass --base-url. To run against another environment, call POST /api/runs/trigger with environmentId. A saved procedure that types a stored password refuses a --base-url on a host that is not the project’s own or one of its environments. Add the host as an environment instead.

Exit codes

Code	Meaning
`0`	The run passed
`1`	The run failed, or it ended without a verdict (cancelled, or stopped by an error before it finished)
`2`	It could not run: bad input, a refusal from the API, a server error, or no result within 10 minutes
`3`	The run needs review: every step passed, but the run could not confirm the request was met

That is what makes the CLI a pipeline step: a failing test fails the build, a run that needs a human stops it with its own code, and an infrastructure problem is distinguishable from a real failure.

The CLI polls for 10 minutes, the default time limit. A saved procedure with a longer time limit can still be running when the CLI gives up and exits 2. Check the run page it printed.

When the API refuses a run (no plan, a quota, your concurrent runs all in use), the CLI prints the error, its code and, where there is something to buy, an upgrade link. With --json the same fields are printed as JSON.

In GitHub Actions

- name: Smoke test
  env:
    TESTORIM_API_KEY: ${{ secrets.TESTORIM_API_KEY }}
  run: npx @testorim/cli run <procedure-id>

To test a pull-request preview, pass its URL:

- name: Smoke test the preview
  env:
    TESTORIM_API_KEY: ${{ secrets.TESTORIM_API_KEY }}
  run: npx @testorim/cli run <procedure-id> --base-url "${{ steps.deploy.outputs.url }}"

For a repository whose pull requests should always be tested, the GitHub integration is less work, because it builds the preview URL itself and reports in the PR.

Configuration precedence

CLI flags
TESTORIM_API_KEY / TESTORIM_API_URL in the environment
~/.testorim/config.json

​Signing in

​Commands

​Flags

​Exit codes

​In GitHub Actions

​Configuration precedence

Signing in

Commands

Flags

Exit codes

In GitHub Actions

Configuration precedence