Skip to main content

GitHub

Run your tests on every pull request and get one sticky comment with the results.

Slack

Failure alerts to a channel you pick.

Email

Failure alerts to a list of recipients.

CLI and API

Trigger runs from any pipeline with an API key.

​
Which one you want

Testorim does not send outbound webhooks. To react to a run programmatically, trigger it yourself with the API or the CLI and read the result. The CLI’s exit code is 0 on pass, 1 on fail, 2 on error and 3 when the run needs review, which is usually all a pipeline needs.

​
Alert behaviour

Slack and email alerts share the same rules.
  • They fire on a failed run and on a run that needs review, worded as such. With Also alert on warnings on, they also fire for a passing run with a step warning.
  • Scheduled runs only skips ad-hoc runs; procedure replays still alert.
  • They are throttled: at most one alert per test every five minutes, so a broken site does not produce an alert storm.
  • They are configured per project, in project settings → Alerts, with a master Alerts enabled switch and a Send test alert button.
  • They need a subscription (a trial counts).
A cancelled run sends no alert. A performance budget breach is recorded on the run as degraded but does not send an alert on its own.

​
Scheduled runs

Project settings → Schedules attaches a cron expression to a saved procedure, in your browser’s timezone, with presets such as every hour. Each schedule can pick an environment or use the project default, and links to its last run. The worker dispatches it server-side; Run now fires one immediately and counts as a scheduled run. How many schedules you can have depends on your plan (1, 5 or 25 per workspace). If the workspace loses access, an unpaid subscription for instance, schedules are paused with that reason and start again by themselves when it is resolved.