Run browser tests with the CLI

The Zerocheck CLI runs the YAML tests in your repository in a Chromium browser on your machine or a CI runner. It's optional: tests you create in the web app run there, with nothing to install.

# zerocheck/tests/discount-code.yaml
version: zerocheck/v1
name: Discount code reaches confirmation
blocks_merge: true
steps:
  - Navigate to /cart
  - Enter "SPRING20" into "Discount code"
  - Click "Apply"
  - Verify text "Discount applied" is visible
  - Enter "4242424242424242" into "Card number"
  - Click "Place order"
  - Verify text "Order confirmed" is visible

npx zerocheck run

How it works

Your dedicated instance reads each test, makes the model calls, chooses each browser action, checks each expected outcome and keeps the results. The CLI drives the browser for it and keeps your test secret values on the machine where it runs.

The CLI is not an offline runner. Every run needs network access to your instance. Credentials and data lists what it sends.

Before you start

• Node.js 20.19 or newer.
• Your instance URL and its access password, supplied during onboarding. There is no default public service.
• A project on that instance.
• zerocheck.yaml and at least one saved YAML test. We set these up with you during onboarding; Settings in the web app shows the zerocheck.yaml text to copy.
• A running app your machine can reach, with the test accounts and secret variables your tests need. Localhost and private networks work because the browser runs where you run the CLI.

Install and sign in

Run these from your app's repository, using your own instance URL:

npm install --save-dev --save-exact zerocheck
npx playwright install chromium
export ZEROCHECK_API="https://your-instance.example.com"
npx zerocheck login
npx zerocheck whoami

The second command downloads the Chromium browser the CLI drives. ZEROCHECK_API must be the origin of your instance: HTTPS, with no path, query or credentials. Plain HTTP is accepted only for localhost.

login prints a URL and a short code. Open the URL, sign in with your instance's access password, then confirm the code and approve the project. The code expires after fifteen minutes. The resulting token is saved in ~/.zerocheck/config.json with owner-only permissions and works only with that instance. whoami shows the instance and project; logout revokes the token and clears it from this machine.

Project files

zerocheck.yaml names the project, its environments and where the tests live. Environments are exactly dev, staging and production:

project: your-project-id
default_environment: dev
test_directory: zerocheck/tests
environments:
  dev:
    url: http://localhost:3000
  staging:
    url: https://staging.example.com

To sign in to your app, map each secret to an environment variable and add login steps to the environment. Set the variable where the CLI runs:

environments:
  dev:
    url: http://localhost:3000
    secrets:
      PASSWORD: ${APP_TEST_PASSWORD}
    login_steps:
      - Enter "{{credential:PASSWORD}}" into "Password"

For a long flow, set testTimeoutMs and stepTimeoutMs under execution, in milliseconds.

Each test is a YAML file in the test directory. See the test format for the fields and step syntax. Adapt this to a route and outcome that exist in your app:

version: zerocheck/v1
name: Release notes are available
blocks_merge: true
steps:
  - Navigate to /releases
  - Verify text "Release notes" is visible

Run your tests

npx zerocheck run
npx zerocheck run --env staging
npx zerocheck run zerocheck/tests/release-notes.yaml
npx zerocheck run --json .zerocheck/latest-run.json

With no path, run executes the YAML files in test_directory against default_environment. Name files or folders to select tests; selection is by path only. Tests run one at a time, each in a fresh browser context.

Option	What it does
--env <name>	Use another environment from zerocheck.yaml.
--json <path>	Write the run record to a file.
--fail-on-flaky	Exit 1 when a blocking test passed only after a retry.
--headed	Show the browser. Runs are headless by default.
--project-dir <dir>	Run a project in another folder.

The terminal prints each test's result and a link to the run. Open the link and sign in to your instance to inspect steps, screenshots and any recording. The names and identifiers below are an example of the format, not a real run:

staging / local: failed (RUN_ID)
  PASS  Release notes are available
  FAIL  Discount code reaches confirmation
    Attempt 1: Verify text "Order confirmed" is visible: <error text>
Results: https://your-instance.example.com/runs/RUN_ID

Exit code	Meaning
0	Every blocking test passed.
1	A blocking test failed, or with --fail-on-flaky a blocking test passed only after a retry.
2	No verdict: a setup or secret problem, a refused start, a cancellation, a run that could not finish, or any selected test that could not complete, including a non-blocking one.

Press Ctrl-C once to cancel: the browser closes and your instance is told. A cancelled run never resumes. A second Ctrl-C stops waiting and exits 130; the next run in that folder tells the instance. The CLI keeps this recovery state in a .zerocheck folder, which it excludes from Git.

Tests in the web app and in your repository

Checklist import, drafting and hosted runs are in the web app. The CLI runs repository YAML files and does not import checklists or start hosted runs.

To bring a web test into your repository, use Download YAML on that test and put the file in your test directory. That creates an independent copy. Edits in the web app and edits in your repository do not synchronize, so decide which copy your team maintains. Repository tests are edited in their files.

Commands

The commands are login, logout, whoami and run, plus --version and --help. This guide describes zerocheck 0.2.1; npx zerocheck --version prints the version you have.

When a run needs attention

Situation	Next step
No instance configured	Set ZEROCHECK_API to your instance before signing in.
No readable zerocheck.yaml	Run from your project folder, or pass --project-dir.
No checks selected	Add YAML tests to the test directory, or name a file that exists.
Chromium is reported as missing	The CLI uses its own pinned browser library. If your repository installs a different version, run npx [email protected] install chromium.
Missing test secrets	Set the environment variables your configured environment references.
A test fails or cannot complete	Open the results link and inspect the failed step and its evidence.
Your instance refuses the start	The CLI exits 2 and nothing runs. Your CLI and instance must be on compatible releases; contact us if the message does not explain the cause.
Your instance is running as many runs as it allows	Try again shortly.
An earlier run did not finish	Run zerocheck run again in the same folder. It settles the earlier run first, then runs the tests you asked for.

Next: run in an existing CI job · use it from a coding agent · credentials and data