> ## Documentation Index > Fetch the complete documentation index at: https://e2e.tester.army/docs/llms.txt > Use this file to discover all available pages before exploring further. # CLI > Commands, flags, output formats, and exit codes. The `e2e` command scaffolds a project, runs, lists, and explores tests, manages the replay cache, subscription logins, and telemetry, and upgrades itself. Every command takes `--help`. ```bash npm npx e2e init # scaffold a project npx e2e run # run every test npx e2e run tests/signup.e2e.ts --headed # one file, with a visible browser ``` ```bash pnpm pnpm dlx e2e init # scaffold a project pnpm exec e2e run # run every test pnpm exec e2e run tests/signup.e2e.ts --headed # one file, with a visible browser ``` ```bash bun bunx e2e init # scaffold a project bunx e2e run # run every test bunx e2e run tests/signup.e2e.ts --headed # one file, with a visible browser ``` The CLI needs Node.js 24.8 or newer, or 22.22.3 or newer on Node.js 22. Any package manager installs it: npm, pnpm, Yarn with or without Plug'n'Play, or Bun. The CLI runs on Node.js, not on the Bun or Deno runtimes; `bunx e2e` runs it on Node.js. Under Yarn Plug'n'Play on Node.js 24.8 through 24.11.0, a `.cts` file fails to load: Node.js rejects the source Yarn's loader hands on. 24.11.1 and later load it. ## Reference ### Help and version | Option | Description | | --- | --- | | `-h`, `--help` | Prints the help of the command it follows and exits 0. `e2e help run` equals `e2e run --help`. | | `-v`, `--version` | Prints the installed `e2e` version and exits 0. | ```console $ npx e2e --version 0.14.0 ``` - Output is colored on a terminal or in CI, and plain when piped or when `NO_COLOR` is set. `FORCE_COLOR` keeps the colors. - `e2e` without a command prints the help on stderr and exits 2. - On a terminal 55 or more columns wide, `e2e --help` opens with the e2e wordmark: the site's lettering in block characters. Piped output and narrower terminals get the text alone. - When a newer `e2e` is on npm, a command run on a terminal ends with the version and the [`e2e upgrade`](#e2e-upgrade) command for the project's package manager. The registry is checked once a day. Nothing prints in CI, when stderr is not a terminal, for `e2e mcp` and `e2e upgrade`, or with `E2E_UPDATE_CHECK_DISABLED=1`, `NO_UPDATE_NOTIFIER=1`, `E2E_TELEMETRY_DISABLED=1`, or `DO_NOT_TRACK=1` set. ### Commands | Command | Description | | --- | --- | | [`e2e init`](#e2e-init) | Scaffolds a project | | [`e2e login`](#e2e-login) | Signs in to a model subscription | | [`e2e logout`](#e2e-logout) | Forgets a stored login | | [`e2e models`](#e2e-models) | Lists a stored login's models | | [`e2e guide`](#e2e-guide) | Prints the agent skill | | [`e2e mcp`](#e2e-mcp) | Serves live sessions over MCP | | [`e2e run`](#e2e-run) | Runs the tests | | [`e2e explore`](#e2e-explore) | Explores the app toward a goal | | [`e2e list`](#e2e-list) | Prints the tests a run selects | | [`e2e cache`](#e2e-cache) | Inspects or clears the replay cache | | [`e2e upgrade`](#e2e-upgrade) | Upgrades e2e and its packages | | [`e2e feedback`](#e2e-feedback) | Sends feedback to the e2e team | | [`e2e telemetry`](#e2e-telemetry) | Shows or switches usage telemetry | An option with no default listed is off unless given. #### `e2e init` Scaffolds a project and adds `e2e` to `devDependencies`. It skips existing config and test files. ```bash npm npx e2e init --yes npx e2e init --yes --gateway openrouter --skills .agents/skills --mcp .mcp.json --install ``` ```bash pnpm pnpm dlx e2e init --yes pnpm dlx e2e init --yes --gateway openrouter --skills .agents/skills --mcp .mcp.json --install ``` ```bash bun bunx e2e init --yes bunx e2e init --yes --gateway openrouter --skills .agents/skills --mcp .mcp.json --install ``` | Option | Description | | --- | --- | | `[directory]` | Where to scaffold; created when missing. Default: the working directory. | | `-y`, `--yes` | Skips the prompts. Each answer not given as a flag takes its default: Web engine, Vercel AI Gateway, skill and MCP config in both locations, no install. | | `--example ` | Starts from an example app: `with-vite`, `with-next`, `with-astro`, `with-angular`, `with-expo`, `with-swiftui`, `with-compose`, `with-kotlin-multiplatform`, `with-flutter`, or `with-maui`. The `with-` prefix is optional. See [Starting from an example](#starting-from-an-example). | | `--engine ` | The engines for a new config: `web`, `mobile`, or `none`. Comma-separated or repeated; `web,mobile` writes a target for each. Default: asked. | | `--gateway ` | The model gateway for agent steps in a new config: `vercel`, `openrouter`, `openai-compatible`, `chatgpt`, `copilot`, `opencode-console`, `grok`, or `none`. Default: asked. | | `--endpoint ` | Base URL for `--gateway openai-compatible`, and implies it. HTTPS, or HTTP on a loopback host. Default: asked; `http://127.0.0.1:11434/v1` with `--yes`. | | `--skills ` | Where to install the agent skill: `.agents/skills`, `.claude/skills`, or `none`. Comma-separated or repeated. Default: asked. | | `--mcp ` | Where to register the MCP server: `.mcp.json`, `.cursor/mcp.json`, or `none`. Comma-separated or repeated. Default: asked. | | `--install`, `--no-install` | Whether to install dependencies. Default: asked; `--no-install` with `--yes`. | - A flag answers its question and skips that prompt. - `--engine`, `--gateway`, and `--endpoint` apply only to a new config. With `--example` they are a usage error. | Exit code | When | | --- | --- | | 0 | The scaffold finished, or the wizard was cancelled. Cancelling leaves files untouched and removes a copied example. | | 2 | An unknown flag value. Exits before any write and lists the valid values. | | 2 | An invalid `package.json`. Exits before any write. | | 2 | No terminal on stdin and stdout, and no `--yes`. The message names the flag. | | 2 | `--example` with a directory that has files in it, or the example download failed. Nothing stays behind. | | 2 | The dependency installation failed. | | 130 | Ctrl-C or SIGTERM while the wordmark is written or the example downloads, or SIGINT during the dependency installation. A partial example is removed. | ##### Prompts On a terminal, the wizard opens with the e2e wordmark, written letter by letter over about a second, the way a pen writes it. With `--yes`, in CI, on a dumb terminal, or on a terminal under 11 rows, it prints at once. Piped output never shows it. For a new project, the wizard asks for: - **The engines.** Select one, both, or none. **Web** (Playwright, the default) adds `@e2e-dev/web`. **Mobile (iOS/Android)** adds `@e2e-dev/mobile`, defaults to iOS on macOS and Android elsewhere, and opens Settings. Selecting none adds no engine, for HTTP tests or your own. - **A model provider or subscription for agent steps:** Vercel AI Gateway, OpenRouter, an OpenAI-compatible endpoint, ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok. None is for tests that use only locators. The wizard adds the required packages and model config. Subscription users then sign in with [`e2e login`](#e2e-login). See [Models](https://e2e.tester.army/docs/models) and [Subscriptions](https://e2e.tester.army/docs/subscriptions). - **Where to install the agent skill** (see [e2e guide](#e2e-guide)). `.agents/skills/` serves Codex, Cursor, Copilot, Gemini CLI, OpenCode, and Zed; `.claude/skills/` serves Claude Code. Both are preselected. - **Where to register the [MCP server](https://e2e.tester.army/docs/reference/mcp).** `.mcp.json` for Claude Code, `.cursor/mcp.json` for Cursor. Both are preselected. After you confirm the file changes, the wizard asks whether to install the selected dependencies. The installer follows the project's `packageManager` field or lockfile, then the invoking package manager, then npm. ##### Files | Creates | Content | | --- | --- | | `package.json` | Created as a private ESM package when missing; otherwise gets only the missing dependencies and scripts. Sibling packages (`@e2e-dev/web`, `@e2e-dev/mobile`) get the caret range of the version released with the CLI, or that exact version when the CLI is a prerelease. Adds a `test:e2e` script that runs `e2e run`, unless one exists | | `e2e.config.ts` | One target per selected engine, with the app in its `app`. Playwright reads `APP_URL` (default `http://localhost:3000`) into `app.url` and shows `app.command` in a comment. Mobile sets `app: { bundleId: 'Settings' }` on iOS and `com.android.settings` on Android, and `workers: 1`. With a gateway, `agents.default` is `{ model, system }` | | `tests/example.e2e.ts` | An app-open test that passes against any page, plus a commented agent test. Device setup gets a Settings check instead. With both engines, `tests/web.e2e.ts` and `tests/mobile.e2e.ts` instead, each test with `requires: ['browser']` or `requires: ['device']` so the other target skips it | | `.gitignore` | Appends `node_modules/`, `.e2e/results/`, `.e2e/cache/`, `.e2e/sessions/`, `.e2e/report.json`, `.e2e/ai-trace.json`, `.e2e/junit.xml`, `.e2e/summary.md`, `.e2e/logs/`, and `.e2e/videos/`. Committing the cache is opt-in; see [commit the replay cache](https://e2e.tester.army/docs/cache#commit-the-replay-cache) | | `.agents/skills/e2e/`, `.claude/skills/e2e/` | `SKILL.md` and one `references/.md` per `e2e guide` topic. With both chosen, the Claude Code directory is a relative symlink (`../../.agents/skills/e2e`) to the copy, the layout `npx skills add` produces | | `.mcp.json`, `.cursor/mcp.json` | Registers `e2e mcp` without changing other servers | ##### Starting from an example `--example` copies one of the [example apps](https://github.com/tester-army/e2e/tree/main/examples) into a new or empty directory, then runs the wizard in its e2e project. That is the directory itself for a web or Expo app, and its `e2e/` folder for a native app. ```bash npm npx e2e init my-app --example with-expo ``` ```bash pnpm pnpm dlx e2e init my-app --example with-expo ``` ```bash bun bunx e2e init my-app --example with-expo ``` - The CLI downloads the example from GitHub, from the `e2e@` tag of the running CLI. When that tag doesn't exist, it uses `main` and warns. - A directory that holds only `.git`, `.DS_Store`, or `Thumbs.db` counts as empty. - The example's config, tests, and scripts stay as they are. The wizard asks only for the skill, the MCP server, and the install, and adds missing `.gitignore` entries. For a native app these files go in `e2e/`, so open your coding agent there. - The e2e project's `package.json` takes the directory's name (with `-e2e` beside a native app). `e2e`, `@e2e-dev/web`, and `@e2e-dev/mobile` get the ranges `init` writes for a new project. - The closing `Next:` line names the example's test script. A native app needs a build first; its `README.md` says how. ##### Re-running init - An existing config is kept, and no packages are added for it. - MCP registration keeps extra settings on the e2e server, such as environment variables and client timeouts, even when it repairs the launch command. A current registration stays byte-for-byte unchanged. - Once a skill directory exists, later runs refresh copies that differ from the installed version. They never add a directory the project did not choose. `--skills` and `--mcp` name the exact locations instead and leave the others as they are. - A `.claude/skills/e2e/` copy from an earlier version becomes the symlink when it holds only the shipped files. A copy with other files stays a copy and is refreshed. - `init` never writes through a skill directory that is a symlink elsewhere, or sits under one. `--yes` skips it with a warning naming the link and its target. An interactive run offers to replace it with a copy, or with the link where the link goes. - A copy with a directory where a file goes, or a file where a directory goes, is skipped with a warning naming the entry. `init` removes nothing to repair it. - An MCP config file reached through a symlink, or under a linked directory, is skipped with the same warning and left unchanged, so the server is not registered there. A plain config is merged as usual. #### `e2e login` Signs in to a subscription and stores the credentials in `$XDG_CONFIG_HOME/e2e/oauth.json`, by default `~/.config/e2e/oauth.json`. See [Subscriptions](https://e2e.tester.army/docs/subscriptions) for the model configuration. ```bash npm npx e2e login openai # ChatGPT, browser sign-in npx e2e login openai --device # sign in with a device code npx e2e login github-copilot --from-gh # use your GitHub CLI login npx e2e login github-copilot --client-id [--enterprise-url ] npx e2e login opencode-console # OpenCode Console, device code npx e2e login spacexai # SuperGrok, device code npx e2e login # choose a provider ``` ```bash pnpm pnpm exec e2e login openai # ChatGPT, browser sign-in pnpm exec e2e login openai --device # sign in with a device code pnpm exec e2e login github-copilot --from-gh # use your GitHub CLI login pnpm exec e2e login github-copilot --client-id [--enterprise-url ] pnpm exec e2e login opencode-console # OpenCode Console, device code pnpm exec e2e login spacexai # SuperGrok, device code pnpm exec e2e login # choose a provider ``` ```bash bun bunx e2e login openai # ChatGPT, browser sign-in bunx e2e login openai --device # sign in with a device code bunx e2e login github-copilot --from-gh # use your GitHub CLI login bunx e2e login github-copilot --client-id [--enterprise-url ] bunx e2e login opencode-console # OpenCode Console, device code bunx e2e login spacexai # SuperGrok, device code bunx e2e login # choose a provider ``` | Option | Description | | --- | --- | | `[provider]` | The subscription: `openai`, `github-copilot`, `opencode-console`, or `spacexai`. Default: a picker. | | `--device` | ChatGPT: shows a code to enter on another device instead of opening a browser. | | `--from-gh` | GitHub Copilot: reuses the signed-in GitHub CLI's token. | | `--client-id ` | GitHub Copilot: the client id of your GitHub OAuth App, with the device flow enabled. | | `--enterprise-url ` | GitHub Copilot: the GitHub Enterprise host. | | Exit code | When | | --- | --- | | 1 | The sign-in failed. | | 2 | An unknown provider. | | 130 | The sign-in was cancelled. | #### `e2e logout` Forgets the stored login of one provider. ```bash npm npx e2e logout # pick one of the stored logins npx e2e logout openai # forget one ``` ```bash pnpm pnpm exec e2e logout # pick one of the stored logins pnpm exec e2e logout openai # forget one ``` ```bash bun bunx e2e logout # pick one of the stored logins bunx e2e logout openai # forget one ``` | Option | Description | | --- | --- | | `[provider]` | The login to forget: `openai`, `github-copilot`, `opencode-console`, or `spacexai`. Default: a picker over the stored logins. | #### `e2e models` Asks the vendor which models the stored login serves and prints one line per model. For OpenCode Console it asks with `OPENCODE_API_KEY` when set. ```bash npm npx e2e models # every stored login npx e2e models openai # one provider ``` ```bash pnpm pnpm exec e2e models # every stored login pnpm exec e2e models openai # one provider ``` ```bash bun bunx e2e models # every stored login bunx e2e models openai # one provider ``` | Option | Description | | --- | --- | | `[provider]` | `openai`, `github-copilot`, `opencode-console`, or `spacexai`. Default: every stored login, plus OpenCode Console when only `OPENCODE_API_KEY` is set. | Each line holds: - the id `chatgpt()`, `copilot()`, `opencodeConsole()`, or `grok()` takes; - the vendor's name for the model; - its notes: reasoning levels, vision, preview, hidden; for Copilot `not enabled` or `no chat or responses` when `copilot()` cannot call it; for OpenCode Console the plan that serves it, Zen or Go. Only a call finds out whether a Copilot account can use a listed model. | Exit code | When | | --- | --- | | 1 | No credential, or none for the named provider. The message names the sign-in command. | | 1 | One of several logins failed to list. The others still print. | | 2 | An unknown provider. | See [Pick a model](https://e2e.tester.army/docs/subscriptions#pick-a-model). #### `e2e guide` Prints the skill that `init` installs; `npx skills add tester-army/e2e` installs the same skill. See [Coding agents](https://e2e.tester.army/docs/coding-agents). ```bash npm npx e2e guide # the overview npx e2e guide setup # one topic ``` ```bash pnpm pnpm exec e2e guide # the overview pnpm exec e2e guide setup # one topic ``` ```bash bun bunx e2e guide # the overview bunx e2e guide setup # one topic ``` | Option | Description | | --- | --- | | `[topic]` | One of `setup`, `writing-tests`, `agent`, `running`, `explore`, `debugging`, `mcp`, or `bug-bash`. Default: the overview. An unknown topic exits 2 and names the valid ones. | #### `e2e mcp` An MCP server on stdio that gives a coding agent a live session on the app. ```bash npm npx e2e mcp [--config ] [--target ] [--headed] [--max-sessions ] ``` ```bash pnpm pnpm exec e2e mcp [--config ] [--target ] [--headed] [--max-sessions ] ``` ```bash bun bunx e2e mcp [--config ] [--target ] [--headed] [--max-sessions ] ``` | Option | Description | | --- | --- | | `--config ` | The config every session loads unless `open_session` names one. Default: the nearest `e2e.config.ts`, `.mts`, `.js`, or `.mjs`. | | `--target ` | The target every session opens on. Default: the only target, or the one `open_session` names. | | `--headed` | Shows the UI in sessions whose `open_session` does not set `headed`, when the engine supports it. | | `--max-sessions ` | Sessions open at once, each with its own browser or device, 1 through 16. Default: 4. | The [MCP server](https://e2e.tester.army/docs/reference/mcp) page covers these options in full, plus the tools and the session rules. #### `e2e run` Runs the tests the config discovers and writes `report.json` under the [output directory](https://e2e.tester.army/docs/reference/config#output). ```bash npm npx e2e run npx e2e run tests/signup.e2e.ts --tag smoke npx e2e run tests/signup.e2e.ts:12 npx e2e run tests/agent npx e2e run 'tests/**/*.smoke.e2e.ts' npx e2e run --target web,webkit --tag smoke,billing --tag-mode all npx e2e run --tag smoke --exclude-tag slow npx e2e run --grep checkout --grep-invert '/refund/i' npx e2e run --last-failed npx e2e run --changed-since origin/main npx e2e run --shard 2/3 npx e2e run --reporter json --workers 4 --retries 2 npx e2e run --max-failures 3 npx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts npx e2e run --reporter list,junit ``` ```bash pnpm pnpm exec e2e run pnpm exec e2e run tests/signup.e2e.ts --tag smoke pnpm exec e2e run tests/signup.e2e.ts:12 pnpm exec e2e run tests/agent pnpm exec e2e run 'tests/**/*.smoke.e2e.ts' pnpm exec e2e run --target web,webkit --tag smoke,billing --tag-mode all pnpm exec e2e run --tag smoke --exclude-tag slow pnpm exec e2e run --grep checkout --grep-invert '/refund/i' pnpm exec e2e run --last-failed pnpm exec e2e run --changed-since origin/main pnpm exec e2e run --shard 2/3 pnpm exec e2e run --reporter json --workers 4 --retries 2 pnpm exec e2e run --max-failures 3 pnpm exec e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts pnpm exec e2e run --reporter list,junit ``` ```bash bun bunx e2e run bunx e2e run tests/signup.e2e.ts --tag smoke bunx e2e run tests/signup.e2e.ts:12 bunx e2e run tests/agent bunx e2e run 'tests/**/*.smoke.e2e.ts' bunx e2e run --target web,webkit --tag smoke,billing --tag-mode all bunx e2e run --tag smoke --exclude-tag slow bunx e2e run --grep checkout --grep-invert '/refund/i' bunx e2e run --last-failed bunx e2e run --changed-since origin/main bunx e2e run --shard 2/3 bunx e2e run --reporter json --workers 4 --retries 2 bunx e2e run --max-failures 3 bunx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts bunx e2e run --reporter list,junit ``` ##### Positional arguments ```bash e2e run [files...] ``` Test files, directories, or globs relative to the project root: | Positional | Selects | | --- | --- | | A file path (`tests/signup.e2e.ts`) | That file. | | A directory (`tests/agent`, `.`) | Every discovered test file beneath it. | | A bare name (`signup.e2e.ts`, `agent/signup.e2e.ts`, `signup`) | Every discovered file whose path ends with it at a segment boundary; the first two both select `tests/agent/signup.e2e.ts`. Without a `/` or `.`, it matches the file name without extensions, so `signup` selects it too. | | Any of these with `:line` (`tests/signup.e2e.ts:12`, `signup:12`) | Only the test declared at that line of each file it names: the line its `test(` call opens on, as an editor or a report's `source` shows it. | | A quoted glob (`'tests/**/*.smoke.e2e.ts'`) | Every discovered file it matches, in the grammar of the config `tests` globs: `*`, `?`, and a whole `**` segment. | - **Lines.** A line that names one member of a serial group runs the group whole. Several lines of one file add up; the file without a line selects it whole. When no test is declared at a named line, `NO_TESTS` names the positional and the lines the file declares tests at. A positional that exists on disk as written is a path, even when it ends in `:12`. - **Globs.** Braces, character classes, and extglobs are `INVALID_GLOB` with a hint, not a pattern that matches nothing. There is no expansion, so name one glob per alternative. `./`, `..`, a trailing `/`, and an absolute path inside the project resolve as for any positional path. An existing file or directory is selected by its path, whatever characters its name holds. - Positionals narrow the config globs and the tag, platform, and capability filters; they never bypass them. Names and globs are case-sensitive on every OS. A path outside the project root is `COLLECTION_ERROR`. - The runner imports every discovered test file, even when you select a few, so a selected test finds its session setup in another file. Other ordinary tests stay unselected. A collection error fails the run when it is in a selected file, or in any file of a run with no positionals. A file the positionals left out that fails to import or register its tests is skipped with a notice. So a half-written file elsewhere does not stop `npx e2e run tests/login.e2e.ts`. If a selected test's session then has no setup test, the error names that file as where the setup may be. - A positional that starts with `-` and names no file is a usage error, exit 2. `pnpm test:e2e -- --headed` produces it: the `--` reaches e2e, which would otherwise take `--headed` for a file. The message names the direct form, `pnpm exec e2e run --headed`. For npm it is `npm exec -- e2e run --headed`; npm needs the separator before the binary to pass its flags through. - When nothing is left to run, `NO_TESTS` names the cause: no file matched the globs (with look-alikes such as `tests/login.test.ts`), a positional matched nothing (with the nearest path), a matched file registered no tests, or every test was filtered out. ##### Flags **Selection** | Option | Description | | --- | --- | | `--config ` | Explicit config path. Default: the nearest `e2e.config.ts`, `.mts`, `.js`, or `.mjs` (see [Loading](https://e2e.tester.army/docs/reference/config#loading)). A missing file is `CONFIG_NOT_FOUND`. | | `--target ` | Targets by name, comma-separated or repeated. Default: every configured target. Only selected targets' app commands start. An unknown ID is `UNKNOWN_TARGET`. | | `--tag ` | Only tests with the tags, comma-separated or repeated: any of them, or every one with `--tag-mode all`. | | `--tag-mode ` | How several tags combine: `any` or `all`. Default: `any`. | | `--exclude-tag ` | Leaves out tests with any of the tags, comma-separated or repeated, whatever `--tag` or a positional selected. A serial group still runs whole when any member is selected; tag the serial `describe` to exclude the flow. | | `--grep ` | Only tests whose title matches one of the regular expressions; repeatable. The title is the describe titles and the test title joined by one space (`checkout pays`), without the file or the tags. A bare pattern is case-sensitive; `'/pattern/i'` adds flags. A pattern that does not compile is a usage error, exit 2. | | `--grep-invert ` | Leaves out tests whose title matches one of the regular expressions; repeatable, with the text and forms of `--grep`. With both, the run keeps what `--grep` matches minus what `--grep-invert` matches. | | `--changed-since ` | Runs the configured [test filters](https://e2e.tester.army/docs/filters) on the change since the ref: every file that differs from the merge base of the ref and `HEAD`, uncommitted and untracked files included. Without it, no filter runs. With no filters configured, a warning says every test runs. | | `--last-failed` | Only the tests the previous run did not pass; the list after these tables says which. | | `--shard ` | Runs one contiguous slice of the selected tests, such as `2/3`, cut after every other flag. Slices differ by at most one test. A serial group counts as one and is never split. Each shard brings only the setup tests its own tests need. Skipped tests are not cut: the shard that runs the nearest test before one reports it, else the first shard, so only one shard does. Every shard runs the same command with a different index. The order is the report order, so a suite shards the same way on every machine. A shard with no test is `NO_TESTS` unless `--pass-with-no-tests`. See [Sharding](https://e2e.tester.army/docs/ci#sharding). | | `--pass-with-no-tests` | Exits 0 instead of `NO_TESTS` when nothing is selected. A run with nothing to run prepares no engine and starts no app or service command. | **Execution** | Option | Description | | --- | --- | | `--headed` | Visible UI when the engine supports it. | | `--agent ` | Runs unpinned tests once per selected agent, comma-separated or repeated. Default: `default`. For a test with `{ agent }`, it selects the shared names when they overlap, else keeps the test's own. An unknown name is `INVALID_CONFIG`. | | `--retries ` | Replaces `config.retries`; an integer 0-10. Default: the config. | | `--workers ` | Replaces `config.workers`; an integer 1-1024. Default: the config. | | `--action-timeout ` | Replaces [`actionTimeout`](https://e2e.tester.army/docs/reference/config#actiontimeout) for this run; an integer 1-2147483647, anything else is a usage error, exit 2. Default: the config, else `30000`. A call's own `timeout` still wins. While you write a test, a low value such as `5000` makes a wrong locator fail with `LOCATOR_NOT_FOUND` after 5 s instead of 30 s. | | `--assertion-timeout ` | Replaces [`assertionTimeout`](https://e2e.tester.army/docs/reference/config#assertiontimeout) for this run; an integer 1-2147483647, anything else is a usage error, exit 2. Default: the config, else `5000`. A matcher's own `timeout` still wins. | | `--max-failures ` | Stops the run once this many tests (at least 1) have failed or timed out. Nothing more is dispatched, and tests running on other workers end `interrupted`. Tests not started, including the rest of the file whose test reached the limit, are skipped with cause `failure-limit` and the reason `run stopped after 3 failures (--max-failures 3)`. Exits 1, the failures' own code, not 130. | | `--max-duration ` | Overrides [`maxDuration`](https://e2e.tester.army/docs/reference/config#maxduration) for one run; an integer 1-2147483647. Default: the config. Exits 1, not 130. | | `--repeat-each ` | Runs every selected test this many times, at least 1. Default: `1`. Each run is its own result, with `repeat` 0 through `n - 1` and its own `id`. The list reporter titles later runs `(repeat #1)`, `(repeat #2)`, and puts their artifacts under a `repeat-` directory. Setup tests run once. With the replay cache on, the first run records and the rest replay; add `--no-cache` to call the model each time. The `list` summary adds a `Repeats` row (`1 of 2 tests passed all 5 runs`) and one line per test that did not pass every run, naming the failed runs. The `markdown` page has the same tally. | | `--no-cache` | Turns the replay cache off for this run, whatever `config.cache` says. | | `--strict-cache` | Turns on `cache.strict` for this run. A step whose recording exists but no longer replays fails with `REPLAY_STALE` instead of handing off to the agent. Retries replay too, and the run never writes the cache. See [Fail on a stale recording](https://e2e.tester.army/docs/cache#fail-on-a-stale-recording). | | `-u`, `--update-snapshots` | Writes each stored screenshot [`toHaveScreenshot`](https://e2e.tester.army/docs/reference/expect#screenshot-matchers) finds missing or different, and passes the matcher. Without it, a missing one is written and fails; in CI it is only attached to the results. | | `-w`, `--watch` | Runs, then reruns when a test file or the config changes, or on a key. See [Watch mode](#watch-mode). | **Output** | Option | Description | | --- | --- | | `--reporter ` | Replaces the built-in IDs in `config.reporters`, comma-separated: `list`, `json`, `junit`, `markdown`. Default: the config. `json` with `list` is `INVALID_CONFIG`. [Reporter objects](https://e2e.tester.army/docs/reference/reporters) in the config keep running. | | `--output ` | The results directory for this run, over the config's [`output`](https://e2e.tester.army/docs/reference/config#output): the report, the reporter files, the AI trace, `results/`, and `sessions/`. Default: the config, else `.e2e`. Resolved from the project root; a directory the run cannot own is `INVALID_CONFIG`. `results/` is cleared once the tests start; a `--last-failed` rerun keeps what the previous report names. | | `--cache-dir ` | The replay cache directory for this run, over [`E2E_CACHE_DIR`](https://e2e.tester.army/docs/reference/environment#replay-cache) and [`cache.dir`](https://e2e.tester.army/docs/reference/config#cache). Default: `E2E_CACHE_DIR`, else `cache.dir`, else `.e2e/cache`. Resolved from the project root; `~` or a leading `~/` is the home directory. The cache mode stays as configured. With a custom `cache.store` it is `INVALID_CONFIG`. | | `--debug` | Phase timings and an agent step table on stderr after the run; see [Debug output](#debug-output). | | `--ai-trace` | Records every model call to `/ai-trace.json`; see [Output](#output). | | `--trace [mode]` | Which tests keep a trace page (`/results//trace.md`): `off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries`. Default: the config. `--trace off` writes none for a test without its own `trace`. See [trace](https://e2e.tester.army/docs/reference/config#trace). | | `--video [mode]` | Which attempts record a video (WebM on a browser engine, MP4 on a device engine), with the modes of `--trace`. Default: the config. Only targets whose engine can record get one; the run names the others in one notice. See [video](https://e2e.tester.army/docs/reference/config#video). | `--trace` and `--video` apply over the config's and every target's value; a test's own `trace` or `video` still wins. A bare flag is `on`: bare `--trace` keeps a page for every test. The value is optional and greedy: `--trace tests/a.e2e.ts` reads the file as a mode and fails. Write `--trace=`, or put test files before the flag. The removed `--artifacts ` fails with the `--output` it maps to: `--artifacts out/artifacts` is `--output out`, and a directory whose parent is the project root or outside it maps to `--output .e2e`. `--last-failed` selects the tests the previous run did not pass. It reads them from the `report.json` this run overwrites, `/report.json` (`--output`, else the config's [`output`](https://e2e.tester.army/docs/reference/config#output)): - Tests that failed, timed out, or were interrupted. - Tests skipped because a setup, a serial predecessor, a hook, or the worker failed, or because `--max-failures` or `--max-duration` stopped the run first. - Every test a failed `beforeAll` or `afterAll` selected in its describe or file, on that target; for a setup, the tests that consume its session. A rerun that runs the hook again reports it again if it still fails. A rerun that runs any test of its scope without it failing resolves it. - A test another filter (`--grep`, `--tag`, `--target`, `--agent`, a positional) leaves out of the rerun stays owed. The rerun's report carries its last result, and the hook failure whose scope holds it, under `run.carried`. The next `--last-failed` selects it again, and the [`@e2e-dev/github`](https://e2e.tester.army/docs/github) page stays red until it passes (⏹️ for a test the run before interrupted). The rerun's own exit code and summary count only what it ran. - The rerun keeps the artifacts the previous report names and writes its own under each test's `rerun-/`, so no attempt overwrites an earlier one's files. - A test is named per target and agent. - No report is `NO_LAST_RUN`. When nothing failed in the previous run, it selects nothing: `NO_TESTS` unless `--pass-with-no-tests`. Out-of-range `--retries` and `--workers` fail with `INVALID_CONFIG` before anything starts, as the config value would. `--target`, `--tag`, `--exclude-tag`, and `--agent` take each name once. An empty value (`--tag ''`, or `--tag "$TAGS"` with the variable unset) is a usage error, exit 2, never a wider selection. An empty `--reporter` is the same usage error, never a run with no reporters. Filters only narrow, and each applies after the one before: positionals, then `.only`, `--tag`, `--exclude-tag`, `--grep`, `--grep-invert`, `--last-failed`, then the platform and capability rules, then `--shard`. When they leave nothing to run, `NO_TESTS` counts the tests each filter removed and names the tags and patterns. It marks each tag no test declares and names the nearest declared one. With `--changed-since`, the configured [test filters](https://e2e.tester.army/docs/filters) run last. ##### Debug output `--debug` appends two tables to stderr. Phase timings are summed across the runner and every worker. The agent step table lists every agent step in execution order: its per-phase split, model calls, tokens, and cost when the provider reported one. ```console $ npx e2e run --debug ... [e2e debug] phase timings (wall 14.2s) phase count total avg max scheduler 1 11.8s 11.8s 11.8s test.body 6 10.1s 1.7s 3.9s session.launch 6 1.1s 183ms 410ms engine.prepare 1 612ms 612ms 612ms collect 1 420ms 420ms 420ms config.load 1 118ms 118ms 118ms realm.import 1 96ms 96ms 96ms session.close 6 88ms 15ms 31ms engine.init 1 74ms 74ms 74ms [e2e debug] agent steps (execution order, model openai/gpt-6-luna-fast, total $0.0312) step total model observe action calls tokens in/out cached cost act "add a todo named groceries" 3.9s 2.8s 410ms 620ms 3 9412/388 61% (5740) $0.0198 assert "the list shows groceries" 1.2s 1.1s 88ms 0ms 1 3120/41 0% (0) $0.0114 ``` `cached` is the share of input tokens the provider served from its prompt cache, `-` when it reports no split. The report records the same per step (`model.cacheReadTokens`, `model.cacheWriteTokens`) and per run (`usage.modelCachedTokens`). ##### Output | Reporter | Writes | | --- | --- | | `list` | Terminal progress, failures with code frames, and a run summary. On a TTY, live progress shows the current agent step, model turns, tool calls, and reasoning excerpts. Untrusted fields are capped at 8 KiB and stripped of control characters. | | `json` | The report document to stdout, and nothing else there, so `e2e run --reporter json \| jq` works. It is the `/report.json` document, described by `schema/report-v1.schema.json` in the package. The shape changes only with the `schemaVersion` field (`report-1` today). | | `junit` | `junit.xml` beside the report. One `` per file, one `` per test-target pair named `title > path [target]`. A test-class error is a ``, any other an ``, a skip a `` with its reason. An interrupted test reached no verdict, so it is a `` whose message starts `interrupted:`. A test a [test filter](https://e2e.tester.army/docs/filters) left out is a `` whose message starts `left out by filter :`; other tests the run did not select are not listed. Run-level errors form a `run` suite. | | `markdown` | `summary.md` beside the report: counts, cost, errors, and failure details, each naming its trace page when one was kept. Exploration runs show findings. [`@e2e-dev/github`](https://e2e.tester.army/docs/github) uses this summary for pull request comments. | Every run that gets to its tests writes, under the [output directory](https://e2e.tester.army/docs/reference/config#output) (`.e2e` by default): - `report.json`, for scripts, whatever the reporters are; - the trace pages its [`trace`](https://e2e.tester.army/docs/reference/config#trace) mode keeps, in each test's directory under `results/`; - the reporters' files beside the report. A run that stops before its tests writes none of them and leaves the previous run's files in place. That covers a config or collection error, `NO_TESTS`, `NO_LAST_RUN`, `UNSUPPORTED_ARTIFACT`, an app command that fails to start, and an interrupt. Its errors are on the terminal and in the exit code. A `junit.xml` or `summary.md` that could not be written is a line on stderr, never a run error. `run.summary` in the report counts the results: - `discovered` is every test-target pair the runner collected, and `selected` the ones this run chose. The left-out pairs are in `results` as `selected: false`, but in no other count. - `passed`, `failed` (timed out included), `interrupted`, `flaky`, and `skipped` count the selected results and add up to `selected`. - `interrupted` is a test the run stopped before its verdict (a signal, `--max-failures`, `--max-duration`, or a run-level error such as `MODEL_UNAVAILABLE`), never a failure. - `executed` is the selected results that ran at least one attempt. With `--ai-trace`, the run also writes `/ai-trace.json`, an AI SDK devtools database. It holds one entry per model round trip: the exact prompt, tool definitions, response, usage, and provider metadata. Observations in it are already redacted, and inline image and file bytes become byte counts. Open it with [unbox-ai](https://github.com/tester-army/unbox-ai): `npx unbox-ai runs .e2e/ai-trace.json` lists its runs. See [Debugging a run](https://e2e.tester.army/docs/debugging#agent-steps). A [test filter](https://e2e.tester.army/docs/filters)'s model calls are not recorded. ##### Watch mode `--watch` runs the selection, prints the summary, and waits instead of exiting. ```bash npm npx e2e run --watch npx e2e run --watch tests/checkout.e2e.ts --headed ``` ```bash pnpm pnpm exec e2e run --watch pnpm exec e2e run --watch tests/checkout.e2e.ts --headed ``` ```bash bun bunx e2e run --watch bunx e2e run --watch tests/checkout.e2e.ts --headed ``` - A saved test file (one the config's [`tests`](https://e2e.tester.army/docs/reference/config#top-level) globs match) runs again alone and whole, in place of the positionals. - A saved source file near the tests (a helper or a fixture: `.ts`, `.js`, or `.json` under the directories the globs name, or beside the config) or a saved config reruns the selection. - Both keep the command line's other filters and the `p` and `t` filters. - Changes during a run wait for it and run next, together. During the first run only the config and its directory are watched, since the test directories come from the config. - On a terminal the screen clears before each rerun, and a banner names the trigger. - Each rerun reads the project's files afresh, including the helpers a test imports. On a terminal, keys drive it as in vitest: | Key | Runs | | --- | --- | | `a`, `enter` | every test the command line selected, with the `p` and `t` filters cleared | | `r` | the selection again, under the `p` and `t` filters | | `f` | the tests the last run did not pass, read from its report as `--last-failed` does, wherever they are; the positionals and the `p` filter do not narrow it | | `u` | the current selection with `--update-snapshots` | | `p` | prompts for a positional (a file, a bare name, a glob) the next runs select with; empty clears it | | `t` | prompts for a title pattern, as `--grep`; empty clears it | | `h` | shows the keys | | `q` | quits | - During a run, any of these keys (and space or `c`) cancels it, as an interrupt would. A second one forces it, and the session waits again. - Ctrl-C or Ctrl-D interrupts the run and ends the session, from inside a `p` or `t` prompt too. Escape does the same outside a prompt, and inside one only closes the prompt. Ctrl-Z suspends the process as usual. - Every rerun is a complete run: once the config loads, it writes `report.json` and the reporter files, and starts and stops the app command, unless it has nothing to run. To keep a dev server up across reruns, start it yourself and set [`reuseExisting`](https://e2e.tester.army/docs/reference/config#command) on the command. - `--last-failed` and the test filters apply to the first run only, so a rerun never calls a filter's model; a rerun that selects nothing passes empty. - On quit, the exit code is that of the last run that finished. - Without a terminal on stdin, only file changes rerun. `--watch` is not for CI: a process that never exits on its own hangs the job, so it is refused when `CI` is set. It is also refused with `--reporter json`, and exits 2 after the first run when the config's `reporters` include `json`. ##### Signals Signals escalate. 1. The first `SIGINT` or `SIGTERM` interrupts. The running test ends `interrupted`; its `afterEach` hooks and session close run within the cleanup budget, and every worker disposes its engine. Tests not started are skipped with cause `infrastructure-unavailable` and the reason `run interrupted before execution`. So the report lists every planned test, and `--last-failed` reruns the interrupted and the unstarted ones. The runner writes the report and exits 130. An interrupt before any test starts (during collection, an engine's `prepare`, or app startup) writes no report and leaves the previous one in place. 2. A second signal forces. Workers dispose their engines at once and are killed when the cleanup budget is spent. The report is still written, and so are `junit.xml` and `summary.md` when the `junit` and `markdown` reporters are on; a custom reporter's `onRunFinished` is abandoned. 3. A third signal exits on the spot, after killing the process groups of every app command the run started. A process attached through `reuseExisting` is left alone. A worker whose runner disappears disposes its engine and exits within the cleanup budget. #### `e2e explore` A run with no test file. The agent gets a goal, plans one exploration step at a time, reports defects through its `report_finding` tool, and ends with an assessment. See [Exploring without a test](https://e2e.tester.army/docs/explore). ```bash npm npx e2e explore npx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off' npx e2e explore --target web --max-steps 4 --headed npx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video ``` ```bash pnpm pnpm exec e2e explore pnpm exec e2e explore 'Explore the checkout flow like a first-time buyer and report anything off' pnpm exec e2e explore --target web --max-steps 4 --headed pnpm exec e2e explore 'Hunt for broken forms and dead links' --agent thorough --video ``` ```bash bun bunx e2e explore bunx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off' bunx e2e explore --target web --max-steps 4 --headed bunx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video ``` | Option | Description | | --- | --- | | `[goal]` | What to explore, in a sentence. Default: `Explore the app and find bugs`. | | `--config ` | Explicit config path, with the errors of `run`. Default: discovered. When no config is found and both stdin and stdout are a terminal, the [`e2e init`](#e2e-init) wizard runs in the working directory first. The exploration follows when nothing is left to do by hand: dependencies installed, a model with its key set or its sign-in done, and a web target (it names the URL it explores). Otherwise init ends with the steps left, then the `e2e explore` command as typed. Without a terminal, no config is `CONFIG_NOT_FOUND`. | | `--target ` | The one target to explore. Default: the first configured target; with several targets and no flag, it explores the first and says so on stderr. An unknown name is `UNKNOWN_TARGET`. | | `--agent ` | The configured agent to explore as. Default: `default`. Its model, `system`, tools, and context are reused. The built-in agent replaces a hand-rolled `StepExecutor`, with a notice. An entry with no model, a custom executor's included, is `MODEL_UNAVAILABLE` before anything starts. An unknown name is `INVALID_CONFIG`. | | `--session ` | Starts signed in. The setup test in the config's test files that declares this session runs first. The exploration restores what it saved, as a test with `{ session }` does. No other test runs. A name no setup test declares is `COLLECTION_ERROR`, naming the declared ones, before any app process starts. See [Start signed in](https://e2e.tester.army/docs/explore#start-signed-in). | | `--max-steps ` | Exploration steps at most, an integer 1-12. Default: `8`. Out of range exits 2. | | `--timeout ` | The run's wall clock, an integer 180000-900000. Default: `600000`. The last minute is kept for the closing assessment. Out of range exits 2. | | `--headed` | Requests visible UI when the engine supports it. | | `--reporter ` | As for `run`. Default: the resolved config value. `list` prints the exploration rows under its summary, `json` carries the record in the report, and `markdown` writes the findings to `/summary.md`. | | `--output ` | As for `run`. Default: the config, else `.e2e`. A finding's evidence screenshot is an attempt artifact under `/results`. | | `--debug` | As for `run`: phase timings and the agent step table. | | `--ai-trace` | As for `run`. | | `--trace [mode]` | As for `run`: a trace page of the exploration. Default: the config. | | `--video [mode]` | As for `run`: one video of the whole exploration (WebM on a browser engine, MP4 on a device engine). Default: the config. An engine that cannot record prints a notice. | For `--trace` and `--video`, a bare flag is `on`, and `retain-on-failure` keeps the page or video only when the exploration did not pass. The exploration is one attempt, so a retry mode records nothing, and a plan-time notice says to pass `--trace on` (or `--video on`). Put the goal before the flag, or write `--trace=` or `--video=`. ##### Report - The run has one result, under the virtual file `explore`, titled by the goal. - The exploration opens the app first: the target's `app.url`, or on a device the [pinned app](https://e2e.tester.army/docs/mobile#point-at-your-app), launched fresh. It installs no build, so a local device target needs `app.bundleId`. - With `--session`, the report also holds the setup test's result. It lists the project's other tests as skipped with cause `filtered`, as `run` lists the tests of a file no positional named. - The exploration runs with the replay cache off and zero retries, whatever the config says. A `--session` setup test runs as it would for `run`, with the config's agents, cache, and retries. - `/report.json` gains `run.explore`: the goal, the budgets, why the run ended (`finished`, `step-limit`, `time`, `stuck`, or `aborted`), the assessment, the steps, and the findings. - Artifacts go to a directory named from the goal's first words and a digest of the goal, target, and agent, such as `/results/explore-check-the-cart-totals-1a2b3c4d5e6f7a8b/`. A rerun of the same goal on the same target and agent writes to the same directory. ##### Step budgets Each step has its own budget: 40 actions and 40 model calls, unless the agent sets `maxSteps` or `maxModelCalls` (1 to 100). A step that reaches its action or time budget is time-boxed, and its findings still count. A failed step does not end the run. Three failed or blocked steps in a row with no new finding do (`stuck`). ##### Findings - A finding of kind `issue` is a functional defect; a `warning` is cosmetic. - Severity runs from `trivial` (1) to `critical` (5). The report lists issues before warnings, the most severe first. - Each finding has its location, the expected and observed behavior, the steps to reproduce, and a screenshot when capture is allowed and succeeds. After a secret fill, findings carry no screenshots; see [screenshots after a secret fill](https://e2e.tester.army/docs/reference/security#screenshots-after-a-secret-fill). ##### Outcomes | Outcome | Run status | Exit code | | --- | --- | --- | | Steps ran, no `issue` finding, and not every step was blocked | passed | 0 | | At least one `issue` finding | failed | 1 | | No step ran and nothing was found | blocked | 1 | | Every step was blocked, with no `issue` finding | blocked | 1, 2, or 3, from the first blocker | | Config, engine, or model provider error | error | 2 or 3 | | Ctrl-C | | 130 | #### `e2e list` A dry run: `list` selects tests exactly as `run` does, prints one line per test-target pair, and exits. It starts no app process, engine, or worker, and writes nothing. ```bash npm npx e2e list npx e2e list tests/signup.e2e.ts npx e2e list --tag smoke --target web npx e2e list --grep checkout --exclude-tag slow npx e2e list --reporter json npx e2e list --changed-since origin/main ``` ```bash pnpm pnpm exec e2e list pnpm exec e2e list tests/signup.e2e.ts pnpm exec e2e list --tag smoke --target web pnpm exec e2e list --grep checkout --exclude-tag slow pnpm exec e2e list --reporter json pnpm exec e2e list --changed-since origin/main ``` ```bash bun bunx e2e list bunx e2e list tests/signup.e2e.ts bunx e2e list --tag smoke --target web bunx e2e list --grep checkout --exclude-tag slow bunx e2e list --reporter json bunx e2e list --changed-since origin/main ``` | Option | Description | | --- | --- | | `[files...]` | As for [`run`](#positional-arguments). | | `--config`, `--target`, `--tag`, `--tag-mode`, `--exclude-tag`, `--grep`, `--grep-invert`, `--changed-since`, `--last-failed`, `--shard`, `--pass-with-no-tests` | As for [`run`](#flags). `--last-failed` reads `report.json` under the config's [`output`](https://e2e.tester.army/docs/reference/config#output). The configured [test filters](https://e2e.tester.army/docs/filters) run as for `run`, and their lines go to stderr. | | `--reporter ` | `list` or `json`. Default: `list`. | | Reporter | Writes | | --- | --- | | `list` | One line per pair in selection order: `file › title [target]`, with the describe path joined by ` › ` (`tests/billing.e2e.ts › checkout › pays [web]`), then each declared tag as `#tag` (`tests/billing.e2e.ts › checkout › pays [web] #smoke #billing`). A skipped test keeps its line, with `(skipped: )` appended, and a test that a test filter left out ends with `(left out by : )`. Setup tests are listed like any other. | | `json` | `{ "pairs": [...] }` to stdout. Each pair carries `id`, `file`, `title`, `titlePath`, `kind` (`test` or `setup`), `tags` (the tags the test declares, `[]` when none), `source` when the declaration is known, `session` when the test consumes one, `sessions`, `serialId` when the test is in a serial group, `target`, `agent`, `disposition` (`run`, `skip`, or `filtered` when a test filter left it out), `reason` when skipped or left out, and `filter`, the test filter's name, when one left it out. | Pairs a selection flag removed are not listed. [`list` from `e2e/runner`](https://e2e.tester.army/docs/reference/runner) runs the same selection in process and also returns the removed pairs. #### `e2e cache` Inspects or clears the replay cache. Each subcommand loads the discovered config or `--config `, reads the directory a run would, and exits 0 on an empty store. ```bash npm npx e2e cache ls npx e2e cache stats npx e2e cache clear npx e2e cache ls --config packages/web/e2e.config.ts ``` ```bash pnpm pnpm exec e2e cache ls pnpm exec e2e cache stats pnpm exec e2e cache clear pnpm exec e2e cache ls --config packages/web/e2e.config.ts ``` ```bash bun bunx e2e cache ls bunx e2e cache stats bunx e2e cache clear bunx e2e cache ls --config packages/web/e2e.config.ts ``` | Command | Prints | | --- | --- | | `e2e cache ls` | One row per entry: the test and target it was recorded for, the first 12 characters of its instruction digest, its age, and how many actions a replay would run (`(truncated)` when recording hit the action cap). Rows are grouped by test, then target, oldest first. | | `e2e cache stats` | The store directory, the number of readable entries, their total size, and the number of unreadable files when there are any. | | `e2e cache clear` | Deletes every cache file, then the directory when nothing else is left in it, and reports how many files went. | | Option | Description | | --- | --- | | `--config ` | Explicit config path. Default: the nearest `e2e.config.ts`, `.mts`, `.js`, or `.mjs` (see [Loading](https://e2e.tester.army/docs/reference/config#loading)). | | `--cache-dir ` | The store directory. Default: `E2E_CACHE_DIR`, else `cache.dir`, else `.e2e/cache/`. | ```text TEST TARGET INSTRUCTION AGE ACTIONS tests/billing.e2e.ts::upgrades web 4f1c0a9d3b62 2h 4 tests/signup.e2e.ts::signs%20up web 9ab7715ce004 3d 7 (truncated) ``` - An entry recorded before entries carried provenance prints `-` in those columns and still replays. - A file the store cannot read (truncated, over the 1 MiB ceiling, or from a newer schema) is counted separately on stderr, treated as a miss by a run, and removed by `clear`. - Files the runner never wrote are left alone. These commands read the file store only. With a custom `cache.store` they exit 2 and say so. #### `e2e upgrade` Upgrades `e2e` to the latest release, a dist-tag, or one version. It also moves the `@e2e-dev/*` packages in the same `package.json`, and installs them with the project's package manager. `e2e update` is the same command. `npx e2e` runs the installed version. When that version has no `upgrade` command yet, run `npx e2e@latest upgrade` one time. ```bash npm npx e2e upgrade # the latest release npx e2e upgrade 0.21.0 # one version npx e2e upgrade nightly # a dist-tag npx e2e update --dry-run # show the plan, change nothing ``` ```bash pnpm pnpm exec e2e upgrade # the latest release pnpm exec e2e upgrade 0.21.0 # one version pnpm exec e2e upgrade nightly # a dist-tag pnpm exec e2e update --dry-run # show the plan, change nothing ``` ```bash bun bunx e2e upgrade # the latest release bunx e2e upgrade 0.21.0 # one version bunx e2e upgrade nightly # a dist-tag bunx e2e update --dry-run # show the plan, change nothing ``` ```console $ npx e2e upgrade Upgrading with npm: e2e 0.19.0 → 0.21.2 @e2e-dev/web 0.13.0 → 0.15.2 @e2e-dev/kernel 0.1.2 → 0.2.1 package.json: "e2e": "^0.21.2", "@e2e-dev/web": "^0.15.2", "@e2e-dev/kernel": "^0.2.1" $ npm install ... Upgraded: e2e 0.21.2, @e2e-dev/web 0.15.2, @e2e-dev/kernel 0.2.1 Release notes: https://github.com/tester-army/e2e/releases/tag/e2e%400.21.2 Updated the agent skill in .agents/skills/e2e/ (12 files) ``` | Option | Description | | --- | --- | | `[version]` | A version such as `0.22.0`, or a dist-tag such as `latest` or `nightly`. Ranges are refused. Default: `latest`. | | `--dry-run` | Prints the versions, the new specs, the install command, and the skill copies to update, and changes nothing. | - **The project.** The nearest `package.json` from the working directory up that lists `e2e`. - **Sibling versions.** Each `@e2e-dev/*` package gets its release under the same dist-tag (`latest` for a named version), when that release's peer ranges accept the new versions. Otherwise it gets its newest release that does. When none does, the command exits 2. A deprecated release is never picked, and a prerelease only when `e2e` gets one. An integration follows the engine it plugs into. - **The package manager.** The one the project's lockfile or `packageManager` field names, or pnpm at a pnpm workspace root. A workspace member with neither uses its workspace root's. A lockfile above a project that no workspace lists doesn't count. - **The install.** The new specs go into `package.json`, each in the section it was in, then one `install` resolves them together. pnpm gets `--no-frozen-lockfile`, and Yarn `YARN_ENABLE_IMMUTABLE_INSTALLS=false`, so the lockfile can change in CI too. Ctrl-C reaches the package manager; the command waits for it to stop. - **A failed install.** `package.json` and any lockfile the install changed go back as they were. `node_modules` can keep some new versions until the next install. - **The saved range.** A named version or a prerelease is pinned exactly. A dist-tag keeps the `^` or `~` of the old spec. - **Left alone.** A package linked with `workspace:`, `file:`, `link:`, `catalog:`, a path, or a git URL, and an `@e2e-dev/*` package the registry doesn't have. When `e2e` itself is linked, the command exits 2. - **Nothing to do.** When every installed version already matches, the command installs nothing. Without `node_modules`, as under Yarn Plug'n'Play, it compares the specs in `package.json` instead. - **The registry.** `npm_config_registry` when the package manager sets it, else `https://registry.npmjs.org/`. - **The agent skill.** A copy in `.agents/skills/e2e` or `.claude/skills/e2e` is updated to the installed version's after the install, and when everything was already up to date. A `.claude/skills/e2e` symlink to the `.agents/skills` copy follows it. Files the new skill no longer ships stay. A symlink to a copy outside the project isn't written through, and without `node_modules`, as under Yarn Plug'n'Play, the command names `e2e init` instead. | Exit code | When | | --- | --- | | 0 | The packages were installed, were already at the target, or `--dry-run` printed the plan. | | 2 | A range, an unknown version or dist-tag, no `package.json` that lists `e2e`, a linked `e2e`, a sibling with no release that fits, or a failed install. | | 3 | A registry request failed or answered with an error status, or sent metadata the command could not read. | | 130 | Ctrl-C or SIGTERM during the install. `package.json` and the lockfile go back as they were. | #### `e2e feedback` Sends one report about e2e itself to the e2e team: something broke, the docs or the skill misled, or a capability is missing. A coding agent may send one when it hits a problem with e2e, not with the app under test. It first shows the user the `--dry-run` output and gets their agreement. ```bash npm npx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout" \ --repro "e2e list --grep checkout lists every test" npx e2e feedback --type docs -m "The mcp topic never says how to close a session" npx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run ``` ```bash pnpm pnpm exec e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout" \ --repro "e2e list --grep checkout lists every test" pnpm exec e2e feedback --type docs -m "The mcp topic never says how to close a session" pnpm exec e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run ``` ```bash bun bunx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout" \ --repro "e2e list --grep checkout lists every test" bunx e2e feedback --type docs -m "The mcp topic never says how to close a session" bunx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run ``` | Option | Description | | --- | --- | | `-m`, `--message ` | Required. One or two sentences on the problem. | | `--type ` | `bug`, `docs`, `feature`, or `other`. Default: `other`. | | `--task ` | What you were trying to do. | | `--expected ` | What you expected to happen. | | `--actual ` | What happened instead, with the error message. | | `--error-code ` | The e2e error code involved, such as `LOCATOR_AMBIGUOUS`. | | `--repro ` | For a bug: steps that reproduce it every time, without app content. | | `--approach ` | What you tried, workarounds included. | | `--command ` | The e2e command or API involved. | | `--agent ` | The coding agent and model sending it. | | `--allow-outdated` | Sends even when a newer e2e release exists. | | `--dry-run` | Prints the event that would be sent and sends nothing. | `--message` takes at most 2000 characters, `--command` and `--agent` 200, and `--task`, `--expected`, `--actual`, `--repro`, and `--approach` 1500. - **A bug needs `--repro`.** Without it, the command sends nothing and prints a prefilled [GitHub issue](https://github.com/tester-army/e2e/issues/new?template=bug_report.yml) instead, where the e2e team can ask for a reproduction. Add a minimal one and submit it yourself. - **Outdated releases are refused.** Before sending, the command asks the npm registry for the latest e2e release. When yours is older, it exits 2 and sends nothing, since the problem may be fixed already. Upgrade with [`e2e upgrade`](#e2e-upgrade), check that it still happens, and send it again, or pass `--allow-outdated`. - **Where it goes.** One `e2e_feedback` event, to the same PostHog project as [telemetry](https://e2e.tester.army/docs/telemetry#feedback), with the anonymous machine facts every telemetry event carries. - **Opting out.** Feedback is sent only when you run the command, so a saved `e2e telemetry disable` does not stop it. `E2E_TELEMETRY_DISABLED=1` and `DO_NOT_TRACK=1` do: with either set, the command exits 2 and sends nothing. So a CI job or a fleet that opted out sends nothing, even when an agent runs it. `E2E_TELEMETRY_DEBUG=1` prints the event like `--dry-run`. | Exit code | When | | --- | --- | | 0 | PostHog accepted the event, and the command printed a reference id. Also `--dry-run`, `E2E_TELEMETRY_DEBUG=1`, and `--type bug` without `--repro`: none of them send, so they exit 0 even with an opt-out variable set. The outdated-release check runs first, so an outdated install exits 2 on these too, unless `--allow-outdated` or an opt-out variable is set. | | 2 | A blank or too long text value, an outdated release without `--allow-outdated`, or `E2E_TELEMETRY_DISABLED=1` or `DO_NOT_TRACK=1` set. | | 3 | PostHog did not confirm the event. The report may still have arrived, so the message names its reference id. | Before sending, the value of every environment variable whose name holds `KEY`, `TOKEN`, `SECRET`, `PASS`, `_PWD`, `CREDENTIAL`, `AUTH`, `COOKIE`, or `PRIVATE` becomes `` when it is six or more characters long. That is the minimum for a config secret; a shorter one would rewrite ordinary words. Bearer tokens, URL credentials, private keys, JWTs, and common API key shapes become ``. Do not paste app data, page content, or test files. #### `e2e telemetry` Shows whether anonymous usage telemetry is on, and switches it. [Telemetry](https://e2e.tester.army/docs/telemetry) lists every property sent. ```bash npm npx e2e telemetry npx e2e telemetry disable E2E_TELEMETRY_DEBUG=1 npx e2e run ``` ```bash pnpm pnpm exec e2e telemetry pnpm exec e2e telemetry disable E2E_TELEMETRY_DEBUG=1 pnpm exec e2e run ``` ```bash bun bunx e2e telemetry bunx e2e telemetry disable E2E_TELEMETRY_DEBUG=1 bunx e2e run ``` | Command | Effect | | --- | --- | | `e2e telemetry`, `e2e telemetry status` | Prints `Status: enabled`, or `Status: disabled` with the reason: the variable that is set, the saved choice, or a preferences directory that cannot be written. `status` is the default. | | `e2e telemetry disable` | Saves the opt-out to `~/.config/e2e/telemetry.json` (`$XDG_CONFIG_HOME/e2e/` when set, `%APPDATA%\e2e\` on Windows) and prints the status. | | `e2e telemetry enable` | Saves the opt-in and prints the status. A variable set in the environment still wins, and the status says so. | - `E2E_TELEMETRY_DISABLED=1` and `DO_NOT_TRACK=1` turn telemetry off without touching the file. - `E2E_TELEMETRY_DEBUG=1` prints every event to stderr and sends nothing. - The first command on a machine, other than `e2e init` and `e2e telemetry`, prints a notice one time. In CI nothing is printed and no file is written. ## Exit codes | Code | Meaning | | ---: | --- | | 0 | All selected tests passed, were flaky, or were explicitly skipped | | 1 | Final test or setup failure, or test timeout | | 2 | CLI, config, collection, dependency, credential, model-config, or policy error | | 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure; `e2e feedback` that could not be delivered; `e2e upgrade` that could not reach the registry | | 4 | Internal runner invariant or unhandled runner error | | 130 | Interrupted by a user or CI signal | - Precedence for a mixed run is `130 > 4 > 3 > 2 > 1 > 0`. The report keeps every individual result; precedence affects only the process exit code. - Argument parsing failures exit 2 with a one-line error on stderr. ## Diagnostics - A message that rejects a name suggests the nearest valid one. - Keys from other runners' configs (`testDir`, `baseURL`, `webServer`, `projects`) say where that fact lives here. - A failing `import` in the config or a test file ends with the install or add command for the project's package manager, or the subpaths the package exports. - A model credential the provider rejects names the variable the provider reads.