Every runner error carries a class and a code. The class sets the exit code,
and the code names the cause. A model never selects either. e2e login,
e2e logout, and e2e models are the exception, under
Exit codes.
tests/upgrade.e2e.ts
import { isAgentError, test } from 'e2e';
test('upgrade to Pro', async ({ agent }) => {
try {
await agent.act('upgrade the workspace to Pro');
} catch (error) {
if (isAgentError(error) && error.code === 'STEP_TIMEOUT') {
// the step ran out of time; the message carries the last judgment
console.error(error.category, error.details);
}
throw error;
}
});Reference
Exit codes
| Class | Exit code |
|---|---|
| test | 1 |
| configuration | 2 |
| infrastructure | 3 |
| internal | 4 |
| interrupted | 130 |
- In a mixed run the highest-precedence code present wins,
130 > 4 > 3 > 2 > 1 > 0. The report keeps every individual result. - A setup assertion failure is a test failure and skips its dependents. Missing setup infrastructure keeps its configuration or infrastructure class.
- The
internalclass and exit code 4 are reserved for runner invariant violations. No first-party code raises a code in that class today. e2e login,e2e logout, ande2e modelsprint a failure’s message with no code and exit 1, or 130 when a sign-in or picker is cancelled.
| Codes | Description |
|---|---|
| Agent codes | The code of an AgentError |
| Runner codes | Plain strings in output and report |
| Test code errors | Thrown from test code |
AgentError
AgentError is the runner-classified agent failure, exported from e2e.
/** Runner-classified agent failure. */
export class AgentError extends Error {
/** Closed union, assigned by the runner, never taken from model text. */
readonly code: AgentErrorCode;
/** The error class in the table above; it sets the exit code. */
readonly category: 'test' | 'configuration' | 'infrastructure' | 'internal' | 'interrupted';
readonly retryable: boolean;
/** Structured facts: locator, expected, observed, match count, wait time. */
readonly details: ErrorDetails | undefined;
/** Sanitized, bounded prose; the same value as `message`. */
readonly explanation: string;
/** Artifact path, present only when evidence was captured and permitted. */
readonly screenshot?: string;
/** True when this failure reports a blocked step, not a product failure. */
readonly blocked: boolean;
}Test files load in their own module realm, so instanceof AgentError can be
false. Use isAgentError from e2e and branch on code, as in the example
at the top of this page.
Agent codes
The members of the closed AgentErrorCode union. blocked: names the
category a blocked verdict reports as the obstacle.
| Code | When it happens | Class | Exit |
|---|---|---|---|
AUTH_CREDENTIAL_UNAVAILABLE | A Secret or credentials.user() names a credential missing from config.credentials, the call ran outside an active runner, or the password’s provider returned no string of at least 6 characters (code points) | configuration | 2 |
AUTH_CREDENTIAL_INVALID | The application rejected a configured credential (blocked: credentials) | configuration | 2 |
SECRET_UNAVAILABLE | A Secret or secrets.get() during a run names a secret missing from config.secrets, the secret’s provider returned no string of at least 6 characters (code points), or an engine resolved a secret it did not declare in its secrets | configuration | 2 |
ENVIRONMENT_UNAVAILABLE | The environment under test is down or unreachable beyond the app itself (blocked: environment) | infrastructure | 3 |
SEED_DATA_MISSING | Data the step depends on does not exist in the environment (blocked: seed_data) | configuration | 2 |
TEST_SETUP_FAILED | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a test.extend() fixture that broke its contract: it returned without calling use(), called it twice, or redefined a fixture the target’s engine contributes | configuration | 2 |
AUTOMATION_UNSUPPORTED | The step needs an interaction the available toolset cannot perform, such as drag-and-drop (blocked: automation) | test | 1 |
MODEL_UNAVAILABLE | No model configured: the built-in agent’s agents.<name>.model holds no AI SDK model instance (the message names the selected agent), or GitHub Copilot does not serve the configured model to this account (model_not_supported; the message names the model). Recorded once as a run-level error, and the run stops. In a test run, a missing model is found when the agent fixture is first acquired, and an unserved model at its first call. e2e explore runs the built-in agent, so it checks for a missing model before anything starts, a custom executor’s entry included. A custom executor without a model is not checked: its act and assert steps run, and waitFor or extract fails with this code on its first call. Evicts no recording. | configuration | 2 |
POLICY_DENIED | The security policy refused an operation: a denied destination, a forbidden fill, pixels after a secret fill, or a read from a secure field. Policy denials lists every case. An unknown tool or an undeclared field goes back to the model as the call’s result, never as this code | configuration | 2 |
APP_UNREACHABLE | The app URL could not be reached, or an app command exited early or exceeded startupTimeout. With a log, the startup error quotes the log’s output since the command started, command.env values redacted, or says there was none. Without one, it says to set a log. A command that fails to spawn reports the spawn error. Under the agent, also any engine fault with no more specific code. On a device, a busy or wedged iOS automation runner reports here; the message names the runner, the session and device, and the recovery. See Mobile | infrastructure | 3 |
APP_ALREADY_RUNNING | readyUrl answered before the target’s app.command was spawned, and reuseExisting was not in effect. Stop that process, or set reuseExisting: true locally | infrastructure | 3 |
MODEL_PROVIDER_FAILED | Provider transport failure (network error, 5xx, rate limit, or no response within 120 s) after the adapter’s transport retries: up to 5 per model call, honoring retry-after, within the remaining step timeout. A stalled request is aborted at 120 s and sent again; an agent.act step’s --debug transcript notes it. The 120 s covers the whole request, so a model that needs longer for one answer fails here too. An error body the provider package cannot read shows as the HTTP status and the body, secrets redacted, cut at 1 KB. Evicts no recording | infrastructure | 3 |
CANCELLED | The runner cancelled the operation without a process signal: an e2e mcp open_session cut short by a client cancel, a closed connection, or the server’s shutdown, after it stopped what it had started | infrastructure | 3 |
MODEL_OUTPUT_INVALID | Structured output still failed validation after the repair budget was spent. Under a decision model: an answer without a valid choice distribution, a choice the request never offered, or a question the provider refused. Decisions are not repaired | test | 1 |
APP_NOT_OPEN | An operation needed an open app: app.open() had not run, or the engine reported invalid state | test | 1 |
LOCATOR_NOT_FOUND | No matching node within the deadline, or an agent action refers to a node that is no longer available | test | 1 |
LOCATOR_AMBIGUOUS | More than one match for a single-node operation. Fails at once, with no polling. | test | 1 |
ACTION_FAILED | Actionability failure, operation timeout, or an action that may already have committed | test | 1 |
STEP_BUDGET_EXHAUSTED | The model-call or action-step budget ran out before a conclusion | test | 1 |
STEP_TIMEOUT | The invocation deadline expired, including an operation that timed out within a poll tick of it. Carries the last judgment where one exists, with the vision hint when that judgment was inconclusive | test | 1 |
CONTEXT_OVERFLOW | A request exceeded the model’s context window. act shortens its history and retries once if that makes the request smaller; judgments fail at once. Lower agents.<name>.maxObservationBytes, or split the instruction into shorter steps. Reported as blocked: automation | test | 1 |
MODEL_REFUSED | The model refused the request (finish reason content-filter) and returned no usable answer. Rephrase the instruction or use another model. Reported as blocked: automation | test | 1 |
ASSERTION_FAILED | agent.assert judged false, or a locator, browser, or value matcher failed | test | 1 |
ASSERTION_INCONCLUSIVE | agent.assert could not judge: the screen did not show enough to decide either way (the relevant part is not on screen, still loading, or unreadable). agent.extract raises it when the model answers that the screen does not show the requested data, instead of returning a placeholder. A failure, never a pass; the explanation says what was missing. Judged from the tree alone with vision off, the message can add a vision hint | test | 1 |
AUTHENTICATION_FAILED | The executor concluded that signing in failed inside the app: a product verdict. AUTH_CREDENTIAL_INVALID is the blocked counterpart, for a configured credential the app rejects | test | 1 |
STEP_NO_CONCLUSION | The executor used every turn without calling complete_step. The step fails with the agent’s last summary | test | 1 |
REPLAY_STALE | Under cache.strict or --strict-cache, the step’s recording exists but no longer replays, so the step fails with no model call instead of handing off to the agent. step.cache.reason is one of invalid-entry, wrong-context, target-not-found, target-ambiguous, viewport-changed, action-failed, action-uncertain, end-mismatch, or no-entry while the cache directory (not a custom cache.store) holds a recording of the step under another key, which the message names. Not retried: every attempt replays the same recording. To fix it, re-record with a read-write run without the flag, with cache.strict off if the config set it, then commit the changed entry. The message names the knob that turned strict on and the configured cache directory (or cache.store) | configuration | 2 |
Runner codes
These appear in terminal output and in the report as plain string codes.
| Code | When it happens | Class | Exit |
|---|---|---|---|
CONFIG_NOT_FOUND | --config <path> names a file that does not exist, or no e2e.config.ts, e2e.config.mts, e2e.config.js, or e2e.config.mjs exists from the working directory up to the repository root | configuration | 2 |
CONFIG_AMBIGUOUS | Both e2e.config.ts and e2e.config.mts in one directory, or both e2e.config.js and e2e.config.mjs in one with no TypeScript config | configuration | 2 |
CONFIG_LOAD_FAILED | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: options a factory the config calls refuses, or a secrets.get() handle turned into a string, is INVALID_CONFIG | configuration | 2 |
INVALID_CONFIG | A factory the config module calls (web(), mobile(), defineEngine(), decisionExecutor(), decisionFilter()) refused its options, with the factory’s own message. Or: an unknown key, out-of-range value, invalid enum, or missing targets; a target key outside { name, platform, engine, app, trace, video }; a target app key that is unknown or malformed; app on a target without an engine; a web() target without app.url or with a device field; a mobile() target without app.bundleId or app.appPath, with app.url (not supported yet), with a link in bundleId, or with an unknown permission name or state (the message names the nearest name); an unknown key in a web() or mobile() option, web({ connect }) and web({ basicAuth }) included (the message names the nearest one); a trace or video that is not a recording mode; the removed artifact kinds list (artifacts: [...] or artifacts.kinds) or artifacts.trace block (the message names the trace mode they meant); the removed artifacts.video block or video artifact kind (the message names the video option); the removed specVersion key; a secrets.get() in an engine option that names no secrets entry (a credential’s password is not one); a secrets.get() reference turned into a string while the config evaluates (a template literal, String()), or given where a string is needed (an app.command args entry or env value; only an engine option that declares secrets takes one); an app.command args entry or env value that is not a string (an undefined env value is dropped instead); an unknown key in app.command (the message names the nearest one); an engine that is not a defineEngine handle; the json reporter combined with list; a duplicate or malformed target name; a secret or credential password shorter than 6 characters (code points); a credential that is not { username, password }, whose username is not a string (only password may be a function), or that has no username in the config or E2E_USER_<NAME>_USERNAME; two credentials or two secrets whose names map to one E2E_USER_* or E2E_SECRET_* override variable (api-key and api_key); an oversized agents.<name>.context; an expect.poll timeout or interval that is not a finite number of milliseconds; a tests value that is not a glob or a list of globs, or a tests list of only ! exclusions; the removed top-level limits key; an agents entry that is a bare StepExecutor (the message names { executor }) or a model instance (the message names { model }); an unknown agent key (the message names the nearest one); the removed agent keys timeout and maxTurns (the message names judgmentTimeout and maxModelCalls); system or tools beside executor; a tools entry that defineTool did not build, or that takes a built-in tool name (a grammar verb, complete_step, locate, test_lines, start_recording, stop_recording, or report_finding); createToolLoopExecutor({ maxTurns }); reuseExisting on an app whose app.url asks for a free port; a port-0 app.url with no app.command to serve it; a filters value that is not a list of { name, filter } (for a bare function such as decisionFilter, the message says to call it); or a filter name that breaks the target name rules, is longer than 256 characters, or is used twice | configuration | 2 |
CONFIG_NOT_DETERMINISTIC | A worker re-resolved the config module and got a different digest than the runner | configuration | 2 |
APP_URL_REQUIRED | A navigation or relative URL on a target that declares no app.url, or an app.command with no app.url or app.readyUrl to poll | configuration | 2 |
INVALID_APP_URL | A base URL that is unparseable or has userinfo, a query, a fragment, a non-HTTP scheme, or plain HTTP on a non-loopback host; port 0 on a host other than a literal loopback address (127.0.0.1, [::1]); or a loopback address the run could not bind a free port on | configuration | 2 |
INVALID_GLOB | An empty pattern; a first segment that starts with ! beyond the one leading ! of a tests exclusion (!!tests/**, ./!tests/**); a bare !; braces, character classes, or extglobs; a backslash separator; an absolute path or a .. segment; a trailing / or .; a tests entry with no wildcard that names a directory (the message names the glob to write); a pattern that names only the project root; or a ** that is not a whole segment | configuration | 2 |
UNKNOWN_TARGET | --target, open_session.target, or openHost (e2e/runner) named a target that is not configured | configuration | 2 |
TARGET_REQUIRED | e2e mcp or openHost (e2e/runner): the config declares several targets and none was named | configuration | 2 |
NO_SESSION | e2e mcp: call or tools before open_session or after the session ended (the message names why), or call, tools, or close_session naming a session id that is not open or is closing | configuration | 2 |
SESSION_REQUIRED | e2e mcp: call, tools, or close_session without a session while several sessions are open. The message lists them | configuration | 2 |
ENGINE_IN_USE | e2e mcp: open_session on a target whose engine instance another open session drives, because the config gets it from a package every session shares. Create the engine in the config, or in a file it imports by path | configuration | 2 |
CONFIG_IN_USE | e2e mcp: open_session on a config other than the one the open sessions use. Credentials and secrets resolve process-wide, so open on that config or close those sessions first | configuration | 2 |
SESSION_OPEN | e2e mcp: open_session while every session slot (--max-sessions, default 4) holds an open, opening, or closing session, or while the server shuts down. Close one first | configuration | 2 |
UNKNOWN_TOPIC | e2e mcp: an e2e://guide/{topic} resource named a topic the skill does not have. The message lists them | configuration | 2 |
UNKNOWN_TOOL | e2e mcp: call or tools named a tool the session’s catalog does not have. The message lists the catalog. An action verb the engine does not declare is UNSUPPORTED_CAPABILITY instead | configuration | 2 |
UNSUPPORTED_ARTIFACT | A video mode set on a target, or on a test that will run there, records on some attempt, and the target’s engine cannot record video. The message names the target, and the test for a test’s mode. A mode from the config root or --video never raises it: it applies to the targets that can record, with a notice naming the rest | configuration | 2 |
UNSUPPORTED_CAPABILITY | A fixture, action, or tier the target’s engine does not declare: a contributed fixture it does not contribute, screen without locate, state restore without state, screenshots without artifacts, app.clearState() on a persistent browser context (the message names the mode), or agent.assert vision with an executor that does not declare vision: true | configuration | 2 |
NO_TESTS | Zero runnable ordinary test-target pairs. The message names the cause: no file matched the tests globs (it lists look-alike files), a positional matched nothing (it names the ! entry that excludes a file the positional names), a file registered no tests, or filters excluded everything. Suppress with --pass-with-no-tests. When test filters leave nothing to run, the run passes instead (exit 0). | configuration | 2 |
RUNNER_BUSY | A suite overlapped another suite, an opening or open host, or a live attempt; or a host opened during a suite. Await the suite and close existing hosts before you retry. | configuration | 2 |
NO_LAST_RUN | --last-failed found no report.json where the run writes it, or a file there that is not a report-1 document. Run once without the flag first. | configuration | 2 |
INVALID_CHANGED_SINCE | --changed-since (changedSince) is not a string, is empty, or starts with -. Or, when the run calls test filters, it names no commit in the checkout or shares no history with HEAD (in a shallow clone the message says to fetch more history, otherwise that the histories are unrelated); or the project is not in a git repository, git is missing, or git did not answer within 30 seconds | configuration | 2 |
ONLY_IN_CI | A focused test reached a CI run | configuration | 2 |
COLLECTION_ERROR | Registration outside collection; an invalid title; a non-function body; an async describe body; nested serial groups; an option forbidden inside a serial group; a setup test inside describe; an invalid session name; a malformed test option, such as tags that is not a list of distinct names; a test or describe option key the call does not take (the message names the nearest one); a test file that fails to import, in a run with no positionals or when the positionals selected it; a session that a selected test consumes, or e2e explore --session names, but no collected setup test produces (the message lists the declared sessions); a consumed session on a target whose engine has no state capability to save it; or a positional path outside the project root | configuration | 2 |
SESSION_MISMATCH | A stored session does not match the current run, target, engine, platform, or app identity | configuration | 2 |
SESSION_EXPIRED | A stored session is past its expiry at restore time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 |
SESSION_INVALID | A stored session failed authentication. Tampered or corrupt. | configuration | 2 |
SESSION_UNAVAILABLE | A test requires a session no selected setup test produced | configuration | 2 |
SESSION_CONTRACT | A setup test saved a session twice, saved an undeclared name, or finished without saving a declared one | test | 1 |
TEST_TIMEOUT | The attempt exceeded the resolved test timeout. The body is not stopped: the harness operation it is in is cancelled, and its fixtures die with the attempt, so it cannot reach the next test. A test that blocks the event loop gets its worker killed once cleanupTimeout (at least 5 s) passes after the timeout. See Hooks | test | 1 |
STEP_NOT_AWAITED | The test body settled while a step it started was still running: a step call without await. The step fails at the line of the call and is cancelled. The attempt fails with this code when the body returned. When the body threw, the attempt keeps the body’s error and records this one under secondaryErrors. An expect.poll matcher still running when the body, an afterEach hook, or a fixture teardown returned fails that phase the same way, at the line of the call, and is cancelled. One left by a beforeAll or afterAll hook fails the hook with HOOK_FAILED | test | 1 |
UNHANDLED_REJECTION | A promise rejected with no test running. Node cannot say which test created a promise, so an unhandled rejection is charged to the test running when it surfaces. Between tests it is this run error, naming the last test that finished in the worker as the likely source. Before any test has finished in the worker, it ends the worker instead: the rejection is the run error, and the test on its way fails with WORKER_CRASH. The body that left it keeps running like one past TEST_TIMEOUT | test | 1 |
HOOK_FAILED | beforeAll or afterAll threw. Skips the suite’s tests. | test | 1 |
RUN_TIMEOUT | The run spent its maxDuration or --max-duration. The running tests end as interrupted, and the rest are skipped with cause time-limit | test | 1 |
LAUNCH_TIMEOUT | Engine init or attempt start exceeded config.launchTimeout | infrastructure | 3 |
CLEANUP_TIMEOUT | Engine attempt end exceeded config.cleanupTimeout, or a route or dialog handler the engine ran for the test was still running when the attempt settled | infrastructure | 3 |
BROWSER_INSTALL_FAILED | Browser provisioning failed | infrastructure | 3 |
WORKER_INIT_FAILED | A worker process never became ready | infrastructure | 3 |
WORKER_EXIT | A worker exited unexpectedly while running a file. The test in flight fails with WORKER_CRASH, and the file’s other tests are skipped. Attempts that finished before the exit stay in the report, including a retry’s earlier attempts and a serial group’s finished members. The crash is recorded as the attempt after them, or on the last attempt when it came after it (an afterAll). When an unhandled rejection ends the worker (see UNHANDLED_REJECTION), the rejection is the run error, not this one | infrastructure | 3 |
WORKER_CRASH | A worker exited during a specific test | infrastructure | 3 |
WORKER_ENTRY_MISSING | The worker entry module could not be resolved. Packaging fault. | infrastructure | 3 |
NODE_UNSUPPORTED | The runtime cannot run e2e’s TypeScript loader: a Node.js outside ^22.22.3 || >=24.8.0, Bun (no module.registerHooks), or Deno. The CLI checks this before it starts and exits 2 with the same sentence. The loader checks again where it registers its hooks, for a host that runs e2e’s runner without the CLI | infrastructure | 3 |
TYPESCRIPT_COMPILER_UNAVAILABLE | oxc, which compiles e2e’s TypeScript, could not be loaded; the message gives the cause. Most often its native bindings for this platform (@oxc-transform/binding-<platform>-<arch>, @oxc-parser/binding-<platform>-<arch>) are not installed. npm leaves them out with --omit=optional, or when package-lock.json was written on another platform (npm/cli#4828). Install again without --omit=optional, or delete package-lock.json and node_modules and run npm install | infrastructure | 3 |
ENGINE_FAILURE | Any engine fault with no more specific mapping | infrastructure | 3 |
WORKER_PROTOCOL | A worker reported the wrong unit, an attempt overlapped a live attempt, openHost tried to acquire an engine instance another host owns, or host.openSession tried to reuse a closed host or overlap an opening, live, or closing session | infrastructure | 3 |
SESSION_KEY_INVALID | A worker received a session key that is not 32 bytes. Packaging or IPC fault. | infrastructure | 3 |
REPORT_WRITE_FAILED | report.json or ai-trace.json could not be written. Recorded as a run error; the in-memory outcome is still complete. A junit.xml that could not be written is a line on the junit reporter’s stderr instead | infrastructure | 3 |
EVALUATE_FAILED | browser.evaluate threw inside the page (@e2e-dev/web) | test | 1 |
INTERRUPTED | A user or CI signal aborted the run. A test cut short by --max-failures or the time limit also ends with this code, and its message names the limit; that run exits 1, not 130 | interrupted | 130 |
Test code errors
| Code | When it happens | Class | Exit |
|---|---|---|---|
INVALID_ARGUMENT | Argument validation in test code refused an argument | test | 1 |
INVALID_LOCATOR | Argument validation in test code refused a locator’s arguments | test | 1 |
ERROR | A value thrown from test code that no other code classifies | test | 1 |