> ## Documentation Index > Fetch the complete documentation index at: https://e2e.tester.army/docs/llms.txt > Use this file to discover all available pages before exploring further. # Reporters > Receive run events and publish results with a custom reporter. A reporter receives the run's events as they happen and the finished run once its report is written. `reporters` in `e2e.config.ts` lists built-in reporters by id beside reporter objects. ```ts title="e2e.config.ts" export default { targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }], reporters: ['list', 'junit', github()], } satisfies E2EConfig; ``` ## Reference ### Built-in reporters | Reporter | Description | | --- | --- | | [`list`](#list) | Terminal progress and run summary | | [`json`](#json) | Report document to stdout | | [`junit`](#junit) | Writes `/junit.xml` | | [`markdown`](#markdown) | Writes `/summary.md` | | [`github()`](#github) | Pull request comment and job summary | | Throws | When | | --- | --- | | `INVALID_CONFIG` | `reporters` is not an array. | | `INVALID_CONFIG` | An id other than `list`, `json`, `junit`, or `markdown`. | | `INVALID_CONFIG` | An entry that is neither an id nor an object with a `name` and at least one handler. | | `INVALID_CONFIG` | `json` together with `list`. | #### `list` Renders terminal progress, failures, and the run summary. ```ts reporters: ['list'], // default ``` - It renders the same events [`onEvent`](#onevent) receives. - The [summary rows](#onrunfinished) reporters return print under its summary. #### `json` Prints the report document to stdout. ```ts reporters: ['json'], ``` #### `junit` Writes `junit.xml` beside the report. ```ts reporters: ['list', 'junit'], ``` - It reads `run.report`, writes `junit.xml` beside `run.reportPath`, and returns one row naming the file. It is the smallest complete example of a reporter. - A second Ctrl-C still lets it write its file, so no file from the previous run is left beside the new report. #### `markdown` Writes `summary.md` beside the report. ```ts reporters: ['list', 'markdown'], ``` It does what `junit` does, second Ctrl-C included, with `summary.md` and through [`renderMarkdownReport`](#rendermarkdownreport). ##### `renderMarkdownReport` Renders a report as the Markdown page the `markdown` reporter writes, so your own reporter can post the same page elsewhere. ```ts const reporter: Reporter = { name: 'wiki', async onRunFinished(run) { const page = renderMarkdownReport(run.report, { traces: run.traces }); await fetch('https://wiki.example.test/runs', { method: 'POST', body: page }); }, }; ``` - `traces` links each block to its trace page by result id. - [`@e2e-dev/github`](https://e2e.tester.army/docs/github) is that reporter for GitHub Actions. #### `github()` `@e2e-dev/github` posts the run as one pull request comment from GitHub Actions, keeps it current on reruns, and writes the same text to the job summary. ```ts const reporter = github({ key: 'chromium' }); ``` - `key` tells one job's comment apart from another's when the workflow, job, and project are the same, as in a matrix. Without it, matrix replicas of one job share a comment, and the last one to finish wins. Never use the run id: a rerun must find the old comment. The key also names the comment in its headline: `e2e chromium: 77 passed`. - It reads the token, the event, and the repository when the run finishes, never when the config loads. - With `--last-failed`, it folds the rerun into [`lastRun`](#onrunfinished) to show both runs as one comment. - Setup and permissions are on [Pull request comments](https://e2e.tester.army/docs/github). ### Custom reporters A reporter object has a `name` and at least one handler. Use `onEvent` for live progress and `onRunFinished` to process the final report and artifacts. ```ts title="e2e.config.ts" const upload: Reporter = { name: 'upload', async onRunFinished(run) { const response = await fetch('https://example.test/runs', { method: 'POST', body: JSON.stringify(run.report), }); const { url } = (await response.json()) as { url: string }; return [{ label: 'Results', text: url }]; }, }; export default { targets: [{ engine: web(), app: { url: 'localhost:3000' } }], reporters: ['list', upload], } satisfies E2EConfig; ``` | Member | Type | | --- | --- | | [`name`](#name) | `string` | | [`onEvent`](#onevent) | `(event: RunEvent) => void` | | [`onRunFinished`](#onrunfinished) | `(run, signal) => Promise<…>` | ```ts interface Reporter { readonly name: string; onEvent?(event: RunEvent): void; onRunFinished?(run: FinishedRun, signal: AbortSignal): Promise; } ``` Cached steps expose `cache.recordedBy: { provider, model }` in `report.json` and on the trace page when the recording has it. Older recordings omit it. The console prints `replayed · recorded with provider/model` only when that identity differs from the configured model. It stays quiet for multi-agent runs and for pinned agents whose model is not in the run header. #### `name` The reporter's label in diagnostics. Required and non-empty. ```ts name: 'upload', ``` #### `onEvent` Receives every run event as it happens, as plain JSON. ```ts onEvent(event) { switch (event.type) { case 'test-finished': console.log(event.result.test.title, event.result.status); break; default: } }, ``` [Events](#events) lists them. `onEvent` must not block and must not throw. Do slow work in `onRunFinished`. A throw quarantines the reporter for the rest of the run and prints one line on stderr naming the reporter, the event type, and the error. The run's outcome is unchanged. #### `onRunFinished` Runs once the report is written and the terminal summary prints, with the finished run and an `AbortSignal`. ```ts async onRunFinished(run, signal) { await fetch('https://example.test/runs', { method: 'POST', body: JSON.stringify(run.report), signal }); return [{ label: 'Results', text: 'https://example.test/runs/0192c' }]; }, ``` ```ts interface FinishedRun { readonly report: Report; // the report-1 document, as written readonly status: RunStatus; // report.run.status readonly exitCode: RunExitCode; readonly projectRoot: string; // what the terminal shows paths relative to readonly reportPath: string | undefined; // absent when the write failed readonly artifactsRoot: string; // what the report's artifact paths are relative to readonly aiTracePath: string | undefined; readonly traces: ReadonlyMap; // result id -> its trace page, from the project root readonly lastRun?: Report; // with --last-failed, the report the rerun selected from } type ReporterSummary = readonly { label: string; text: string }[]; ``` - All reporters run concurrently, each with a one-minute deadline. - A timeout, exception, or invalid return value is reported on stderr. The run's status, exit code, and report do not change. - One Ctrl-C still lets reporters finish. A second Ctrl-C stops waiting for them, except [`junit`](#junit) and [`markdown`](#markdown). - `signal` aborts when the budget runs out or the run is forced to stop. - The runner writes the trace pages, `results//trace.md`, before it calls `onRunFinished`. Hand `signal` to every request and timer. The CLI exits without waiting for a reporter it abandoned, cutting off what it still does; under [`e2e/runner`](https://e2e.tester.army/docs/reference/runner), that work keeps your process alive. ##### Summary rows The rows a reporter resolves with print under the `list` summary, in the same layout as its own rows. The `json` reporter does not carry them. ``` Test Files 2 passed (2) Tests 6 passed (6) Start at 14:02:11 Duration 8.41s JUnit .e2e/junit.xml Results https://example.test/runs/0192c… ``` ##### Artifacts - Artifact paths in the report are POSIX paths relative to `artifactsRoot`. - `path`, `size`, and `sha256` are optional: an artifact whose file never appeared, or that a redaction withheld, is recorded without them. Check for `path` before reading. - A video a hosted service keeps has an `http(s)` `url` in place of `path`. ##### Reruns with `--last-failed` - The rerun's own report lists the tests it left out as unselected. A reporter that keeps one place current, the way `@e2e-dev/github` keeps a pull request comment, folds the rerun into `lastRun` to show both runs as one. - A test the rerun owed but another filter left out is in the rerun's `run.carried`. It holds the test's last result, the serial group its attempts live on, and the `beforeAll` or `afterAll` failure whose scope holds it, from whichever earlier run reported them. - A fold shows the carried result in place of the unselected one and counts a carried failure or hook error against the outcome. - The artifacts both reports name are on disk under `artifactsRoot`. ##### Test filters - A test a [test filter](https://e2e.tester.army/docs/filters) left out is an unselected result with `skip.cause` `filtered` and `skip.filter` naming the filter. - `run.filters` holds one record per filter that was called, in order: `name`, `candidates`, `excluded` (the tests left out once serial groups took back their members), `note` (cut to 1024 characters; for a filter that failed or timed out, the warning saying why), and `outcome`, set when the filter left nothing out: `failed` when it threw or returned an invalid result, `timed-out` when it ran past 5 minutes. A filter left with no tests is not called and has no record. - Once the filters have decided, `run.changes` holds the change they judged, by counts: `base` (the merge base), `files`, and `skipped` (why no filter ran, such as `no files changed since abc1234`). It is present when filters are configured and the run passed `--changed-since`, except under `e2e explore`, on `--watch` reruns, and when the run ends before the filters decide. On `--last-failed`, or with no tests to judge, it holds `skipped` alone. - A failure while the filters decide, such as `INVALID_CHANGED_SINCE` for a ref that names no commit, is in `run.errors` with `phase` `filter`. A malformed `changedSince` has `phase` `config`. ## Events `onEvent` receives one of these facts. Each event also has a sequence number, `seq`, and an ISO timestamp, `at`. ```ts type RunEventFact = | { type: 'run-started'; runId; projectId; projectRoot; artifactsRoot; ci; targets; agents?; model?; judge? } | { type: 'plan'; total; files: { file; target; tests }[] } | { type: 'notice'; target; message } | { type: 'output'; target; pair; stream: 'stdout' | 'stderr'; text } | { type: 'setup'; step: SetupStep; state: 'started' } | { type: 'setup'; step: SetupStep; state: 'finished'; durationMs; outcome?: 'reused' | 'failed' | 'timed-out' } | { type: 'test-started'; testId; agent; repeat; title; file; serialId; target } | { type: 'step'; testId; agent; repeat; target; progress: StepProgress } | { type: 'test-finished'; result: RunEventResult } | { type: 'serial-group'; group: SerialGroupRecord } | { type: 'explore'; progress: ExploreProgress } | { type: 'run-error'; error: SerializedError } | { type: 'run-interrupted'; mode: 'graceful' | 'forced' } | { type: 'run-stopped'; failures; limit } | { type: 'run-finished'; status; exitCode; reportPath?; aiTracePath?; traces? }; ``` - `run-started` comes first and `run-finished` last. - `judge` on `run-started` is the run agent's judge model, present only when judgments use a model other than `model`. - `setup` brackets the work before the tests start. Its `step.kind` is `collect`, `filter` (one [test filter](https://e2e.tester.army/docs/filters), by its `name`), `prepare`, `service`, or `app`. `outcome` is `reused` when a service or app command attached to a process already serving its URL, `failed` when a filter threw or returned an invalid result, and `timed-out` when a filter ran past 5 minutes. - A `notice` names what its line is about in `target`: a target, `app`, `collect`, `filter` for a test filter's progress and summary, or `run`. - `output` is one write a test made to stdout or stderr; `pair` names the test running at the time and is undefined between tests. - `test-finished` carries the full result record, including every attempt's artifacts. - `explore` appears only in an `e2e explore` run: `started`, `planning`, `step-started`, `step-finished`, `finding`, and `finished`. The record of all of it is `run.explore` in the report. - New event types are added over time. Handle the types you know and ignore the rest; leave the `default` branch of a `switch` empty. ### Tests and attempts - A test pinned to several agents starts and finishes once per agent, so key running state by test id, target, and agent together. - Ordinary tests emit `test-started` once for that combination. Serial members emit it on each group attempt they execute. - Use step progress [identity](#step-progress) to distinguish retries. - An attempt that fails before any step emits no step progress and appears in the final result. Secrets are redacted before events are built. Error messages are only stripped of control characters, so treat them as untrusted. ### Step progress The `phase` field describes step progress: - `start` and `end` mark the step's boundaries. - `event` reports a completed model turn, tool call, or observation. - `activity` reports work starting: `observe`, `action`, or `model`. - `replay` reports when replay from the cache starts or returns control to the agent. ```ts type StepProgress = ( | { phase: 'start'; kind: StepKind; api: string; label: string } | { phase: 'end'; kind: StepKind; api: string; label: string; status: 'passed' | 'failed' | 'blocked' | 'timed-out' | 'cancelled'; durationMs: number; modelCalls: number; error?: SerializedError; explanation?: string; } | { phase: 'event'; api: string; event: StepEvent } | { phase: 'activity'; api: string; activity: 'observe' | 'model' | 'action' } | { phase: 'replay'; api: string; active: boolean } ) & { identity?: StepProgressIdentity }; ``` An `end` phase's `error` and `explanation` are redacted. Keep its `status`, including `blocked` and `cancelled`. Do not treat every non-passed step as a product failure. Every step phase the runner emits includes `progress.identity`: ```ts { attemptId: string; attemptIndex: number; stepId: string; stepIndex: number } ``` - The IDs match the final report. Both indexes start at zero. - A retry has a new attempt ID and a higher attempt index; its step indexes start again at zero. - Serial-group members share the group's attempt ID and index. Each member's step IDs stay distinct. - Use `stepId` to match start, activity, event, replay, and end phases, including nested steps. - `identity` is optional in the type so a reporter can also accept older streams. ## What `--reporter` does to reporter objects `--reporter` replaces only the built-in reporters; configured reporter objects still run. ```bash npm npx e2e run --reporter list,junit ``` ```bash pnpm pnpm exec e2e run --reporter list,junit ``` ```bash bun bunx e2e run --reporter list,junit ``` - A config with only reporter objects has no built-in terminal output. ## Where a reporter runs Handlers run in the runner process. Workers also load the config, so reporter construction must have no side effects. - Read tokens and start requests inside handlers. - Adding a reporter leaves [cached traces](https://e2e.tester.army/docs/cache#what-must-match) valid. ## Types `Reporter`, `FinishedRun`, `ReporterSummary`, `Report`, `RunEvent`, and `RunEventOf` are exported from the main entrypoint, and so is `renderMarkdownReport(report, options)`. ## Related [Pull request comments](https://e2e.tester.army/docs/github) sets up `@e2e-dev/github`. The [CLI reference](https://e2e.tester.army/docs/reference/cli#output) lists what the built-in reporters write.