e2e/engine is the contract an engine implements to drive a user interface,
such as a browser or simulator. An engine implements shared types for nodes,
locators, actions, and errors. defineEngine validates and freezes the
implementation, derives its capabilities, and returns a handle for targets
in the config. Use test.extend fixtures for resources outside the UI, such
as APIs or databases.
This sketch assumes a connect adapter for the platform:
Reference
defineEngine
Validates an engine spec, freezes it, computes its
capabilities, and returns a branded handle.
isEngineHandle tells such a handle from any other value.
Every function on the handle is bound to the spec it came from, so a class instance is as valid a body as an object literal.
defineEngine throws INVALID_CONFIG when the spec breaks one of these
rules:
artifacts.startTrace and artifacts.stopTrace were removed from the
contract. An engine published before their removal that declares them still
passes validation; the runner never calls them.
Engine
The body of one target. Every member but name, version, and
spiVersion is optional; the members an engine implements decide its
capabilities.
OperationContext
Capabilities
The members an engine implements set its capabilities. A screen,
Locator, or app call that needs a missing one fails with
UNSUPPORTED_CAPABILITY. Test APIs lists what needs which
member. Three gaps end differently:
- A test that
requiresa fixture the engine lacks is skipped (capability-unavailable). - A selected test that uses a
test.setupsession, on an engine withoutstate, is a collection error before the setup runs. - A
videomode on an engine without video recording prints a notice and records no video.
Agent verbs
The agent offers the model only the verbs the engine’s declarations unlock. An undeclared verb is absent from the toolset.
clearandfocusunlock no verb.tap_atandhover_atgo throughperformAt(thetapandhoverpointer kinds). Without that kind they go throughperformwhen the point lands on a node the tree lists.type_at,press_at, andselect_atgo throughperformwhen the point lands on a node the tree lists. Elsewheretype_atandpress_atfall back tokeyboard.
Test APIs
The test-facing APIs each engine member backs. A screen or Locator
method for an undeclared kind, or a call with modifiers and no
tapModifiers, fails with UNSUPPORTED_CAPABILITY before it reaches the
engine.
engine.capabilities
The computed ReadonlySet of capability names. Selection, the report, and
UNSUPPORTED_CAPABILITY messages read it.
Selection also grades a target by the kind of app it declares, not by its
engine. A target with app.url has browser, and one with app.bundleId or
app.appPath has native-app, so a test can requires: ['native-app'].
Keyboard
Lifecycle
The hooks the runner calls around workers and attempts. Every hook is
optional. The Engine declaration’s JSDoc above states each hook’s contract.
init and startAttempt run within launchTimeout. finish,
settleAttempt, endAttempt, and dispose run within cleanupTimeout.
prepare has no timeout. Both budgets come from the
config.
prepare
Runs once per target in the runner process, after collection and before workers, app commands, and the run’s clock start.
A failure ends the run before any test:
finish
Runs once per run and target in the runner process, after every worker of
the target is gone. Release what prepare acquired for the run.
The finish hooks of all prepared targets share one cleanupTimeout.
init
Runs once per worker, before the first step. Boot devices here, not in a step budget.
init never learns the app URL; URLs arrive through session.open only. A
failure is an infrastructure error.
startAttempt
Runs before each attempt, to set up the per-test isolation that
endAttempt tears down.
On failure or timeout, context.signal aborts and endAttempt runs.
A value from resolveSecret is protected as text only. Reports, logs, and
observations redact it and the attempt’s text downloads are rewritten, but
unlike a fill it withholds no screenshot or model pixels.
settleAttempt
Runs after the body has settled, and again after teardown, before the verdict. It reports a failure on a path no step awaited, such as a request interceptor or a native dialog handler.
endAttempt
Runs after each attempt and tears down what startAttempt set up. A failure
marks cleanup failed; the test keeps its status.
dispose
Runs one time per worker, including when init never ran.
Missing or aborted context.reuseSignal requires final cleanup. A live
signal permits keeping run-owned resources for a replacement worker.
An engine that keeps resources must observe revocation until its hook settles.
Final shutdown, a forced interrupt, or parent disconnect revokes that permission.
The cleanup budget still uses context.signal and context.timeoutMs.
Cleanup is best effort within that budget. The runner does not guarantee cleanup after a worker exits or after disposal settles with reuse permission.
Observation
observe returns one fresh semantic snapshot of the surface under test.
Return the surface as it is right now. The runner mints the revision and
stamps it onto every ref. When pixels are present, it checks that
maskedRegionCount covers every secure node. A valid empty tree may have a
childless root and include pixels.
Truncated trees
Set truncated when a node limit or an inaccessible subtree leaves content
out. The truncated JSDoc above says how the runner treats it.
Pixel fallback
When semantic capture times out, an engine may return masked pixels in place
of the tree, but only when pixelFallback is true and it can prove
screenshot masking without that capture.
- Return
treeUnavailable: true, freshpixels, and arootcontaining only its stableref. That ref supports viewport actions; it supplies no semantic node evidence. Retire earlier node refs. - The flag tells missing semantic evidence from a valid empty tree. Never infer absence from it.
- Without permission, pixels, or independent masking proof, propagate the capture failure. Cancellation, policy denial, and hard engine failures propagate too.
- The runner permits fallback for
actand for judgments withvision: trueor'only', before any secret fill. It includes the screenshot even if the initial observation requested only a tree. - A step that loses semantic evidence cannot record or finish by replaying a trace.
- Engines whose masks depend on their semantic capture cannot provide this fallback.
EngineSnapshot is an extendable interface, so TypeScript can’t enforce
these rules. The runner checks the shape at the engine boundary, at runtime:
Actions
perform carries out one LocatorAction on a node; performAt carries out
one PointerAction at a viewport point. An engine declares the kinds it
honors in actions and pointerActions.
The grammar is closed. Anything beyond it is an agent-side defineTool.
Locator actions
Pointer actions
Pointer actions use viewport coordinates when no semantic node is available.
dragToends at another point.swipestarts a scroll gesture at the given point.swipeTois the one kind with no node form: the finger’s path from the point totarget, a touch swipe on a device and a pointer drag on a document platform. It differs fromdragTo, where a platform holds before it drags.
Keys
The key grammar of press, shared by every engine, and its parser.
App
The app under test is declared on the target (targets: [{ engine, app }],
see the app under test); an engine
only drives it. The harness checks the shape of every field. Then it hands
the declaration to validateApp at config load.
CommandConfig is exported from e2e, not e2e/engine.
validateApp
Checks the declared app at config load, synchronously.
- It runs for every target the engine serves, with
{}for a target that declares noapp. @e2e-dev/webrequiresurl;@e2e-dev/mobilerequiresbundleIdorappPathand refusesurl.
EngineAppInfo
The resolved app, as EnginePrepareInfo.app and EngineInitInfo.app: the
site of the URL and the device fields as the target declared them. Apply the
site to a URL with sameSite.
Session
Steering hooks behind the universal app fixture and the agent’s navigate
and back verbs.
Errors
An engine throws EngineError with one code of a closed set. The runner maps
each code to the error a test sees.
EngineError
Timeouts at a deadline
An OPERATION_TIMEOUT on a locate or read that started with under 100 ms of
a waiting call’s deadline left, after one that completed, ends the wait
instead:
A negated assertion passes there if its negation had held long enough by the
time the read was cut off. pollCondition polls by these
rules.
Runner error classes
An EngineError a contributed fixture method throws is mapped by the table
above, so a fixture over a session that ended reports APP_NOT_OPEN the way
an observation would. A method can also fail a step with the runner’s own
error classes, exported from the same entry.
E2EError itself is not exported; construct one of the three.
Shared semantics
Helpers that keep behavior consistent across engines. Every engine reproduces these semantics; import them instead of carrying a copy.
Text patterns
TextPattern matching for locate and text matchers.
resolveExpression
The reference locate. Resolves one LocatorExpression over a semantic
tree, in document order.
textandlabelanswer with the innermost match when a container echoes a descendant’s text.- An engine whose tree is the whole truth (a device) calls it; a browser engine reproduces it.
- A
selectorexpression needs theselectoroption. - A
frameexpression isFRAME_NOT_FOUND. - A
unionexpression matches either branch in document order, without duplicate nodes.
urlMatches
toHaveURL-style comparison of a current URL against an expected one.
ignoreCase works as on the locator matchers.
Site helpers
siteOf reads a hostname’s registrable domain, the form
EngineAppInfo.site takes. sameSite tells whether a URL is on a site.
pollCondition
Assertion polling and the negation grace window.
- Bound each read
evaluatemakes bydeadline(context.operation(deadline.remaining())). A read on the action timeout can outlast the assertion’s own. - An
OPERATION_TIMEOUTfrom anevaluateends the poll under the rules in Timeouts at a deadline.onTimeout(cause)builds the failure with it.
validateJsonValue
Checks that a value is plain JSON.
- A top-level
undefinedpasses. rules.onSecretreplaces thePOLICY_DENIEDaSecretraises with an error of its own;rules.maxDepthcaps the nesting.
rejectUnknownKeys
Throws for the first own key of value outside keys: the check the runner
runs on its own config objects, for an engine factory’s options.
- The message names the nearest known key when one is a plausible typo, and the known keys otherwise.
- Pass
'INVALID_ARGUMENT'ascodefor an object a test passes to a fixture method.
rejectUnknownOptions
Refuses an option key a fixture method or matcher does not take, so a JavaScript caller’s unsupported option fails instead of running on the default.
- Pass
INVALID_LOCATORascodefor a locator option. optionsmust be a plain object orundefined; a class instance or an array is refused.- Non-enumerable own keys count as keys.
obj
Drops undefined-valued keys, so a result built from optional inputs
satisfies exact optional property types.
Timing helpers
raceAbortabandons the wait whensignalaborts. Pass a thunk to check cancellation before dispatch.withTimeoutraces a promise against a timeout and throws whatonTimeoutbuilds.withinCleanupBudgetnever rejects. It resolves when the promise settles, failures included, whensignalaborts, or whentimeoutMspasses.
isSecret
Tells a secrets.get() handle from a string, for an option that takes
either.
State
Platform-neutral state capture and restore, behind test.setup sessions and
the session: option.
Snapshots are credentials: cookies, tokens, and storage that authenticate
the app under test. An engine must not persist, cache, or log a snapshot on
its own, must not echo snapshot contents in an error message, and must treat
restore as a replacement, never a merge, so a restored session cannot leak
into the next attempt.
Artifacts
Evidence capture: screenshots under the attempt directory, and videos there or at a provider.
screenshot is required and returns the path of the file it wrote.
Videos
The runner calls startVideo only for an attempt whose
video mode records.
startVideoruns afterstartAttemptand any session restore, within the launch budget.stopVideoruns beforeendAttempt, within the cleanup budget.- When the video mode is set explicitly, a
startVideothat fails fails the launch and astopVideothat fails fails cleanup. With the default mode, a failure of either drops the video quietly. - The runner decides which recordings are kept and deletes a file an attempt does not keep.
Provider recordings
An engine that leases its browser or device from a hosted service can let the
service record the attempt in place of its own capture. That is how
BrowserProvider.record and
DeviceProvider.record work. e2e/engine exports
the shared shape and stopProviderRecording, so every engine turns a
provider’s recording into a VideoSegment the same way.
isProviderRecording(value)tells whether what a provider’srecordresolved to is a recording: a parseablestartedAtand astop. Check it before keeping one, so a malformed recording fails the start, named, instead of the stop.stopProviderRecordingcreates the attempt’svideodirectory and callsstopwith it. A file must be a plain name that exists in that directory; a link must be anhttp(s)URL with a media type.- It returns
{ path: 'video/<file>', startedAt }or{ url, mediaType, startedAt }forstopVideoto hand back. - The target argument’s type is exported as
ProviderRecordingTarget.
Redaction
- Videos are unmasked and carry
redaction: 'incomplete'. - A
downloadan engine attaches is the app’s bytes. When the session knows a secret value, the runner rewrites a text one (.txt,.csv,.json,.html) through the ledger and labels itcomplete. - Every other download keeps its bytes and carries
redaction: 'incomplete'.
Contributed fixtures
An engine contributes named surfaces to the test fixtures through
fixtures. Each key becomes a fixture and a capability name.
The factory runs once per attempt with an EngineFixtureContext:
Declaring operations
context.fixture(name, surface, operations) declares which methods are
recorded steps. Each operation takes:
- Nested maps describe namespaces.
- Recording decorates the object in place, so create a fresh surface in each factory invocation.
- The runner opens the step before calling a declared method, so attachments and failures from synchronous and asynchronous work land on the same step. Work arriving after its step has closed cannot attach to a later one.
- Type the fixture with
test.extend<{ name: Type }>().
Expectations
expectable attaches an expectation surface so expect(fixture) returns it.
Declare matcher recording with context.fixture('expect', matchers, operations).
Types
Exported names
EngineCleanupContext, EngineDisposeContext, OperationContext,
TextPattern, SemanticQuery, LocatorExpression,
NodeRef, SemanticNode, ViewportPoint, ViewportSize,
ObservationPixels, LocatorAction, LocatorActionKind, PointerAction,
PointerActionKind, ScrollDirection, Momentum, SelectOption, Key,
KeyModifier, KeyName, ParsedKey, ENGINE_SPI_VERSION,
LOCATOR_ACTION_KINDS, POINTER_ACTION_KINDS, KEY_MODIFIERS,
KEY_NAMES, parseKey, ENGINE_ERROR_CODES, RETRYABLE_ENGINE_ERROR_CODES,
Secret, isSecret, TextMatch, JsonValue, JsonValueRules,
ResolveExpressionOptions, PollConditionOptions, WithoutUndefined,
FixtureOperation, OBSERVED_NAME_LIMIT, and OBSERVED_TEXT_LIMIT are
exported from e2e/engine.
SemanticNode
Observed field limits
- An engine that cuts an observed field cuts it at exactly its limit. A field exactly at its limit may have been cut; a shorter one, or a longer one the engine left whole, is complete.
- Keep the start and cut the end: the runner redacts the leading part of a secret that a field exactly at its limit ends with, which a whole-value match cannot see.
- Single-node reads are unbounded and always carry the full value.
Queries
The selector expression kind carries a platform-native selector string;
frame scopes a query into a nested document.
A SemanticQuery with visible: true must match only nodes whose
states.hidden is false. Use the same predicate the engine reports on
SemanticNode, so getByText('x', { visible: true }) and toBeVisible()
never disagree. Apply it wherever the query sits in the expression, under a
scope, filter, or index too.
Related
Writing an engine walks through building one.
@e2e-dev/web (reference) and @e2e-dev/mobile
(reference) implement this contract with the same
public defineEngine.