E2e Runner

With the Docker pool, Chrome runs inside a container, so localhost there means the container — not your machine. host.docker.internal bridges to your host. On Linux (Docker Engine, not Docker Desktop) add --add-host=host.docker.internal:host-gateway, or use your LAN IP. Options 1 & 2 don't have this — the browser is local, so plain localhost just works.

cp node_modules/@matware/e2e-runner/opencode.json ./mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/

See OPENCODE.md for details.

Each .json file in e2e/tests/ contains an array of tests. Each test has a name and sequential actions:

[  {    "name": "homepage-loads",    "actions": [      { "type": "goto", "value": "/" },      { "type": "assert_visible", "selector": "body" },      { "type": "assert_url", "value": "/" },      { "type": "screenshot", "value": "homepage.png" }    ]  }]

Suite files can have numeric prefixes for ordering (01-auth.json, 02-dashboard.json). The --suite flag matches with or without the prefix, so --suite auth finds 01-auth.json.

Click by text — when click uses text instead of selector, it searches across common interactive and content elements:

button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"],[role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6

{ "type": "click", "text": "Sign In" }

These actions handle common patterns in React/MUI apps that normally require verbose evaluate boilerplate:

// Before: 5 lines of evaluate boilerplate{ "type": "evaluate", "value": "const input = document.querySelector('#search'); const nativeSet = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, 'value').set; nativeSet.call(input, 'term'); input.dispatchEvent(new Event('input', {bubbles: true})); input.dispatchEvent(new Event('change', {bubbles: true}));" }
// After: 1 action{ "type": "type_react", "selector": "#search", "value": "term" }

All subsequent actions run in the active tab:

{ "type": "click", "text": "Open report" }{ "type": "wait_for_tab", "text": "report" }{ "type": "assert_text", "text": "Quarterly results" }{ "type": "close_tab" }

Test-level retry — retry an entire test on failure. Set globally via config or per-test:

{ "name": "flaky-test", "retries": 3, "timeout": 15000, "actions": [...] }

Tests that pass after retry are flagged as flaky in the report and learning system.

Action-level retry — retry a single action without rerunning the entire test. Useful for timing-sensitive clicks and waits:

{ "type": "click", "selector": "#dynamic-btn", "retries": 3 }{ "type": "wait", "selector": ".lazy-loaded", "retries": 2 }

Set globally: actionRetries in config, --action-retries <n> CLI, or ACTION_RETRIES env var. Delay between retries: actionRetryDelay (default 500ms).

Tests that share state (e.g., two tests modifying the same record) can race when running in parallel. Mark them as serial:

{ "name": "create-patient", "serial": true, "actions": [...] }{ "name": "verify-patient-list", "serial": true, "actions": [...] }

Serial tests run one at a time after all parallel tests finish — preventing interference without slowing down independent tests.

The simplest approach — log in via the UI like a real user:

{  "hooks": {    "beforeEach": [      { "type": "goto", "value": "/login" },      { "type": "type", "selector": "#email", "value": "[email protected]" },      { "type": "type", "selector": "#password", "value": "test-password" },      { "type": "click", "text": "Sign In" },      { "type": "wait", "selector": ".dashboard" }    ]  },  "tests": [...]}

For SPAs with JWT, skip the login form by injecting the token directly:

{ "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." }

Or set it globally in config:

// e2e.config.jsexport default {  authToken: 'eyJhbGciOiJIUzI1NiIs...',  authStorageKey: 'accessToken',};

Each test runs in a fresh browser context, so auth state is automatically clean between tests.

More strategies: Cookie-based auth, HTTP header injection, OAuth/SSO bypasses, reusable auth modules, and role-based testing — see docs/authentication.md

Extract common flows into parameterized modules:

// e2e/modules/login.json{  "$module": "login",  "description": "Log in via the UI login form",  "params": {    "email": { "required": true, "description": "User email" },    "password": { "required": true, "description": "User password" }  },  "actions": [    { "type": "goto", "value": "/login" },    { "type": "type", "selector": "#email", "value": "{{email}}" },    { "type": "type", "selector": "#password", "value": "{{password}}" },    { "type": "click", "text": "Sign In" },    { "type": "wait", "value": "2000" }  ]}

Use in tests:

{  "name": "dashboard-loads",  "actions": [    { "$use": "login", "params": { "email": "[email protected]", "password": "secret" } },    { "type": "assert_text", "text": "Dashboard" }  ]}

Modules support parameter validation (required params fail fast), conditional blocks ({{#param}}...{{/param}}), nested composition, and cycle detection.

Run actions at lifecycle points. Define globally in config or per-suite:

{  "hooks": {    "beforeAll": [{ "type": "goto", "value": "/setup" }],    "beforeEach": [{ "type": "goto", "value": "/" }],    "afterEach": [{ "type": "screenshot", "value": "after.png" }],    "afterAll": []  },  "tests": [...]}

Important: beforeAll runs on a separate browser page that is closed before tests start. Use beforeEach for state that tests need (cookies, localStorage, auth tokens).

Skip exploratory or draft tests from --all runs:

// e2e.config.jsexport default {  exclude: ['explore-*', 'debug-*', 'draft-*'],};

Individual suite runs (--suite) are not affected by exclude patterns.

claude plugin marketplace add fastslack/mtw-e2e-runnerclaude plugin install e2e-runner@matware

This gives Claude 17 MCP tools, a workflow skill, 4 slash commands (/e2e-runner:run, /e2e-runner:create-test, /e2e-runner:verify-issue, /e2e-runner:capture), and 3 specialized agents (test-analyzer, test-creator, test-improver).

MCP-only install (tools only, no skill/commands/agents):

claude mcp add --transport stdio --scope user e2e-runner \  -- npx -y -p @matware/e2e-runner e2e-runner-mcp

cp node_modules/@matware/e2e-runner/opencode.json ./mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/

See OPENCODE.md for details.

Pool start/stop are CLI-only — not exposed via MCP.

Describe what the page should look like — AI judges pass/fail from screenshots:

{  "name": "dashboard-loads",  "expect": "Patient list with at least 3 rows, no error messages, sidebar with navigation links",  "actions": [    { "type": "goto", "value": "/dashboard" },    { "type": "wait", "selector": ".patient-list" }  ]}

After test actions complete, the runner auto-captures a verification screenshot. The MCP response includes the screenshot hash — Claude Code retrieves it and visually verifies against your expect description. No API key required.

Turn GitHub and GitLab issues into executable E2E tests. Paste an issue URL and get runnable tests — automatically.

How it works:

1. Fetch — Pulls issue details (title, body, labels) via gh or glab CLI
2. Generate — AI creates JSON test actions based on the issue description
3. Run — Optionally executes the tests immediately to verify if a bug is reproducible

# Fetch and displaye2e-runner issue https://github.com/owner/repo/issues/42
# Generate a test file via Claude APIe2e-runner issue https://github.com/owner/repo/issues/42 --generate
# Generate + run + reporte2e-runner issue https://github.com/owner/repo/issues/42 --verify# -> "BUG CONFIRMED" or "NOT REPRODUCIBLE"

In Claude Code, just ask:

"Fetch issue #42 and create E2E tests for it"

Bug verification logic: Generated tests assert the correct behavior. Test failure = bug confirmed. All tests pass = not reproducible.

Auth: GitHub requires gh CLI, GitLab requires glab CLI. Self-hosted GitLab is supported.

Live execution — monitor tests in real-time with step-by-step progress, durations, and active worker count.

[Dashboard - Live test execution]

Test suites — browse all suites across projects. Run a single suite or all tests with one click.

[Dashboard - Test suites grid]

Run history — track pass-rate trends with the built-in chart. Click any row to expand full detail.

[Dashboard - Run history]

Run detail — PASS/FAIL badges, screenshot thumbnails with copyable hashes (ss:77c28b5a), formatted console errors, and network request logs.

[Dashboard - Run detail]

Screenshot gallery — browse all captured screenshots with hash search (action, error, and verification captures).

[Dashboard - Screenshot gallery]

Pool status — Chrome pool health: available slots, running sessions, memory pressure.

[Dashboard - Pool status]

The runner learns from every test run — building knowledge about your test suite over time. Query insights via the e2e_learnings MCP tool:

Storage & export:

• SQLite (~/.e2e-runner/dashboard.db) — default, zero setup
• Neo4j knowledge graph — optional, for relationship-based analysis. Manage via e2e_neo4j MCP tool or docker compose
• Markdown report (e2e/learnings.md) — auto-generated after each run

Test narration: Each test run generates a human-readable narrative of what happened step by step, visible in the CLI output and the dashboard.

Explicit assertion — place assert_no_network_errors after critical page loads:

{ "type": "goto", "value": "/dashboard" },{ "type": "wait", "selector": ".loaded" },{ "type": "assert_no_network_errors" }

Global flag — set failOnNetworkError: true to automatically fail any test with network errors:

e2e-runner run --all --fail-on-network-error

When disabled (default), the runner still collects and reports network errors — the MCP response includes a warning when tests pass but have network errors.

Full network logging — all XHR/fetch requests are captured with URL, method, status, duration, request/response headers, and response body (truncated at 50KB). Viewable in the dashboard with expandable request detail rows.

MCP drill-down flow:

1. e2e_run          → compact networkSummary + runDbId2. e2e_network_logs(runDbId)                     → all requests (url, method, status, duration)3. e2e_network_logs(runDbId, errorsOnly: true)   → only failed requests4. e2e_network_logs(runDbId, includeHeaders: true) → with headers5. e2e_network_logs(runDbId, includeBodies: true)  → full request/response bodies

The e2e_run response stays compact (~5KB) regardless of how many requests were captured. Use e2e_network_logs with the returned runDbId to drill into details on demand.

Capture screenshots of any URL on demand — no test suite required:

e2e-runner capture https://example.come2e-runner capture https://example.com --full-page --selector ".loaded" --delay 2000

Via MCP, the e2e_capture tool supports authToken and authStorageKey for authenticated pages — it injects the token into localStorage before navigating.

Every screenshot gets a deterministic hash (ss:a3f2b1c9). Use e2e_screenshot to retrieve any screenshot by hash — it returns the image with metadata (test name, step, type).

{  "tests": [    {      "name": "checkout flow (heavy JS, real Chrome)",      "driver": "browserless",      "actions": [...]    },    {      "name": "scrape product page (lightweight)",      "driver": "obscura",      "fallbackDriver": "cdp",      "actions": [...]    }  ]}

driver is optional. If set, only pools whose detected driver matches become candidates. fallbackDriver is explicit opt-in — without it, a missing target driver fails the test with a clear message. Pool busyness does not trigger fallback; the runner waits inside the filtered set.

Force a driver for a whole run (CLI overrides win over per-test fields — useful for A/B benchmarks):

e2e-runner run --all --driver obscurae2e-runner run --all --driver obscura --fallback-driver cdp

Create e2e.config.js in your project root:

export default {  baseUrl: 'http://host.docker.internal:3000',  concurrency: 4,  retries: 2,  actionRetries: 1,  testTimeout: 30000,  outputFormat: 'both',  failOnNetworkError: true,  exclude: ['explore-*', 'debug-*'],
  hooks: {    beforeEach: [{ type: 'goto', value: '/' }],  },
  environments: {    staging: { baseUrl: 'https://staging.example.com' },    production: { baseUrl: 'https://example.com', concurrency: 5 },  },};

Config priority (highest wins):

1. CLI flags
2. Environment variables
3. Config file (e2e.config.js or e2e.config.json)
4. Defaults

When --env <name> is set, the matching profile overrides everything.