Smolanalytics

npx smolanalytics test --url https://yourapp.com --test "the pricing page shows a monthly price"

A test is one sentence in a markdown file. An agent opens a real browser, reads the page through its accessibility tree, works out how to do what the sentence says, and returns a verdict. There is no test code, so there is no selector to update when a button moves.

A run that passes is recorded, and every run after it replays that recording with no model call at all. The agent comes back only when the recording stops fitting your app.

Five verdicts, never blurred: passed, failed, stale, errored, flaky. Only failed is a claim about your app. A recording that stopped fitting is stale, never a bug report, and a runner that could not finish is errored, never counted against you.

On every pull request

test --suite tests/ --url <preview> --comment runs a folder of sentences and leaves one comment on the pull request, edited in place on every push: what broke, the file most likely responsible and the evidence for that guess. Exit codes are 0 (clean), 1 (your app failed a test) and 2 (the runner could not finish). The workflow file is in the docs.

In your editor (MCP)

{ "mcpServers": { "smolanalytics": { "command": "npx", "args": ["-y", "smolanalytics", "mcp"] } } }

Three tools: run_tests, can_i_ship and list_tests. Listed on the official MCP Registry as io.github.Arjun0606/smolanalytics.

Evidence rather than claims

smolanalytics.com/proof is a GitHub Action that runs this product's own suite against two live sites every night and publishes every verdict, including the red ones. Nothing on that page is written by hand.

What it costs

The CLI runs on your machine or your CI runner, on your own ANTHROPIC_API_KEY. A first run of a test costs roughly what one agent conversation costs; a replay costs nothing. The hosted side at smolanalytics.com is $19/month with 100 tested pull requests included, after a 14-day trial with no card. This is commercial software, not open source.

Full CLI reference: cli/README.md. Questions: smolanalytics.com/contact.