agentcheck

io.github.agentwaresPublished by GitHub account agentwaresv0.1.1Updated Oct 27, 2026

Synthetic checks, nightly regression replay and model-drift alerts for AI agents

VerifiedStreamable HTTPWeb executableDeveloper ToolsSecurity & MonitoringAI & ML

Overview

AI-generated overview

Remote MCP server for agentcheck that lets an assistant monitor AI agents, MCP servers and APIs with synthetic checks, nightly replay and incident lookups.

What you can do
  • Read public status pages and monitored targets: overall state, uptime, last scores, open incidents and badge URLs
  • Preview any public Atlassian Statuspage page once, including components and unresolved incidents
  • Look up model retirements and API version sunsets across provider deprecation pages
  • Enroll targets (HTTP endpoints, remote MCP servers, A2A agents), add checks and run them immediately
  • Record production traces, promote a trace into a replayable nightly check, and list open and recent incidents
  • Fetch machine-readable pricing tiers and add-ons
When to use it
Useful when you want an assistant to watch AI agents, MCP servers or API integrations for regressions, model or API deprecations, and outages, and to turn real production traces into recurring checks. The free read-only tools work without an account for quick status and pricing lookups.
Requirements
Runs as a hosted remote endpoint over Streamable HTTP; nothing to install locally. Write tools need an API key sent as an Authorization bearer token, created in the agentcheck dashboard. The status and pricing tools work without a key. Network access to the agentcheck service is required.
Before you install
The Authorization header carries a live API key; keep it out of logs and shared configs. Write tools create targets and checks, run checks that may incur LLM judge cost, and record production prompts, answers and tool calls to a third-party service. Check pricing before enabling rubric or baseline judges and nightly replay.

Installation

In SourceWeft

  1. Open agentcheck in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.agentwares%2Fagentcheck/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "agentcheck": {
      "type": "http",
      "url": "https://agentwares-agentcheck.vercel.app/api/mcp"
    }
  }
}

README

This repository has no README.

Tools

11

agentcheck_get_status

Current status of a public monitored target: overall state, uptime over 24h/7d/30d, last check time, last nightly scores, open incidents and the badge/status URLs. Use the owner (GitHub login) and target slug from the status page URL https://agentwares-agentcheck.vercel.app/<owner>/<slug>. No API key needed.

Parametersslugrequiredownerrequired

agentcheck_get_pricing

Machine-readable pricing for agentcheck: tiers with monthly USD price, target limits, check interval and features, plus per-run add-ons. Same data as /pricing.json. No API key needed.

Takes no parameters.

agentcheck_model_retirements

Is this model retiring? Pass model IDs as your code or config names them (gpt-5-mini, claude-sonnet-4-5-20250929, gemini-2.5-flash, openai/o3-mini). For each, returns every announced shutdown on OpenAI's, Anthropic's and Google's own deprecation pages (OpenAI API, Claude API, Gemini API, Vertex AI): status, shutdown date, recommended replacement, the replacement's list-price ratio per token, the API changes the provider documents, the source URL with the date it was read, and a page with the details. retiring: false means no shutdown for that ID is on those pages. Language models only. No API key needed.

Parametersmodelsrequired

agentcheck_api_sunsets

When does the API version this code pins stop working, and what happens then? Pass provider and version pairs as your code pins them (shopify 2025-10, meta v21.0, google-ads v22, hubspot v1, twilio-flex 2.16.2, twilio-notify v1, keap xml-rpc, exchange-online ews, stripe 2024-06-20). For each, returns every dated end the provider's own versioning, changelog or retirement page lists: status, end date (and whether the provider gives only a month), what happens after it (error, silent fall-forward to another version, app delisting, or unsupported), whether the failure is silent, the replacement version, the source URL with the date it was read, and a page with the details. Where the provider's own pages give different dates for the same end, other_dates lists each one with its page and date_note says which the counts run to. A pair with no entry comes back with the reason: older than every listed version, no end date published yet, or a provider that publishes none (Stripe). Covers Shopify, HubSpot, Meta Graph and Marketing APIs, Google Ads and Content API for Shopping, Twilio Flex UI and Notify, Thryv's Keap XML-RPC, and Exchange Web Services (EWS) in Exchange Online. No API key needed.

Parameterspinsrequired

agentcheck_statuspage_preview

Read a public Atlassian Statuspage page once: its name, overall status, components (id, name, status, group) and unresolved incidents, from the page's own /api/v2/summary.json. Pass any address on the page: status.example.com, https://status.example.com/history or the JSON URL. Use it before POST /api/v1/statuspage, which turns the components you give a URL into a live agentcheck status page checked by agentcheck itself, with no signup; the answer carries that request's body. Same read as /statuspage/preview: only public addresses, 5-second timeout, 512 KB cap. Private pages cannot be read. No API key needed.

Parametersurlrequired

agentcheck_create_target

Enroll something to monitor: an http endpoint (JSON or OpenAI-style chat), a remote MCP server (Streamable HTTP url) or an A2A agent (origin with /.well-known/agent-card.json). Pass `checks` to create checks in the same call (POST /api/v1/probe proposes three). Returns the target id, the public status page, the badge SVG URL and a README snippet. The first check runs on the next scheduled tick, within five minutes; call agentcheck_run_now to run immediately. Free tier: 1 target, hourly; Starter+: 5-minute checks. Requires an API key.

ParametersurlauthkindrequirednameslugalertschecksformatheadersisPublicmodelVarcorpusUrl+3 more

agentcheck_add_check

Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key.

ParameterskindrequirednamerequiredinputgoldenrequiredtargetIdrequiredintervalSec

agentcheck_run_now

Run every check of one of your targets immediately (outside the schedule) and return pass/fail per check with latency, judge cost and any incident opened or closed. Use it right after enrolling, or to confirm a fix. Requires an API key.

ParameterstargetIdrequired

agentcheck_record

Record one production interaction with your agent (the prompt or messages, the final answer, the tools it called, optionally the model) as a trace on a target. Call it from your agent or from a proxy in front of it after each task; promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions. Requires an API key.

ParametersnameinputrequiredmodeloutputrequiredtargetIdrequiredtoolCalls

agentcheck_promote_trace

Turn an imported or recorded trace into a replayable check whose golden is the recorded outcome (tool sequence + final-answer rubric). The check runs daily and in the nightly replay (Pro). Returns the check. Requires an API key.

ParameterstraceIdrequired

agentcheck_list_incidents

Open and recent incidents across your targets (or one target): when they opened/closed, the failing check and the cause. Requires an API key.

ParameterstargetId

Version history

1
  1. v0.1.1LatestSep 16, 2026

Score

75% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts

    Declare a GitHub repository with a README, or serve resource_documentation in the OAuth protected-resource metadata.

  • License10 pts

    Declare a license in the repository or the registry entry.