ai.openmarkPublished by openmark.aiOfficial publisherv1.0.0Updated Oct 25, 2026
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
OpenMark's hosted MCP server (openmark.ai), published by openmark.ai; API key required; Streamable HTTP; 9 tools.
No AI overview yet: these facts come from the registry entry and the server itself.
Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.
Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.
Read https://sourceweft.com/mcp/ai.openmark%2Fopenmark/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.Add this to your client's mcpServers config.
{
"mcpServers": {
"openmark": {
"type": "http",
"url": "https://openmark.ai/mcp",
"headers": {
"Authorization": "Bearer <YOUR_TOKEN>"
}
}
}
}Task YAML format
How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name.
topicCheck a task
Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.
task_yamlrequiredList models
Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.
searchimages_onlymax_output_pricePrepare file attachments
Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass the upload_uid to estimate_cost / run_benchmark.
file_namesEstimate a run's cost
Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmark. Use it to drop an expensive model, or to compare settings (e.g. max_tokens), before running.
modelsrequiredpresetfail_fasttask_yamlrequiredmax_tokensupload_uidcustom_timeoutstability_runstemp_preferencetimeout_profiletemperature_discoveryRun a benchmark
Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_credits to only start within a budget. Optional run settings match the app's Configure & Run panel (preset, stability_runs, max_tokens, temperature_discovery, temp_preference, timeout_profile, custom_timeout, fail_fast).
modelsrequiredpresetfail_fasttask_yamlrequiredmax_tokensupload_uidmax_creditscustom_timeoutstability_runstemp_preferencetimeout_profiletemperature_discoveryGet benchmark results
Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the others.
job_idrequiredtoleranceshow_missesoptimize_forAccount, credits and plans
The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.
Takes no parameters.
Watch a task for better models
Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal rate.
modeljob_idrequired83% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
Declare a GitHub repository with a README, or serve resource_documentation in the OAuth protected-resource metadata.
A hosted service is used under its provider's terms, so no license is expected here. This check is left out of the percentage.