Codabench Mcp

io.github.harshitAgrPublished by GitHub account harshitAgrv0.1.1Updated Oct 25, 2026

MCP server for the Codabench REST API — drives a full ML-benchmark participant workflow.

VerifiedSTDIODesktop onlyData & AnalyticsAI & ML

Overview

AI-generated overview

Local MCP server for the Codabench REST API that lets an assistant find ML competitions, download data, submit results and read leaderboards.

What you can do
Search and read competitions, phases, rules and tasks, and fetch your profile details via the Codabench API tools, including a generic GET-only request tool for other endpoints. Download competition datasets with streaming and SHA-256 verification, subject to a size cap. Submit to a phase through Codabench's multi-step upload flow, then poll the submission until it finishes. List and inspect your submissions and their logs. Read competition leaderboards.
When to use it
Useful when you are taking part in a Codabench machine-learning competition and want an assistant to look up rules and phases, pull datasets, submit predictions and track results without switching to the web UI.
Requirements
Runs locally as a Python package (uvx codabench-mcp or uv). Requires a Codabench account and an API token in the CODABENCH_API_TOKEN environment variable. Network access to codabench.org is needed; CODABENCH_BASE_URL, CODABENCH_ALLOW_WRITE_RAW and CODABENCH_MAX_DOWNLOAD_BYTES are optional settings.
Before you install
The token in CODABENCH_API_TOKEN grants access to your Codabench account and is sent as an Authorization header, so keep it out of shared configs. Submitting to a phase writes data to Codabench and may count against competition limits. Setting CODABENCH_ALLOW_WRITE_RAW to 1 lets the generic request tool send non-GET calls, which can change remote state. Dataset downloads can be large and are capped by CODABENCH_MAX_DOWNLOAD_BYTES.

Installation

In SourceWeft

  1. Open Codabench Mcp in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

codabench-mcp

MCP server for the Codabench REST API. Lets an AI agent drive a full participant ML-benchmark workflow: discover competitions, read rules, download data, submit, poll, read leaderboards.

Features

Curated tools for the participant path:

  • search_competitions, get_competition, list_competition_phases, get_phase, get_competition_rules, list_competition_tasks, get_task
  • download_dataset (streaming + SHA-256)
  • list_my_submissions, get_submission, get_submission_logs
  • submit_to_phase (handles Codabench's 3-step upload flow)
  • poll_submission (backoff + timeout + non-error "still_running")
  • get_leaderboard, get_my_profile
  • codabench_request — generic REST escape hatch (GET-only by default)

Authentication

You need a Codabench API token. Two ways to get one:

1. Swagger UI (recommended)

  1. Visit https://www.codabench.org/api/docs/
  2. Find the api-token-auth endpoint → Try it out
  3. POST your username/password
  4. Copy the token value from the response

2. curl

bash
curl -X POST https://www.codabench.org/api/api-token-auth/ \  -H "Content-Type: application/json" \  -d '{"username":"YOUR_USERNAME","password":"YOUR_PASSWORD"}'

The response is {"token": "..."}.

Install

bash
uvx codabench-mcp

To run the bleeding-edge main branch instead of the last release:

bash
uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcp

Or for development:

bash
git clone https://github.com/harshitAgr/codabench-mcp.gitcd codabench-mcpuv syncuv run codabench-mcp  # requires CODABENCH_API_TOKEN

Configure an MCP client

Claude Desktop (claude_desktop_config.json)

json
{  "mcpServers": {    "codabench": {      "command": "uvx",      "args": ["codabench-mcp"],      "env": {        "CODABENCH_API_TOKEN": "paste-your-token-here"      }    }  }}

Claude Code (plugin marketplace)

shell
/plugin marketplace add harshitAgr/codabench-mcp/plugin install codabench-mcp@codabench-mcp

Then export your token in the shell where you launch Claude Code:

bash
export CODABENCH_API_TOKEN=paste-your-token-here

Claude Code (manual)

If you'd rather skip the plugin layer:

bash
claude mcp add codabench \  --env CODABENCH_API_TOKEN=paste-your-token-here \  -- uvx codabench-mcp

Environment variables

VariableRequiredDefaultPurpose
CODABENCH_API_TOKENyes—DRF token, sent as Authorization: Token <token>
CODABENCH_BASE_URLnohttps://www.codabench.orgOverride for tests
CODABENCH_ALLOW_WRITE_RAWno0Set to 1 to allow non-GET methods through codabench_request
CODABENCH_MAX_DOWNLOAD_BYTESno5368709120 (5 GB)Cap for download_dataset

Development

bash
uv syncuv run pytest                              # unit tests, no networkuv run ruff check .                        # lintuv run ruff format --check .               # format check

License

MIT — see LICENSE.

mcp-name: io.github.harshitAgr/codabench-mcp

Source: README.md at commit 6ee6313

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.1.1LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts