AssemblyAI

AssemblyAI MCP Server

A remote MCP server for the AssemblyAI API. It gives any MCP-capable agent five tools covering async speech-to-text, Speech Understanding (translation, speaker identification, custom formatting, chaptered summarization, action items), and Guardrails (PII redaction, content moderation, profanity filtering) — served over Streamable HTTP and authenticated per-request, either by signing in with AssemblyAI (OAuth) or with the caller's own AssemblyAI API key.

Works with any MCP client that supports MCP authorization (Claude Code, Claude.ai, Cursor, MCP Inspector, and others) or can send a Bearer header. Built with Next.js 16 + mcp-handler + the official @modelcontextprotocol/sdk; deploys as a Vercel serverless function.

Quickstart

Sign in with AssemblyAI: add the server's /mcp endpoint with no headers. The client opens AssemblyAI's sign-in and consent page, and the server uses an API key it gets for that connection; nothing to copy or paste. (This works against AssemblyAI's hosted deployment; a self-hosted copy uses API keys, see Auth.)

claude mcp add --transport http assemblyai https://<host>/mcp

With an API key: point your MCP client at the /mcp endpoint with your AssemblyAI API key as the Bearer token.

Claude Code

claude mcp add --transport http assemblyai https://<host>/mcp \  --header "Authorization: Bearer $ASSEMBLYAI_API_KEY"

Generic remote-MCP client config

{  "mcpServers": {    "assemblyai": {      "url": "https://<host>/mcp",      "headers": { "Authorization": "Bearer <ASSEMBLYAI_API_KEY>" }    }  }}

No hosted instance handy? npm install && npm run dev serves it at http://localhost:3000/mcp (see Local dev).

Then prompt the agent, e.g. "transcribe this audio file and summarize it: https://assembly.ai/wildfires.mp3".

Tools

Typical agent flow: submit_transcript → poll get_transcript until completed → optionally understand_transcript / summarize_transcript → delete_transcript when the user asks for cleanup. When the need is known up front, request Speech Understanding features inline at submit time instead of post-hoc.

submit_transcript parameter surface

Accepts the full public async transcription param surface with API-native names, grouped roughly as:

• Prompting — prompt (a contextual description of the audio, not instructions), keyterms_prompt
• Language — language_code, language_codes, language_detection, language_detection_options, language_confidence_threshold
• Models — speech_models (defaults to the pinned ["universal-3-5-pro", "universal-2"] priority list), temperature
• Diarization — speaker_labels, speaker_options
• Guardrails — redact_pii (+ redact_pii_policies, redact_pii_sub, redact_pii_audio, redact_pii_return_unredacted, redact_static_entities), filter_profanity, content_safety (+ content_safety_confidence), speech_threshold
• Speech Understanding, inline — speech_understanding.request.{translation, speaker_identification, custom_formatting, summarization, action_items}, run during transcription
• Other — sentiment_analysis (English only), entity_detection, iab_categories, auto_highlights, multichannel, disfluencies, custom_spelling, punctuate, format_text, audio_start_from, audio_end_at, domain (medical-v1 for Medical Mode), and remove_audio_tags, plus webhooks (webhook_url, webhook_auth_header_name/_value)

Omitted params are never sent — AssemblyAI's own defaults apply. See the submit endpoint docs for parameter semantics.

get_transcript output

Everything renders into content[].text as labeled sections, because some MCP clients surface only that to the model: --- text --- and --- utterances --- always; sentiment, entities, translations (whole-transcript and per-utterance), speaker identification, custom formatting, chaptered summary, action items, content safety, topics, highlights, unredacted text, the redacted-audio URL, and processing warnings when requested/present. Raw fields are mirrored in structuredContent.

Auth

Every request carries a Bearer token, which is one of two things.

An OAuth access token (Sign in with AssemblyAI). A request with no token gets a 401 whose WWW-Authenticate names /.well-known/oauth-protected-resource, which names AssemblyAI's authorization server. The client registers itself there, signs the member in through the consent page (scope aai:api), and sends the access token it gets back. AssemblyAI's API takes API keys, not access tokens, so src/oauth.ts exchanges the token for the API key of the member's connection to that client (POST /api/v2/oauth/api-key) before the request reaches withMcpAuth, and the request continues with that key. The exchange also proves which server is asking: the deployment's Vercel OIDC token goes in X-AAI-Service-Token, and AssemblyAI only accepts its own production deployment of this project, so a self-hosted copy can't exchange tokens and uses API keys instead. Keys are cached per access token for at most five minutes, so a connection revoked from the dashboard stops working within minutes. A refused token answers 401 (403 for a missing scope) and sends the client back to sign-in; an outage answers 503, which doesn't.

An AssemblyAI API key (pass-through). MCP clients send the API key as Authorization: Bearer <key> (most client configs prepend Bearer automatically). AssemblyAI's REST API expects the raw key, so withMcpAuth extracts the token, strips the prefix, and threads it into every tool callback via extra.authInfo.token. No keys are written to disk or shared between requests — each request runs on the key it arrived with.

Configuration

Every setting defaults to AssemblyAI's production endpoints, so a production deployment needs none. A deployment that signs in against another AssemblyAI environment sets all four together, since that environment's keys only work against its own API.

The service token needs no setting: Vercel provides it to the deployment (VERCEL_OIDC_TOKEN), and @vercel/oidc reads it.

Extending the tool surface (additive-only contract)

MCP clients discover tools at runtime (list_tools); client configs store only URL + auth, not a tool-schema snapshot. So new tools and new optional parameters appear automatically — no client reconfiguration, and for platforms with a connection-test step, no re-test. To keep that guarantee, all changes here are additive:

1. Tool names are permanent — only add tools, never rename or remove.
2. New inputs are always optional, with defaults that preserve current behavior.
3. get_transcript text output only gains labeled sections — existing sections (--- text ---, --- utterances ---, --- sentiment ---, --- entities ---) are never renamed or reshaped, because text-only MCP clients read content[].text. (A legacy --- summary --- section is still rendered if present, but summaries now come from the summarize_transcript tool, not get_transcript.) The same protection covers the newer sections added for Speech Understanding and guardrails: --- translation:<lang> ---, --- translated_utterances ---, --- speaker_identification ---, --- custom_formatting ---, --- su_summary ---, --- action_items ---, --- content_safety ---, --- topics ---, --- highlights ---, --- unredacted_text ---, --- redacted_audio ---, and --- warnings ---.
4. structuredContent is additive only — new fields, never removed/renamed.
5. Avoid deprecated AssemblyAI transcript params (auto_chapters, summarization, summary_model, summary_type); use LLM Gateway instead. Mirror AssemblyAI's official MCP tool shapes where they exist.

Local dev

Requires Node 24.x.

npm installnpm test                # in-process smoke + OAuth exchange tests (no network)npm run dev             # next dev on http://localhost:3000

MCP endpoint: http://localhost:3000/mcp. OAuth protected resource metadata: http://localhost:3000/.well-known/oauth-protected-resource. Sign-in needs a deployment AssemblyAI accepts (see Auth); locally, use an API key.

Smoke test with MCP Inspector

npx @modelcontextprotocol/inspector

Connect to http://localhost:3000/mcp via Streamable HTTP, paste your AssemblyAI key as the Bearer token. Call submit_transcript with a public mp3 URL, then get_transcript with the returned id until completed.

Smoke test with curl

# No auth — expect 401curl -i -X POST http://localhost:3000/mcp \  -H "Accept: application/json, text/event-stream" \  -H "Content-Type: application/json" \  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'
# With your AssemblyAI key as the Bearer token — expect 200 + Mcp-Session-Idcurl -i -X POST http://localhost:3000/mcp \  -H "Authorization: Bearer $ASSEMBLYAI_API_KEY" \  -H "Accept: application/json, text/event-stream" \  -H "Content-Type: application/json" \  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'

Live end-to-end test (real AssemblyAI, real transcription)

ASSEMBLYAI_API_KEY=... npm run test:live                       # against localhost:3001ASSEMBLYAI_API_KEY=... MCP_URL=https://<host>/mcp npm run test:live

Drives the running server with the MCP SDK client over Streamable HTTP: tool discovery, all submit_transcript feature flags, speech_model_used assertion, summarize_transcript, inline Speech Understanding (translation + chaptered summary), guardrails (content safety), post-hoc Speech Understanding via understand_transcript, graceful failure on a bad audio URL, and teardown via delete_transcript. Every transcript id is recorded as soon as submit_transcript returns it and deleted from a finally block, so the suite tears down every transcript it successfully creates even when the run fails midway. Costs a few cents of transcription credit per run.

Sign-in test against a deployment

MCP_URL=https://<deployment>/mcp VERCEL_PROTECTION_BYPASS=<secret> npm run test:oauth

Signs in the way an MCP client does, discovering everything from the deployment: the 401 challenge, the protected resource metadata, the authorization server's metadata, client registration, then PKCE through the consent page in your browser (click Allow). Then it lists tools and makes one tool call with the access token, and checks a tampered token is refused. VERCEL_PROTECTION_BYPASS is the deployment's Protection Bypass for Automation secret, for preview deployments behind Vercel Authentication. No API key needed and no transcription credit spent.

Project layout

assemblyai-mcp/├── app/│   ├── mcp/[[...path]]/route.ts             # mcp-handler entry, wrapped with withOAuthExchange + withMcpAuth│   └── .well-known/oauth-protected-resource/route.ts├── src/│   ├── server.ts                            # registerTools factory│   ├── oauth.ts                             # Sign in with AssemblyAI: access token → API key exchange│   ├── log.ts                               # JSON logging helpers + key hashing│   ├── tool-errors.ts                       # shared 401/404/429 error mapper│   ├── assemblyai.ts                        # fetch wrapper (submit, get, delete, retry)│   ├── llm-gateway.ts                       # LLM Gateway client (summarize, understand)│   ├── transcript-sections.ts               # section renderers for tool text output│   ├── speech-understanding-schema.ts       # shared speech_understanding zod schema│   ├── tools/│   │   ├── submit-transcript.ts│   │   ├── get-transcript.ts│   │   ├── summarize-transcript.ts│   │   ├── understand-transcript.ts│   │   └── delete-transcript.ts│   ├── smoke-test.ts                        # InMemoryTransport, mocked fetch│   ├── oauth-test.ts                        # token exchange, stubbed (npm test)│   ├── oauth-preview-test.ts                # sign-in against a deployment (npm run test:oauth)│   └── live-test.ts                         # real API, Streamable HTTP client├── package.json├── tsconfig.json├── next.config.js├── vercel.json└── README.md

Logging

JSON to stdout, same pattern as AssemblyAI's docs MCP server. Every tool emits one event with event, tool, keyHash, latencyMs, status. High-signal events worth alerting on: bad_audio_url, assemblyai_rejected_key, tool_error, understanding_error, delete_error. Never logs the raw API key — only sha256(key).slice(0,12) — and never logs transcript content.

Deployment

Deploys to Vercel (vercel deploy once the project is linked). Vercel Firewall provides rate limiting and DDoS protection. The MCP endpoint lives at https://<host>/mcp with maxDuration: 60 per the handler config.

Reference: prior Python implementation

The Python FastMCP-based implementation that preceded this TypeScript port is preserved in the first commit of this repo (git log --reverse --oneline | head -1). It serves as the source of truth for the design (auth bridge, error handling, response shapes); this TypeScript port carries the design forward onto a serverless-friendly platform.