ChatbotX

ChatbotX MCP Server

Model Context Protocol server for ChatbotX. Gives AI agents (Claude, Cursor, ChatGPT, etc.) access to your ChatbotX workspace through tools that are automatically generated from the ChatbotX OpenAPI spec — no manual tool definitions needed.

Published skill/agent distribution docs live in the separate chatbotx-agent package; this repo owns the MCP server implementation and README.

How it works

On startup the server fetches {CHATBOTX_API_URL}/public-spec.json and registers one MCP tool per API operation the spec marks x-mcp.visibility: "default". CHATBOTX_API_URL is the origin including the /api path prefix (e.g. https://app.chatbotx.io/api). Adding a new default-visible API endpoint in ChatbotX automatically makes it available as a tool — the spec is re-fetched in the background whenever it's older than CHATBOTX_SPEC_TTL_MS (default 5 minutes), so a new/changed operation shows up on the next tools/list call without a server restart.

OAuth 2.1

Remote MCP clients should authenticate through OAuth 2.1. The production endpoints are https://app.chatbotx.io/mcp/messages for Streamable HTTP and https://app.chatbotx.io/mcp/sse for legacy SSE — the same URL for every account, so it works as a Claude or ChatGPT marketplace connector. The user picks the target workspace on the consent screen, and every token the client holds acts on that workspace; to switch, disconnect and connect again. Only workspace owners and superAdmin members can authorize. An explicit ?workspace_id=<id> (or x-workspace-id header) may select a workspace only for legacy connections that predate the picker. A different workspace than the one consented for a new connection is rejected; disconnect and connect again to switch. The builder settings block Settings → Integrations → ChatbotX MCP provides the exact connection URL and lets each user revoke their OAuth grants.

Legacy workspace tokens remain supported for stdio and existing connections.

Default tools vs. the full API

ChatbotX's public API has ~350 operations. Listing all of them as MCP tools overwhelms an agent's context and its ability to pick the right one, so tools/list returns only a curated default set (see below) plus three meta-tools:

Use search_tools when the task needs something outside the default set (e.g. deleting a resource, managing AI agents, coupons, products) — then invoke it with call_tool.

search_tools's catalog (tool names, summaries, descriptions) is entirely English. The calling client must translate the user's intent into an English "action + resource" query before searching; the ranker does not translate. When an accented Latin or non-Latin query reaches the server, search_tools returns a hint asking the client to translate and retry in English.

• Name normalization: a dotted or camelCase operation label (contacts.get, contactsGet) is accepted and re-derived to the executable snake_case name — you don't have to pass the exact string search_tools returned.
• Unknown-tool suggestions: an unrecognized name returns the closest matching tool names instead of a bare error, so a model can self-correct without another search_tools round trip.
• Pre-flight argument check: a call_tool invocation missing a field the selected tool's inputSchema marks required is rejected immediately with the missing field names, instead of waiting for the real API's 422. A common wrapper mistake (nesting every field under body/params/input instead of passing them at the top level) is called out explicitly.
• Contact identifier auto-prefix: a bare email, +-prefixed or local 0-prefixed phone number, or numeric id passed as identifier to a contact tool is automatically prefixed (email:/phone:/id:) before the request is sent, matching what the API actually requires.

Human approval

Every operation carries an x-mcp.hitlPolicy from the public OpenAPI spec; reads use none. Mutating operations return CONFIRMATION_REQUIRED with the policy, proposed arguments, and current state when available; show that response to the user, then repeat the same call with _confirm: true only after explicit approval. _confirm is a compatibility guardrail, not an authorization boundary.

Writes that are read-only computations declare hitlPolicy: "none" in the backend. Destructive writes also carry the MCP destructiveHint annotation. When a current-state read fails for any reason, currentState is omitted and currentStateUnavailable explains why; the failed read never blocks confirmation.

Every tools/list definition also includes a human-readable title and explicit readOnlyHint, destructiveHint, idempotentHint, and openWorldHint annotations. Dynamic tool titles come from the OpenAPI operation summary. The loader marks confirm_outbound operations as open-world by default; an x-mcp.openWorldHint override takes precedence.

Scope-based filtering

tools/list is further narrowed to what the calling token can actually use, resolved once per token via GET /v1/token (introspectToken, cached per token value for CHATBOTX_SPEC_TTL_MS):

• A token missing a scope never sees that scope's tools (they still exist for search_tools/call_tool, which always hit the real API and get a real 403 if unauthorized).
• A read_only token only sees tools whose readOnlyHint annotation is true — every GET by default, plus any POST explicitly marked x-mcp.readOnlyHint: true for an endpoint that reads without writing (no default tool currently uses this).
• capabilities_get and token_get are always visible regardless of scope — an agent needs them to discover what it can do and what its token allows before anything else works.
• If token introspection itself fails (network blip, unreachable API), filtering fails open — tools/list falls back to the full default set. The actual API call still enforces the token's real permissions either way.

Discovery tools an agent should call first

Available tools

Tool names are derived from the OpenAPI operationId converted to snake_case (e.g. tags.list → tags_list). The current default set has 52 tools:

Capabilities

AI Agents

Analytics

Broadcasts

Connections

For connections_create with provider: "api", the response includes a one-time secret.token. Store it before discarding the response; it is never retrievable again.

Contacts

Conversations

Error Logs

Flows

Keywords

Messages

Mini Apps

Pipelines

Sequences

Everything else — deletes, less-common resources (coupons, products, webhooks, saved replies, tags/triggers/inboxes/custom-fields management, workspace members, etc.), connection lifecycle management beyond the initial connect (disconnect, reconnect, refresh, verify, rename) — is reachable via search_tools → call_tool, not tools/list.

Operations whose security names no workspace-token scheme are not tools at all — isWorkspaceTokenOperation (src/openapi-loader.ts) drops them, so neither tools/list nor search_tools returns them. That covers deprecated operations, channel-token operations (/v1/channels/api/*), and the mobile-app-only member surface (/v1/me*, /v1/realtime/connect-token, marked by MOBILE_APP_TOKEN_ONLY_PATHS in apps/builder/src/lib/orpc/public-spec.ts): an MCP OAuth or workspace token has no member identity and would always get 401.

conversations_list accepts view: "compact" to drop ad attribution and thread-control metadata from each conversation's contact inboxes (default full); flows_list accepts enableInInbox: true to return only flows that can be started from the inbox.

Prerequisites

• Node.js >= 24
• A workspace token only for the stdio and workspace-token connection options.

Quick start

Option A — OAuth (recommended, remote)

Use the builder-hosted MCP endpoint. The client opens a browser for sign-in and consent; no workspace token belongs in the client configuration.

Claude Code:

claude mcp add chatbotx -t http -s user \  "https://app.chatbotx.io/mcp/messages"

Claude Desktop / Cursor:

{  "mcpServers": {    "chatbotx": {      "type": "http",      "url": "https://app.chatbotx.io/mcp/messages"    }  }}

ChatGPT: add https://app.chatbotx.io/mcp/sse as a custom connector. ChatGPT completes OAuth in the browser.

The load balancer must forward /mcp/* to the MCP server without stripping the /mcp prefix. The builder serves protected-resource discovery under /.well-known/oauth-protected-resource/mcp/*.

Option B — stdio (workspace token, local use)

Claude spawns the server process on demand. No server needs to be running. The workspace token is supplied via CHATBOTX_API_KEY.

The package is published to npm as chatbotx-mcp — npx -y chatbotx-mcp fetches and runs it with no local checkout or build required. Use a local node /path/to/dist/index.mjs instead when developing against this repo.

Claude Code CLI:

claude mcp add chatbotx \  -e CHATBOTX_API_KEY=<your-token> \  -e CHATBOTX_API_URL=https://your-instance.com/api \  -e CHATBOTX_MCP_TRANSPORT=stdio \  -s user \  -- npx -y chatbotx-mcp

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{  "mcpServers": {    "chatbotx": {      "command": "npx",      "args": ["-y", "chatbotx-mcp"],      "env": {        "CHATBOTX_API_KEY": "<your-token>",        "CHATBOTX_API_URL": "https://your-instance.com/api",        "CHATBOTX_MCP_TRANSPORT": "stdio"      }    }  }}

Running from a local checkout instead (e.g. node /path/to/dist/index.mjs) works the same way — swap command/args accordingly.

Option C — SSE (workspace token, shared / remote access)

Run the server once and multiple clients connect via URL. Pass the workspace token in the request header.

# Start the serverpnpm start
# Add to Claude Code CLIclaude mcp add chatbotx \  -t sse \  -H "x-workspace-token: <your-token>" \  -s user \  "https://your-mcp-server.com/sse"

Claude Desktop:

{  "mcpServers": {    "chatbotx": {      "type": "sse",      "url": "https://your-mcp-server.com/sse",      "headers": {        "x-workspace-token": "<your-token>"      }    }  }}

Option D — ChatGPT.com (workspace-token SSE)

ChatGPT.com connects via SSE. The server must be publicly reachable. Because ChatGPT does not support custom request headers, pass the workspace token directly in the URL:

1. Start the server with CHATBOTX_MCP_TRANSPORT=sse (or both).
2. In ChatGPT Settings → Connectors → Add custom connector, set the URL:
   https://your-mcp-server.com/sse?workspace_token=<your-token>

   When using this URL in a shell command, always quote it to prevent ? being interpreted as a glob:
   bash

   claude mcp add chatbotx -t sse -s user "https://your-mcp-server.com/sse?workspace_token=<your-token>"

3. Set CHATBOTX_MCP_SERVER_INSTRUCTIONS so ChatGPT uses tools instead of its training data. Leave it unset unless replacing the full built-in workflow, including OAuth checkpoints.

Token resolution order

Configuration

Copy .env.example to .env and fill in the values:

cp .env.example .env

Scripts

# Development with watch modepnpm dev:mcp
# Build for productionpnpm build
# Run built serverpnpm start
# Type checkpnpm check-types
# List loaded tools (requires CHATBOTX_API_URL and CHATBOTX_API_KEY in .env)dotenv -e .env -- tsx src/test-tools.ts

Evaluating tool selection

Two layers of evaluation cover search_tools/call_tool accuracy — a fast offline gate and a full LLM-driven business eval.

eval:search — offline ranking gate (no API key, seconds)

Scores every prompt in the eval corpus (evals/cases.ts) through the real searchTools() ranker against a real generated spec, with no LLM in the loop. Use this after any change to src/server/search/ or to a public route's summary/description.

1. Dump the current OpenAPI spec (the builder test already has a hook for this — see apps/builder/__tests__/public-spec-operations.test.ts):
   bash

   MCP_EVAL_SPEC_OUTPUT=/absolute/path/public-spec.json \  pnpm --filter builder test -- public-spec-operations.test.ts

2. Run the gate against it:
   bash

   pnpm --filter chatbotx-mcp eval:search --spec /absolute/path/public-spec.json [--min-top1 0.65] [--min-top3 0.85] [--verbose]

   Prints per-locale top-1/top-3/empty-result rates, but enforces the configured thresholds only for English. --verbose also lists every prompt that missed the top 3, with the ranker's actual top matches, for targeted ranking or description fixes.

eval:business — full LLM-driven business eval

Spins up a synthetic HTTP sandbox implementing the generated spec's operations, connects a real MCP client, and drives gpt-4o-mini/gpt-4.1-mini (or any @ai-sdk/openai-supported model) through the selected corpus. business contains 240 prompts (48 families × 5 locales: vi, vi-unaccented, colloquial, en, mixed); multilingual is the 80-case smoke matrix (16 families × en, vi-natural, es, fr, zh). Requires OPENAI_API_KEY.

pnpm --filter chatbotx-mcp eval:business \  --spec /absolute/path/public-spec.json \  --out /absolute/path/out/baseline \  --phase smoke \  --seed 20260923 \  --models gpt-4o-mini,gpt-4.1-mini \  [--repeat N] \  [--corpus business|multilingual] \  [--exposure default|meta-only|both] \  [--cases family-a,family-b] \  [--server-source /absolute/path/to/apps/mcp-server]

• --repeat N runs every selected case under every model and exposure N times. Smoke runs default to 3; baseline and candidate runs default to 1.
• --exposure meta-only restricts the model's tool set to search_tools/call_tool only (no directly-listed default tools) — the path a client using only the default connection payload takes for any request outside the ~45 default-visible tools. both (the default) runs every case under both exposures.
• Each episode is graded on semantic arguments, ordered tool sequences, its expected final HTTP state, API-error claims, post-failure mutations, and step exhaustion. The summary reports the pass rate per model × exposure × locale × family; episodes also record searchRank, callToolErrorCount, and unknownToolCount.
• The smoke sandbox includes contact, conversation reply, flow, broadcast, appointment, and analytics fixtures. No public appointment-availability operation exists, so unavailable-slot coverage exercises appointments_book returning 422.
• Compare two runs (regressions fail the command):
  bash

  pnpm --filter chatbotx-mcp eval:business --compare /absolute/path/out/baseline /absolute/path/out/candidate

  Reports any case that regressed from pass to non-pass, plus the average searchRank and total callToolErrorCount delta between the two runs.

Evaluation output is written under evals/out/ (gitignored) unless --out points elsewhere.

Project structure

src/├── index.ts                # Entry point — loads spec, starts transport(s)├── env.ts                  # Environment variable schema├── guides/                 # Packaged markdown operational guides├── http.ts                 # Timed fetch helper for OpenAPI, token, and tool requests├── openapi-loader.ts       # Fetches OpenAPI spec → DynamicTool list, x-mcp│                           # visibility/scope/HITL parsing, scope-filtered getVisibleTools()├── token-introspection.ts  # GET /v1/token → cached {permission, scopes} per token├── test-tools.ts           # Dev utility — prints loaded tools└── server/    ├── create-mcp-server.ts   # MCP server factory, resources, tools/list + tools/call    ├── hitl-gate.ts           # Two-step confirmation gate for mutations    ├── meta-tools.ts          # search_tools / call_tool / get_guide definitions    ├── execute-tool.ts        # Shared HTTP dispatch, confirmation, argument normalization    ├── search/                # search_tools ranking: normalization and IDF scoring    ├── sse-server.ts          # SSE / Streamable HTTP transport    └── stdio-server.ts        # stdio transport
evals/├── cases.ts           # Eval corpus: business-intent prompts × 5 locale variants├── sandbox.ts         # Synthetic HTTP server implementing the generated spec├── run.ts             # eval:business — drives an LLM through the MCP server└── search-audit.ts    # eval:search — offline ranking gate, no LLM

Troubleshooting

Tools not showing up

• Check that CHATBOTX_API_URL is reachable and {CHATBOTX_API_URL}/public-spec.json returns a valid OpenAPI spec.
• Only operations the spec marks x-mcp.visibility: "default" appear in tools/list — everything else is reachable via search_tools/call_tool. See "Default tools vs. the full API" above.
• The spec/tool list refreshes automatically every CHATBOTX_SPEC_TTL_MS (default 5 minutes); a brand-new operation may take that long to appear without a restart.
• A token missing a required scope, or a read_only token calling a write endpoint's tool, will not see that tool — see "Scope-based filtering" above.
• Check stderr output on startup — the server logs Loaded N tools from OpenAPI spec.

Port already in use

lsof -ti:3333 | xargs kill -9

Self-signed certificate errors

CHATBOTX_ALLOW_SELF_SIGNED_CERT=true

SSE connection fails in Claude

• Prefer stdio mode for local use — it has no network dependency.
• For SSE, verify the server is running and the URL/port are reachable from the client.

ChatGPT uses its training knowledge instead of calling tools

• Set CHATBOTX_MCP_SERVER_INSTRUCTIONS to explicitly instruct ChatGPT to use tools (see .env.example).
• Make sure the connector URL includes ?workspace_token=<your-token> so authentication is handled automatically.