Incident Response MCP for Apache Airflow

Apache Airflow MCP Server

[MCP] [PyPI] [Python] [Airflow] [CI] [Security] [License] [Ruff]

Independent community project; not affiliated with or endorsed by the Apache Software Foundation.

Published through PyPI, GHCR, and the official MCP Registry. Reproducible scan reports, dependency audits, SBOMs, and provenance are attached to each immutable release.

Connect Claude, Cursor, VS Code, or another MCP client to your Apache Airflow deployments and help agents diagnose failed DAGs.

Paste an Airflow UI link from a PagerDuty/Datadog alert and ask "why did this fail?" — the agent resolves the URL, finds the failed tasks, pulls log errors filtered and bounded before the MCP response, and can re-trigger or clear runs. Write tools carry destructive-operation annotations that MCP clients can use to request confirmation.

Highlights

• 🔍 Incident-response first — resolve Airflow UI URLs straight to the failing task, filter logs by error level with context lines, follow try_number semantics correctly (sensors included)
• 🏢 Multi-instance — one server for same-API-family dev/staging/prod targets, with per-instance credentials and an SSRF guard that rejects unknown hosts
• 🔒 Safety controls — opt-in read-only mode (AIRFLOW_MCP_READ_ONLY=true) that never registers write tools; write tools annotated as destructive; configured credentials are redacted from instance responses and operation logs
• 📉 Token-efficient — log tailing, level filtering, byte caps, and truncation metadata designed for LLM context windows
• 🧭 Airflow 2 and 3 — API v1/v2 adapters with live E2E against Airflow 2.11 and 3.3, including JWT auth
• 📎 Traceable — every response carries a request_id that matches the structured server logs

Quickstart

1. Install

uv tool install apache-airflow-mcp-server \  --with 'apache-airflow-client==3.3.0'  # replace 3.3.0 with your Airflow version

Airflow 2.11? Use --with 'apache-airflow-client==2.10.0' instead; 2.10.0 is the final generated v1 client and targets Airflow 2's stable API. See Airflow compatibility before using a different release.

2. Connect your MCP client

The fastest path is a single instance configured entirely with environment variables — no config file needed.

claude mcp add airflow \  --env AIRFLOW_MCP_HOST=https://airflow.example.com \  --env AIRFLOW_MCP_USERNAME=admin \  --env AIRFLOW_MCP_PASSWORD=your-password \  --env AIRFLOW_MCP_READ_ONLY=true \  -- uvx --from apache-airflow-mcp-server \  --with apache-airflow-client==3.3.0 airflow-mcp --transport stdio

{  "mcpServers": {    "airflow": {      "command": "uvx",      "args": ["--from", "apache-airflow-mcp-server", "--with", "apache-airflow-client==3.3.0", "airflow-mcp", "--transport", "stdio"],      "env": {        "AIRFLOW_MCP_HOST": "https://airflow.example.com",        "AIRFLOW_MCP_USERNAME": "admin",        "AIRFLOW_MCP_PASSWORD": "your-password",        "AIRFLOW_MCP_READ_ONLY": "true"      }    }  }}

{  "mcpServers": {    "airflow": {      "command": "uvx",      "args": ["--from", "apache-airflow-mcp-server", "--with", "apache-airflow-client==3.3.0", "airflow-mcp", "--transport", "stdio"],      "env": {        "AIRFLOW_MCP_HOST": "https://airflow.example.com",        "AIRFLOW_MCP_USERNAME": "admin",        "AIRFLOW_MCP_PASSWORD": "your-password",        "AIRFLOW_MCP_READ_ONLY": "true"      }    }  }}

{  "servers": {    "airflow": {      "type": "stdio",      "command": "uvx",      "args": ["--from", "apache-airflow-mcp-server", "--with", "apache-airflow-client==3.3.0", "airflow-mcp", "--transport", "stdio"],      "env": {        "AIRFLOW_MCP_HOST": "https://airflow.example.com",        "AIRFLOW_MCP_USERNAME": "admin",        "AIRFLOW_MCP_PASSWORD": "your-password",        "AIRFLOW_MCP_READ_ONLY": "true"      }    }  }}

AIRFLOW_MCP_HOST=https://airflow.example.com \AIRFLOW_MCP_USERNAME=admin AIRFLOW_MCP_PASSWORD=your-password \AIRFLOW_MCP_READ_ONLY=true \airflow-mcp --transport http --host 127.0.0.1 --port 8765

{ "mcpServers": { "airflow": { "url": "http://127.0.0.1:8765/mcp" } } }

3. Ask your agent something

"Why did the latest run of etl_pipeline fail?"

"https://airflow.example.com/dags/etl_pipeline/grid — what happened here, and is it safe to clear?"

"Show the failed task's error context and tell me the smallest recovery action."

Airflow compatibility

The server talks to Airflow through the generated apache-airflow-client. For Airflow 3, match the client release to your Airflow release: generated models can change within a major version, and a newer client is not guaranteed to deserialize an older server's responses correctly. Airflow 2.11 uses the final v1 client release, 2.10.0, against Airflow 2's stable API.

When api_version isn't set, the server assumes the API matching the installed client (v1 for a 2.x client, v2 for 3.x). Set AIRFLOW_MCP_API_VERSION (or api_version: in the registry) explicitly to catch a major-version mismatch early.

One server process can currently load only one generated client major. All instances in a registry must therefore use the same API family; run separate MCP server processes for Airflow 2 and Airflow 3. Mixed-version support requires a future client-adapter change and is not advertised as working today.

Airflow 3 notes:

• Auth: bearer tokens are passed through as JWTs; basic credentials are automatically exchanged for a JWT via POST /auth/token and refreshed periodically (AIRFLOW_MCP_TOKEN_REFRESH_SECONDS, default 3600 — keep it below your deployment's JWT expiry, and note there is no automatic re-auth on 401 yet).
• execution_date ordering maps to logical_date, datasets map to assets, and UI links use the Airflow 3 route scheme. Tool names and the core workflow stay stable; documented fields and options can differ by API family.
• Clear options that no longer exist in Airflow 3 (include_subdags/include_parentdag, and the include_*/reset_dag_runs options of airflow_clear_dag_run) are rejected with INVALID_INPUT rather than silently narrowing a destructive operation.

Both client families are exercised on relevant pull requests and main pushes. Bug reports from real Airflow deployments are very welcome!

Configuration

Single instance (env vars only)

* provide either username+password or a token.

Multiple instances (registry YAML)

Point AIRFLOW_MCP_INSTANCES_FILE at a YAML registry (it takes precedence over the single-instance env vars). Values may reference environment variables with ${VAR}:

data-stg:  host: https://airflow.data-stg.example.com/  api_version: v1        # Airflow 2  verify_ssl: true  auth:    type: basic    username: ${AIRFLOW_DATA_STG_USERNAME}    password: ${AIRFLOW_DATA_STG_PASSWORD}
data-prod:  host: https://airflow.data-prod.example.com/  api_version: v1        # Keep one client/API family per server process  auth:    type: bearer    token: ${AIRFLOW_DATA_PROD_TOKEN}

Every tool accepts either an instance key (data-stg) or a ui_url — a full http(s) Airflow UI URL whose host is resolved against the registry, with unknown hosts rejected (SSRF guard). ui_url also auto-fills dag_id/dag_run_id/task_id when the link contains them. If both instance and ui_url are passed and disagree, the call fails with INSTANCE_MISMATCH rather than guessing.

Kubernetes tip: mount the registry from a Secret at /config/instances.yaml and set AIRFLOW_MCP_INSTANCES_FILE=/config/instances.yaml.

Server options

Read-only mode

The quickstarts set AIRFLOW_MCP_READ_ONLY=true: write tools (trigger, clear, pause/unpause) are never registered. This prevents MCP mutations, but read tools can still disclose sensitive logs, configuration, rendered fields, and DAG-run data; use least-privilege Airflow credentials.

To enable recovery operations deliberately, set AIRFLOW_MCP_READ_ONLY=false. Write tools then carry MCP destructiveHint annotations that clients can use when deciding whether to request confirmation. Annotations are advisory, so do not enable writes unless the Airflow credentials and MCP client's approval behavior are appropriate for the target environment.

Tools

Discovery & URL utilities

Read

Write (annotated destructive so clients can require approval; hidden entirely in read-only mode)

Every success payload includes a request_id for log correlation; failures raise a structured ToolError with {code, message, request_id, context}.

The incident workflow

This is the flow the tools were designed around — going from an alert link to a diagnosis in four calls:

# 1. Alert contains an Airflow UI link → resolve itairflow_resolve_url("https://airflow.example.com/dags/etl_pipeline/grid?dag_run_id=...")#    → {instance, dag_id, dag_run_id, ...}
# 2. Which tasks failed in this run?airflow_list_task_instances(dag_id="etl_pipeline", dag_run_id="scheduled__2026-01-01",                            state=["failed"])
# 3. Get attempt metadata (authoritative try_number, retries, timings)ti = airflow_get_task_instance(dag_id="etl_pipeline",                               dag_run_id="scheduled__2026-01-01",                               task_id="transform_data")
# 4. Pull only the error lines, with context, capped for the LLMairflow_get_task_instance_logs(dag_id="etl_pipeline",                               dag_run_id="scheduled__2026-01-01",                               task_id="transform_data",                               try_number=ti["attempts"]["try_number"],                               tail_lines=500, filter_level="error", context_lines=5)

Log responses include truncated, auto_tailed (logs >100MB tail automatically), match_count, and byte/line stats so the agent knows exactly what it's looking at. Host-segmented logs are flattened with --- [worker-1] --- headers; Airflow 3 structured logs are rendered as plain lines.

Note on try_number: reschedule-mode sensors could increment it on every reschedule through Airflow 2.9; Airflow 2.10+ no longer does. Always read it from airflow_get_task_instance rather than guessing—the derived retries_consumed/retries_remaining fields are heuristics.

Deployment

Docker

docker run -p 127.0.0.1:8765:8765 \  -e AIRFLOW_MCP_HOST=https://airflow.example.com \  -e AIRFLOW_MCP_USERNAME=admin \  -e AIRFLOW_MCP_PASSWORD=your-password \  -e AIRFLOW_MCP_READ_ONLY=true \  ghcr.io/madamak/apache-airflow-mcp-server:latest

Or build locally with docker build -t airflow-mcp .

The release image contains the lockfile's Airflow 3.3 client and serves streamable HTTP on :8765 (/mcp endpoint, /health for probes). The MCP HTTP endpoint has no built-in caller authentication: keep it loopback-bound or place it behind an authenticated private proxy. Mount a same-API-family registry YAML for multi-instance setups. Airflow 2 deployments should use the pinned local installation path above until a separate v1 image is published.

CI audits the exact image's installed Python dependencies and scans both the read-only and write-enabled MCP tool surfaces with a pinned Cisco MCP Scanner release's YARA analyzer. The security workflow fails on incomplete scans or any untriaged YARA finding. Starting with v1.0.1, release assets include the machine-readable scan reports, and release image manifests carry attached SBOM and provenance attestations covering their broader package inventory. These are automated checks, not a security certification or substitute for deployment-specific review.

FastMCP tooling

A fastmcp.json is included so FastMCP-aware tooling can auto-discover the entrypoint and deployment defaults.

Development

uv sync                 # install dependenciesuv run pytest           # unit tests (no real network; the Airflow client is mocked)uv run ruff check .     # lintuv run ruff format .    # formatuv run airflow-mcp --transport stdio   # run locally./scripts/e2e.sh af2    # end-to-end against Airflow 2.11./scripts/e2e.sh af3    # end-to-end against Airflow 3.3                        # Both seed failures/noisy logs and drive every tool                        # through MCP. Set E2E_KEEP=1 to keep the instance up.

CI runs the unit suite against both apache-airflow-client families on Python 3.10–3.13. Relevant pull requests, main pushes, nightly runs, and releases also exercise live dockerized Airflow 2.11 and 3.3. See CONTRIBUTING.md for guidelines and AGENTS.md if you're pointing a coding agent at this repo (it's written for that).

Contributing

Issues and PRs are welcome — especially:

• Reports from real Airflow incident-response workflows and version combinations
• Bounded-log, diagnosis-safety, and URL-first workflow improvements
• Client setup recipes for more MCP hosts and deployment types

If this server saves you a debugging session, a ⭐ helps other Airflow teams find it.

License

Apache 2.0 — see LICENSE.

Apache Airflow and Airflow are registered trademarks of The Apache Software Foundation. This independent project is not affiliated with or endorsed by the ASF.