
AISIX AI Gateway
io.github.api7由 GitHub 帳號 api7 發布v1.3.0更新於 Oct 24, 2026
Self-hosted AI gateway that governs MCP servers, LLM traffic, and A2A agents behind one endpoint
概覽
自架式 AI 閘道,以單一 OpenAI 相容 API 介接各家大型語言模型,並把已註冊的 MCP 伺服器、A2A 代理與 REST API 當成工具對外提供。
- 可以做什麼
- 以 OpenAI 相容介面與 Anthropic Messages 介面代理大型語言模型流量,並轉換到 OpenAI、Anthropic、Bedrock、Vertex 或 Azure OpenAI 上游
- 依輪詢、一致性雜湊、容錯移轉、最低成本、最低延遲、最少負載、集成模型與語意路由等策略在模型間路由與容錯
- 在單一端點後彙整已註冊的上游 MCP 伺服器,由閘道保管憑證,依伺服器劃分工具命名空間並依呼叫方控管存取,還能把 REST API 的 OpenAPI 描述公開為 MCP 工具
- 透過 JSON-RPC 彙整 A2A 代理,並把代理卡片中的位址改寫為閘道位址
- 驗證呼叫方 API 金鑰或 OIDC/JWT 權杖,執行速率與並行限制、內容護欄與完全比對的回應快取
- 透過 Prometheus 指標、存取日誌、用量事件以及 OTLP、Datadog、物件儲存匯出器觀測流量
- 適用情境
- 適合多個團隊或應用需要統一的、受控管的多模型入口,或希望把多個 MCP 伺服器與 A2A 代理收斂到一個自行維運、具驗證機制的閘道之後。也適合偏好宣告式設定而非託管控制面的平台團隊。
- 執行需求
- 以本機容器映像(ghcr.io/api7/aisix)或靜態二進位檔形式執行於你自己的基礎設施。需要設定檔與 resources.yaml,用來宣告供應商金鑰、模型、呼叫方金鑰、MCP 伺服器與代理;供應商憑證透過 OPENAI_API_KEY 等環境變數參照提供,呼叫方金鑰透過 CALLER_API_KEY 等變數提供。多副本部署可選用 etcd,共用限流與快取可選用 Redis。
安裝
在 SourceWeft 中
- 開啟 儀表板中的 AISIX AI Gateway,將其新增到工作區。
- 為需要使用其工具的對話啟用該服務。
Desktop only,透過 STDIO。 STDIO 服務會啟動本機處理程序,因此需要 SourceWeft 桌面主機。
其他 MCP 客戶端
參照 儲存庫 中的啟動說明。
README
AISIX OSS
The open-source, Rust-native AI gateway for LLMs and AI agents
One OpenAI-compatible API in front of every model. Route, govern, secure, cache, and observe all your LLM and AI-agent traffic from a single control point — shipped as one static binary with low per-request overhead. Run it in your infrastructure for free, forever.
Built by the original creators of Apache APISIX.
[License: Apache 2.0] [Built with Rust] [Docs] [Discord] [Website]
Start free · Documentation · Quickstart · AISIX Enterprise and AISIX Cloud · Roadmap
[AISIX OSS architecture — one OpenAI- or Anthropic-compatible API in front of OpenAI, Anthropic, Gemini/Vertex, Bedrock, Azure OpenAI, and DeepSeek, with API key auth, rate and token limits, guardrails, caching, routing and failover, and observability in between]
AISIX OSS is the open-source product built around the Rust-native AISIX gateway. The gateway puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible endpoint. It gives platform teams one place to route, govern, secure, and observe LLM traffic, with first-class SSE streaming and low gateway overhead.
It runs as a single static binary — low cold-start, lock-free config reads, and hot
configuration reloads with no restarts: declare resources in one resources.yaml and
reload on SIGHUP, or point the gateway at etcd for a multi-replica cluster. Run the
gateway in your infrastructure, or connect it to AISIX Enterprise or
AISIX Cloud
for centralized management with team governance, budgets, audit, and a dashboard.
AISIX OSS (this repo) is the open-source product. It runs without a control plane using declarative configuration or etcd. The same gateway binary serves as the data plane in the commercial products: customers host the control plane with AISIX Enterprise, while API7 hosts it with AISIX Cloud. In all three products, the gateway runs in your environment and calls providers directly; live AI traffic does not pass through either control plane. The proxy API is identical throughout. Talk to us about AISIX Enterprise or AISIX Cloud →
⚡ Quickstart
One container. No control plane, no database, no configuration store — the gateway reads
every dynamic resource from one declarative resources.yaml.
Then call the gateway exactly like OpenAI:
Edit resources.yaml and send SIGHUP (docker kill -s HUP aisix) to apply changes with
no restart — an invalid file is rejected whole and the last good configuration keeps
serving. Check a file before booting with aisix validate --resources resources.yaml.
Full walkthrough: the
Gateway Quickstart ·
every field: the resources file reference.
For a multi-replica cluster, point the gateway at etcd instead — resources_file and
etcd are mutually exclusive.
✨ Why AISIX
- One API, every model. Speak the OpenAI or Anthropic wire format in; the gateway
translates to whichever provider each model points at. Point an OpenAI or Claude SDK at
one
base_urland switch models without changing code. - A real gateway, in Rust. Single static binary, low cold-start, lock-free config reads on the hot path, native streaming.
- Open source, free forever. Apache-2.0 licensed and built to run in your infrastructure. Choose AISIX Enterprise or AISIX Cloud when you want centralized management through a control plane and dashboard.
- Production controls built in. Routing & failover, rate limits, guardrails, caching, and observability ship in the box. (Budgets and spend caps are AISIX Enterprise and AISIX Cloud features — the gateway enforces the control plane's decisions.)
🧩 Features — available today
Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.
- OpenAI-compatible proxy (
:3000) —chat/completions,completions,responses,embeddings,rerank,images/{generations,edits},audio/{speech,transcriptions,translations},videos(submit → poll → fetch),files,batches,fine_tuning/jobs,realtime,GET /v1/models, plus a root-level/passthrough/:provider/*escape hatch. Native SSE streaming, tool/function calling, JSON mode, vision/multimodal input, and reasoning-content support. - Anthropic Messages API —
POST /v1/messagesas a first-class route, working against any configured upstream: requests and responses (including streaming) are translated both ways when a model points at a non-Anthropic provider. - Routing & failover — virtual/routing models with six strategies:
round_robin(smooth weighted round-robin),consistent_hash(session affinity keyed by header / cookie / API key / client IP),failover, plus metric-basedleast_cost,least_latency, andleast_busy. Per-targetprioritytiers (active/backup pools), retry budgets, cooldowns, tag-conditional targets, and per-attempt timeouts. - Ensemble models — fan one request out to a panel of models concurrently, then have a judge model synthesize a single answer, with a minimum-successful-responses threshold.
- Semantic routing — one virtual model that dispatches by the meaning of each request: it embeds the prompt, scores it against per-route example utterances, and routes to the best match (or a default). See the semantic routing docs.
- Rate limiting & concurrency — RPS/RPM/RPH/RPD + TPM/TPD + concurrency caps,
AND-combined across caller keys, models, and policy scopes (
api_key/model/team/member/team_member). Counters are per-process by default, or shared across replicas with the Redis backend. - Guardrails — content-policy enforcement on input and output, in-process or through a
provider: keyword/regex, built-in PII detection and redaction, Presidio, Lakera, OpenAI
Moderation, AWS Bedrock Guardrails, Azure AI Content Safety (Prompt Shield + text
moderation), and two Alibaba Cloud services. A block returns
422 content_filter; monitor mode records what would have happened without blocking. - Caching — exact-match response cache with per-policy TTL and model/key scope matchers; memory and Redis backends; cost-saved telemetry on every hit. Separately, automatic prompt caching can be enabled per direct Anthropic model to inject cache breakpoints, so callers get provider-side prompt discounts without changing their requests.
- MCP gateway — front registered upstream MCP servers at
/mcpwith gateway-held credentials, per-server tool namespaces, and per-caller access. It serves every Streamable HTTP revision from2025-03-26through stateless2026-07-28without downstream sessions. Upstreams useinitializeby default orserver/discoverwithprotocol_version: "2026-07-28". CI runs the official MCP suite's applicable tools-only protocol scenarios. Also exposes a REST API as MCP tools from its OpenAPI description. - A2A agent gateway — front A2A (Agent-to-Agent) agents at
/a2a/:agent, serving each agent's card with URLs rewritten to the gateway, over JSON-RPC 2.0. - Inbound authentication — caller API keys (SHA-256 hashed, model allowlists, expiry, rotation), or OIDC/JWT bearer tokens validated against registered providers (Entra ID, Okta, Google Workspace, or any OIDC issuer) with JWKS caching.
- Observability — Prometheus
/metrics, structured per-request access logs, usage events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver), plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob) telemetry. - Declarative configuration — one
resources.yamlcarries all ten resource collections (provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies, observability exporters, rate-limit policies, OIDC providers), validated against the same JSON Schemas the gateway uses at runtime.aisix validatechecks a file offline;SIGHUPreloads it atomically. - Operational endpoints —
/livezand/readyzon the proxy listener;/status/config,/status/ready,/status/models, and Prometheus/metricson a dedicated metrics listener (:9090). The admin listener (:3001) additionally serves a read-only resource surface, OpenAPI 3 with a Scalar UI, and a playground. Resources are managed declaratively — through theresources_file(reloaded on SIGHUP) or direct etcd writes — not through the admin listener; its former write endpoints were removed.
🔌 Supported providers
AISIX dispatches through five native adapter families — distinct wire-protocol bridges, not one generic relabel. Whatever the upstream protocol, the client-facing API stays OpenAI-shaped.
Plus specialized handling for vendor quirks (e.g. DeepSeek reasoning content) and dedicated rerank / embeddings vendors (Cohere, Jina). Details in adapter protocol families.
☁️ Choose an AISIX Product
Same gateway binary, same proxy API — in every form the gateway runs in your environment. AISIX Enterprise adds a customer-hosted commercial control plane. AISIX Cloud adds an API7-hosted commercial control plane.
The AISIX Enterprise and AISIX Cloud dashboard — overview metrics, multi-provider models, guardrails, budgets, and observability exporters across all your gateways.
▶ Try the live dashboard demo — aisix-demo.api7.ai
→ Want centralized governance, budgets, and a dashboard? Talk to API7 about AISIX Enterprise or AISIX Cloud, or book a demo.
🏗️ Architecture
A single Cargo workspace; the aisix-server crate builds one binary named aisix that
wires the crates together.
🗺️ Roadmap
Highlights on the roadmap; tracked live in issues:
- Semantic (embedding-similarity) response caching
- More observability sinks — Langsmith, Helicone, Slack alerts
- Prompt templates managed as gateway resources
- Llama-Guard as a guardrail provider
Shipped since this list was last written: the MCP gateway, the A2A agent gateway, OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio, PII, and OpenAI Moderation guardrails — see Features above.
🛠️ Development
Prerequisites: the Rust toolchain pinned in rust-toolchain.toml. Docker is only needed
for the tests that exercise etcd, Redis, or provider emulators.
💬 Community
- Discord — discord.gg/dUmRZ7Rvf
- Issues & discussions — github.com/api7/aisix/issues
- Contributing — CONTRIBUTING.md
- Website — api7.ai/ai-gateway
If AISIX is useful to you, a ⭐ helps other engineers find it.
📄 License
來源:README.md,提交 ad58900
工具
0版本歷史
2- v1.3.0最新Sep 21, 2026
- v0.12.0Sep 16, 2026
評分
已滿足適用檢查項的 50%
該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。
- 近 30 天內可連通25 分
請保持端點能夠回應 MCP 握手;SourceWeft 每 30 天探測一次。
- 已宣告認證方式10 分
請在註冊中心條目中宣告憑證請求標頭,或保持端點能夠回應探測,以便判定其認證方式。
- 已列出工具15 分
請讓伺服器在無憑證時回應 tools/list,或發布可在沙箱中無憑證啟動的軟體包。
- 描述必需15 分
- 圖示10 分
- 文件15 分
- 授權條款10 分