AISIX AI Gateway

io.github.api7由 GitHub 帳號 api7 發布v1.3.0更新於 Oct 24, 2026

Self-hosted AI gateway that governs MCP servers, LLM traffic, and A2A agents behind one endpoint

概覽

AI 產生的概覽

自架式 AI 閘道,以單一 OpenAI 相容 API 介接各家大型語言模型,並把已註冊的 MCP 伺服器、A2A 代理與 REST API 當成工具對外提供。

可以做什麼
  • 以 OpenAI 相容介面與 Anthropic Messages 介面代理大型語言模型流量,並轉換到 OpenAI、Anthropic、Bedrock、Vertex 或 Azure OpenAI 上游
  • 依輪詢、一致性雜湊、容錯移轉、最低成本、最低延遲、最少負載、集成模型與語意路由等策略在模型間路由與容錯
  • 在單一端點後彙整已註冊的上游 MCP 伺服器,由閘道保管憑證,依伺服器劃分工具命名空間並依呼叫方控管存取,還能把 REST API 的 OpenAPI 描述公開為 MCP 工具
  • 透過 JSON-RPC 彙整 A2A 代理,並把代理卡片中的位址改寫為閘道位址
  • 驗證呼叫方 API 金鑰或 OIDC/JWT 權杖,執行速率與並行限制、內容護欄與完全比對的回應快取
  • 透過 Prometheus 指標、存取日誌、用量事件以及 OTLP、Datadog、物件儲存匯出器觀測流量
適用情境
適合多個團隊或應用需要統一的、受控管的多模型入口,或希望把多個 MCP 伺服器與 A2A 代理收斂到一個自行維運、具驗證機制的閘道之後。也適合偏好宣告式設定而非託管控制面的平台團隊。
執行需求
以本機容器映像(ghcr.io/api7/aisix)或靜態二進位檔形式執行於你自己的基礎設施。需要設定檔與 resources.yaml,用來宣告供應商金鑰、模型、呼叫方金鑰、MCP 伺服器與代理;供應商憑證透過 OPENAI_API_KEY 等環境變數參照提供,呼叫方金鑰透過 CALLER_API_KEY 等變數提供。多副本部署可選用 etcd,共用限流與快取可選用 Redis。
安裝前請注意
閘道持有上游供應商憑證與呼叫方 API 金鑰,需妥善保護資源檔與環境變數。9090 埠上的指標/狀態監聽器沒有驗證,應只綁定回環或私有網路。它會代理真實 AI 流量並可能在上游產生費用,預算與支出上限僅商業版提供。設定變更需改寫 resources.yaml 並送出 SIGHUP 才會生效;etcd 中的明文憑證不會加密儲存。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 AISIX AI Gateway,將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Desktop only,透過 STDIO。 STDIO 服務會啟動本機處理程序,因此需要 SourceWeft 桌面主機。

其他 MCP 客戶端

參照 儲存庫 中的啟動說明。

README

AISIX OSS

The open-source, Rust-native AI gateway for LLMs and AI agents

One OpenAI-compatible API in front of every model. Route, govern, secure, cache, and observe all your LLM and AI-agent traffic from a single control point — shipped as one static binary with low per-request overhead. Run it in your infrastructure for free, forever.

Built by the original creators of Apache APISIX.

[License: Apache 2.0] [Built with Rust] [Docs] [Discord] [Website]

Start free · Documentation · Quickstart · AISIX Enterprise and AISIX Cloud · Roadmap


[AISIX OSS architecture — one OpenAI- or Anthropic-compatible API in front of OpenAI, Anthropic, Gemini/Vertex, Bedrock, Azure OpenAI, and DeepSeek, with API key auth, rate and token limits, guardrails, caching, routing and failover, and observability in between]

AISIX OSS is the open-source product built around the Rust-native AISIX gateway. The gateway puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible endpoint. It gives platform teams one place to route, govern, secure, and observe LLM traffic, with first-class SSE streaming and low gateway overhead.

It runs as a single static binary — low cold-start, lock-free config reads, and hot configuration reloads with no restarts: declare resources in one resources.yaml and reload on SIGHUP, or point the gateway at etcd for a multi-replica cluster. Run the gateway in your infrastructure, or connect it to AISIX Enterprise or AISIX Cloud for centralized management with team governance, budgets, audit, and a dashboard.

AISIX OSS (this repo) is the open-source product. It runs without a control plane using declarative configuration or etcd. The same gateway binary serves as the data plane in the commercial products: customers host the control plane with AISIX Enterprise, while API7 hosts it with AISIX Cloud. In all three products, the gateway runs in your environment and calls providers directly; live AI traffic does not pass through either control plane. The proxy API is identical throughout. Talk to us about AISIX Enterprise or AISIX Cloud →

⚡ Quickstart

One container. No control plane, no database, no configuration store — the gateway reads every dynamic resource from one declarative resources.yaml.

yaml
# config.yamlresources_file: /etc/aisix/resources.yamlproxy:  addr: "0.0.0.0:3000"admin:  enabled: false          # a declarative gateway needs no admin listenerobservability:  metrics:    prometheus:      enabled: true      addr: "0.0.0.0:9090"
yaml
# resources.yaml_format_version: "1"
provider_keys:  - display_name: openai-main    provider: openai    api_key: ${OPENAI_API_KEY}        # interpolated from the environment
models:  - display_name: my-model    provider: openai    model_name: gpt-4o-mini    provider_key: openai-main
api_keys:  - display_name: local-dev    key_env: CALLER_API_KEY           # hashed at load; the plaintext is never stored    allowed_models: ["my-model"]
bash
export OPENAI_API_KEY="YOUR_PROVIDER_KEY"export CALLER_API_KEY="YOUR_CALLER_KEY"
docker run -d --name aisix \  -v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \  -v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \  -e OPENAI_API_KEY -e CALLER_API_KEY \  -p 3000:3000 -p 127.0.0.1:9090:9090 \  ghcr.io/api7/aisix:latest        # proxy → :3000, metrics + status → :9090#                                  ^ the metrics/status listener is unauthenticated;#                                    keep it on loopback or a private network

Then call the gateway exactly like OpenAI:

bash
curl http://localhost:3000/v1/chat/completions \  -H "Authorization: Bearer $CALLER_API_KEY" \  -H "Content-Type: application/json" \  -d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'

Edit resources.yaml and send SIGHUP (docker kill -s HUP aisix) to apply changes with no restart — an invalid file is rejected whole and the last good configuration keeps serving. Check a file before booting with aisix validate --resources resources.yaml.

Full walkthrough: the Gateway Quickstart · every field: the resources file reference. For a multi-replica cluster, point the gateway at etcd instead — resources_file and etcd are mutually exclusive.

✨ Why AISIX

  • One API, every model. Speak the OpenAI or Anthropic wire format in; the gateway translates to whichever provider each model points at. Point an OpenAI or Claude SDK at one base_url and switch models without changing code.
  • A real gateway, in Rust. Single static binary, low cold-start, lock-free config reads on the hot path, native streaming.
  • Open source, free forever. Apache-2.0 licensed and built to run in your infrastructure. Choose AISIX Enterprise or AISIX Cloud when you want centralized management through a control plane and dashboard.
  • Production controls built in. Routing & failover, rate limits, guardrails, caching, and observability ship in the box. (Budgets and spend caps are AISIX Enterprise and AISIX Cloud features — the gateway enforces the control plane's decisions.)

🧩 Features — available today

Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.

  • OpenAI-compatible proxy (:3000) — chat/completions, completions, responses, embeddings, rerank, images/{generations,edits}, audio/{speech,transcriptions,translations}, videos (submit → poll → fetch), files, batches, fine_tuning/jobs, realtime, GET /v1/models, plus a root-level /passthrough/:provider/* escape hatch. Native SSE streaming, tool/function calling, JSON mode, vision/multimodal input, and reasoning-content support.
  • Anthropic Messages API — POST /v1/messages as a first-class route, working against any configured upstream: requests and responses (including streaming) are translated both ways when a model points at a non-Anthropic provider.
  • Routing & failover — virtual/routing models with six strategies: round_robin (smooth weighted round-robin), consistent_hash (session affinity keyed by header / cookie / API key / client IP), failover, plus metric-based least_cost, least_latency, and least_busy. Per-target priority tiers (active/backup pools), retry budgets, cooldowns, tag-conditional targets, and per-attempt timeouts.
  • Ensemble models — fan one request out to a panel of models concurrently, then have a judge model synthesize a single answer, with a minimum-successful-responses threshold.
  • Semantic routing — one virtual model that dispatches by the meaning of each request: it embeds the prompt, scores it against per-route example utterances, and routes to the best match (or a default). See the semantic routing docs.
  • Rate limiting & concurrency — RPS/RPM/RPH/RPD + TPM/TPD + concurrency caps, AND-combined across caller keys, models, and policy scopes (api_key / model / team / member / team_member). Counters are per-process by default, or shared across replicas with the Redis backend.
  • Guardrails — content-policy enforcement on input and output, in-process or through a provider: keyword/regex, built-in PII detection and redaction, Presidio, Lakera, OpenAI Moderation, AWS Bedrock Guardrails, Azure AI Content Safety (Prompt Shield + text moderation), and two Alibaba Cloud services. A block returns 422 content_filter; monitor mode records what would have happened without blocking.
  • Caching — exact-match response cache with per-policy TTL and model/key scope matchers; memory and Redis backends; cost-saved telemetry on every hit. Separately, automatic prompt caching can be enabled per direct Anthropic model to inject cache breakpoints, so callers get provider-side prompt discounts without changing their requests.
  • MCP gateway — front registered upstream MCP servers at /mcp with gateway-held credentials, per-server tool namespaces, and per-caller access. It serves every Streamable HTTP revision from 2025-03-26 through stateless 2026-07-28 without downstream sessions. Upstreams use initialize by default or server/discover with protocol_version: "2026-07-28". CI runs the official MCP suite's applicable tools-only protocol scenarios. Also exposes a REST API as MCP tools from its OpenAPI description.
  • A2A agent gateway — front A2A (Agent-to-Agent) agents at /a2a/:agent, serving each agent's card with URLs rewritten to the gateway, over JSON-RPC 2.0.
  • Inbound authentication — caller API keys (SHA-256 hashed, model allowlists, expiry, rotation), or OIDC/JWT bearer tokens validated against registered providers (Entra ID, Okta, Google Workspace, or any OIDC issuer) with JWKS caching.
  • Observability — Prometheus /metrics, structured per-request access logs, usage events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver), plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob) telemetry.
  • Declarative configuration — one resources.yaml carries all ten resource collections (provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies, observability exporters, rate-limit policies, OIDC providers), validated against the same JSON Schemas the gateway uses at runtime. aisix validate checks a file offline; SIGHUP reloads it atomically.
  • Operational endpoints — /livez and /readyz on the proxy listener; /status/config, /status/ready, /status/models, and Prometheus /metrics on a dedicated metrics listener (:9090). The admin listener (:3001) additionally serves a read-only resource surface, OpenAPI 3 with a Scalar UI, and a playground. Resources are managed declaratively — through the resources_file (reloaded on SIGHUP) or direct etcd writes — not through the admin listener; its former write endpoints were removed.

🔌 Supported providers

AISIX dispatches through five native adapter families — distinct wire-protocol bridges, not one generic relabel. Whatever the upstream protocol, the client-facing API stays OpenAI-shaped.

Adapter familyReachesWire shape · auth
openaiOpenAI + any OpenAI-compatible vendor — DeepSeek, Groq, Mistral, Together, Fireworks, Perplexity, vLLM, Ollama, or self-hosted OpenAI-compatible endpointsOpenAI chat completions · Bearer
anthropicAnthropic ClaudeAnthropic Messages · x-api-key
bedrockAWS Bedrock — Anthropic, Meta Llama, Mistral, Cohere, Amazon Titan/Nova, AI21Bedrock Converse + /invoke · SigV4
vertexGoogle Vertex AI (Gemini)Vertex :generateContent · OAuth2
azure-openaiAzure OpenAIAzure deployments · api-key / Entra ID

Plus specialized handling for vendor quirks (e.g. DeepSeek reasoning content) and dedicated rerank / embeddings vendors (Cohere, Jina). Details in adapter protocol families.

☁️ Choose an AISIX Product

Same gateway binary, same proxy API — in every form the gateway runs in your environment. AISIX Enterprise adds a customer-hosted commercial control plane. AISIX Cloud adds an API7-hosted commercial control plane.

The AISIX Enterprise and AISIX Cloud dashboard — overview metrics, multi-provider models, guardrails, budgets, and observability exporters across all your gateways.

▶ Try the live dashboard demo — aisix-demo.api7.ai

AISIX OSS (this repo)AISIX EnterpriseAISIX Cloud
LicenseApache-2.0CommercialCommercial
Control planeNoneCustomer-hostedAPI7-hosted
ConfigurationDeclarative resources.yaml, or etcd for a clusterDashboard + AISIX Cloud Admin APIDashboard + AISIX Cloud Admin API
TenancySingle instance / namespaceOrganization-scoped environments, teams, and membersOrganization-scoped environments, teams, and members
Provider keys${VAR} references in resources.yaml; plaintext credentials in etcdEncrypted at rest; operator-triggered in-place rotationEncrypted at rest; operator-triggered in-place rotation
Budgets— (rate and token limits only)Per key, provider, environment, organization, and teamPer key, provider, environment, organization, and team
RBAC and auditLocal admin key; no organization layerOrganization roles, invitations, and audit historyOrganization roles, invitations, and audit history
OperationsYou operate the gatewayYou operate the control plane and gatewaysAPI7 operates the control plane; you operate gateways

→ Want centralized governance, budgets, and a dashboard? Talk to API7 about AISIX Enterprise or AISIX Cloud, or book a demo.

🏗️ Architecture

A single Cargo workspace; the aisix-server crate builds one binary named aisix that wires the crates together.

text
crates/├── aisix-core           Config, snapshot, resource model, resources.yaml source, errors├── aisix-etcd           Config provider + watch supervisor├── aisix-gateway        Hub & bridge, SSE parser, provider trait├── aisix-proxy          /v1/*, /mcp, /a2a handlers, routing, middleware├── aisix-admin          Read-only resource surface + playground + OpenAPI├── aisix-provider-*     openai · anthropic · azure-openai · bedrock · vertex├── aisix-mcp            MCP gateway — server registry, tool ACL, transports├── aisix-a2a            A2A agent gateway — agent cards, JSON-RPC bridge├── aisix-ratelimit      fixed-window + token accounting + concurrency (local | redis)├── aisix-cache          memory + redis backends├── aisix-redis          shared Redis connection for cache + rate limits├── aisix-guardrails     pre/post content-policy hooks├── aisix-obs            tracing, metrics, access log, exporters└── aisix-server         the `aisix` binary — bootstrap + CLI

🗺️ Roadmap

Highlights on the roadmap; tracked live in issues:

  • Semantic (embedding-similarity) response caching
  • More observability sinks — Langsmith, Helicone, Slack alerts
  • Prompt templates managed as gateway resources
  • Llama-Guard as a guardrail provider

Shipped since this list was last written: the MCP gateway, the A2A agent gateway, OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio, PII, and OpenAI Moderation guardrails — see Features above.

🛠️ Development

Prerequisites: the Rust toolchain pinned in rust-toolchain.toml. Docker is only needed for the tests that exercise etcd, Redis, or provider emulators.

bash
cargo check --workspacecargo fmt --checkcargo clippy --workspace -- -D warningscargo test --workspace
# Coverage (matches the CI gate)cargo llvm-cov --workspace --lcov --output-path lcov.info
# Run locally against a resources.yaml (no etcd needed). Copy the Quickstart's two files# and change resources_file to the local path, e.g. resources_file: ./resources.yamlcargo run -p aisix-server --bin aisix -- --config config.local.yaml
# Check a resources file without starting a listenercargo run -p aisix-server --bin aisix -- validate --resources resources.yaml

💬 Community

If AISIX is useful to you, a ⭐ helps other engineers find it.

📄 License

Apache 2.0.

來源:README.md,提交 ad58900

工具

0
SourceWeft 尚未在沙箱中啟動過該軟體包,因此其工具未知。

版本歷史

2
  1. v1.3.0最新Sep 21, 2026
  2. v0.12.0Sep 16, 2026

評分

已滿足適用檢查項的 50%

該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通25 分

    請保持端點能夠回應 MCP 握手;SourceWeft 每 30 天探測一次。

  • 已宣告認證方式10 分

    請在註冊中心條目中宣告憑證請求標頭,或保持端點能夠回應探測,以便判定其認證方式。

  • 已列出工具15 分

    請讓伺服器在無憑證時回應 tools/list,或發布可在沙箱中無憑證啟動的軟體包。

  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分