AISIX AI Gateway

io.github.api7由 GitHub 账号 api7 发布v1.3.0更新于 Oct 24, 2026

Self-hosted AI gateway that governs MCP servers, LLM traffic, and A2A agents behind one endpoint

概览

AI 生成的概览

自托管的 AI 网关,用统一的 OpenAI 兼容 API 接入各家大模型,并把已注册的 MCP 服务器、A2A 代理和 REST API 作为工具对外提供。

可以做什么
  • 以 OpenAI 兼容接口和 Anthropic Messages 接口代理大模型流量,并转换到 OpenAI、Anthropic、Bedrock、Vertex 或 Azure OpenAI 上游
  • 按轮询、一致性哈希、故障转移、最低成本、最低延迟、最少负载、集成模型和语义路由等策略在模型间路由与容错
  • 在一个端点后聚合已注册的上游 MCP 服务器,由网关保管凭据,按服务器划分工具命名空间并按调用方控制访问,还可把 REST API 的 OpenAPI 描述暴露为 MCP 工具
  • 通过 JSON-RPC 聚合 A2A 代理,并把代理卡片中的地址改写为网关地址
  • 校验调用方 API 密钥或 OIDC/JWT 令牌,执行速率与并发限制、内容护栏和精确匹配响应缓存
  • 通过 Prometheus 指标、访问日志、用量事件以及 OTLP、Datadog、对象存储导出器观测流量
适用场景
适合多个团队或应用需要统一的、受管控的多模型入口,或希望把多个 MCP 服务器和 A2A 代理收敛到一个自运维的带认证网关之后。也适合偏好声明式配置而非托管控制面的平台团队。
运行要求
以本地容器镜像(ghcr.io/api7/aisix)或静态二进制形式运行在你自己的基础设施中。需要配置文件和 resources.yaml,用于声明供应商密钥、模型、调用方密钥、MCP 服务器和代理;供应商凭据通过 OPENAI_API_KEY 等环境变量引用提供,调用方密钥通过 CALLER_API_KEY 等变量提供。多副本部署可选 etcd,共享限流与缓存可选 Redis。
安装前请注意
网关持有上游供应商凭据和调用方 API 密钥,需妥善保护资源文件与环境变量。9090 端口上的指标/状态监听器无认证,应仅绑定回环或私有网络。它会代理真实 AI 流量并可能在上游产生费用,预算与支出上限仅在商业版中提供。配置变更需改写 resources.yaml 并发送 SIGHUP 生效;etcd 中的明文凭据不会加密存储。

安装

在 SourceWeft 中

  1. 打开 控制台中的 AISIX AI Gateway,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Desktop only,通过 STDIO。 STDIO 服务会启动本地进程,因此需要 SourceWeft 桌面宿主。

其他 MCP 客户端

参照 仓库 中的启动说明。

README

AISIX OSS

The open-source, Rust-native AI gateway for LLMs and AI agents

One OpenAI-compatible API in front of every model. Route, govern, secure, cache, and observe all your LLM and AI-agent traffic from a single control point — shipped as one static binary with low per-request overhead. Run it in your infrastructure for free, forever.

Built by the original creators of Apache APISIX.

[License: Apache 2.0] [Built with Rust] [Docs] [Discord] [Website]

Start free · Documentation · Quickstart · AISIX Enterprise and AISIX Cloud · Roadmap


[AISIX OSS architecture — one OpenAI- or Anthropic-compatible API in front of OpenAI, Anthropic, Gemini/Vertex, Bedrock, Azure OpenAI, and DeepSeek, with API key auth, rate and token limits, guardrails, caching, routing and failover, and observability in between]

AISIX OSS is the open-source product built around the Rust-native AISIX gateway. The gateway puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible endpoint. It gives platform teams one place to route, govern, secure, and observe LLM traffic, with first-class SSE streaming and low gateway overhead.

It runs as a single static binary — low cold-start, lock-free config reads, and hot configuration reloads with no restarts: declare resources in one resources.yaml and reload on SIGHUP, or point the gateway at etcd for a multi-replica cluster. Run the gateway in your infrastructure, or connect it to AISIX Enterprise or AISIX Cloud for centralized management with team governance, budgets, audit, and a dashboard.

AISIX OSS (this repo) is the open-source product. It runs without a control plane using declarative configuration or etcd. The same gateway binary serves as the data plane in the commercial products: customers host the control plane with AISIX Enterprise, while API7 hosts it with AISIX Cloud. In all three products, the gateway runs in your environment and calls providers directly; live AI traffic does not pass through either control plane. The proxy API is identical throughout. Talk to us about AISIX Enterprise or AISIX Cloud →

⚡ Quickstart

One container. No control plane, no database, no configuration store — the gateway reads every dynamic resource from one declarative resources.yaml.

yaml
# config.yamlresources_file: /etc/aisix/resources.yamlproxy:  addr: "0.0.0.0:3000"admin:  enabled: false          # a declarative gateway needs no admin listenerobservability:  metrics:    prometheus:      enabled: true      addr: "0.0.0.0:9090"
yaml
# resources.yaml_format_version: "1"
provider_keys:  - display_name: openai-main    provider: openai    api_key: ${OPENAI_API_KEY}        # interpolated from the environment
models:  - display_name: my-model    provider: openai    model_name: gpt-4o-mini    provider_key: openai-main
api_keys:  - display_name: local-dev    key_env: CALLER_API_KEY           # hashed at load; the plaintext is never stored    allowed_models: ["my-model"]
bash
export OPENAI_API_KEY="YOUR_PROVIDER_KEY"export CALLER_API_KEY="YOUR_CALLER_KEY"
docker run -d --name aisix \  -v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \  -v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \  -e OPENAI_API_KEY -e CALLER_API_KEY \  -p 3000:3000 -p 127.0.0.1:9090:9090 \  ghcr.io/api7/aisix:latest        # proxy → :3000, metrics + status → :9090#                                  ^ the metrics/status listener is unauthenticated;#                                    keep it on loopback or a private network

Then call the gateway exactly like OpenAI:

bash
curl http://localhost:3000/v1/chat/completions \  -H "Authorization: Bearer $CALLER_API_KEY" \  -H "Content-Type: application/json" \  -d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'

Edit resources.yaml and send SIGHUP (docker kill -s HUP aisix) to apply changes with no restart — an invalid file is rejected whole and the last good configuration keeps serving. Check a file before booting with aisix validate --resources resources.yaml.

Full walkthrough: the Gateway Quickstart · every field: the resources file reference. For a multi-replica cluster, point the gateway at etcd instead — resources_file and etcd are mutually exclusive.

✨ Why AISIX

  • One API, every model. Speak the OpenAI or Anthropic wire format in; the gateway translates to whichever provider each model points at. Point an OpenAI or Claude SDK at one base_url and switch models without changing code.
  • A real gateway, in Rust. Single static binary, low cold-start, lock-free config reads on the hot path, native streaming.
  • Open source, free forever. Apache-2.0 licensed and built to run in your infrastructure. Choose AISIX Enterprise or AISIX Cloud when you want centralized management through a control plane and dashboard.
  • Production controls built in. Routing & failover, rate limits, guardrails, caching, and observability ship in the box. (Budgets and spend caps are AISIX Enterprise and AISIX Cloud features — the gateway enforces the control plane's decisions.)

🧩 Features — available today

Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.

  • OpenAI-compatible proxy (:3000) — chat/completions, completions, responses, embeddings, rerank, images/{generations,edits}, audio/{speech,transcriptions,translations}, videos (submit → poll → fetch), files, batches, fine_tuning/jobs, realtime, GET /v1/models, plus a root-level /passthrough/:provider/* escape hatch. Native SSE streaming, tool/function calling, JSON mode, vision/multimodal input, and reasoning-content support.
  • Anthropic Messages API — POST /v1/messages as a first-class route, working against any configured upstream: requests and responses (including streaming) are translated both ways when a model points at a non-Anthropic provider.
  • Routing & failover — virtual/routing models with six strategies: round_robin (smooth weighted round-robin), consistent_hash (session affinity keyed by header / cookie / API key / client IP), failover, plus metric-based least_cost, least_latency, and least_busy. Per-target priority tiers (active/backup pools), retry budgets, cooldowns, tag-conditional targets, and per-attempt timeouts.
  • Ensemble models — fan one request out to a panel of models concurrently, then have a judge model synthesize a single answer, with a minimum-successful-responses threshold.
  • Semantic routing — one virtual model that dispatches by the meaning of each request: it embeds the prompt, scores it against per-route example utterances, and routes to the best match (or a default). See the semantic routing docs.
  • Rate limiting & concurrency — RPS/RPM/RPH/RPD + TPM/TPD + concurrency caps, AND-combined across caller keys, models, and policy scopes (api_key / model / team / member / team_member). Counters are per-process by default, or shared across replicas with the Redis backend.
  • Guardrails — content-policy enforcement on input and output, in-process or through a provider: keyword/regex, built-in PII detection and redaction, Presidio, Lakera, OpenAI Moderation, AWS Bedrock Guardrails, Azure AI Content Safety (Prompt Shield + text moderation), and two Alibaba Cloud services. A block returns 422 content_filter; monitor mode records what would have happened without blocking.
  • Caching — exact-match response cache with per-policy TTL and model/key scope matchers; memory and Redis backends; cost-saved telemetry on every hit. Separately, automatic prompt caching can be enabled per direct Anthropic model to inject cache breakpoints, so callers get provider-side prompt discounts without changing their requests.
  • MCP gateway — front registered upstream MCP servers at /mcp with gateway-held credentials, per-server tool namespaces, and per-caller access. It serves every Streamable HTTP revision from 2025-03-26 through stateless 2026-07-28 without downstream sessions. Upstreams use initialize by default or server/discover with protocol_version: "2026-07-28". CI runs the official MCP suite's applicable tools-only protocol scenarios. Also exposes a REST API as MCP tools from its OpenAPI description.
  • A2A agent gateway — front A2A (Agent-to-Agent) agents at /a2a/:agent, serving each agent's card with URLs rewritten to the gateway, over JSON-RPC 2.0.
  • Inbound authentication — caller API keys (SHA-256 hashed, model allowlists, expiry, rotation), or OIDC/JWT bearer tokens validated against registered providers (Entra ID, Okta, Google Workspace, or any OIDC issuer) with JWKS caching.
  • Observability — Prometheus /metrics, structured per-request access logs, usage events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver), plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob) telemetry.
  • Declarative configuration — one resources.yaml carries all ten resource collections (provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies, observability exporters, rate-limit policies, OIDC providers), validated against the same JSON Schemas the gateway uses at runtime. aisix validate checks a file offline; SIGHUP reloads it atomically.
  • Operational endpoints — /livez and /readyz on the proxy listener; /status/config, /status/ready, /status/models, and Prometheus /metrics on a dedicated metrics listener (:9090). The admin listener (:3001) additionally serves a read-only resource surface, OpenAPI 3 with a Scalar UI, and a playground. Resources are managed declaratively — through the resources_file (reloaded on SIGHUP) or direct etcd writes — not through the admin listener; its former write endpoints were removed.

🔌 Supported providers

AISIX dispatches through five native adapter families — distinct wire-protocol bridges, not one generic relabel. Whatever the upstream protocol, the client-facing API stays OpenAI-shaped.

Adapter familyReachesWire shape · auth
openaiOpenAI + any OpenAI-compatible vendor — DeepSeek, Groq, Mistral, Together, Fireworks, Perplexity, vLLM, Ollama, or self-hosted OpenAI-compatible endpointsOpenAI chat completions · Bearer
anthropicAnthropic ClaudeAnthropic Messages · x-api-key
bedrockAWS Bedrock — Anthropic, Meta Llama, Mistral, Cohere, Amazon Titan/Nova, AI21Bedrock Converse + /invoke · SigV4
vertexGoogle Vertex AI (Gemini)Vertex :generateContent · OAuth2
azure-openaiAzure OpenAIAzure deployments · api-key / Entra ID

Plus specialized handling for vendor quirks (e.g. DeepSeek reasoning content) and dedicated rerank / embeddings vendors (Cohere, Jina). Details in adapter protocol families.

☁️ Choose an AISIX Product

Same gateway binary, same proxy API — in every form the gateway runs in your environment. AISIX Enterprise adds a customer-hosted commercial control plane. AISIX Cloud adds an API7-hosted commercial control plane.

The AISIX Enterprise and AISIX Cloud dashboard — overview metrics, multi-provider models, guardrails, budgets, and observability exporters across all your gateways.

▶ Try the live dashboard demo — aisix-demo.api7.ai

AISIX OSS (this repo)AISIX EnterpriseAISIX Cloud
LicenseApache-2.0CommercialCommercial
Control planeNoneCustomer-hostedAPI7-hosted
ConfigurationDeclarative resources.yaml, or etcd for a clusterDashboard + AISIX Cloud Admin APIDashboard + AISIX Cloud Admin API
TenancySingle instance / namespaceOrganization-scoped environments, teams, and membersOrganization-scoped environments, teams, and members
Provider keys${VAR} references in resources.yaml; plaintext credentials in etcdEncrypted at rest; operator-triggered in-place rotationEncrypted at rest; operator-triggered in-place rotation
Budgets— (rate and token limits only)Per key, provider, environment, organization, and teamPer key, provider, environment, organization, and team
RBAC and auditLocal admin key; no organization layerOrganization roles, invitations, and audit historyOrganization roles, invitations, and audit history
OperationsYou operate the gatewayYou operate the control plane and gatewaysAPI7 operates the control plane; you operate gateways

→ Want centralized governance, budgets, and a dashboard? Talk to API7 about AISIX Enterprise or AISIX Cloud, or book a demo.

🏗️ Architecture

A single Cargo workspace; the aisix-server crate builds one binary named aisix that wires the crates together.

text
crates/├── aisix-core           Config, snapshot, resource model, resources.yaml source, errors├── aisix-etcd           Config provider + watch supervisor├── aisix-gateway        Hub & bridge, SSE parser, provider trait├── aisix-proxy          /v1/*, /mcp, /a2a handlers, routing, middleware├── aisix-admin          Read-only resource surface + playground + OpenAPI├── aisix-provider-*     openai · anthropic · azure-openai · bedrock · vertex├── aisix-mcp            MCP gateway — server registry, tool ACL, transports├── aisix-a2a            A2A agent gateway — agent cards, JSON-RPC bridge├── aisix-ratelimit      fixed-window + token accounting + concurrency (local | redis)├── aisix-cache          memory + redis backends├── aisix-redis          shared Redis connection for cache + rate limits├── aisix-guardrails     pre/post content-policy hooks├── aisix-obs            tracing, metrics, access log, exporters└── aisix-server         the `aisix` binary — bootstrap + CLI

🗺️ Roadmap

Highlights on the roadmap; tracked live in issues:

  • Semantic (embedding-similarity) response caching
  • More observability sinks — Langsmith, Helicone, Slack alerts
  • Prompt templates managed as gateway resources
  • Llama-Guard as a guardrail provider

Shipped since this list was last written: the MCP gateway, the A2A agent gateway, OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio, PII, and OpenAI Moderation guardrails — see Features above.

🛠️ Development

Prerequisites: the Rust toolchain pinned in rust-toolchain.toml. Docker is only needed for the tests that exercise etcd, Redis, or provider emulators.

bash
cargo check --workspacecargo fmt --checkcargo clippy --workspace -- -D warningscargo test --workspace
# Coverage (matches the CI gate)cargo llvm-cov --workspace --lcov --output-path lcov.info
# Run locally against a resources.yaml (no etcd needed). Copy the Quickstart's two files# and change resources_file to the local path, e.g. resources_file: ./resources.yamlcargo run -p aisix-server --bin aisix -- --config config.local.yaml
# Check a resources file without starting a listenercargo run -p aisix-server --bin aisix -- validate --resources resources.yaml

💬 Community

If AISIX is useful to you, a ⭐ helps other engineers find it.

📄 License

Apache 2.0.

来源:README.md,提交 ad58900

工具

0
SourceWeft 尚未在沙箱中启动过该软件包,因此其工具未知。

版本历史

2
  1. v1.3.0最新Sep 21, 2026
  2. v0.12.0Sep 16, 2026

评分

已满足适用检查项的 50%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通25 分

    请保持端点能够响应 MCP 握手;SourceWeft 每 30 天探测一次。

  • 已声明认证方式10 分

    请在注册中心条目中声明凭据请求头,或保持端点能够响应探测,以便判定其认证方式。

  • 已列出工具15 分

    请让服务器在无凭据时响应 tools/list,或发布可在沙箱中无凭据启动的软件包。

  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分