Inferrail

[Inferrail logo]

Inferrail

Spending limits, receipts and paid execution for AI agents.
Give each AI job a dollar budget that's enforced before the provider is called, and a cost receipt that never stores what the model said.

[CI] [PyPI version] [License: Apache-2.0]

Quickstart · For agents · How it works · Docs · tryinferrail.com

[Recording of a real Inferrail run. The gateway starts with its local dashboard. An agent script gives one AI job, contract-review-42, a $0.04 spending limit and makes six model calls: four are answered and two are blocked with HTTP 402 before reaching the model. The dashboard lists each call with its cost, then shows the job spent $0.0325 of its $0.0400 budget, with 2 requests blocked before reaching the provider.]

A real run of the gateway and dashboard with a local stand-in model: one job with a $0.04 limit, four calls answered, two refused with HTTP 402 before reaching the model. How it was recorded

What works today

Developer preview: interfaces may change before 1.0.

Also experimental: AP invoice-exception recovery, a hosted trial gateway, and Work Economics / Economic Authority (example; Base Sepolia testnet only).

Quickstart

Requires Python 3.11+.

Try it offline. No API key, no network calls, no provider charges:

pip install inferrailinferrail demoinferrail report --by customer --receipts ./inferrail-demo-receipts.jsonl

The demo sends scripted requests through the real engine to a fake provider with made-up prices labelled DEMO, and prints what they cost by customer.

Give one run a budget, inside your Python app. No config file, no second process:

import inferrailfrom openai import OpenAI
base_url = inferrail.start()   # the gateway, on a background thread in this processclient = OpenAI(base_url=base_url, api_key="unused")   # your key stays in OPENAI_API_KEYclient.chat.completions.create(    model="gpt-4o-mini",   # any model your account can use; `inferrail models` lists priced ones    max_tokens=200,    messages=[{"role": "user", "content": "Summarize this contract."}],    extra_headers={        "X-Inferrail-Attribute-Work-Id": "contract-review-42",   # the run        "X-Inferrail-Budget-Usd": "0.50",                        # its dollar ceiling    },)

Every call with the same work id shares that budget, including parallel calls; one that would push the run past it gets HTTP 402 before it reaches OpenAI. inferrail work contract-review-42 shows what the run cost.

More: run it as a shared gateway with the local dashboard, framework snippets (LangChain, LangGraph, OpenAI Agents SDK, CrewAI, Haystack, LlamaIndex, Microsoft Agent Framework), and other clients, including the Anthropic SDK.

For agents

Start at tryinferrail.com/llms.txt: what's live, with links to each service's guide and the OpenAPI contract.

Web Read sells one public web page as clean markdown, with its word count, a SHA-256 of the content and a receipt. $0.002 per delivered page, paid over x402 (USDC on Base) or MPP (Tempo); no account or API key. The payment is checked first and settled only after the page is delivered, so refusals (robots.txt, non-public addresses, non-text, over 2 MB, target-site errors) are never charged. Retrying the same url and request_id from the same wallet within 10 minutes returns the page already paid for.

curl -i -X POST https://inferrail-cost-gateway.onrender.com/read/v1/page \  -H 'content-type: application/json' -d '{"url": "https://example.com/"}'# HTTP 402 with the price; an x402 or MPP client pays and repeats the request

Guides: Web Read · Hosted · service source.

MCP. Ask your agent what a job cost, from your local receipts:

claude mcp add inferrail \  -e INFERRAIL_RECEIPTS_PATH=/absolute/path/to/inferrail-receipts.jsonl \  -- uvx --with "mcp>=2.0" inferrail mcp

Two read-only tools, get_spend (known cost and tokens by provider, model, route or any tag such as work_id) and get_health. They don't run inference, write files or pay for anything. Other clients: docs/integrations.md.

How it works

The gateway routes each request by model to a configured provider, checks any budget in scope, forwards the call, and writes one receipt per request, including requests a budget refused. A cost is recorded only when the provider reports usage and a price is on file; otherwise it's unknown, never a guessed $0.

Attribution tags are stored as sent, so keep secrets and message content out of them. inferrail verify-payload-free prints the live receipt schema, and canary tests check that message bodies never reach receipts; neither is a security audit. The optional usage beacon sends nothing unless an endpoint is configured (details). Architecture: docs/ARCHITECTURE.md.

Current support

• Endpoints: POST /v1/chat/completions (OpenAI-compatible) and POST /v1/messages (Anthropic-compatible), with streaming and tool calls, plus GET /v1/models.
• Providers: OpenAI, Anthropic, and endpoints compatible with either. Built-in prices cover OpenAI and Anthropic models; others need a price in your config, or their cost stays unknown.
• Not supported: the OpenAI Responses API, embeddings, image generation, audio and Realtime, batch, and native Gemini or Bedrock APIs. Fields the gateway can't account for are rejected with a clear error, never silently dropped.
• Scope: only calls through a running gateway are counted.

Exact contract and non-goals: docs/PRODUCT.md. Running agent jobs that spend real money? Open a production setup request.

Documentation

• Give one AI agent run a dollar budget
• Integrations: SDKs, frameworks, attribution, MCP, voice
• Self-hosting: install, configuration, storage, budgets, dashboard
• Product scope and architecture
• Machine-readable: llms.txt, openapi.json (the gateway), config.schema.json

Contributing, security, license

Questions and bugs: GitHub issues. Security or privacy vulnerabilities: report privately, as described in SECURITY.md. Contributing: CONTRIBUTING.md (pytest needs no API key or network). License: Apache-2.0.