Aether

Project Aether — M2M Inference Clearinghouse (MVP)

AI agents buy and sell inference units (tokens of a named model, e.g. llama-3.1-70b-instruct) on a continuous double-auction spot market. Credentials never change hands. Buyers hold escrowed allocations, and the clearinghouse proxies every request to the seller's own stateless endpoint using a single-use, body-bound delivery JWT. Sellers are paid in nano-dollars for exactly the tokens the proxy streamed.

Going to production: see docs/PRODUCTION.md for what the live platform does, how money moves, the launch checklist, and the roadmap.

Agents can connect with any OpenAI-compatible client:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="<client_id>.<client_secret>")  # api_key from POST /v1/agentsclient.chat.completions.create(    model="llama-3.1-70b-instruct",    messages=[{"role": "user", "content": "hi"}],    max_tokens=64,    extra_body={"max_price_usd_per_mtok": "0.50"},)

Each call market-buys any missing capacity within the price cap, escrows it, streams the answer, and settles per delivered token.

Scope note. Abstracting keys behind a proxy does not by itself make it permissible to resell a proprietary API (OpenAI, Anthropic, Google…). If a seller's endpoint just forwards to their own vendor key, that is still resale of access, which those providers' terms generally prohibit. The model this MVP is built for is sellers running models they have the rights to serve (open-weight models on their own or rented GPUs), or providers who explicitly allow resale.

Architecture

            OAuth2 client_credentials                    ┌──────────────────────────────┐ Buyer ──── POST /oauth/token ──► JWT (buy_inference) ──►│  FastAPI clearinghouse       │ agent      POST /v1/orders  (bid, escrowed first) ─────►│  auth.py      RS256 JWT/JWKS │            POST /v1/inference (SSE) ◄──── stream ──────│  exchange.py  clearing+escrow│                                                         │  proxy_router.py (asyncio)   │ Seller ─── POST /v1/orders (ask) ──────────────────────►│                              │ agent  ◄── POST /v1/generate + delivery JWT ───────────│                              │   (verifies via /.well-known/jwks.json)                 └──────┬──────────────┬────────┘                                                                │              │                          Redis: Lua matching engine (ZSET book),│              │ PostgreSQL: agents, orders,                          match Stream (durable fill log),        │              │ trades/allocations, double-entry                          Pub/Sub market data, job checkpoints   ▼              ▼ ledger (source of truth)

Order → allocation → delivery → settlement

1. Escrow before matching. A bid locks price_cap × qty from available into escrow before it reaches the book.
2. Match. One Lua script matches atomically by price, then time (FIFO), with self-trade prevention, and appends the result to a Redis Stream. Fills execute at the maker's price, and price improvement is refunded at once.
3. Apply exactly once. Each stream event is applied to PostgreSQL once, guarded by applied_match_events. The request applies its own event inline; a consumer-group worker applies anything left over after a crash. PostgreSQL is authoritative, and on startup the Redis book is rebuilt from it.
4. Route. /v1/inference reserves max_tokens on the cheapest escrowed allocation (row-locked, so concurrent requests can't overdraw it) or returns 402. It then mints a 60-second delivery JWT (aud=aether-seller:<id>, jti single use, body_sha256 binding) and streams the seller's SSE response back.
5. Checkpoint / retry. Delivered tokens are batched into a Redis Stream. If the seller drops the connection, times out, returns 5xx, or ends without a done marker (spot preemption), the proxy resumes on the same allocation with resume_from and prefix. After max_attempts_per_allocation it fails over to another escrowed allocation. The buyer sees one continuous, gap-free stream.
6. Settle. Buyer escrow moves to seller available plus the house:fees account for exactly the delivered tokens. The rest of the reservation is released. This runs even if the buyer disconnects. If the proxy process dies, a sweep settles orphaned jobs from the checkpoint once their heartbeat lapses.

Proxy choice: Python asyncio. The router is I/O-bound: it relays chunks and runs a few short transactions per request. One event loop with a pooled httpx.AsyncClient multiplexes thousands of streams and shares the ledger code with no RPC hop. A Go data plane becomes worthwhile once per-chunk CPU work (tokenizer-based metering) dominates.

Data model (app/models.py)

All money is integer nano-USD (1e-9). Prices are integer nano-USD per token: $0.35 per 1M tokens = 350. That puts the tick at $0.001/1M, and every cost is an exact integer product with no rounding. Fees are computed cumulatively per allocation (floor(gross_total × bps / 10⁴)), so they never drift across many micro-settlements.

GET /v1/audit (sandbox) checks the invariants: the ledger sums to zero, every transaction balances, cached balances equal the ledger, and each agent's escrow equals its open-bid escrow plus its allocation escrow.

JWT payloads (RS256, kid = RFC 7638 thumbprint, public keys at /.well-known/jwks.json)

Access token (from POST /oauth/token, grant_type=client_credentials, HTTP Basic or form auth):

{  "iss": "http://clearinghouse:8000", "sub": "agt_7bf7…", "aud": "aether-clearinghouse",  "iat": 1790508813, "nbf": 1790508813, "exp": 1790509713, "jti": "0dbd3f97…",  "client_id": "agt_cli_f17d…", "scope": "buy_inference", "tv": 0, "typ": "access"}

Scopes: buy_inference (bid, consume, release allocations) and sell_compute (ask, register endpoint). A token can be revoked by jti (POST /oauth/revoke), or all of an agent's tokens at once via tv.

Delivery token (proxy → seller, one per seller call):

{  "iss": "…", "sub": "job_0d41…", "aud": "aether-seller:agt_2edb…", "exp": "iat + 60",  "jti": "…", "typ": "delivery", "trade_id": "trd_…", "instrument": "llama-3.1-70b-instruct",  "max_tokens": 150, "resume_from": 25, "body_sha256": "<sha256 of the exact request body>"}

Files

Running it

docker compose up -d --build --wait      # postgres, redis, clearinghouse, seller-a, seller-bdocker compose run --rm --no-deps buyer  # the buyer agent's end-to-end simulationdocker compose logs -f clearinghouse seller-aopen http://localhost:8000/docs          # OpenAPI UIdocker compose down -v                   # tear down, wiping volumes

Local development and tests (need redis-server on PATH; the tests use SQLite plus a throwaway Redis):

python -m venv .venv && . .venv/bin/activate && pip install -r requirements-dev.txtpytest -q

Install the secret-scan pre-commit hook once per clone (needs gitleaks on PATH, e.g. sudo apt-get install gitleaks):

git config core.hooksPath scripts/git-hooks

CI also runs gitleaks on every push and PR (.github/workflows/gitleaks.yml) as a second layer. Never git add -A / git add . in this repo — stage explicit paths so runtime state (.env, data/) can't ride along with a commit.

Expected output (from an actual docker compose run --rm --no-deps buyer, abridged)

== 2. The M2M boundary holds =============================================sell attempt with a buy_inference token   -> HTTP 403 insufficient_scope (Bearer realm="aether", error="insufficient_scope", scope="sell_compute")inference before escrowing any funds      -> HTTP 402 no_escrowed_allocationtampered JWT signature                    -> HTTP 401 invalid_token
== 4. Order book for llama-3.1-70b-instruct ==============================side   $/1M tokens     tokens  ordersASK            0.4       5000       1ASK           0.35        400       1
== 5. Market BID 1000 tokens, protection cap $0.50/1M (escrow before matching)order ord_6645… status=filled filled=1000/1000 cancelled_remainder=0  FILL    400 tokens @ $0.35/1M from seller agt_9b4c3e5d7982… -> allocation trd_1790507830419_0_0… escrow=$0.000140000  FILL    600 tokens @ $0.4/1M from seller agt_b242131598ab… -> allocation trd_1790507830419_0_1… escrow=$0.000240000wallet: available=$0.999620000 escrow=$0.000380000 (price improvement vs cap already refunded)
== 6. 3 concurrent inference streams through the proxy router ============[job1] routed job=job_83cb… -> seller=agt_9b4c3e5d7982… trade=trd_1790507830419_0_0… @ $0.35/1M reserved=150[job3] routed job=job_1d8c… -> seller=agt_b242131598ab… trade=trd_1790507830419_0_1… @ $0.4/1M reserved=150[job1] interrupted: stream ended without completion marker (instance preempted)[job1] RESUME on seller=agt_9b4c3e5d7982… from checkpoint token #25[job1] interrupted: stream ended without completion marker (instance preempted)[job1] FAILOVER to seller=agt_b242131598ab… from checkpoint token #50[job1] done: 150 tokens, cost $0.000057500, attempts=3, failovers=1, finish=length[job1]   segment seller=agt_9b4c3e5d7982… trade=trd_1790507830419_0_0… tokens=50[job1]   segment seller=agt_b242131598ab… trade=trd_1790507830419_0_1… tokens=100
== 8. Micro-transaction ledger (buyer, newest first) =====================  escrow_release  available    $0.000105000  allocation released by buyer  settlement      escrow      $-0.000040000  job_798b…: 100 tokens  settlement      escrow      $-0.000017500  job_798b…: 50 tokens  escrow_release  available    $0.000060000  price improvementfinal wallet: available=$0.999825000 escrow=$0.000000000
== 9. Clearinghouse audit (double-entry invariants) ======================{ "ledger_sum_nanos": 0, "unbalanced_transactions": [], "cached_balance_mismatches": [],  "escrow_backing_mismatches": [], "house_fees_usd": "$0.000001750", "ok": true }
SIMULATION PASSED

Clearinghouse log for the same run:

aether.exchange: FILL llama-3.1-70b-instruct 400 tokens @ $0.35/1M buyer=agt_7bf7… seller=agt_2edb… (taker=bid)aether.proxy: ROUTE job=job_0d41… buyer=agt_7bf7… -> seller=agt_2edb… trade=trd_…_0_0 reserved=150aether.proxy: job=job_0d41… seller=agt_2edb… interrupted at token 25 (attempt 1): stream ended without completion marker (instance preempted)aether.proxy: job=job_0d41… seller=agt_2edb… interrupted at token 50 (attempt 2): stream ended without completion marker (instance preempted)aether.proxy: SETTLED job=job_0d41… tokens=150 cost=$0.000057500 segments=[('trd_…_0_0', 50), ('trd_…_0_1', 100)] status=completed

API summary

Known gaps before real money

The full list, with priorities, is in docs/PRODUCTION.md. The most important:

• Metering trust. The proxy counts SSE token events, so a dishonest seller could split output into more "tokens". Billing should use the instrument's tokenizer.
• Prompt tokens are not billed yet, and nothing yet verifies which model a seller actually runs.
• Operations. The schema is created with create_all (Alembic migrations needed before the first schema change), and the Lua scripts assume a single Redis primary.
• Regulation. Real money flows through Stripe Connect as the platform, but holding balances and paying sellers still needs a legal review in your jurisdiction.