ABS Data — Australian Bureau of Statistics data for your AI assistant
Ask the question you actually have. Add one web address to Claude, ChatGPT or
Copilot, say what you care about, and your assistant finds the right ABS data,
fetches it live, and explains it — with its caveats. Free, anonymous and
read-only.
Start here: https://abs-data-front-door.aicolab.workers.dev — setup guides
for each AI tool, things to ask, and a feedback form.
An independent, unofficial connector — not an ABS product, and not endorsed by
the Australian Bureau of Statistics. Data: ABS, CC BY 4.0.
How it was built
Before offering anything, this project measured what the ABS Data API actually
serves — every dataflow, by retrieval — so the connector only ever offers tables
and options that have data, and a selection that doesn't exist comes back with
the ones that do.
[The ABS Data API cartography: unmasking the 4.21% reality — a comic-style map of the findings and the front door]
- 621,536,714 series exist across 1,227 dataflows. ABS's metadata implies
14,761,557,818 — a density of 4.21%, overstated 23.8×.
- The per-dimension metadata is exact: every advertised option exists, and
nothing real is unadvertised. The entire overstatement is combinatorial.
- 81.0% of series sit in flows whose sparsity is principled and verified in
the keys — almost entirely hierarchical geography encoded as separate
dimensions (
STATE is a function of REGION in 664 flows). The remaining
19% is not reducible to a single-dimension dependency.
detail=serieskeysonly — the endpoint that should answer "what exists" — is
broken (HTTP 200, malformed JSON). Eight of fifteen documented behaviours do
not conform. See reports/.
DESIGN.md is the full record: verified API findings, every decision and why,
and the corrections made along the way.
Layout
Running it
Deploying (pnpm run deploy in packages/worker builds the site first — not
pnpm deploy, which is pnpm's own command) needs, once:
the feedback migration on the remote D1 (wrangler d1 execute abs-catalogue --remote --file=../schema/migrations/0003_public_micromacro.sql) and two secrets
(wrangler secret put FEEDBACK_ADMIN_TOKEN, wrangler secret put FEEDBACK_SALT).
Rate limits, per client address: 600 requests/min overall; 120/min for data
pulls (each is a live ABS call); 20/min for Code Mode execute; 5/min for
feedback (and 20 per day, in D1). Over the limit,
HTTP doors answer 429 with Retry-After; an MCP tool call gets a tool error.
Using the SDK from TypeScript:
The probe is resumable: rerunning skips flows already recorded as complete. For
the largest census flows run with NODE_OPTIONS=--max-old-space-size=20480.
Gotcha: wrangler keys the local D1 state by database_id. Changing the id in
wrangler.toml (e.g. after d1 create) gives wrangler dev a fresh, empty
local database — re-run the local import.
Principles
- Observed-only. Declared metadata steers the crawler and feeds the defect
report; it never enters what a caller sees. Availability is empirical: observed
options from the catalogue, exact combination checks live against ABS's
availableconstraint (which agrees with the probe on every dimension of every
flow).
- One contract, derived doors. Every crawl regenerates one contract; the MCP
verbs, the HTTP routes, the OpenAPI document and the docs all read or render
it. Nothing hardcodes the surface, so nothing drifts.
- Only calls that return data. A selection is verified before it is
forwarded; an invalid one comes back with the observed valid options — as an
MRTR
input_required on MCP, a typed 422 on HTTP.