LLM Audit

app.llmaudit由 llmaudit.app 发布官方发布者v1.1.0更新于 Nov 2, 2026

Measure whether AI assistants actually recommend a brand, from measured answers.

已验证Streamable HTTP可网页运行商业与电商AI 与机器学习

概览

AI 生成的概览

测量 AI 助手在买家寻求推荐时是否真的推荐某个品牌,并报告实测回答而非模型自述。

可以做什么
  • 为品牌域名启动可见度测量并获得 runId
  • 轮询运行状态,在服务商回答后获取结论、赢得的买家问题数量以及被提及的竞品
  • 查看实测地图,即实际提出的买家问题及每个回答的胜出品牌
  • 查阅说明测量方式的 methodology 资源
适用场景
适合想了解品牌在 AI 助手推荐中的出现情况,或想知道哪些竞品被提及的场景。注意 README 说明免费测量服务已于 2026-10-29 关闭:现在发起新检查只会返回关闭状态,仅能读取该日期之前开始的测量。
运行要求
可使用远程 streamable HTTP 端点,或对仅支持本地命令的客户端通过 npx 运行 npm 包。未声明需要认证。自行部署服务器需设置 API 基础地址、服务器密钥、Supabase URL 与服务角色密钥以及每月运行上限等环境变量。
安装前请注意
README 称每次免费测量会消耗服务商的 API 密钥,约 0.05 至 0.15 美元,且该服务现已关闭,改为需要账号的付费扫描。自行部署需要服务器密钥和 Supabase 服务角色密钥等敏感凭据。每个域名每 30 天仅限一次测量。

安装

在 SourceWeft 中

  1. 打开 控制台中的 LLM Audit,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable,通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体,或 SourceWeft 对话中。智能体会读取该服务器的安装指南,先向你展示发布者、端点和登录方式,经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/app.llmaudit%2Fllmaudit/install.md,并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前,先告诉我它的发布者、端点和登录方式,等我确认。不要把 API 密钥写进命令或我会提交的文件;问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "llm-audit": {
      "type": "http",
      "url": "https://mcp.llmaudit.app/mcp"
    }
  }
}

README

askedthrice-mcp

MCP server for askedthrice.com. Lets anyone's assistant measure whether a brand shows up when buyers ask an AI for a recommendation.

Closed since 2026-10-29. Asked Thrice no longer runs free measurements; the product is a paid scan that needs an account. start_visibility_check returns status: "closed" with the link to askedthrice.com and does not call the API. get_visibility_check still reads measurements started before that date. The sections below describe how it worked while it was open.

Hosted at https://mcp.askedthrice.com/mcp (streamable HTTP). Listed in the official MCP registry as com.askedthrice/askedthrice. The stdio bridge for clients that only launch local commands is the npm package askedthrice-mcp, in npm/.

Full spec: specs/features/geo-13-mcp-server.md in the private llm-rank-tracker repo.

Install

json
{"mcpServers":{"askedthrice":{"url":"https://mcp.askedthrice.com/mcp"}}}

or, for command-only clients:

json
{"mcpServers":{"askedthrice":{"command":"npx","args":["-y","askedthrice-mcp"]}}}

What it exposes

  • start_visibility_check starts the measurement and returns a runId.
  • get_visibility_check collects it. Returns running until the providers answer.
  • askedthrice://methodology explains how it is measured.

Two tools rather than one because a measurement takes up to two minutes: the audit is 15 to 60 s and the measured map another 15 to 60 s.

What it returns, and what it does not

It returns the measured map: the buyer questions that were asked live and who won each answer. Never the providers' self-report, which is what each model believes about the brand. The two numbers drift apart a lot in practice, and the self-report is always the optimistic one; that is why the tool contract does not expose it in any field.

It never returns a 0 to 100 score: a verdict in three bands and counts.

Money

Every free measurement spends askedthrice's API keys, on the order of USD 0.05 to 0.15. Three guards, all covered by tests:

  1. One free measurement per domain every 30 days, checked BEFORE the API is called.
  2. MCP_MONTHLY_RUN_LIMIT (default 200) as the channel ceiling, separate from the web funnel budget.
  3. Rate limiting in prod counts by client identity, not by this server's IP, which rotates on serverless.

Env

VariablePurpose
LLMAUDIT_API_BASE_URLAPI base. Default https://askedthrice.com
MCP_SERVER_KEYKey shared with llm-rank-tracker. Without it prod buckets by IP
SUPABASE_URL / SUPABASE_SERVICE_ROLE_KEYmcp_runs table. Without them the store is in memory and does NOT work on serverless
MCP_MONTHLY_RUN_LIMITMonthly ceiling of runs for the channel

Channel usage

npm run stats reads mcp_runs and reports how much of the monthly ceiling is left, who called, which domains were measured and which one currently holds the 30-day quota.

sh
node --env-file <file.env> scripts/stats.mjsnode --env-file <file.env> scripts/stats.mjs --days 30 --history 180

Needs SUPABASE_URL (or NEXT_PUBLIC_SUPABASE_URL) and SUPABASE_SERVICE_ROLE_KEY. --days is the daily detail, --history how far back it looks; the current month is always included in full, because that number has to match the guard's.

The first thing it prints, when there are any, is the orphaned runs: reserved rows that never got their audit_id. They return "Unknown runId" to the client forever while holding their domain's free quota and counting against the monthly ceiling. Delete them by run_id.

Vercel Web Analytics cannot measure this: it works by injecting a script into HTML, and here there is no HTML and no browser to run it, every request is a JSON-RPC call from an MCP client. It would read zero forever.

Docker (directory checks)

Dockerfile builds the HTTP server and serves it on 0.0.0.0:3001, which is what Glama and similar directories run to verify the server starts and answers introspection. Port and host are read by xmcp.config.ts at build time, so they go as ENV before npm run build, not at docker run. Without SUPABASE_* the run store is in memory: fine for a check, not for serving traffic.

Development

sh
npm run dev     # xmcp devnpm test        # vitest, no networknpm run build   # xmcp build

Tests run against real fixtures in fixtures/ (already-paid runs from prospecting). None of them touch the network or call a provider.

Publishing a new registry version

server.json is published with mcp-publisher using DNS auth on askedthrice.com. The key lives outside the repo as a PEM; the CLI wants the raw ed25519 seed in hex:

sh
HEX=$(openssl pkey -in <key.pem> -noout -text | sed -n '/^priv:/,/^pub:/p' | grep -v '^priv:\|^pub:' | tr -d ' :\n')mcp-publisher login dns --domain askedthrice.com --private-key "$HEX"mcp-publisher publish server.json

Bump version in server.json on every publish; the registry rejects a repeat. When the npm package changes version, update packages[0].version too.

来源:README.md,提交 0c5e67c

工具

2

get_visibility_check

Get visibility check

Collect the result of a measurement started with start_visibility_check. Returns status running while the providers are still answering, so poll every 20 to 30 seconds. When ready it reports a verdict, how many of the buyer questions the brand won, which competitors were named instead, and which providers actually answered.

输入参数runId必填

start_visibility_check

Start visibility check

Measure whether a brand shows up when buyers ask AI assistants for a recommendation. Asks real buyer questions to OpenAI, Anthropic and Gemini and reports what they answered, not what they claim they would answer. Free, no signup, one measurement per domain every 30 days. Returns a runId: call get_visibility_check with it after about a minute. If the domain was already measured this month the response says quota_reached and includes when it ran.

输入参数brand必填website必填category必填languagelocationcompetitors

版本历史

1
  1. v1.1.0最新Sep 16, 2026

评分

已满足适用检查项的 100%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分