
prompt-protection
io.github.mughalhere由 GitHub 帳號 mughalhere 發布v4.4.0更新於 Nov 5, 2026
Scan prompts, tool definitions and model output for injection, and guard agent tool calls.
概覽
一個 JavaScript 函式庫,可掃描提示詞、工具定義與模型輸出中的注入,並透過追蹤資料來源來管控代理的工具呼叫。
- 可以做什麼
- 為文字評分:分析、驗證或剔除使用者輸入、對話紀錄、工具定義與模型輸出中的惡意片段
- 管控工具呼叫:標記工具結果為不可信來源,分類目標動作,並阻擋、標記、要求確認或放行每次呼叫
- 固定工具定義並設定呼叫次數、重複次數與深度預算,以偵測工具漂移與失控迴圈
- 以聚光標記不可信片段,並用金絲雀與相似度檢查偵測系統提示詞外洩
- 記錄雜湊鏈稽核日誌,並為每次決策發出 OpenTelemetry 事件
- 適用情境
- 適合正在建置會呼叫工具的 JavaScript 或 TypeScript 代理的開發者,希望在不可信的工具結果與網路、電子郵件、命令執行、檔案或付款等目標之間加一層預設拒絕的策略層。也適合在提示詞或模型輸出進入模型或呈現給使用者之前先行篩檢。
- 執行需求
- 以 npm 套件 prompt-protection 安裝,並在自己的代理程式碼中呼叫;它是函式庫而非獨立服務。支援 Node 20/22/24、Bun、Deno 2 以及邊緣或瀏覽器執行環境。不需要 API 金鑰、環境變數或網路存取。
安裝
在 SourceWeft 中
- 開啟 儀表板中的 prompt-protection,將其新增到工作區。
- 為需要使用其工具的對話啟用該服務。
Desktop only,透過 STDIO。 STDIO 服務會啟動本機處理程序,因此需要 SourceWeft 桌面主機。
其他 MCP 客戶端
參照 儲存庫 中的啟動說明。
README
prompt-protection
A provenance-tracked tool-call guard for JavaScript agents. Tool results are tainted, destinations the user named are trusted, and every tool call the model proposes is checked before it runs: untrusted data does not reach a network, email, exec, file or payment sink. Fail-closed, in-process, zero runtime dependencies. Text detection, spotlighting and canaries ship as components, with their benchmark numbers published whether they flatter the library or not.
[CI] [npm] [npm downloads] [License: MIT] [TypeScript] [Zero dependencies] [Prompt Protection MCP on mcprush] [XERJ Corpus Hub contributor]
Listed in the Agent Threat Rules ADOPTERS registry (Tier 2, adapter), the official MCP registry as io.github.mughalhere/prompt-protection, MCPrush, and the XERJ Corpus Hub.
What it is, and what it isn't
Prompt injection stopped being a text-classification problem the moment models started calling tools. The damage in an agent is a consequence, not a sentence: a URL lifted from an email ends up in an outbound request, an attendee address from a calendar entry becomes the recipient of send_email, a payload in a README lands in exec. Version 3 tracks where data came from and refuses to let untrusted data reach a dangerous sink. That is the capability model from Google DeepMind's CaMeL paper (arXiv 2503.18813), ported to a JavaScript tool-calling loop without the custom interpreter.
It is a policy layer inside your process. It is not an isolation boundary. It cannot see a flow that passes through the model's hidden state, a paraphrase that shares no identifiers with its source, or a source you never registered. The Limitations section lists what it misses, with the dataset rows that show it.
60-second agent quickstart
Or drive it by hand:
Taint survives three places it used to leak (4.1):
Every decision carries reasons: DecisionReason[] (stable codes, docs/CONFORMANCE.md) and depth.
Static structure (4.2): pin what the agent may call, and bound how often.
Under a lock, readOnlyHint: true makes a tool a non-sink and an unannotated tool the name heuristics cannot place resolves to unknown (treated as exfil and exec) unless you set annotationsDefault: 'heuristic' or name its sink in sinks.
Operability (4.3): pick a preset, observe before you enforce, and read the reason chain.
One call per framework:
Output side: analyzeOutput(text, { renderAllowlist: ['ourcompany.com', '*.cdn.ourcompany.com'], conversation }) blocks markdown/HTML that would render or link an unlisted host, a data: URI, or a query parameter carrying high-entropy or conversation-derived data (off unless the allowlist is set). redact(text) and analyzeOutput(text, { redact: 'all' }) replace secrets and PII (gitleaks / Presidio shapes, checksums validated in code).
Trust across boundaries (4.4, preview): a label is only worth what signs it.
Architecture
The default policies run in this order. plan-violation blocks any tool outside the plan() allow-list. injection-source-flow blocks when a source that itself scored as injection flows anywhere. untrusted-to-exfil-sink blocks untrusted data reaching network, email or messaging tools, and untrusted-to-exec does the same for exec and file writes. untrusted-to-payment asks for confirmation. injection-then-sink flags a sink call made in the same turn as an injection-scored source even when no flow was detected, because paraphrase is exactly the case the flow detectors miss. args-injection flags when the arguments themselves read as injection. Each of these is one of the patterns in Design Patterns for Securing LLM Agents (arXiv 2506.08837): Action-Selector via plan(), Plan-Then-Execute, Context-Minimisation via spotlighting.
Spotlighting (prompt-protection/spotlight, arXiv 2403.14720) marks untrusted spans by delimiting, datamarking or base64-encoding them, and gives you the system-prompt sentence that tells the model what the marker means. With wrapTools({ spotlight: 'datamark' }) the model sees marked text while the guard taints the original, and arguments are unmarked before flow detection so a copied span still matches.
Canaries (prompt-protection/canary) put a token in the system prompt and look for it in the output in exact, normalized, spaced, base64, hex, reversed and partial forms. There is also a shingle-similarity check between the output and the system prompt. Plain verbatim canaries were shown to fail against paraphrase (arXiv 2506.19109). Similarity closes part of that gap. Not all of it.
Text detection is a component, not the product: 106 input rules, 21 output rules and 9 tool-poisoning rules score text that has to be scored, and the Benchmark section says how well. An embedded 33 KB int8 n-gram classifier lives on prompt-protection/ml (preview tier, off by default, for reasons the benchmark explains). prompt-protection/lite is the rules-only entry at 20 KB gzipped.
Benchmark
npm run bench runs the shipped build against every set below and writes bench/results.json. The same run is a CI gate. Recall and false-positive rate are shown as regex / ml / hybrid; the shipped default is regex.
Some of these numbers are bad, and they are here on purpose. On NotInject the rules do well: 97.1% of short benign queries that merely contain "ignore" or "instruction" pass through. On the hard-negative set I wrote myself they false-positive on 19.4% of benign text. Questions about prompt injection, fiction, "grant admin access on Netflix" all trip them. They catch 14.6% of the attacks written to avoid canonical phrases. That is what pattern matching tops out at, and it is why provenance is the primary mechanism now. Both figures are CI gates at their current baseline; they can only go down from here.
The in-the-wild "regular" set is noisy. It includes SEO prompts that open with "Please ignore all previous instructions", so its false-positive column overstates. I kept it because it is external and unmodified.
The embedded model ships for transparency, not for use. It is trained on Apache and MIT datasets (deepset, gandalf, hackaprompt, SPML, plus about 17k mined benign rows) with a reproducible pipeline described in training/REPORT.md. In-distribution it looks great: 3-fold CV F1 0.98. Held out by dataset it does not: leave-one-dataset-out F1 0.53, in-the-wild AUROC 0.67. Adding hackaprompt in a second round lifted recall on unseen attacks from 4% to 19% on our set and lifted in-the-wild false positives from 17% to 24% with it. A bag of hashed n-grams does not transfer across jailbreak genres, so ml defaults to 'off'. If you want it anyway, analyzePrompt(text, { ml: 'escalate' }). Python and JS produce identical features and logits on 64 golden vectors under test, and the weights are 33 KB gzipped.
Latency: rule scan p99 about 0.1 ms, guard checkToolCall p99 about 2 ms over the agent-flows rows (per-row minimum of 5 passes after a warm-up pass; gate: 5.4 ms) and under 50 ms with 64 registered sources of 3 KB, classifier about 0.15 ms. Bundle: core 68 KB gzipped with the weights included, lite 23 KB, guard 80 KB (gate: 87 KB).
Datasets
datasets/ is CC-BY-4.0 and disjoint from the test fixtures. It is also on the Hugging Face Hub as promptprotection/agent-security-datasets, with a card built from the benchmark results. attacks.jsonl has 130 rows across nine categories and 14 languages. benign-hard.jsonl has 155 benign prompts carrying trigger vocabulary, in NotInject's four categories plus developer jargon and security documentation. agent-flows.jsonl has 120 tool-call scenarios with the expected guard decision and the reason; the 20 multi-step rows carry a steps array (memory write/read, fork, approve, clock) run by datasets/steps.cjs, the interpreter shared by the tests and the bench. node datasets/validate.mjs checks schema, uniqueness and disjointness from the fixtures.
Limitations
Semantic paraphrase. Tainted prose rewritten so it shares no identifiers and no six-word shingles with its source is invisible to the guard. injection-then-sink covers the same-turn case only when the source itself scores as injection. For derivations your application can see (structured-output fields, summaries), guard.derive(value, fromSourceIds) carries the label explicitly; nothing carries it for a paraphrase the model performs internally.
Recipient ambiguity. "Reply to them" leaves the recipient derived from the tool result, which has the same flow shape as attacker exfiltration. The default blocks. Call guard.trust(sender) first, or swap untrusted-to-exfil-sink for a confirm policy (af-065, af-072).
Unregistered sources, internal exfiltration, hidden state. The guard only knows what you taint(). A tool that leaks on its own side, or a flow the model carries without copying text, is out of reach.
Encodings the normaliser does not undo (rot13, chunk reordering, translation) defeat containment.
Rule over-defence and the model's generalisation gap are covered in the benchmark section. Both are measured and gated. Neither is solved.
Standards: ATR, OWASP, ATLAS
prompt-protection/atr loads Agent Threat Rules packs as customRules and emits findings in the ATR ScanResult shape. ATR is the Sigma-style open standard for agent threats, adopted by Microsoft, Cisco, MISP and SigmaHQ.
The engine follows the spec's mandatory behaviours (no short-circuit, scan_target and agent_source filtering, draft and deprecated rules skipped, the enforce lane) and tests/atr/conformance.test.ts checks each one. Conditions the engine cannot express faithfully are skipped with a reason in report, never approximated: condition: all, the named and behavioural formats, PCRE-only syntax. Native rules carry OWASP LLM Top 10 (2025) and MITRE ATLAS ids in mappings, with a CI floor on coverage. One real difference from the reference engine: we scan normalised text while it scans raw, so zero-width and case-sensitive ATR rules behave differently here. docs/ATR.md lists the deltas.
Observability
Composing with policy-as-code (Vercel AI SDK)
prompt-protection/adapters/vercel-guardrail implements the GuardrailProvider shape proposed in vercel/ai#13434: a pre-call decision, context for the approval card, and hash-chained receipts. It composes with @ai-sdk/policy-opa:
Failure semantics
Every shipped regex is fuzzed with recheck in CI (npm run test:redos). That is 148 of them, across the input, output and tool rules, the sink heuristics, identifier extraction and normalisation. At 3.1.0 the result is 148 safe, 0 allowlisted, 0 vulnerable. The first run found 28 quadratic-or-worse patterns in 3.0.0 and one more once the fuzzer had enough time. I rewrote all 29 and no benchmark number moved.
The library fails closed. If anything inside it throws (a rule, a policy, a sink resolver, a classifier adapter) the verdict is block, with a synthetic internal-error match and result.error set, and the logger receives the event with error. Set failMode: 'open' to let input through instead; the error is still reported. A logger that throws never changes a verdict.
Runtime compatibility
Verified in CI on every push (scripts/compat/):
The library makes no network calls and reads no environment: grep -rE "fetch\(|XMLHttpRequest|sendBeacon" dist/ returns nothing.
Install
Text detection (component)
When there is a string to score rather than a tool call to check: user input before it reaches the model, model output before it reaches the user, a tool definition before it is registered.
API
verifyPrompt(prompt, options?)
Throws PromptInjectionError if the prompt is detected as malicious.
prompt may be a string or a chat message array ({ role, content }[]). Message arrays are scanned on untrusted roles by default (user, tool, function) so system instructions are not mixed into the score.
stripPrompt(prompt, options?)
Returns the prompt with malicious spans removed. Safe to pass to your LLM.
analyzePrompt(prompt, options?)
Returns full analysis without throwing. Use this when you want to inspect results yourself.
createProtectionSession(options?)
Opt-in multi-turn protection. Remembers recently blocked prompts and escalates deferred follow-ups like "process the last prompt", even when the blocked text never entered chat history.
Passing a full ChatMessage[] transcript still works without a session when prior attack text remains in the array. Use a session when you only scan the latest turn, or when blocked messages are dropped from history.
React: usePromptProtection({ enableSession: true }).
Express / Next.js: pass session or getSession(req) on the middleware options.
analyzeOutput(output, options?)
Scans an LLM response for signs of compromise: system prompt leakage, credential exposure, injection relay patterns targeting downstream systems, and PII.
OutputAnalysisOptions mirrors AnalyzeOptions: threshold (default: 40), flagThreshold, customRules, disabledCategories, disabledRuleIds, logging, allowlists.
Logging
Ship protection events to any service with a zero-dep logger interface:
Events include type, score, action, severity, categories, ruleIds, and optional truncated promptPreview / contentPreview.
Allowlists (false-positive control)
verifyPromptAsync(prompt, options)
AI-assisted verification. Sync block always wins; the adapter may only escalate allow/flag → block.
Options
All functions accept an options object:
Threat Categories
Input categories (used by analyzePrompt / verifyPrompt / stripPrompt)
Output categories (used by analyzeOutput)
Custom Rules
Express Middleware
Next.js App Router
React Hook
Severity Levels
Every AnalysisResult (from analyzePrompt) and OutputAnalysisResult (from analyzeOutput) includes a severity field. Bands are fixed and independent of your custom threshold:
AI Adapters
Claude Adapter
Uses claude-haiku-4-5-20251001 for fast, cheap classification.
Requires @anthropic-ai/sdk:
OpenAI Adapter
Uses gpt-4o-mini by default. Drop-in replacement for the Claude adapter.
Requires openai:
Agentic AI: tool poisoning & MCP
Agentic systems face a distinct attack, tool poisoning, where malicious instructions hide inside a tool/function definition (its description or parameter docs). The agent reads them; the user never sees them.
scanToolDefinition(tool, options?)
Accepts both the OpenAI (parameters) and MCP / Anthropic (inputSchema) tool shapes.
MCP server
Ships an MCP server so an agent can scan its own inputs, tool definitions, and outputs as tools. Requires the optional peer @modelcontextprotocol/sdk.
Seven tools. scan_prompt, scan_tool_definition and scan_output scan text. register_source and check_tool_call put the tool-call guard behind MCP, so a client can label an untrusted tool result and then ask whether a proposed call is safe; check_tool_call also accepts inline sources for a stateless check. spotlight_text and detect_canary expose the marking and leak-detection helpers. Verdicts carry rule ids and policy names, never the matched text.
Embed it in your own server via createProtectionMcpServer() from prompt-protection/mcp. The server is listed in the MCP registry as io.github.mughalhere/prompt-protection.
Vercel AI SDK
Requires the optional peer ai (>=4). Throws PromptInjectionError on a blocked prompt.
Threshold Tuning
- High-security apps (customer-facing LLM chat): keep default
35 - Developer tools (false positives are costly): raise to
50–65 - Zero tolerance (financial, medical): lower to
20–25
Browser Usage
Works without a bundler in modern browsers:
How Detection Works
- Normalize: Unicode NFKC, strip zero-width chars, collapse whitespace
- URL-decode: handle
%20-style encoding - Base64-decode: detect and decode embedded base64 segments (≥ 20 chars)
- Homoglyph substitution:
0→o,1→i,@→a,$→s, Cyrillic look-alikes, etc. - Pattern match: 106 regexes across 8 input threat categories (+ 21 output rules)
- Score:
100 × (1 − e^(−raw/15))with 25% diminishing returns for repeated same-rule hits - Threshold: score ≥ 35 → malicious
FAQ
Does this work with LangChain / the OpenAI SDK / the Anthropic SDK?
Yes. It operates on plain strings and on { role, content }[] chat arrays, so it sits in front of any LLM client. Call verifyPrompt (or analyzePrompt) on the user text before you build the request; there is no framework coupling.
How is this different from an LLM-based classifier like Lakera or an LLM Guard model? Those reason about intent and catch novel, semantically-rephrased attacks that regex cannot. This runs in-process in under a millisecond, calls nothing, sends nothing off-box, and has zero dependencies. They are complementary: use this as a cheap deterministic first layer and escalate only the survivors to a model, the bundled Claude / OpenAI adapters do exactly that, and the local verdict always wins. See the comparison.
What is the performance cost? Sub-millisecond for typical prompt sizes: it is regex matching over normalized text, no I/O and no model. Safe to run synchronously on every request.
Can it catch every attack? No, and nothing can. It is pattern-based: it defeats obfuscation (homoglyphs, zero-width characters, base64, percent-encoding) and covers the known attack shapes well, but a genuinely novel phrasing can pass. Treat it as one layer alongside least-privilege tool access, human approval for consequential actions, and output scanning.
Does it send my prompts anywhere? No. The core is fully offline. The only network calls are the optional Claude / OpenAI adapters, which you wire in explicitly and which are off by default.
Does it run in the browser / at the edge? Yes: no Node built-ins, no bundler required. The live demo is the library running client-side. It works in Cloudflare Workers and other edge runtimes.
What about false positives?
Tunable. Use flagThreshold for a review band that logs without blocking, raise threshold for developer-facing tools, and exclude known-good phrases with allowlistPatterns / allowlistRuleIds. See Threshold Tuning.
Versioning
Three API tiers, defined in docs/API_STABILITY.md: stable (root entry, /guard, /mcp, the framework adapters and middlewares; full semver), preview (/ml, /atr, /audit, /otel, /canary, /spotlight, /adapters/vercel-guardrail, /lite; may change in a minor, always listed in the CHANGELOG), internal (/internal; no contract). ThreatCategory and the event unions are open, so new categories arrive in minors. Rules version separately as RULES_VERSION.
1.0, 2.0 and 3.0 were cut as majors for milestone reasons. From 4.0 a major means an incompatible change to the stable tier, at least six months apart, with a migration document.
Migration to 4.0
Import-path changes only; nothing renamed, no behaviour change. See docs/migrations/v4.md.
Contributing
See CONTRIBUTING.md for a guide on adding detection rules, writing tests, and submitting pull requests.
Migration from 2.x
analyzePromptverdicts are unchanged by default (ml: 'off'). Opt into the classifier with{ ml: 'escalate' }.ThreatCategorygains'data-flow';ProtectionEvent.typegainstool-call.blocked | tool-call.flagged | tool-call.allowedanddirectiongains'tool-call': add cases to exhaustive switches.AnalysisResult.ml?andOutputAnalysisResult.canary?are new optional fields;analyzeOutputacceptscanaryandsystemPrompt.- New subpaths:
prompt-protection/guard,/spotlight,/canary,/ml,/lite. Root exports gaincreateGuard,spotlight,createCanary,mlClassifier,analyzePromptWith,normalize. - MCP server is 3.0.0 with
register_source,check_tool_call,spotlight_text,detect_canary. Vercel middleware acceptsguardandonBlock. - Tool names starting
http_now classify asnetworksinks.
License
MIT: see LICENSE
來源:README.md,提交 91cfa19
工具
0版本歷史
6- v4.4.0最新Nov 5, 2026
- v4.3.0Oct 29, 2026
- v4.2.0Oct 13, 2026
- v4.1.0Oct 7, 2026
- v4.0.0Sep 21, 2026
- v3.1.1Sep 16, 2026
評分
已滿足適用檢查項的 50%
該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。
- 近 30 天內可連通25 分
請保持端點能夠回應 MCP 握手;SourceWeft 每 30 天探測一次。
- 已宣告認證方式10 分
請在註冊中心條目中宣告憑證請求標頭,或保持端點能夠回應探測,以便判定其認證方式。
- 已列出工具15 分
請讓伺服器在無憑證時回應 tools/list,或發布可在沙箱中無憑證啟動的軟體包。
- 描述必需15 分
- 圖示10 分
- 文件15 分
- 授權條款10 分