Llm Output Quality Monitor

io.github.lazymac2xPublished by GitHub account lazymac2xv1.0.0Updated Nov 2, 2026

Cloudflare Workers MCP server: llm-output-quality-monitor

VerifiedStreamable HTTPWeb executableDeveloper ToolsAI & ML

Overview

AI-generated overview

Remote MCP endpoint that lets an assistant score and validate LLM responses for quality, drift, hallucination risk, schema conformance and consistency.

What you can do
  • Validate LLM responses for length, format and structure
  • Detect quality drift between a current and a previous response
  • Score hallucination risk 0-100 from linguistic signals such as contradictory assertions, unsourced claims and uncertainty markers
  • Validate JSON responses against a schema
  • Check consistency across multiple LLM responses
When to use it
Useful when you are building or evaluating LLM pipelines and want an assistant to run quick heuristic checks on model output: format and schema validation, regression drift between versions, and consistency across several responses.
Requirements
A remote streamable HTTP endpoint at api.lazy-mac.com; no packages, environment variables, headers or authentication are declared. Network access to that host is required.
Before you install
The hallucination scorer is explicitly pattern-based and heuristic, not a semantic detector, so scores should not be treated as proof of correctness. Responses are sent to a third-party hosted service; avoid submitting confidential model output. No authentication is declared, so do not assume the endpoint is access-controlled.

Installation

In SourceWeft

  1. Open Llm Output Quality Monitor in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.lazymac2x%2Fllm-output-quality-monitor/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "llm-output-quality-monitor": {
      "type": "http",
      "url": "https://api.lazy-mac.com/llm-output-quality-monitor/mcp"
    }
  }
}

README

llm-output-quality-monitor-api

Cloudflare Workers MCP server: llm-output-quality-monitor. REST + MCP (JSON-RPC 2.0). Live: api.lazy-mac.com/llm-output-quality-monitor

Source: README.md at commit 79735fd

Tools

5

quality_validator

Validate LLM response quality based on length, format, and structure

ParametersresponserequiredmaxLengthminLengthstrictFormat

drift_detector

Detect quality drift between current and previous LLM responses

ParametersthresholdcurrentResponserequiredpreviousResponserequired

hallucination_scorer

Pattern-based heuristic risk scoring for LLM responses (0-100). Detects linguistic signals such as contradictory assertions, unsourced claims, and uncertainty markers. Not a semantic hallucination detector.

Parameterscontextresponserequired

schema_enforcer

Validate JSON response against schema

Parametersschemarequiredresponserequired

consistency_check

Check consistency across multiple LLM responses

Parametersresponsesrequired

Version history

1
  1. v1.0.0LatestSep 16, 2026

Score

90% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

    Declare a license in the repository or the registry entry.