
Evalgate
io.github.ipezygjPublished by GitHub account ipezygjv0.4.1Updated Oct 24, 2026
Statistical checks an agent runs before trusting an AI eval number (is #1 real, judge bias, more).
The publisher's own server: EvalGate
Overview
A local MCP server that runs statistical checks on AI evaluation results before an assistant treats a reported score as trustworthy.
- What you can do
- Runs statistical checks on AI evaluation numbers, such as whether a top-ranked result is real and whether a judge is biased, before an assistant relies on the score. It is a local, desktop-only process with no tools listed in the manifest, so the exact operations are not documented here. It is intended to add scrutiny to eval comparisons rather than to run or host evaluations itself.
- When to use it
- Use it when an assistant is asked to interpret or compare AI benchmark or eval scores and you want statistical sanity checks on whether a reported ranking or improvement is meaningful. It suits analysis of eval results, not running the evaluations themselves.
- Requirements
- Runs as a local process on the user's machine, installed from the PyPI package eval-integrity over stdio. Desktop only; no web execution. No authentication, environment variables or headers are declared.
Installation
In SourceWeft
- Open Evalgate in the dashboard and add it to a workspace.
- Enable the server for the chats that should use its tools.
Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.
Other MCP clients
Follow the launch instructions in the repository.
README
This repository has no README.
Tools
0Version history
1- v0.4.1LatestSep 16, 2026
Score
25% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
- Reached in the last 30 days25 pts
Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.
- Auth declared10 pts
Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.
- Tools listed15 pts
Answer tools/list without a credential, or ship a package that starts in a sandbox without one.
- DescriptionRequired15 pts
- Logo10 pts
- Documentation15 pts
Declare a GitHub repository with a README, or serve resource_documentation in the OAuth protected-resource metadata.
- License10 pts
Declare a license in the repository or the registry entry.
Other Evalgate integrations
EvalGate
com.evalgateOfficial publisher
EvalGate is a remote MCP server for inspecting AI evaluation projects, runs, baselines and connector health in an authenticated organization.
EvalGate Docs
com.evalgateOfficial publisher
Read-only remote MCP server that lets an assistant list, search and read EvalGate's published documentation pages.

EvalGate
Published by GitHub account evalgate
Remote EvalGate MCP server that lets an assistant inspect AI evaluations, runs, baselines and connector health for an organization.