skillcheck

io.github.Noem836Published by GitHub account Noem836v0.1.2Updated Oct 29, 2026

Check an agent skill before you install it: which of your skills will it clash with?

VerifiedStreamable HTTPWeb executableDeveloper ToolsAI & ML

Overview

AI-generated overview

Read-only remote MCP server that statically checks agent skills in a public GitHub repo and warns which installed skills a new skill would clash with.

What you can do
  • Check a public GitHub repo of SKILL.md agent skills for frontmatter and YAML errors, name/folder mismatches, weak or slash-command-only descriptions, nested SKILL.md files and duplicate names.
  • Preflight a skill before installing it: pass the repo plus the skills the user already has (name and description) to get name clashes that would shadow an installed skill, likely rivals for the same requests, and how much the skill listing grows.
  • Flag pairs of skills in one repo whose descriptions overlap enough to compete.
When to use it
Use it before installing a skill or skill pack from GitHub, or before publishing one, when you want a quick static sanity check and a warning about clashes with skills already installed. It does not measure real trigger behaviour; that needs the project's local CLI.
Requirements
Remote streamable HTTP endpoint; no install, account, API key or environment variables declared. The checked repository must be public on GitHub. The assistant supplies the user's existing skill names and descriptions as input.
Before you install
Read-only static checks; the README says real trigger measurement and any upload of results happen only through the separate local CLI, which uses the user's own CLI login. Skill names and descriptions of installed skills are sent to the hosted service as input, so avoid passing anything sensitive in them.

Installation

In SourceWeft

  1. Open skillcheck in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.Noem836%2Fskillcheck/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "skillcheck": {
      "type": "http",
      "url": "https://skillcheck-alpha.vercel.app/mcp"
    }
  }
}

README

skillcheck

Find out which of your skills is stealing the request.

Your skill passed eval. Does it fire in your setup, next to everything else you installed?

Official skill evals load one skill in a clean environment. In real use it sits next to dozens of other skills and plugins, and one of them quietly takes the request. In the author's setup, prompts that fired 21/21 in an isolated claude plugin eval missed 5 times in the real harness.

skillcheck runs your prompts through your real claude and codex CLIs with everything you have installed, and reports which skill fired, which one stole the request, how flaky each prompt is, and whether your description fix helped.

New: check a skill before you install it. preflight tells you which of your installed skills a new one will take requests from, by running your existing skills' prompts with and without it in a throwaway folder. Your real skill folders are not touched. Word overlap only suggests suspects (a narrow skill can take requests with little shared wording), so name the skills you care about with --skills:

bash
python3 skillcheck.py preflight path/to/new-skill            # instant: name clashes, likely rivals, listing sizepython3 skillcheck.py preflight path/to/new-skill --measure  # real runs: your skills' fire rate without → with itpython3 skillcheck.py preflight path/to/new-skill --measure --skills my-a,my-b   # pick which of yours to measure

Example report: two skills went from 0/4 and 1/4 to 4/4 after rewriting their descriptions.

Install

As a Claude Code plugin (the skillcheck skill walks you through scan → run → fix → re-run):

/plugin install skillcheck --marketplace Noem836/skillcheck

Or as one file (Python 3 standard library; PyYAML is used for frontmatter checks if installed). Pin a release and check its hash:

bash
curl -fsSLO https://raw.githubusercontent.com/Noem836/skillcheck/v0.1.2/skillcheck.pyshasum -a 256 skillcheck.py   # compare with the release notes

Use

bash
python3 skillcheck.py scan                          # static checks, no model callspython3 skillcheck.py gen --skills my-skill         # draft should-fire / near-miss prompts → suite.jsonpython3 skillcheck.py run --agent claude --tag before --runs 3python3 skillcheck.py report --tag before           # fire rate, false fires, who took it# edit the description, thenpython3 skillcheck.py run --agent claude --tag after --runs 3python3 skillcheck.py compare --before before --after afterpython3 skillcheck.py html --tag before,after       # one-file HTML reportpython3 skillcheck.py upload --tag before,after     # optional share link

For AI agents (remote MCP, no install)

A hosted, read-only MCP server runs the static checks on any public GitHub repo, which is useful before installing or publishing a skill pack:

bash
claude mcp add --transport http skillcheck https://skillcheck-alpha.vercel.app/mcp

Tools:

  • preflight_skill(github_url, installed): call it before installing a skill. Pass the skills the user already has (name and description, which an agent can read from its own skill listing). It returns the name clashes that would shadow an installed skill, which installed skills the new one is most likely to take requests from, and how much the skill listing grows.
  • check_skill_repo(github_url) returns the frontmatter/YAML errors, name mismatches, weak or slash-only descriptions, nested SKILL.md files, duplicate names, and pairs of skills whose descriptions overlap enough to compete. Real trigger measurement still needs the CLI, because it runs in your setup.

What it checks

  • Discovery: Claude Code (~/.claude/skills, .claude/skills, enabled plugins, skillOverrides) and Codex (~/.agents/skills, ~/.codex/skills, .agents/skills)
  • Static: missing/broken frontmatter, name vs folder mismatch, too short/long descriptions, nested SKILL.md, slash-only wording, disable-model-invocation, duplicate names
  • Real triggers: each prompt run N times; the first skill loaded is recorded and the run is stopped
  • Competition: which other skill grabbed a should-fire prompt, and where near-miss prompts went

Safety

Prompt suites and third-party skills may contain prompt injection, so runs are locked down: Claude runs with --tools Skill only; Codex runs with --ignore-user-config, no web search, no apps/plugins, read-only sandbox. python3 check_isolation.py verifies this with a canary file.

Runs use your own CLI login and subscription. Nothing is sent anywhere unless you run upload, which sends skill names and hit/miss rows only — never prompt texts.

Report server

server.py is the share-link server (stdlib, sqlite for local runs, Vercel Blob when configured). check_web.py [url] attacks a running instance (XSS, oversized/deep JSON, slow clients, id guessing, rate-limit spoofing). Deploy on Vercel; the Dockerfile remains for self-hosting.

FAQ

Why isn't my Claude Code skill triggering? Usually another installed skill wins the request, or the description reads as slash-command only ("use as /name"). Run scan, then run + report and look at the taken by column. See the guide.

My skill passes claude plugin eval but doesn't fire in my normal setup. Why? The official eval loads the skill in a clean environment. skillcheck measures it next to everything you actually have installed.

Which of my skills is stealing requests from another? report lists, per skill, which other skill loaded on its should-fire prompts.

Does it work with Codex CLI? Yes: run --agent codex. Skill activation is read from the SKILL.md read in codex exec --json events.

How do I test SKILL.md descriptions before publishing to a marketplace? gen drafts should-fire and near-miss prompts, run measures, you edit, then compare shows before/after. The remote MCP gives a quick static check of the whole repo.

Does it send my prompts or code anywhere? No. Runs use your own CLI login locally. upload is opt-in and sends skill names and hit/miss rows only.

Limits

  • Claude Code and Codex only. Gemini CLI and Cursor: activation can't be read reliably yet — PRs welcome.
  • A run takes about a minute per prompt; 10 skills × 6 prompts × 3 runs is a few hours of subscription usage.

MIT license.

Source: README.md at commit aea4304

Tools

2

check_skill_repo

Static check of every SKILL.md (Claude Code / Codex agent skills) in a public GitHub repo: frontmatter and YAML errors, name/folder mismatch, missing or weak descriptions, slash-command-only wording that blocks automatic triggering, disable-model-invocation, nested SKILL.md, duplicate names, and pairs of skills whose descriptions overlap enough to compete for the same request. Use before installing or publishing a skill pack. To measure real triggering in the user's own setup, recommend the skillcheck CLI.

Parametersgithub_urlrequired

preflight_skill

Call BEFORE installing a skill or skill pack from GitHub. Pass the repo and the skills the user already has (name + description, e.g. from your own skill listing). Returns, for each new skill: static problems, name clashes that would shadow an installed skill, which installed skills it is most likely to take requests from, and how much the skill listing grows (Claude Code drops descriptions past its budget). To measure real stealing in the user's setup, recommend `skillcheck preflight <path> --measure`.

Parametersinstalledrequiredgithub_urlrequired

Version history

1
  1. v0.1.2LatestOct 29, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts