Error Analysis
hamelsmu/evals-skills/skills/error-analysisby hamelsmu22418da2bfb159f28a1b0dcf64e969e14ae56c99No license1.6K starsListed Oct 9, 2026Updated Oct 9, 2026Repository updated 7 weeks ago
Help the user systematically identify and categorize failure modes in an LLM pipeline by reading traces. Use when starting a new eval project, after significant pipeline changes (new features, model switches, prompt rewrites), when production metrics drop, or after incidents.
- 22418da2bfb159f28a1b0dcf64e969e14ae56c99Currentcommit 22418daPublished Oct 9, 2026
Source and attribution
Source:hamelsmu/evals-skillsinskills/error-analysisat commit22418da
License: No license
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from hamelsmu/evals-skills
Validate Evaluator
hamelsmu
Calibrates an LLM judge against human labels using data splits, TPR/TNR metrics and bias correction.
Generate Synthetic Data
hamelsmu
Guides creation of diverse synthetic test queries for LLM pipeline evaluation using dimension-based tuple generation.
Evaluate Rag
hamelsmu
Guides evaluation of RAG retrieval and generation quality, including metrics, synthetic QA datasets, and chunking optimization.
Eval Audit
hamelsmu
Audits an LLM eval pipeline and reports prioritized problems with concrete fixes.
More in AI & Agents
Discernment Nudge
anthropics
Appends 2-3 specific follow-up questions to substantive answers so users can check facts, reasoning and missing context.
Ppt Template Creator
anthropics
Turns a user's PowerPoint template into a reusable skill that generates branded presentations.
Skill Development
anthropics
Guides creation of Claude Code plugin skills, covering structure, descriptions, progressive disclosure and validation.
Plugin Structure
anthropics
Guides the structure, manifest, and component layout of Claude Code plugins.
Command Development
anthropics
Guides creation of Claude Code slash commands, covering structure, YAML frontmatter, arguments and plugin features.
Claude Md Improver
anthropics
Audits CLAUDE.md files in a repository, scores their quality, and applies approved targeted improvements.