eval-harness
The eval-harness skill provides structured frameworks for systematically testing agent and skill performance through benchmarking, quality scoring, and regression testing. Use it after creating or modifying agents and skills, during periodic quality audits, and before promoting skills to production to ensure outputs meet accuracy, completeness, and consistency thresholds.
git clone --depth 1 https://github.com/a5c-ai/babysitter /tmp/eval-harness && cp -r /tmp/eval-harness/library/methodologies/everything-claude-code/skills/eval-harness ~/.claude/skills/eval-harnessSKILL.md
- Define test cases with known-correct outputs - Run agent against each test case - Score: accuracy, completeness, relevance - Compare against baseline performance - Track performance over time ### 2. Skill Quality Testing - Verify skill instructions produce expected outcomes - Test edge cases and boundary conditions - Measure consistency across multiple runs - Check for harmful or incorrect outputs - Validate against ground truth ### 3. Regression Suite - Collection of previously-passing test cases - Run after any agent/skill modification - Flag regressions with before/after comparison - Maintain pass rate threshold (>= 95%) ### 4. Process Verification - End-to-end process execution with known inputs - Verify each phase produces expected outputs - Check task ordering and dependency satisfaction - Measure total execution time ## Quality Scoring ### Accuracy Score (0-100) - Correctness of output vs expected - Partial credit for partially correct outputs - Penalty for hallucinated or fabricated content ### Completeness Score (0-100) - Coverage of required output elements - Missing sections flagged and scored - Bonus for useful additional context ### Consistency Score (0-100) - Run same input 3 times - Compare outputs for semantic similarity - Flag inconsistencies ### Composite Score - (accuracy * 0.4 + completeness * 0.3 + consistency * 0.3) - Threshold: 80 to pass ## When to Use - After creating new agents or skills - After modifying existing agents or skills - Periodic quality audits - Before promoting skills to production ## Agents Used - Used by process-level evaluation orchestrators - No specific agent dependency (evaluates other agents)
Coordinate Crew (persistent) and Polecat (transient) agents using Gas Town's hook-based work distribution and GUPP principle.
System and API design guidance covering component boundaries, data flow, integration patterns, and scalability considerations.
This skill should be used when the user asks to "find skills in the wild", "assimilate popular workflows", "discover SKILL.md files in repos", "research external skills", "find workflow patterns", "survey the skill landscape", "what skills exist out there", or wants to investigate public repositories for extractable processes, babysitter plugins, and reusable procedural insights. Searches GitHub for SKILL.md files, classifies repos by archetype, and maintains structured research under docs/reference-repos/.
>-
This skill should be used when the user asks to "babysit issues", "work on assigned issues", "check a5c-agent issues", "process babysitter issues", or wants to find and work on open GitHub issues assigned to a5c-agent in the babysitter repo.