freemcplab / Verzeichnis / Playground / SkillCompass

● Community Evol-ai ⚡ Sofort

SkillCompass

Name: SkillCompass (Claude Skill)
Author: Evol-ai

Evaluate Agent Skill quality — find the weakest link, fix it, and prove the fix worked with before/after metrics.

SkillCompass scores your Agent Skills on clarity, activation rate, downstream correctness, and context cost. It highlights the skill most likely to be hurting your agent's performance, suggests a fix, and re-runs the evaluation so you can show the improvement. Useful when you have a shelf of skills and don't know which are actually earning their context weight.

▶ Demo ansehen 📦 Installieren 💡 Anwendungsfälle

Warum nutzen

Hauptfunktionen

Multi-dimensional skill scoring (clarity, activation, correctness, cost)
'Weakest link' identification across a bundle
Suggests concrete SKILL.md edits
Before/after eval harness
JavaScript tooling — runs locally

Live-Demo

In der Praxis

skillcompass-skill.replay ▶ bereit

0/0

Installation

Repository des Autors · Offizielle Dokumentation zu Claude Skills

Anwendungsfälle

Praxisnahe Nutzung: SkillCompass

Find the skill pulling your agent's performance down

👤 Skill authors with 5+ skills installed ⏱ ~45 min advanced

Wann einsetzen: You feel the agent has gotten worse, not better, as you added skills.

Ablauf

Diagnose the loser

For the weakest skill, what specifically is wrong?✓ Kopiert

→ Concrete critique (vague description, conflicting with other skill, etc.)
Propose a fix

Suggest a minimal edit to SKILL.md to fix it.✓ Kopiert

→ Small, reviewable diff
Re-evaluate

Re-run the eval and show before/after.✓ Kopiert

→ Metrics improved, with evidence

Ergebnis: A measurably better skill bundle, with a reproducible eval process.

Fallstricke

Gaming the eval metric instead of helping real tasks — Include task-level downstream metrics (actual agent outcomes), not just text-level

Kombinieren mit: skill-optimizer-skill · filesystem

Review a new skill before you publish it

👤 Skill authors shipping their first bundle ⏱ ~20 min intermediate

Wann einsetzen: Before pushing to GitHub and telling the world about your skill.

Ablauf

Score the draft

Evaluate my draft skill at ./my-skill/.✓ Kopiert

→ Dimension scores
Fix obvious issues

Apply the low-hanging suggestions✓ Kopiert

→ Edits in SKILL.md

Ergebnis: A publication-ready skill rather than a rough draft.

Fallstricke

Chasing a perfect score — Ship when scores plateau — diminishing returns

Kombinationen

Mit anderen MCPs für 10-fache Wirkung

skillcompass-skill + skill-optimizer-skill

Two complementary tools: SkillCompass ranks, skill-optimizer drills into SKILL.md patterns

Use SkillCompass to pick the worst skill; use skill-optimizer to deeply analyze its SKILL.md.✓ Kopiert

skillcompass-skill + filesystem

Operate across the full ~/.claude/skills/ directory

Evaluate every skill in ~/.claude/skills/ and give me a CSV.✓ Kopiert

Werkzeuge

Was dieses MCP bereitstellt

Werkzeug	Eingaben	Wann aufrufen	Kosten
skill-scoring	skill path(s)	Periodic audits	eval compute
weakest-link-id	bundle scores	After scoring	0
fix-suggestion	weak skill + critique	Before editing	0
before-after-eval	pre/post SKILL.md	After applying fixes	eval compute

Kosten & Limits

Was der Betrieb kostet

API-Kontingent: none beyond your LLM provider (evals use LLM calls)
Tokens pro Aufruf: evals can be heavy — budget 20–100k tokens for a full bundle scan
Kosten in €: free, MIT
Tipp: Run on one skill at a time during iteration; bundle runs only for audits

Sicherheit

Rechte, Secrets, Reichweite

Credential-Speicherung: none at skill level

Datenabfluss: none beyond your LLM provider

Fehlerbehebung

Häufige Fehler und Lösungen

Node errors on install

Ensure Node 20+; npm i inside the skill directory.

Prüfen: node -v

Evals are inconsistent run-to-run

Fix the task seed and use a non-stochastic sample; record provider+model.

Alternativen

SkillCompass vs. andere

Alternative	Wann stattdessen	Kompromiss
skill-optimizer-skill	You want a single skill analyzed deeply rather than a bundle ranked	Depth over breadth
manual review	You have 1–2 skills total	Doesn't scale

Mehr

Ressourcen

📖 Offizielle README auf GitHub lesen

🐙 Offene Issues ansehen

🔍 Alle 400+ MCP-Server und Skills durchsuchen