SkillCompass MCP — ユースケース, インストール & ライブデモ

なぜ使うのか

主な機能

Multi-dimensional skill scoring (clarity, activation, correctness, cost)
'Weakest link' identification across a bundle
Suggests concrete SKILL.md edits
Before/after eval harness
JavaScript tooling — runs locally

ライブデモ

実際の動作

skillcompass-skill.replay ▶ 準備完了

0/0

インストール

クライアントを選択

~/Library/Application Support/Claude/claude_desktop_config.json · Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "skillcompass-skill": {
      "command": "git",
      "args": [
        "clone",
        "https://github.com/Evol-ai/SkillCompass",
        "~/.claude/skills/SkillCompass"
      ],
      "_inferred": true
    }
  }
}

Claude Desktop → Settings → Developer → Edit Config を開く。保存後、アプリを再起動。

~/.cursor/mcp.json · .cursor/mcp.json

{
  "mcpServers": {
    "skillcompass-skill": {
      "command": "git",
      "args": [
        "clone",
        "https://github.com/Evol-ai/SkillCompass",
        "~/.claude/skills/SkillCompass"
      ],
      "_inferred": true
    }
  }
}

Cursor は Claude Desktop と同じ mcpServers スキーマを使用。プロジェクト設定はグローバルより優先。

VS Code → Cline → MCP Servers → Edit

{
  "mcpServers": {
    "skillcompass-skill": {
      "command": "git",
      "args": [
        "clone",
        "https://github.com/Evol-ai/SkillCompass",
        "~/.claude/skills/SkillCompass"
      ],
      "_inferred": true
    }
  }
}

Cline サイドバーの MCP Servers アイコンをクリックし、"Edit Configuration" を選択。

~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "skillcompass-skill": {
      "command": "git",
      "args": [
        "clone",
        "https://github.com/Evol-ai/SkillCompass",
        "~/.claude/skills/SkillCompass"
      ],
      "_inferred": true
    }
  }
}

Claude Desktop と同じ形式。Windsurf を再起動して反映。

~/.continue/config.json

{
  "mcpServers": [
    {
      "name": "skillcompass-skill",
      "command": "git",
      "args": [
        "clone",
        "https://github.com/Evol-ai/SkillCompass",
        "~/.claude/skills/SkillCompass"
      ]
    }
  ]
}

Continue はマップではなくサーバーオブジェクトの配列を使用。

~/.config/zed/settings.json

{
  "context_servers": {
    "skillcompass-skill": {
      "command": {
        "path": "git",
        "args": [
          "clone",
          "https://github.com/Evol-ai/SkillCompass",
          "~/.claude/skills/SkillCompass"
        ]
      }
    }
  }
}

context_servers に追加。保存時に Zed がホットリロード。

claude mcp add skillcompass-skill -- git clone https://github.com/Evol-ai/SkillCompass ~/.claude/skills/SkillCompass

ワンライナー。claude mcp list で確認、claude mcp remove で削除。

ユースケース

実用的な使い方： SkillCompass

Find the skill pulling your agent's performance down

👤 Skill authors with 5+ skills installed ⏱ ~45 min advanced

使うタイミング： You feel the agent has gotten worse, not better, as you added skills.

前提条件

Node 20+ — nvm install 20
Skill cloned and installed — git clone https://github.com/Evol-ai/SkillCompass ~/.claude/skills/SkillCompass; npm i

フロー

Run the evaluator

Score all skills in ~/.claude/skills/ — show me the weakest link.✓ コピーしました

→ Ranked skill list with per-dimension scores
Diagnose the loser

For the weakest skill, what specifically is wrong?✓ コピーしました

→ Concrete critique (vague description, conflicting with other skill, etc.)
Propose a fix

Suggest a minimal edit to SKILL.md to fix it.✓ コピーしました

→ Small, reviewable diff
Re-evaluate

Re-run the eval and show before/after.✓ コピーしました

→ Metrics improved, with evidence

結果： A measurably better skill bundle, with a reproducible eval process.

注意点

Gaming the eval metric instead of helping real tasks — Include task-level downstream metrics (actual agent outcomes), not just text-level

組み合わせ： skill-optimizer-skill · filesystem

Review a new skill before you publish it

👤 Skill authors shipping their first bundle ⏱ ~20 min intermediate

使うタイミング： Before pushing to GitHub and telling the world about your skill.

フロー

Score the draft

Evaluate my draft skill at ./my-skill/.✓ コピーしました

→ Dimension scores
Fix obvious issues

Apply the low-hanging suggestions✓ コピーしました

→ Edits in SKILL.md

結果： A publication-ready skill rather than a rough draft.

注意点

Chasing a perfect score — Ship when scores plateau — diminishing returns

組み合わせ

他のMCPと組み合わせて10倍の力を

skillcompass-skill + skill-optimizer-skill

Two complementary tools: SkillCompass ranks, skill-optimizer drills into SKILL.md patterns

Use SkillCompass to pick the worst skill; use skill-optimizer to deeply analyze its SKILL.md.✓ コピーしました

skillcompass-skill + filesystem

Operate across the full ~/.claude/skills/ directory

Evaluate every skill in ~/.claude/skills/ and give me a CSV.✓ コピーしました

ツール

このMCPが提供する機能

ツール	入力	呼び出すタイミング	コスト
skill-scoring	skill path(s)	Periodic audits	eval compute
weakest-link-id	bundle scores	After scoring	0
fix-suggestion	weak skill + critique	Before editing	0
before-after-eval	pre/post SKILL.md	After applying fixes	eval compute

コストと制限

運用コスト

APIクォータ: none beyond your LLM provider (evals use LLM calls)
呼び出しあたりのトークン: evals can be heavy — budget 20–100k tokens for a full bundle scan
金額: free, MIT
ヒント: Run on one skill at a time during iteration; bundle runs only for audits

セキュリティ

権限、シークレット、影響範囲

認証情報の保管： none at skill level

データ送信先： none beyond your LLM provider

トラブルシューティング

よくあるエラーと対処法

Node errors on install

Ensure Node 20+; npm i inside the skill directory.

確認： node -v

Evals are inconsistent run-to-run

Fix the task seed and use a non-stochastic sample; record provider+model.

代替案

SkillCompass 他との比較

代替案	代わりに使う場面	トレードオフ
skill-optimizer-skill	You want a single skill analyzed deeply rather than a bundle ranked	Depth over breadth
manual review	You have 1–2 skills total	Doesn't scale

その他

リソース

📖 GitHub の公式 README を読む

🐙 オープンな issue を見る

🔍 400以上のMCPサーバーとSkillsを見る