/ Verzeichnis / Playground / web-eval-agent
← Alle Server ↗ GitHub
● Community refreshdotdev ⚡ Sofort

web-eval-agent

von refreshdotdev · refreshdotdev/web-eval-agent

Natural-language end-to-end tests for web apps — the agent drives a browser, captures console/network, and writes up findings.

web-eval-agent (refreshdotdev) lets you describe a user task in plain English; the MCP opens a browser, performs the flow, and reports screenshots, console logs, and network traffic. Useful for exploratory UX testing without writing Playwright scripts. Note: the original project is discontinued — the team has moved on to withrefresh.com — but the MCP remains functional under its existing license.

▶ Demo ansehen 📦 Installieren 💡 Anwendungsfälle

Warum nutzen

Hauptfunktionen

Live-Demo

In der Praxis

web-eval-agent.replay ▶ bereit
0/0

Installieren

Wählen Sie Ihren Client

~/Library/Application Support/Claude/claude_desktop_config.json  · Windows: %APPDATA%\Claude\claude_desktop_config.json
{
  "mcpServers": {
    "web-eval-agent": {
      "command": "uvx",
      "args": [
        "web-eval-agent"
      ],
      "_inferred": true
    }
  }
}

Öffne Claude Desktop → Settings → Developer → Edit Config. Nach dem Speichern neu starten.

~/.cursor/mcp.json · .cursor/mcp.json
{
  "mcpServers": {
    "web-eval-agent": {
      "command": "uvx",
      "args": [
        "web-eval-agent"
      ],
      "_inferred": true
    }
  }
}

Cursor nutzt das gleiche mcpServers-Schema wie Claude Desktop. Projektkonfiguration schlägt die globale.

VS Code → Cline → MCP Servers → Edit
{
  "mcpServers": {
    "web-eval-agent": {
      "command": "uvx",
      "args": [
        "web-eval-agent"
      ],
      "_inferred": true
    }
  }
}

Klicken Sie auf das MCP-Servers-Symbol in der Cline-Seitenleiste, dann "Edit Configuration".

~/.codeium/windsurf/mcp_config.json
{
  "mcpServers": {
    "web-eval-agent": {
      "command": "uvx",
      "args": [
        "web-eval-agent"
      ],
      "_inferred": true
    }
  }
}

Gleiche Struktur wie Claude Desktop. Windsurf neu starten zum Übernehmen.

~/.continue/config.json
{
  "mcpServers": [
    {
      "name": "web-eval-agent",
      "command": "uvx",
      "args": [
        "web-eval-agent"
      ]
    }
  ]
}

Continue nutzt ein Array von Serverobjekten statt einer Map.

~/.config/zed/settings.json
{
  "context_servers": {
    "web-eval-agent": {
      "command": {
        "path": "uvx",
        "args": [
          "web-eval-agent"
        ]
      }
    }
  }
}

In context_servers hinzufügen. Zed lädt beim Speichern neu.

claude mcp add web-eval-agent -- uvx web-eval-agent

Einzeiler. Prüfen mit claude mcp list. Entfernen mit claude mcp remove.

Anwendungsfälle

Praxisnahe Nutzung: web-eval-agent

Smoke-test a web deploy with web-eval-agent

👤 Solo devs, small teams without a Playwright suite ⏱ ~10 min beginner

Wann einsetzen: You deployed and want a quick 'did I break anything obvious' check.

Voraussetzungen
  • Free API key from operative.sh/mcp — Sign up, copy the key
  • Playwright deps — npx playwright install (the MCP will prompt if missing)
Ablauf
  1. Describe the test
    On staging.example.com, verify I can: sign up with a new email, create a project, log out. Report what fails.✓ Kopiert
    → Pass/fail with screenshots
  2. Dig into failures
    For the failed step, show the console errors and the network request that returned 500.✓ Kopiert
    → Stack-level evidence

Ergebnis: Post-deploy confidence in 2 minutes.

Fallstricke
  • Test accounts clutter your prod DB — Always run against staging; if prod, use a dedicated QA account and clean up
Kombinieren mit: sentry

Exploratory UX evaluation of a new flow

👤 Designers, PMs ⏱ ~20 min intermediate

Wann einsetzen: You want an outside perspective on a flow without scheduling user tests.

Ablauf
  1. Describe user intent, not steps
    As a first-time user, try to share a project with a colleague. Note every friction point.✓ Kopiert
    → Free-form UX critique with screenshots of each confusion
  2. Contrast with the happy path
    Now do the same flow as a power user who knows the UI. How much faster? What confused the noob but not the expert?✓ Kopiert
    → Comparative friction map

Ergebnis: Cheap UX heuristics before putting real users in front of it.

Test features behind login with persisted browser state

👤 Anyone testing authenticated flows ⏱ ~15 min intermediate

Wann einsetzen: Your feature requires login; you don't want the agent handling your password.

Ablauf
  1. Seed the session
    Call setup_browser_state opening https://app.example.com/login — I'll sign in myself.✓ Kopiert
    → Interactive browser opens; you log in; session saved
  2. Run the test using the saved state
    Test the billing settings page: load it, verify the current plan shows, try downgrading.✓ Kopiert
    → Test runs with your authenticated session

Ergebnis: Authed testing without sharing credentials with the agent.

Kombinationen

Mit anderen MCPs für 10-fache Wirkung

web-eval-agent + sentry

Run an eval, any new errors go to Sentry for post-hoc review

Run the signup eval, then check Sentry for new error events captured during that window.✓ Kopiert
web-eval-agent + playwright

Prototype with web-eval-agent, harden into Playwright for CI

Convert the working web-eval-agent test into a Playwright spec I can run in CI.✓ Kopiert

Werkzeuge

Was dieses MCP bereitstellt

WerkzeugEingabenWann aufrufenKosten
web_eval_agent url: str, task: str, headless_browser?: bool Any natural-language web test LLM calls + browser time
setup_browser_state url?: str Once per service, to persist logged-in state 0

Kosten & Limits

Was der Betrieb kostet

API-Kontingent
Free tier from operative.sh
Tokens pro Aufruf
A full eval can be 5-20k tokens (screenshots described)
Kosten in €
Free for low volume
Tipp
For repetitive tests, graduate them to Playwright; use web-eval-agent for exploration

Sicherheit

Rechte, Secrets, Reichweite

Credential-Speicherung: operative.sh API key in env; browser state saved locally
Datenabfluss: Target sites + operative.sh for eval orchestration

Fehlerbehebung

Häufige Fehler und Lösungen

Browser fails to launch

Install Playwright deps: npx playwright install-deps

Session keeps expiring

Some sites rotate cookies; re-run setup_browser_state. Or use Playwright's storageState for finer control

Agent misunderstands the task

Be specific: URLs, selectors or text to look for, expected outcomes

Alternativen

web-eval-agent vs. andere

AlternativeWann stattdessenKompromiss
Playwright MCPYou want scriptable, reproducible testsYou write the code
Browserbase MCPYou need cloud-hosted browsers for CIPaid per minute

Mehr

Ressourcen

📖 Offizielle README auf GitHub lesen

🐙 Offene Issues ansehen

🔍 Alle 400+ MCP-Server und Skills durchsuchen