QA Copilot MCP Server

A Model Context Protocol server that hands an AI assistant a small set of concrete QA/test-intelligence tools instead of asking it to guess: summarize a Playwright run, find flaky tests across several runs, explain a failure heuristically, scaffold a Playwright test stub, and read a GitHub repo's recent CI status. Point Claude Desktop, Claude Code, or any other MCP-compatible host at it and those become real tool calls, not free-text guesses.

It's the "give the AI real data, not vibes" piece of a QA automation portfolio: the other repos in this portfolio (healthcare-qa-automation-framework, healthcare-data-quality-framework) produce Playwright JSON reports and CI runs; this server is what turns those into something an assistant can actually query.

Why it's built this way

• The protocol wiring is a thin layer over plain functions. Every tool's real logic lives in src/playwright/, src/github/, and src/tools/ as small, pure(-ish), independently unit-tested modules. src/index.ts only registers them with the MCP SDK - so the hard-to-get-wrong part (report parsing, flaky detection, heuristics) is covered by fast unit tests, and the protocol wiring is covered by a separate end-to-end smoke test (below).
• GitHub access is pluggable, the same pattern as the LLM integrations elsewhere in this portfolio. GitHubClient is an interface; RealGitHubClient wraps fetch against the GitHub REST API, FakeGitHubClient returns canned data. That makes get_repo_ci_status fully unit-testable offline, while the real client is genuine, runnable code - an MCP host running on a developer's machine has normal internet access, even in environments (like this repo's own CI sandbox) that don't.
• Test-stub generation is deterministic, not LLM-based, on purpose. Given a named sequence of data-testid interactions, generate_playwright_test_stub always produces the same output - instant, free, reproducible. A different tool for a different job (generating test cases from a spec or user story, where an LLM actually adds value) is a separate project in this portfolio.
• A protocol-level smoke test, not just unit tests. smoke-test.mjs spawns the built server as a real child process and drives it with the official MCP Client over stdio - tools/list, then a tools/call per tool - so CI proves the actual wire protocol works, not just the functions behind it.

Tools

Tool	What it does
summarize_test_run	Parses a Playwright JSON reporter output into pass/fail/timeout/skipped/flaky counts, duration, and full failure details.
list_flaky_tests	Compares the same tests across 2+ Playwright JSON reports and flags any whose outcome wasn't consistent.
explain_failure	Heuristic, pattern-matched root-cause guess for a raw error message. Fully offline.
generate_playwright_test_stub	Deterministically scaffolds a runnable Playwright test from a named sequence of data-testid interactions.
get_repo_ci_status	Fetches a public GitHub repo's recent Actions runs and summarizes pass/fail/in-progress counts.

Architecture

src/
  index.ts                    MCP server: registers all 5 tools, connects over stdio
  playwright/reportParser.ts  Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
  github/githubClient.ts      GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
  tools/
    explainFailure.ts         Heuristic pattern-matching (ported from the triage_failure.py
                               script in healthcare-qa-automation-framework)
    generateTestStub.ts       Deterministic Playwright test-file template generator
    getRepoCiStatus.ts        Wraps GitHubClient into pass/fail/in-progress counts
test/                         Vitest unit tests for every module above
fixtures/                     Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs                End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml      Type-check, unit tests, build, protocol smoke test

Tech stack

TypeScript · Node.js · @modelcontextprotocol/sdk · Zod · Vitest · GitHub REST API · GitHub Actions

Running it locally

npm install
npm run build       # compiles to dist/
npm test            # unit tests (27 tests, fully offline)
npm run smoke        # builds, then drives the real server over stdio via the MCP client

Using it from an MCP host

Point any MCP-compatible client at the built server, e.g. in Claude Desktop's claude_desktop_config.json:

{
  "mcpServers": {
    "qa-copilot": {
      "command": "node",
      "args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
    }
  }
}

Set GITHUB_TOKEN in the environment to raise get_repo_ci_status's unauthenticated GitHub rate limit (60 requests/hour without one, 5,000/hour with one) - it works fine without a token for occasional use.

CI

Every push and pull request to main runs a single job: type-check (tsc --noEmit), the unit test suite (vitest run), a production build, and the protocol-level smoke test (node smoke-test.mjs) against the built output. No network access is assumed or required anywhere in CI - get_repo_ci_status is exercised only against FakeGitHubClient in the unit tests, and the smoke test treats a network-free get_repo_ci_status result as a pass as long as the server reports it cleanly via MCP's isError channel instead of crashing.

What this is not

This isn't a general-purpose GitHub or CI dashboard - it's a small, focused set of tools built specifically around the Playwright-report and GitHub-Actions shapes this portfolio's other repos actually produce, meant to demonstrate how an AI assistant gets wired up with real, verifiable test data instead of being asked to reason about test results from a paraphrase.