MCP Incident Responder

An AI-native incident response server built on the Model Context Protocol (MCP). Exposes diagnostic tools that any MCP-compatible LLM agent can discover and call over HTTP using standard JSON-RPC 2.0.

---

Roadmap

Phase	What	Status
Phase 1	Build MCP server + 3 tools	✅ Done
Phase 2	Test all JSON-RPC calls in Bruno	✅ Done
Phase 3	Connect LLM agent	Next
Phase 4	Add auth (Bearer token)	Enhancement
Phase 5	Connect real data (CloudWatch, etc.)	Enterprise extension

---

Phase 1 — Build the MCP Server ✅

What we built

A Python MCP server that runs on http://localhost:8002 and exposes three tools and one prompt template over a single HTTP endpoint: POST /mcp.

Project structure:

mcp-incident-responder/
├── server.py              ← MCP server entry point
├── tools/
│   ├── __init__.py        ← Registers all tools with the server
│   ├── system_status.py   ← Tool: getSystemStatus
│   ├── recent_errors.py   ← Tool: getRecentErrors
│   └── create_ticket.py   ← Tool: createTicket
├── data/
│   └── mock_data.py       ← Simulated metrics and logs
└── requirements.txt

---

The 3 Tools

Tool	Type	What it does
getSystemStatus	Read-only	Returns CPU, memory, uptime, and health status for all services
getRecentErrors	Read-only	Returns recent error logs, optionally filtered by service name
createTicket	Action	Creates an incident ticket in the (simulated) ticketing system

The mock data in data/mock_data.py simulates a realistic degraded system:

• auth-service → healthy
• payment-service → degraded (CPU ~92%, memory ~90%, DB connection pool exhausted)
• notification-service → down (crashed with OutOfMemoryError)

---

The Prompt Template

The server also exposes a prompt called incidentAnalysisTemplate that gives an LLM a structured workflow:

ASSESS → INVESTIGATE → CORRELATE → DIAGNOSE → RECOMMEND → ACT

An LLM agent can fetch this prompt from the server and follow it automatically — the server defines the workflow, not the client.

---

Running the server

pip install "mcp[cli]" fastapi uvicorn
python server.py

Server starts on http://localhost:8002. Health check available at GET /health.

---

Phase 1 Verification — The MCP Handshake

The first thing any MCP client does is send an initialize request. This is capability discovery — the client asks "who are you and what can you do?" before calling any tools.

Request (POST http://localhost:8002/mcp):

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "protocolVersion": "2025-03-26",
    "capabilities": {
      "roots": { "listChanged": true },
      "sampling": {}
    },
    "clientInfo": {
      "name": "BrunoClient",
      "title": "Incident Responder Test",
      "version": "1.0.0"
    }
  }
}

Response (200 OK):

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2025-03-26",
    "capabilities": {
      "experimental": {},
      "prompts": { "listChanged": false },
      "resources": { "subscribe": false, "listChanged": false },
      "tools": { "listChanged": false }
    },
    "serverInfo": {
      "name": "Incident Responder",
      "version": "1.27.2"
    }
  }
}

What this exchange means

The client introduces itself and declares the protocol version it speaks. The server responds with its own identity (Incident Responder, SDK v1.27.2) and confirms it supports the same protocol version. It also advertises its capability categories: tools, prompts, and resources.

No tools were called yet — this is purely the handshake. The client now knows what's available and can start making targeted requests.

Phase 1 is complete. The server is running, the tools are registered, and the MCP handshake succeeds.

---

Phase 2 — Manual Testing with Bruno ✅

Phase 2 is about proving every part of the server works by manually firing each JSON-RPC request and reading the response. You are the brain here — you decide what to call next, read the output, and reason about it. In Phase 3, an LLM takes over that role automatically.

The 6 requests, in order

#	Method	What it does	Result
1	initialize	Handshake — discover capabilities	✅ 200 OK
2	tools/list	List all registered tools	✅ 200 OK
3	tools/call → getSystemStatus	Check live service health	✅ 200 OK
4	tools/call → getRecentErrors	Retrieve filtered error logs	✅ 200 OK
5	tools/call → createTicket	Create an incident ticket	✅ 200 OK
6	prompts/list	List available prompt templates	✅ 200 OK

---

Request 2 — tools/list

After the handshake, ask the server to enumerate every tool it has registered.

Request:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}

Response (200 OK) — server returns all 3 tools with their input schemas so the client knows exactly what arguments each tool accepts.

---

Request 3 — tools/call → getSystemStatus

Call the first tool to get a live snapshot of all services.

Request:

{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "getSystemStatus",
    "arguments": {}
  }
}

Response — the mock data returns a degraded system:

• auth-service → healthy
• payment-service → degraded (CPU ~92%, memory ~90%)
• notification-service → down (port 8080 refused)
• overall_status → critical

---

Request 4 — tools/call → getRecentErrors

Drill into the logs for payment-service to understand why it is degraded.

Request:

{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "tools/call",
  "params": {
    "name": "getRecentErrors",
    "arguments": {
      "service_name": "payment-service",
      "limit": 5
    }
  }
}

Response — ordered from most recent:

1. ERROR — DB connection pool exhausted (max 100 reached)
2. CRITICAL — Transaction timeout after 30 s, DB unresponsive
3. WARN — Connection pool at 90%, approaching limit

Root cause is clear: the database is overloaded and the connection pool is full.

---

Request 5 — tools/call → createTicket

With the diagnosis confirmed, create a formal incident ticket.

Request:

{
  "jsonrpc": "2.0",
  "id": 5,
  "method": "tools/call",
  "params": {
    "name": "createTicket",
    "arguments": {
      "title": "payment-service DB connection pool exhausted",
      "severity": "critical",
      "description": "DB connection pool reached max (100). Transactions timing out after 30s. notification-service also down with OOM crash. Immediate DB scaling and connection pool tuning required."
    }
  }
}

Response:

{
  "success": true,
  "message": "Ticket INC-1001 created successfully",
  "ticket": {
    "ticket_id": "INC-1001",
    "severity": "critical",
    "status": "open",
    "assigned_to": "on-call-team"
  }
}

---

Request 6 — prompts/list

Discover the structured workflow templates the server exposes.

Request:

{
  "jsonrpc": "2.0",
  "id": 6,
  "method": "prompts/list",
  "params": {}
}

Response:

{
  "jsonrpc": "2.0",
  "id": 6,
  "result": {
    "prompts": [
      {
        "name": "incidentAnalysisTemplate",
        "description": "A structured template for analyzing IT incidents. Guides the LLM through a systematic diagnosis workflow."
      }
    ]
  }
}

A prompt in MCP is not the user's message — it is a workflow template the server publishes. An LLM agent can fetch incidentAnalysisTemplate at runtime and follow its prescribed steps:

ASSESS → INVESTIGATE → CORRELATE → DIAGNOSE → RECOMMEND → ACT

Different servers publish different workflows. The agent adapts automatically without any code change on the client side.

---

The full incident workflow — what just happened

Request 1: initialize
├── Client: "Hi server, what can you do?"
└── Server: "I have tools, prompts, and resources"

Request 2: tools/list
├── Client: "Show me your tools"
└── Server: "getSystemStatus, getRecentErrors, createTicket"

Request 3: tools/call → getSystemStatus
├── Client: "What's the current system health?"
└── Server: "payment-service DEGRADED, notification-service DOWN"

Request 4: tools/call → getRecentErrors
├── Client: "Show me payment-service errors"
└── Server: "DB connection pool exhausted, transaction timeouts"

Request 5: tools/call → createTicket
├── Client: "Create critical ticket with diagnosis"
└── Server: "INC-1001 created, assigned to on-call-team" ✅

Request 6: prompts/list
├── Client: "What workflow templates do you have?"
└── Server: "incidentAnalysisTemplate (ASSESS→INVESTIGATE→...→ACT)"

Phase 2 vs Phase 3 — the key difference

Phase 2 (done): You are the brain. You read each response, decided what to call next, and wrote the ticket description manually.

Phase 3 (next): The LLM is the brain. Given a single user message — "There's an alert, what's going on?" — it will call the same sequence of tools automatically, reason over each response, and produce the ticket without any manual input.

The LLM fills in the same arguments you typed, but generates them from reasoning over previous tool outputs. The server does not change at all between Phase 2 and Phase 3.

Phase 2 is complete. All 6 JSON-RPC methods verified, full incident lifecycle exercised end-to-end.