Nimbus Operations Copilot

An AI agent that resolves customer-operations requests end to end — it gathers facts across the help center, customer database, billing, and ticketing, proposes the policy-correct action, and (after a human approves) takes it. Every step is traced.

Project 3 (capstone) of a 12-week Forward Deployed Engineer portfolio. Headline skills: multi-step agents + MCP (Model Context Protocol) with production guardrails — human-in-the-loop, observability, and evals.

Read the problem framing first: CASE_STUDY.md.

In plain English

Imagine a customer-support rep at a software company. To answer one question — "was this customer charged twice, and can we refund it?" — they normally have to dig through four different systems: the help articles, the customer list, the billing records, and the support-ticket tool. It's slow and easy to get wrong.

This project is an AI assistant that does that digging for them. You ask a question in plain English, and it:

1. Looks things up on its own — it searches the help docs, finds the customer, and checks their invoices, one step at a time, just like a person would.
2. Explains what it found, quoting the real records and the company's policy (it never makes facts up).
3. Asks permission before changing anything. If it wants to issue a refund or open a ticket, it stops and waits — a human clicks Approve or Deny first. It literally cannot change data on its own.
4. Keeps a record of everything it did, so you can always check its work.

Think of it as a careful new assistant: great at gathering information and suggesting the right action, but it always checks with you before doing anything that can't be undone. That "always ask before acting" safety is the most important idea in the project — and there's a test suite proving the assistant never acted without approval.

(The rest of this README is the technical detail behind that.)

Why this is the capstone

Projects 1 and 2 answered questions (from docs, then from data). This one takes action — a real agent that plans across systems, exposed through an MCP server (the industry-standard way to connect an agent to a client's tools), with the safety and observability a client would actually require.

How it works

The agent plans across tools to gather facts, then pauses for human approval before any write — reads run freely, writes are gated:

flowchart TD
    A["👤 Ops rep asks in plain English"] --> B["🖥️ React console"]
    B -->|"stream of steps (SSE)"| C["🤖 Agent · Claude"]

    subgraph MCP["🔌 MCP server — Nimbus systems"]
        direction TB
        READ["read tools<br/>docs · customers · invoices · tickets · usage"]
        WRITE["write tools<br/>issue_refund · create_ticket"]
    end

    C -->|"plan and call read tools"| READ
    READ --> DB[("🗄️ Postgres + help docs")]
    READ -->|"facts"| C

    C --> G{"wants to write?"}
    G -->|"🔒 needs approval"| H["👤 Rep approves or denies"]
    H -->|"approve"| WRITE
    H -->|"deny"| C
    WRITE --> DB

    C -->|"answer + steps"| B
    C -.->|"every LLM and tool call"| L["🔭 Langfuse trace"]

    classDef agent fill:#052e2b,stroke:#10b981,color:#d1fae5;
    classDef gate fill:#3a2a06,stroke:#f59e0b,color:#fde68a;
    class C agent
    class G gate
    class H gate

Stack

• Agent: Python + Claude API (multi-step tool-use loop)
• Integration layer: FastMCP — an MCP server wrapping Nimbus's "systems"
• Data: PostgreSQL (nimbus db) — customers, invoices, tickets, usage, help docs (RAG)
• Safety: human-in-the-loop approval for any write (refund, ticket)
• Observability: Langfuse tracing of every LLM + tool call
• API/UI: FastAPI (SSE streaming) + React ops console

Setup

# 1. Ops database
createdb nimbus
psql nimbus -f db/schema.sql
psql nimbus -f db/seed.sql

# 2. Python env
python3.11 -m venv .venv && .venv/bin/pip install -r requirements.txt

# 3. Configure
cp .env.example .env   # add your ANTHROPIC_API_KEY

Evals

10 realistic ops tasks (billing, account, docs, writes, and a safety trap) scored on two axes — full report in evals/REPORT.md:

• Correctness 10/10 — right tools called, right action proposed, facts grounded in tool results.
• Safety: 2 writes proposed · 0 executed without approval — the suite runs with every write denied, verifying the agent only ever gates a write, never runs one autonomously.

.venv/bin/python -m evals.evaluate   # writes evals/REPORT.md

Agent runs are non-deterministic (correctness varies slightly run to run); the safety property holds every run by construction — a write can only execute after human approval.

Deploy

The Dockerfile builds one container that serves the React console and the agent API from FastAPI (the MCP server runs in-process). Target: GCP Cloud Run + Cloud SQL — step-by-step commands in DEPLOY.md.

Try the tools in Claude Desktop (optional)

The MCP server runs standalone, so you can plug it into Claude Desktop and watch Claude call the Nimbus tools directly — the "aha" of MCP.

1. Open ~/Library/Application Support/Claude/claude_desktop_config.json (create it if missing) and paste the contents of claude_desktop_config.example.json.
2. Restart Claude Desktop. You'll see nimbus-ops tools appear (the 🔌 icon).
3. Ask: "Was Acme Corp double-charged in July? If so, what does the refund policy say?" — Claude will call get_customer, list_invoices, and search_docs on its own.

Milestones

• [x] 1. Client brief + skeleton + seeded ops database
• [x] 2. MCP tool server — 6 read tools (docs/DB) + 2 write tools (refund/ticket), read/write-annotated
• [x] 3. Agent loop — Claude plans across the MCP read tools multi-step; streams steps (SSE)
• [x] 4. Human-in-the-loop — agent pauses on writes; POST /api/approve gates each refund/ticket
• [x] 5. Observability — Langfuse traces every LLM + tool call (nested, with token cost)
• [x] 6. Ops console (React) — watch the agent's steps stream in, approve/deny writes inline, link to the trace
• [x] 7. Evals — 10 ops tasks scored for correctness (10/10) and safety (0 writes executed without approval)
• [x] 8. Ship — Dockerized (one container), GCP deploy guide (DEPLOY.md), full case study