DevPulse PM Agent — MCP Challenge

This repository contains the implemented solution for the DevPulse PM Agent challenge.

All 4 MCP tool functions are fully implemented inside the tools/ directory, along with validation scripts and cross-validation reports.

---

Repo structure

├── server.py               # MCP server entry point
├── agent_config.json       # run contract — tells the agent how to launch your server
├── requirements.txt        # pip dependencies (pinned)
├── env_vars.json           # env var declarations
├── .gitignore              # excludes .venv, __pycache__, .env, etc.
├── validate.py             # local validation script
├── cross_validate.py       # cross-validation against oracle
├── oracle_comparison.py    # oracle comparison utility
├── data/                   # sample data for local development
│   ├── product_backlog.json
│   ├── customer_feedback.json
│   ├── sprint_history.json
│   ├── sample_roster.json
│   └── sample_dependencies.json
└── tools/
    ├── __init__.py
    ├── prioritize_backlog.py   # implemented
    ├── analyze_feedback.py     # implemented
    ├── assess_capacity.py      # implemented
    └── map_dependencies.py     # implemented

Note: team_roster.json and dependency_map.json are not in the repo. Your server fetches them at startup from the shared MCP data server (see below).

---

The 4 tools you must implement

prioritize_backlog

Rank backlog items using a scoring method (rice, value_effort, or customer_signal). Accepts optional filters (squad, status, tags) and a dependency check flag.

analyze_feedback

Extract and rank themes from customer feedback. Supports filtering by time range, customer tier, and source. Supports grouping by theme, customer, or source.

assess_capacity

Calculate available sprint capacity per engineer. Inputs include sprint ID, squad filter, carry-over flag, and skill-fit check flag.

Note on data sources for assess_capacity: This tool uses both sprint_history.json (for sprint-level metadata such as velocity and planned points) and team_roster.json (fetched from the MCP server, for per-engineer capacity, PTO, and skill data). Both sources are passed directly into assess_capacity_impl via the sprints and roster arguments.

map_dependencies

Trace dependency chains for backlog items up to a configurable depth. Handle circular dependencies and flag external blockers.

---

Data sources

Your tools receive data from two sources:

1. Local files — injected by the eval agent via PM_AGENT_DATA

Three files are provided at eval time by setting the PM_AGENT_DATA env var to a directory:

File	Used by
product_backlog.json	prioritize_backlog, map_dependencies
customer_feedback.json	analyze_feedback
sprint_history.json	assess_capacity

server.py already reads these with DATA_DIR = Path(os.getenv("PM_AGENT_DATA", "./data")). You do not need to change this.

2. Remote MCP server — team roster and dependency map (can also be called Nimbus Oracle)

team_roster.json and dependency_map.json are served from a shared MCP data server. server.py fetches them at startup using the MCP_DATA_URL env var:

MCP_DATA_URL=https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp

This is already wired in server.py and pre-set in env_vars.json. You do not need to change anything — roster and deps are passed directly into your *_impl functions just like the local data files.

Network access: The MCP data server is hosted on the Lilly internal network. You must be on the Lilly corporate network or connected via VPN to reach it during local development. If the server is unreachable, server.py will log a warning and fall back to empty roster and deps lists — your tools must handle empty lists gracefully without crashing.

For local development with Claude Desktop, add the mcpServers block to claude_desktop_config.json.

Mac path: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows path: %APPDATA%\Claude\claude_desktop_config.json (Paste this path into File Explorer's address bar to open the folder directly.)

Open the file and paste below "mcpServers" block directly above the coworkUserFilesPath line, replacing YOUR_LILLY_EMAIL@lilly.com and 'L-User ID' accordingly. Please make sure to take care of alignment and syntax properly. There will be other content as well apart from 'coworkUserFilesPath'. Keep everything else same and do not change anything (We have tried below two versions during testing and one of them worked (for different machines). In case if both does not work, you will need to troubleshoot to fix it for your system since system settings may differ from person to person. Also, you need NOT change the default path for your 'coworkUserFilesPath'):

{
  "mcpServers": {
    "pm-data-agent": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp",
        "--header", "X-User-Email: YOUR_LILLY_EMAIL@lilly.com"
      ]
    }
  },
  "coworkUserFilesPath": "/Users/YOUR-L-USER/Claude",

OR

{
    "coworkUserFilesPath": "/Users/YOUR-L-USER/Claude",
    "mcpServers": {
    "pm-data-agent": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp",
        "--header", "X-User-Email: YOUR_LILLY_EMAIL@lilly.com"
      ]
    }
  },
}

Enabling the MCP connector in Claude Desktop

After saving claude_desktop_config.json, follow these steps to activate the connector:

1. Fully quit Claude Desktop — do not just close the window. On Mac, right-click the Claude icon in the Dock and select Quit. On Windows, right-click the Claude icon in the system tray and select Quit.
2. Relaunch Claude Desktop.
3. Open a new conversation and click the 🔌 (plug/tools) icon in the bottom-left of the chat input box. You should see pm-data-agent listed as an available tool server.
4. If pm-data-agent does not appear, open Claude → Settings → Developer (Mac: Cmd+, → Developer tab; Windows: Ctrl+, → Developer tab) and check the MCP server logs for errors. Common causes:
   • npx is not installed — run npm install -g npx to fix.
   • mcp-remote package is not available — run npx mcp-remote --version to verify.
   • VPN is not connected — the MCP server URL is only reachable on the Lilly network.
5. Once the connector is active, Claude Desktop will automatically pass roster and dependency_map data to your server at startup whenever you start a session.

---

Data schema

product_backlog.json

id, title, description, status, priority, effort_points,
business_value_score, confidence_score, requester, requested_date,
last_updated, tags, dependencies, squad_assignment, acceptance_criteria

customer_feedback.json

id, customer_id, customer_name, customer_tier, customer_status,
arr, source, date, text, sentiment_score

team_roster.json (from MCP server)

name, role, squad, total_capacity_points, sprint_allocation_percent,
pto_days_this_sprint, skills, carry_over_items, current_sprint_assignments

dependency_map.json (from MCP server)

source_item_id, target_item_id, dependency_type,
external_team, external_eta, notes

sprint_history.json

sprint_id, sprint_number, start_date, end_date, planned_points,
completed_points, items_planned, items_completed, items_carried_over,
velocity, notes

Never hardcode IDs, names, or numbers from the sample data — your tools must work correctly on any conformant dataset with the same schema.

---

agent_config.json — run contract

Do not modify the schema of agent_config.json. The agent reads this file to know how to launch your server. The fields are:

Field	Description
challenge_id	Identifies the challenge — do not change
type	Always "python"
runtime_version	Python version to use — must match your venv (3.11, 3.12, or 3.13)
entry	Entry point file — always "server.py"
run_command	Command to start the server — always "python server.py"
requirements	Dependency file — always "requirements.txt"
env_file	Env vars file — always "env_vars.json"
data_env_var	Env var name for the local data directory — always "PM_AGENT_DATA"
required_env	List of env var names your code requires at runtime
timeout_seconds	Max seconds allowed for server startup

The only fields you should change are runtime_version (if you use Python 3.12 or 3.13) and required_env (if your implementation needs additional env vars like LLM credentials).

MCP_DATA_URL is already in required_env — the eval runtime injects it automatically.

---

Environment variables (env_vars.json)

Declare any environment variables your implementation needs in env_vars.json.

The default env_vars.json contains:

{
    "PM_AGENT_DATA": "",
    "MCP_DATA_URL": "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp"
}

PM_AGENT_DATA is overridden at eval time with the blind dataset directory. MCP_DATA_URL default points to the shared data server — leave it as-is unless told otherwise.

If your implementation needs additional variables (e.g. LLM credentials), add them here with empty values — the runtime will inject the actual values. Also add their names to required_env in agent_config.json.

Never hardcode credentials or secrets in source code.

---

Supported Python versions

Use Python 3.11, 3.12, or 3.13. Declare your version in agent_config.json:

{ "runtime_version": "3.11" }

---

Dependencies (requirements.txt)

Pin your dependencies to exact versions compatible with your chosen Python version. The requirements.txt in this repo uses placeholder versions (0.0.0) — replace them with real pinned versions before submitting.

Example for Python 3.11 (Below are just examples for declaring versions in requirements.txt. The actual may differ depedning on the python version you choose to go ahead with):

mcp==1.9.4
pydantic==2.11.5
openai==1.93.0
requests==2.32.3
python-dotenv==1.1.0
httpx==0.28.1

Important: httpx is required — server.py uses it to connect to the MCP data server at startup. Unpinned or incorrect versions can cause your server to fail to start. Always test pip install -r requirements.txt from a fresh virtual environment with your declared Python version before submitting.

---

Quick start (local dev)

Mac / Linux:

python3.11 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Run the server locally (fetches roster+deps from MCP server at startup)
python server.py

Windows (PowerShell):

python3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# Run the server locally (fetches roster+deps from MCP server at startup)
python server.py

The server fetches team_roster and dependency_map from the MCP data server on startup. Local files (product_backlog.json, customer_feedback.json, sprint_history.json) are read from ./data/.

---

Testing your tools locally

You do not need Claude Desktop to verify your implementations. You can call the *_impl functions directly from a Python script using the sample data files:

# test_tools.py — run from the repo root with your venv active
import json
from pathlib import Path

from tools.prioritize_backlog import prioritize_backlog_impl
from tools.analyze_feedback import analyze_feedback_impl
from tools.assess_capacity import assess_capacity_impl
from tools.map_dependencies import map_dependencies_impl

DATA = Path("data")
backlog  = json.loads((DATA / "product_backlog.json").read_text())
feedback = json.loads((DATA / "customer_feedback.json").read_text())
sprints  = json.loads((DATA / "sprint_history.json").read_text())

# roster and deps come from the MCP server at runtime; use empty lists offline
roster = []
deps   = []

print("=== prioritize_backlog ===")
print(prioritize_backlog_impl("value_effort", {}, True, backlog, feedback, deps))

print("=== analyze_feedback ===")
print(analyze_feedback_impl(None, None, None, "theme", feedback))

print("=== assess_capacity ===")
print(assess_capacity_impl(None, "all", True, False, roster, backlog, sprints))

print("=== map_dependencies ===")
print(map_dependencies_impl([], True, 3, backlog, deps))

Run it with:

python test_tools.py

Each function should return a dict without raising a NotImplementedError or crashing.

---

Critical checklist before submitting

• [x] All 4 tools are implemented and return valid dicts (no NotImplementedError)
• [x] requirements.txt has real pinned versions (no 0.0.0 placeholders), including httpx, tested with your Python version
• [x] agent_config.json has the correct runtime_version (3.11, 3.12, or 3.13)
• [x] env_vars.json is present and lists all env var names your code reads
• [x] Local data tools read from PM_AGENT_DATA, not hardcoded paths
• [x] MCP_DATA_URL is in required_env in agent_config.json
• [x] Tools do not crash on unexpected inputs — handle edge cases gracefully (including empty roster or deps lists)
• [x] Same call returns the same result every time — no randomness unless seeded; if using an LLM, sort final output deterministically by score so rankings are stable
• [x] If using an LLM, credentials are declared in env_vars.json and required_env in agent_config.json
• [x] .venv/, __pycache__/, and .env are not committed — verify your .gitignore covers these before pushing