multi-model-mcp

An MCP server that exposes tools for sub-agent style reasoning across multiple LLM providers. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.

Tools

Tool	Description
ask_model	Send a prompt to one configured model
ask_many	Send the same prompt to multiple models in parallel
reason_together	Multi-step reasoning: independent → critique, debate, or red-team
critique_answer	Ask models to critique a draft answer
pick_best_answer	Have a judge model rank candidate answers
list_models	List all configured model aliases

reason_together strategies

• independent_then_critique (default): All models answer independently → critic synthesizes
• debate: Models see each other's answers and refine over N rounds → critic synthesizes
• red_team: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes

Setup

1. Install

Requires Python ≥ 3.11 and uv.

git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv sync

2. Configure models

Copy and edit models.yaml — it ships with common models pre-configured. Each entry is a model alias pointing to a LiteLLM model string:

models:
  gpt:
    litellm_model: gpt-4.1
    api_key_env: OPENAI_API_KEY

  claude:
    litellm_model: claude-sonnet-4-5
    api_key_env: ANTHROPIC_API_KEY

  local:
    litellm_model: ollama/qwen3:latest
    api_base: http://localhost:11434   # no key needed

Add any provider LiteLLM supports: Groq (groq/llama-3.3-70b-versatile), Mistral, Together AI, DeepSeek, OpenRouter (openrouter/...), etc.

3. Set API keys

cp .env.example .env
# edit .env with your keys

Only keys for providers you actually use are required.

4. Register with Claude Code

Add to your project's .mcp.json (or ~/.claude.json for global):

{
  "mcpServers": {
    "multi-model": {
      "command": "uv",
      "args": [
        "run",
        "--project", "/path/to/multi-model-mcp",
        "multi-model-mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GEMINI_API_KEY": "...",
        "MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
      }
    }
  }
}

Or if you install it:

uv tool install .

Then use "command": "multi-model-mcp" without args.

Example Claude Code usage

# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.

# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.

# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"

# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2

# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2

# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]

# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"

Configuration reference

models.yaml fields

Field	Required	Description
litellm_model	Yes	LiteLLM model string (e.g. gpt-4.1, gemini/gemini-2.5-pro, ollama/qwen3:latest)
description	No	Human-readable label
api_key_env	No	Env var name holding the API key
api_base	No	Override base URL (needed for Ollama, proxies)
timeout	No	Per-call timeout in seconds (default: 60)
max_retries	No	Retry attempts on rate limit / timeout (default: 2)

LiteLLM model strings by provider

Provider	Example model string
OpenAI	gpt-4.1, gpt-4o, o4-mini
Anthropic	claude-sonnet-4-5, claude-opus-4-8
Google Gemini	gemini/gemini-2.5-pro, gemini/gemini-2.5-flash
Groq	groq/llama-3.3-70b-versatile
Ollama	ollama/qwen3:latest, ollama/llama3.3
OpenRouter	openrouter/anthropic/claude-sonnet-4-5
Mistral	mistral/mistral-large-latest
DeepSeek	deepseek/deepseek-chat
Together AI	together_ai/meta-llama/Llama-3-70b-chat-hf

See LiteLLM providers docs for the full list.

Design notes

• No key leakage: API keys are never logged; errors are sanitized before returning.
• Failure isolation: one model failing in ask_many / reason_together does not crash the call.
• Synthesis ≠ truth: reason_together presents the critic's output as a synthesized answer, not ground truth.
• No hidden reasoning exposed: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
• Easy to extend: add any LiteLLM-supported model in models.yaml with no code changes.