LLMconcil

An MCP server that lets Claude Code ask several other models the same question, then hands back a structured comparison of what they said.

The idea is stolen from OpenRouter's Fusion, including the part most people get wrong: the second model does not merge the answers. It compares them and reports where they agree, where they contradict each other, and what none of them brought up. Claude writes the final answer from that. A merged answer hides which parts were unanimous and which came from one model having a bad day; a comparison doesn't.

Claude Code
   │  fusion_deliberate(prompt, context=[{path, lines}, ...])
   ▼
 panel ──┬─→ google/gemini-3.1-pro-preview   (+ Google Search grounding)
         ├─→ deepseek/deepseek-v4-pro
         ├─→ x-ai/grok-4.5
         └─→ minimax/minimax-m3
   │       four independent answers, in parallel
   ▼
 judge  ──→ moonshotai/kimi-k3
   │       { consensus, contradictions, partial_coverage,
   │         unique_insights, blind_spots }
   ▼
 Claude Code writes the final answer

Worth it for architecture trade-offs, "is this actually a good idea", library choices, anything where being confidently wrong is expensive. Not worth it for tactical questions with one right answer — you'd be paying four models to agree.

Setup

Needs Python 3.11+ and an OpenRouter API key. Everything else is optional.

git clone https://github.com/azeur365/LLMconcil
cd LLMconcil
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
cp .env.example .env      # then fill in OPENROUTER_API_KEY

Register it with Claude Code:

claude mcp add llmconcil -- "$PWD/.venv/bin/llmconcil"

fusion_deliberate shows up as a tool from there.

The council

council.toml decides who sits on the panel and who judges. This is the shipped default, the one in the diagram above:

[[panel]]
model = "google/gemini-3.1-pro-preview"
search = true

[[panel]]
model = "deepseek/deepseek-v4-pro"

[[panel]]
model = "x-ai/grok-4.5"

[[panel]]
model = "minimax/minimax-m3"

[judge]
model = "moonshotai/kimi-k3"

model is an OpenRouter slug. The file is re-read on every call, so you can change the line-up without restarting the server.

Pick models that disagree: four siblings from the same lab produce four near-identical answers and a judge with nothing to report. And keep the judge's vendor off the panel. blind_spots only means something coming from a model that didn't answer the prompt itself.

search = true turns on Grounding with Google Search for that seat. It only works on the AI Studio path, so it needs a Gemini model and a GEMINI_API_KEY. On any other seat it does nothing, and a Gemini seat that falls back to OpenRouter answers without grounding.

That asymmetry is deliberate. OpenRouter has its own web plugin, and it is not wired up here: it bills per request, and on a model whose provider has no native search it substitutes a third-party engine. Losing citations on a fallback is a smaller problem than a config flag that quietly changes which search engine answered and what it cost.

Gemini and Google AI Studio

Everything runs on OpenRouter, with one exception you can opt into.

If GEMINI_API_KEY is set, Gemini panellists are sent to Google directly instead of through OpenRouter. Nothing else changes: same slug in council.toml, same model, same grounding mechanism, same shape coming back. But the calls come out of your Gemini quota, which matters if your subscription includes credit that OpenRouter can't spend.

Without the key, Gemini rides on OpenRouter like everything else.

The key has to have billing enabled. AI Studio's free tier won't serve the Pro models this is pointed at, so there's no free path to fall back to and the code doesn't pretend otherwise: AI Studio answers are reported as unpriced, never as free.

When the call doesn't land, for any reason, it falls back to OpenRouter rather than dropping the panellist. Quota exhaustion and "this model is experiencing high demand" are both routine, and neither is worth losing an answer over. meta.aistudio_fallbacks records what went wrong, so a key that never works shows up instead of quietly spending OpenRouter credit.

The judge always goes through OpenRouter. It needs response_format: json_schema, which isn't worth reproducing on the genai SDK (where it also can't be combined with grounding) to save a few cents.

meta.served_by tells you which backend actually answered for each model.

The tool

fusion_deliberate(prompt, context?, panel?, judge?, temperature?, reasoning_effort?)

context takes file refs ({path, lines}, read server-side), inline snippets ({text}), and images ({image}, png/jpg/gif/webp). Curate it. Everything you attach is sent to every panellist, so an irrelevant file costs you N times and dilutes the analysis. There's a 200k-token budget; over it the call is rejected with a message naming the offending files rather than silently truncating. Paths are confined to LLMCONCIL_ROOT, and binary or oversized (>5 MB) files are refused.

Attaching an image drops the panellists that can't see it, listed in meta.skipped_no_vision. Capability comes from OpenRouter's /models catalogue, fetched once and only when an image is attached. If that fetch fails, nothing is dropped and a text-only model fails on its own, visibly.

panel / judge override council.toml for one call. Both take bare slugs, so they carry no per-model options: a model named that way runs without search.

Returns {status, analysis, responses, failed_models, failure_reason, meta}. The raw panel answers come back alongside the analysis, so Claude can go read what a model actually said instead of trusting the judge's summary of it.

meta.cost_usd is what OpenRouter billed, read off the response rather than estimated from a price table. Calls it didn't bill (anything AI Studio served) are listed in meta.cost_excludes instead of being counted as free.

Here is analysis from a real run, asking whether a small team should pin exact dependency versions. Trimmed to one entry per key:

{
  "consensus": [
    "Pin exact versions, via a committed lockfile that freezes the full tree."
  ],
  "contradictions": [
    {
      "topic": "What should be declared in the manifest?",
      "stances": [
        {
          "model": "deepseek/deepseek-v4-pro",
          "stance": "Ranges. The lockfile does the actual pinning."
        },
        {
          "model": "x-ai/grok-4.5",
          "stance": "Exact versions, so intent survives someone deleting the lockfile."
        }
      ]
    }
  ]
}

That contradiction is the whole point. A merged answer would have picked one of those two and thrown the other away, and you would never have known the question was contested.

Configuration

Everything lives in .env, and everything but the first line has a default that works.

variable	default	what it does
OPENROUTER_API_KEY	—	required; serves the whole council
GEMINI_API_KEY	unset	routes Gemini to AI Studio instead of OpenRouter
LLMCONCIL_COUNCIL	./council.toml	where to read the line-up from
LLMCONCIL_ROOT	working dir	file refs outside this are rejected
LLMCONCIL_MAX_CONTEXT_TOKENS	200000	curation budget for attached context
LLMCONCIL_STALL_TIMEOUT	180	seconds of silence before a stream is killed
LLMCONCIL_JUDGE_CONTEXT_MAX	32000	above this, the judge doesn't re-read the context
LLMCONCIL_PROVIDER_SORT	throughput	OpenRouter provider routing; price, latency, none

Failure modes

A panellist that dies takes its own answer down and nothing else: it lands in failed_models with a reason and the rest of the council carries on. Same for the judge. You still get the panel answers, with an empty analysis and status: "error".

Every streamed call has a stall timeout (LLMCONCIL_STALL_TIMEOUT, default 180s) that fires when no token arrives within the window. It's deliberately a stall timeout and not a total one: a model reasoning hard for six minutes is working, a model silent for three is stuck, and a fixed deadline can't tell them apart. Keepalive frames don't reset it, or a stuck stream would keep itself alive forever.

Layout

file	role
server.py	the MCP server and the fusion_deliberate tool definition
fusion.py	orchestration — panel fan-out, then the judge
council.py	reads council.toml
backends.py	which service serves which model
openrouter.py	streaming client, stall timeout, model catalogue
gemini.py	the Google AI Studio path
context.py	file reading, line ranges, token budget
schema.py	the JSON schema the judge has to fill in

Known rough edges

• No tests. The failure paths are handled but only manually exercised.
• The judge's structured output leans on response_format: json_schema. Models vary in how well they honour it, so there's a tolerant parser behind it that digs the first balanced JSON object out of the reply. A judge that ignores the schema entirely yields an empty analysis rather than garbage.
• A near-200k context will overflow panellists with smaller windows. They fail individually and land in failed_models, but nothing warns you upfront.
• The /models catalogue is fetched once and kept for the life of the process, so a model that gains vision after your server started won't be recognised until you restart it.

License

MIT