llm-chat-mcp

MCP server for interacting with LLM models configured in Continue.dev's config.yaml.

What it does

Exposes 4 tools that let the agent inspect your configured models and send chat requests to them:

• llm_chat_list_models — list all models in your config with their names and ids
• llm_chat_get_model_params — inspect parameters (temperature, topP, etc.) for any model
• llm_chat_get_model_prompt — read the system prompt configured for a model
• llm_chat_send_request — chat with a model, optionally overriding parameters or loading prompts from files

Installation

pip install -e .

Or run directly from source without install.

CLI Usage

python -m llm_chat_mcp --default-model "GLM-5.2-FP8"
python -m llm_chat_mcp --config /path/to/config.yaml --default-model "ModelName"
python -m llm_chat_mcp --help

Argument	Description	Default
--config PATH	Path to config.yaml	~/.continue/config.yaml
--default-model NAME	Default model for send_request	(none)
--timeout SECONDS	Request timeout in seconds	19
--relative_paths_base PATH	Base directory for resolving relative output file paths	(process cwd)
--auto-output-dir PATH	Directory for auto-generated output files when response exceeds auto_file_threshold and no output_file_path is set	(OS temp directory)

Configuration

• Config is re-read on every request — no restart needed
• Auth uses apiKey from each model entry in config.yaml
• If no model is specified (neither in CLI nor tool call), an error tells you how to set one

llm_chat_send_request parameters

The main tool. Sends a chat completion request to an LLM and returns the response.

Input parameters

Parameter	Type	Default	Description
model_selector	str	(CLI default)	Model name or id from config.yaml
prompt_text	str	(none)	Prompt text to send
prompt_files	list	(none)	List of file specs (string path or {path, start_line, end_line} dict)
include_line_numbers	bool	true	Prefix each file line with N:
system_prompt	str	(from config)	Override system message; "" suppresses entirely
temperature, topP, topK, minP	float	(from config)	Sampling parameters
maxTokens, presencePenalty, frequencyPenalty	int/float	(from config)	Generation parameters
extraParams	dict	(none)	Extra body properties merged into API request
details	bool	false	Include reasoning/thinking content in response
timeout	float	(CLI default)	Per-request timeout override
output_file_path	str	(none)	Write full response to this file (relative paths resolved against --relative_paths_base)
append	bool	false	Append to output_file_path instead of overwriting; returns appended_line_start/appended_line_end
inline_preview_chars	int	500	Max chars of content returned inline; preview applies only when a file is written
auto_file_threshold	int	8000	Auto-write response to file when response_chars exceeds this and no output_file_path is set; 0 disables

Response structure

Always returned as JSON:

{
  "content": "<preview, full content, or empty>",
  "truncated": true,
  "metadata": {
    "model_name": "...",
    "model": "...",
    "elapsed_seconds": 1.23,
    "request_sent": {...},
    "response_headers": {...},
    "response_chars": 1234,
    "output_file": "...",
    "auto_output_file": "...",
    "created_dirs": [...],
    "appended_line_start": 201,
    "appended_line_end": 250
  }
}

Output strategy

The tool chooses one of three strategies based on parameters and response size:

1. Explicit file (output_file_path set): full response written to the file. If append=true, the response is appended and appended_line_start/appended_line_end (1-based, inclusive) are returned so the caller can read only the appended slice via extract_lines.
2. Auto file (no output_file_path, auto_file_threshold > 0, response_chars > threshold): full response written to an auto-generated file in --auto-output-dir (or OS temp). Filename format: llm_output_<YYYYMMDD_HHMMSS>_<6-char-uuid>.json.
3. Inline only (no file written): full content returned in the content field.

Inline preview

When a file is written (explicit or auto), the content field contains a preview of the response:

• If inline_preview_chars > 0 and len(content) > inline_preview_chars: truncated preview with suffix [truncated, full response in <file_path>].
• If inline_preview_chars > 0 and len(content) <= inline_preview_chars: full content (fits in preview).
• If inline_preview_chars == 0: empty content (file has the full response).

When no file is written, the full content is returned inline regardless of inline_preview_chars — this prevents data loss.

Continue.dev Integration

Add to .continue/mcpServers/llm-chat.yaml:

name: LLM Chat MCP server
version: 0.2.0
schema: v1
mcpServers:
  - name: LLM Chat MCP server
    command: python
    args:
      - "-m"
      - "llm_chat_mcp"
      - "--default-model"
      - "GLM-5.2-FP8"
      - "--timeout"
      - "570"
      - "--relative_paths_base"
      - "/path/to/your/workspace"
      - "--auto-output-dir"
      - "/path/to/your/workspace/.continue/skills/large-tasks/tmp-outputs"
    env:
      PYTHONPATH: "/path/to/llm-chat-mcp"

Then reload Continue.dev.

Project Structure

llm-chat-mcp/
├── pyproject.toml          # Dependencies: mcp, pyyaml, httpx
├── README.md               # This file
├── llm_chat_mcp/
│   ├── __init__.py
│   ├── __main__.py         # CLI entry point + tool registration
│   ├── config.py           # Config loading, model resolution
│   └── api.py              # API client, error handling
└── tests/
    └── test_output_strategies.py  # Tests for append, inline_preview, auto_file

Testing

python tests/test_output_strategies.py

Tests mock the API call and verify the file-writing and response assembly logic. Covers all combinations of output_file_path, append, inline_preview_chars, and auto_file_threshold.