local-mcp

Run MCP tools against local MLX models on your Mac.

local-mcp routes MCP tool calls, CLI prompts, and dashboard testing traffic to your own OpenAI-compatible local model servers. v5 adds better hardware detection, a one-command init flow, CLI streaming, and improved routing visibility in the dashboard.

Quick Start

# 1. Install dependencies
pip install mlx-lm

# 2. Detect your hardware and write a starter config
npx local-mcp init

# 3. Start the two recommended MLX servers printed by init
python3 -m mlx_lm.server --model mlx-community/Qwen3.5-9B-MLX-4bit --port 8081
python3 -m mlx_lm.server --model mlx-community/Qwen2.5-1.5B-Instruct-4bit --port 8083

# 4. Register with Claude Code
claude mcp add local-mcp -- npx local-mcp serve

# 5. Open the dashboard
npx local-mcp dashboard

You can inspect hardware fit without writing config:

npx local-mcp fit

You can also ask the local model directly from the terminal. Output streams as tokens arrive:

npx local-mcp ask "Explain why mmap helps large-model inference"

Architecture

                    ┌──────────────────────────────┐
                    │      Hardware Scanner        │
                    │   local-mcp fit / init       │
                    └──────────────┬───────────────┘
                                   │ fit report
                                   ▼
┌───────────────┐   HTTP    ┌──────────────────────┐   HTTP    ┌───────────────┐
│ local-mcp CLI │──────────▶│  OpenAI-compatible   │──────────▶│  MLX Models   │
│ ask / fit /   │           │   local endpoints    │           │ smart + fast  │
│ init          │           └──────────────────────┘           └───────────────┘
└───────────────┘
        │
        │ stdio
        ▼
┌───────────────┐            HTTP                   ┌───────────────┐
│  MCP Server   │─────────────────────────────────▶│  MLX Models   │
│ Claude/Codex  │                                  │ smart + fast  │
└───────────────┘                                  └───────────────┘

┌───────────────┐   HTTP    ┌──────────────────────┐
│  Dashboard    │──────────▶│  OpenAI-compatible   │
│ status/routing│           │   local endpoints    │
└───────────────┘           └──────────────────────┘

CLI

npx local-mcp ask "your question"
npx local-mcp ask --fast "classify this quickly"
npx local-mcp ask --reason "work through this carefully"
npx local-mcp fit
npx local-mcp init
npx local-mcp bench
npx local-mcp status
npx local-mcp dashboard
npx local-mcp serve
npx local-mcp start

Bin aliases:

local-mcp-fit
local-mcp-init

Hardware Fit

local-mcp fit scores curated models against detected machine RAM.

• perfect: model RAM footprint is under 50% of system RAM
• good: under 70%
• marginal: under 85%
• too_large: likely poor experience or unsafe to run alongside normal apps

On macOS, v5 also improves CPU detection and available-memory reporting:

• CPU uses system_profiler, sysctl -n machdep.cpu.brand_string, and sysctl -n hw.model
• Apple Silicon detection checks both sysctl -n hw.optional.arm64 and uname -m
• Available RAM uses vm_stat free + inactive + speculative pages instead of raw free pages only

Machine Recommendations

Machine	Smart	Fast
Mac mini 24GB	Qwen3.5-9B	Qwen2.5-1.5B
MacBook Pro 36GB	Qwen3-14B or Phi-4	Qwen2.5-1.5B
MacBook Pro 64GB	Qwen3.5-27B	Qwen2.5-7B

Dashboard

The dashboard at http://localhost:4242 includes:

• Status with live smart/fast endpoint health and latency
• Model library with curated MLX models
• Hardware fit view with recommended smart/fast assignments
• Routing controls with health badges, reset-to-defaults, and last-saved timestamp
• Setup wizard, logs, and prompt template editing

Configuration

local-mcp init writes ~/.local-mcp/config.json with recommended models. Environment variables can still override the config file.

Example:

{
  "endpoints": {
    "smart": {
      "url": "http://localhost:8081",
      "model": "mlx-community/Qwen3.5-9B-MLX-4bit"
    },
    "fast": {
      "url": "http://localhost:8083",
      "model": "mlx-community/Qwen2.5-1.5B-Instruct-4bit"
    }
  },
  "routing": {
    "ask": "smart",
    "reason": "smart",
    "classify": "fast",
    "summarize": "fast",
    "code_review": "smart",
    "explain": "smart",
    "extract": "fast",
    "translate": "fast",
    "diff_analysis": "smart"
  }
}

MCP Clients

Claude Code:

claude mcp add local-mcp -- npx local-mcp serve

Generic stdio config:

{
  "mcpServers": {
    "local-mcp": {
      "command": "node",
      "args": ["/absolute/path/to/local-mcp/dist/index.js", "serve"]
    }
  }
}

Troubleshooting

• If ask does not stream, local-mcp falls back to normal non-streaming JSON responses automatically.
• If fit reports Unknown CPU, make sure sysctl, uname, and system_profiler are available in your shell.
• If an endpoint is down, open the dashboard Routing tab to confirm which tier assignments currently point at an unhealthy server.