Codex Local Vision Router

DeepSeek V4 runs the Codex session through DwarfStar while Qwen3.5 in LM Studio handles image inspection through the local MCP tool.

Use one model as the main Codex agent and a separate local vision-language model for image inspection.

This project exposes a read-only MCP tool that:

1. reads an image from an absolute local path;
2. sends it to an OpenAI-compatible vision endpoint such as LM Studio;
3. returns only a bounded text observation to Codex.

It is intended for custom Codex providers that work well with text and tools but reject input_image content.

Architecture

user prompt with local path
        |
        v
primary Codex model (text + tools)
        |
        | MCP: local_vision.describe_image
        v
Local Vision Router
        |
        | image bytes over loopback HTTP
        v
LM Studio + local vision model
        |
        | plain-text observation
        v
primary Codex model

Privacy defaults

• The default endpoint is http://127.0.0.1:1234/v1.
• Non-loopback endpoints are rejected unless LOCAL_VISION_ALLOW_REMOTE=1 is explicitly set.
• Image bytes, base64 payloads, prompts, and API responses are not logged.
• The returned tool result does not include the local image path.
• No credentials, local configuration, model files, source images, or session data are included. The documentation contains only the intentionally published demonstration screenshot above.

A configured remote endpoint can receive image contents. Read SECURITY.md before enabling one.

Requirements

• Node.js 20 or newer
• Codex CLI
• LM Studio, or another compatible /v1/chat/completions server
• A loaded vision model

The tested model is TracNetwork/Qwen3.5-4B-4bit-mlx, loaded through LM Studio. LM Studio exposes it to the local API as qwen3.5-4b-mlx in the tested setup; use the exact ID returned by /v1/models if yours differs. Other vision models can be selected with --model.

Quick start

1. Start the vision endpoint

In LM Studio:

1. Load a vision-capable model.
2. Open Developer.
3. Start the local server on port 1234.

Confirm the model is visible:

curl http://127.0.0.1:1234/v1/models

2. Clone and check the router

git clone <repository-url>
cd codex-local-vision-router
node scripts/doctor.mjs

3. Register the MCP server

For normal Codex:

node scripts/register.mjs --model qwen3.5-4b-mlx

For an isolated custom-provider installation, set the same CODEX_HOME used by its launcher:

CODEX_HOME="$HOME/.codex-custom" \
  node scripts/register.mjs --model qwen3.5-4b-mlx

PowerShell:

$env:CODEX_HOME = "$HOME\.codex-custom"
node .\scripts\register.mjs --model qwen3.5-4b-mlx

The registration command refuses to replace an existing local_vision entry unless --replace is supplied.

4. Mark the primary model as text-only

Copy the root keys from templates/text-only-primary-config.toml into the active $CODEX_HOME/config.toml. Place them before the first TOML table.

This disables the built-in image tool and instructs Codex to use local_vision.describe_image.

5. Start a new Codex session

Do not resume a session that already contains an incompatible image block.

Run /mcp and confirm local_vision is connected. Then test:

Use local_vision.describe_image to inspect /absolute/path/to/image.jpg.
Describe the main subject, composition, visible text, and uncertainty.

Use a filesystem path rather than attaching or dragging the image into a text-only session.

End-to-end diagnostics

node scripts/doctor.mjs \
  --image /absolute/path/to/image.jpg \
  --prompt "Transcribe the largest visible heading."

Configuration

Variable	Default	Purpose
LOCAL_VISION_BASE_URL	http://127.0.0.1:1234/v1	Compatible API base URL
LOCAL_VISION_MODEL	qwen3.5-4b-mlx	Vision model ID
LOCAL_VISION_TIMEOUT_MS	120000	Request timeout
LOCAL_VISION_MAX_IMAGE_BYTES	52428800	Maximum image size
LOCAL_VISION_MAX_OUTPUT_TOKENS	4096	Generation ceiling
LOCAL_VISION_MAX_ANSWER_CHARS	2000	Structured answer limit
LOCAL_VISION_ALLOW_REMOTE	unset	Set to 1 to permit remote endpoints
LOCAL_VISION_API_KEY	unset	Optional endpoint bearer token

Supported extensions: PNG, JPEG, WebP, GIF, and AVIF.

Remove

codex mcp remove local_vision

Remove the text-only routing block from config.toml only if the primary provider can safely receive images.

Development

npm run check
npm test

The MCP server has no runtime npm dependencies.

See docs/SETUP.md for the detailed provider workflow and docs/PUBLISHING.md for publishing this repository.

Codex plugin bundle

The repository is also a valid Codex plugin bundle. A marketplace can point to this repository, and codex plugin add will install both the routing skill and the bundled MCP server. The plugin defaults to the loopback LM Studio endpoint and qwen3.5-4b-mlx.

Use the clone-and-register flow above when a different endpoint, model ID, or CODEX_HOME needs to be selected at install time.

License

MIT