GIMP Local MCP

GIMP Local MCP is a local-first Model Context Protocol server for controlling GIMP 3 through an LLM. The LLM chooses an operation through structured MCP calls; GIMP owns the image manipulation, rendering, file loading, and exporting. Images and files remain on the user’s machine.

This v0.1 implementation favors a small, composable architecture over hundreds of one-off wrappers.

Philosophy

• GIMP owns pixels. This project does not generate replacement pixels or call cloud image services.
• Routine edits use ergonomic tools with stable image and layer IDs.
• The live GIMP PDB is the long-term path to broad coverage, with structured invocation rather than raw Scheme input.
• File writes are explicit and never overwrite existing targets by default.
• The bridge preserves GIMP’s undo history and groups multi-step operations when the operation needs multiple PDB calls.

Architecture

Natural language
      |
      v
MCP host (Codex, Claude, custom client)
      |
      | MCP stdio / JSON-RPC
      v
gimp-local-mcp
  ergonomic tools ---> GimpService ---> safe Scheme serializer
                              |                  |
                              +--> PDB catalog/invoker
                              +--> structured GEGL filter gateway
                              +--> VisionClient ---> trusted local JSONL worker
                                                   |
                                                   v
                                    Script-Fu TCP framing (127.0.0.1:10008)
                                                   |
                                                   v
                                             GIMP 3 Script-Fu server

The official MCP Python SDK is used for the server. Script-Fu socket and framing logic is isolated in gimp/transport.py and gimp/protocol.py; tool functions do not manipulate sockets directly.

Requirements

• Python 3.10 or newer
• GIMP 3.x
• A running GIMP 3 Script-Fu server

GIMP is not required for the ordinary unit test suite.

GIMP 3 setup

1. Start GIMP 3.
2. Open Filters → Development → Script-Fu → Start Server.
3. Keep the listener on 127.0.0.1 and port 10008 unless you have a specific reason to change it.
4. Start the MCP server after the Script-Fu server is listening.

GIMP’s Script-Fu server accepts a three-byte G/16-bit-length request frame and returns a four-byte G/error/16-bit-length response frame. GIMP allows one Script-Fu client at a time; this server maintains one reconnectable client connection.

Installation

From a checkout:

python -m pip install -e '.[dev]'

For a runtime-only installation:

python -m pip install .

Run the server

gimp-local-mcp serve

With no subcommand, gimp-local-mcp also starts the stdio MCP server. Stdout is reserved for MCP protocol traffic; diagnostics go to stderr.

Check the local connection independently:

gimp-local-mcp doctor

doctor prints Python details, the configured endpoint, local-only status, Script-Fu reachability, GIMP version, and open-image count. A connection refusal is expected until GIMP’s Script-Fu server is started.

Codex configuration

Add a project-local MCP entry in the Codex configuration used for this checkout. The exact location depends on the Codex client version; do not modify global configuration automatically.

[mcp_servers.gimp-local-mcp]
command = "gimp-local-mcp"
args = ["serve"]

If the command is not on the client’s PATH, use the absolute path to the virtual-environment executable or launch it with Python:

[mcp_servers.gimp-local-mcp]
command = "python"
args = ["-m", "gimp_local_mcp.cli", "serve"]

Configuration

Defaults work with a standard local GIMP setup:

Environment variable	Default	Meaning
GIMP_MCP_HOST	127.0.0.1	Script-Fu host
GIMP_MCP_PORT	10008	Script-Fu port
GIMP_MCP_TIMEOUT	10	TCP operation timeout in seconds
GIMP_MCP_LOG_LEVEL	INFO	stderr log level
GIMP_MCP_MAX_RESPONSE_BYTES	65535	bounded response body size
GIMP_MCP_ALLOW_REMOTE	false	explicit opt-in for non-loopback hosts
GIMP_MCP_VISION_PROVIDER	none	optional trusted local provider name, such as clipseg
GIMP_MCP_VISION_COMMAND	unset	trusted local worker command; never an MCP argument
GIMP_MCP_VISION_TIMEOUT	60	local worker response timeout in seconds, capped at 600
GIMP_MCP_VISION_DEVICE	auto	provider device policy: auto, cpu, or cuda

Remote connections are rejected unless GIMP_MCP_ALLOW_REMOTE=true is set. Localhost-only operation is the secure default.

MCP tools

The current server registers 57 tools across these groups:

• Session: gimp_status, gimp_capabilities, list_open_images, get_active_image, get_current_context, get_image_info
• Files and images: open_image, create_image, save_xcf, export_image, close_image
• Layers: list_layers, get_layer_tree, get_layer_info, get_selected_layers, set_selected_layers, create_layer, create_layer_group, duplicate_layer, rename_layer, delete_layer, set_layer_visibility, set_layer_opacity, set_layer_mode, move_layer, merge_down
• Transforms: resize_image, resize_canvas, crop_image, rotate_image, flip_image
• Selection: select_all, select_none, invert_selection, select_rectangle, select_ellipse, select_layer_alpha
• Masks and subject isolation: get_layer_mask_info, create_layer_mask, set_layer_mask_enabled, isolate_subject, isolate_subject_vision, separate_subject_to_layers, separate_concepts_to_layers
• Optional local vision: vision_status, segment_subject
• Adjustments and undo: brightness_contrast, hue_saturation, desaturate, undo, redo
• Non-destructive filters: apply_gaussian_blur_filter, apply_brightness_contrast_filter, list_drawable_filters
• PDB: search_pdb, describe_pdb_procedure, invoke_pdb_procedure

invoke_pdb_procedure accepts JSON-compatible structured values, including { "scheme_symbol": "RGB" } for a GIMP enum. It does not accept Scheme source, Python, shell commands, or arbitrary evaluation. Runtime PDB counts and documentation are used when available. Procedure descriptions now include a bounded typed-metadata state (available, partial, unavailable, or malformed) plus argument/return records when a trusted structured adapter reports them. The default Script-Fu TCP adapter reports argument metadata as unavailable because Script-Fu does not expose a stable GimpProcedure/GParamSpec representation; no signatures or types are guessed. Named-argument validation is performed only when trustworthy names are actually available.

GIMP 3 also exposes non-destructive drawable filters through special Script-Fu bindings rather than ordinary PDB procedures. The filter gateway uses the documented gimp-drawable-append-new-filter binding with structured named GEGL properties, then reads the actual filter ID and state back through GIMP. The current high-level slices are Gaussian blur and brightness/contrast; they preserve the existing destructive adjustment tool names.

Layer masks use a reusable gateway over the GIMP bindings gimp-layer-create-mask, gimp-layer-add-mask, gimp-layer-get-mask, and the mask state accessors. Existing masks are reported and never replaced by the public creation tool. isolate_subject duplicates the explicit source layer, measures the perimeter, and seeds a contiguous-color background selection from either light perimeter samples or all perimeter samples when high-key evidence is strong. It inverts that selection into a mask and leaves the source layer intact. The operation returns baseline and refined observable mask proxies: border transparency, transparent and opaque proportions, partial-alpha proportion, edge-transition samples, a sampled retained bounding box, and retention relative to the baseline's confident samples. These are not semantic accuracy scores.

Optional local semantic vision

Semantic vision is deliberately out of process. VisionClient speaks a bounded JSONL protocol to a trusted command configured by the operator; the core installation does not install PyTorch, CUDA, Transformers, SAM, or model weights. A worker returns structured candidates and lossless grayscale PNG mask artifacts. Diagnostics go to worker stderr, while protocol traffic remains stdout-only.

CLIPSeg through tools/vision/clipseg_worker.py remains the lightweight fallback. It supports local free-text concept masks and fits CPU-only systems much better than SAM 3. It loads checkpoints with local_files_only=True during ordinary operation; downloading is a separate explicit setup step. tools/vision/sam3_worker.py is now an adapter for the official Meta SAM 3 image API. It supports the official text and positive-box grounding paths, multiple scored masks, and boxes only after a real checkpoint self-test succeeds. Point prompting belongs to SAM 3's separate interactive predictor and is not claimed by this adapter. No normal request downloads a checkpoint.

Create a separate provider environment; this does not change pip install .:

python3.13 -m venv .venv-vision
.venv-vision/bin/python -m pip install --upgrade pip
.venv-vision/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv-vision/bin/python -m pip install transformers accelerate pillow psutil
.venv-vision/bin/python tools/vision/clipseg_worker.py --download-model

export GIMP_MCP_VISION_PROVIDER=clipseg
export GIMP_MCP_VISION_COMMAND="$PWD/.venv-vision/bin/python $PWD/tools/vision/clipseg_worker.py"
export GIMP_MCP_VISION_DEVICE=auto
export GIMP_MCP_VISION_OFFLOAD=auto
export GIMP_MCP_VISION_GPU_RESERVE_MIB=256
export GIMP_MCP_VISION_DTYPE=auto
export GIMP_MCP_VISION_TIMEOUT=120
gimp-local-mcp doctor

CLIPSeg converts its coarse semantic logits into a soft mask with conservative defaults of 0.2 for the probability threshold and 2.0 for the sigmoid slope. Operators can tune those generic controls with GIMP_MCP_CLIPSEG_MASK_THRESHOLD (0.01–0.99) and GIMP_MCP_CLIPSEG_MASK_SLOPE (0.25–8.0). Higher thresholds reject weaker surrounding regions; higher slopes make the transition firmer. These settings refine a semantic mask but do not turn it into a fur/hair alpha matte.

Device and offload are separate policies:

• GIMP_MCP_VISION_DEVICE=auto|cpu|cuda chooses the execution device.
• GIMP_MCP_VISION_OFFLOAD=auto|none|sequential-cpu controls weight placement.
• GIMP_MCP_VISION_GPU_RESERVE_MIB preserves dynamic headroom from the free VRAM measured at worker startup. GIMP_MCP_VISION_MAX_VRAM_MIB can impose an optional planning cap.
• GIMP_MCP_VISION_DTYPE=auto|float32|float16|bfloat16 accepts reduced precision only after a real execution probe. AUTO remains FP32 on Pascal-class GPUs without Tensor Cores.

AUTO first executes an actual CUDA matmul; discovery alone is insufficient. It subtracts the configured reserve from current free VRAM and uses full CUDA only when estimated model storage plus workspace fits. Otherwise it uses Hugging Face Accelerate's cpu_offload when the provider supports it, or CPU. Sequential CPU offload keeps primary parameter storage in system RAM, attaches layer-wise execution hooks, moves parameters to CUDA as their modules execute, and removes them after use. It lowers persistent VRAM residency but does not lower system-RAM requirements. Disk offload is a separate slower tradeoff and is intentionally not the default.

AUTO has a bounded OOM path: full CUDA may retry sequential offload, then CPU; explicit device and offload selections fail instead of silently violating operator intent. Vision completes before any document mutation, so provider OOM cannot leave partial GIMP edits. doctor reports the placement reason, CUDA kernel probe and compiled architectures, startup free VRAM, reserve/budget, Torch allocated/reserved peaks, process RSS, and evidence from Accelerate hooks and persistent parameter devices. Torch allocator peaks and nvidia-smi process usage are different measurements.

For an explicit official SAM 3 setup, install Meta's package according to its prerequisites, obtain access to the gated checkpoint, authenticate outside MCP, and run:

.venv-sam3/bin/python tools/vision/sam3_worker.py --download-checkpoint
export GIMP_MCP_SAM3_CHECKPOINT=/absolute/path/printed/by/setup/sam3.1_multiplex.pt
export GIMP_MCP_VISION_PROVIDER=sam3
export GIMP_MCP_VISION_COMMAND="$PWD/.venv-sam3/bin/python $PWD/tools/vision/sam3_worker.py"

The worker passes load_from_HF=False to the official builder during normal operation and reports unavailable if the explicit file is absent. Use the bounded comparison command for one provider and input across placements:

python tools/vision/provider_check.py \
  --benchmark cpu,full-cuda,sequential-cpu-offload,auto

segment_subject(image_id, layer_id, prompt="red fox") snapshots a duplicate of the current GIMP image to a uniquely named temporary PNG, invokes the local worker, validates candidate artifacts, and removes the snapshot and artifacts after returning bounded summaries. It does not mutate the document. isolate_subject_vision uses the same bridge, duplicates the explicit source layer, imports the mask through a temporary RGBA layer/selection, and attaches it through the existing layer-mask gateway. isolate_subject(strategy="auto", prompt="red fox") prefers a capable semantic worker and falls back to the existing high-key contiguous-color strategy when no provider is available. Explicit high-key-background and border-color retain the heuristic.

separate_subject_to_layers(image_id, layer_id, prompt="red fox") leaves a persistent group with Subject — red fox and Background, hides but does not modify the original source, and selects the subject layer. The background mask is the exact 8-bit complement of the subject/foreground-union artifact; both layers use duplicates of the source. separate_concepts_to_layers accepts up to eight concepts, preserves distinct provider instances when returned, bounds the total generated layers to 24, reports mask overlap without assigning ownership, and derives the remainder from the soft-alpha foreground union. Provider failures happen before GIMP mutation; mask-import failures roll back generated layers, the group, source visibility, and selected-layer state.

GIMP 3.2 stores imported partial mask samples in its image-space encoding. In live validation, complementary 128/127 artifact values read back near 188/187; after decoding to linear alpha their sum was 1.0 within 0.005. Binary endpoints remained exact. This is reported as bridge behavior and is not described as learned matting.

The current refinement boundary is IdentityMaskRefiner: provider output is retained, but this is not learned alpha matting. CLIPSeg emits a coarse semantic probability mask; SAM 3 emits binary segmentation masks. Semantic/object discovery, segmentation, optional future edge/alpha refinement, artifact validation, and GIMP mask creation remain distinct stages. Coverage, border transparency, partial-alpha, edge-transition, and sampled bounding-box values are observable proxies only. The fox-on-snow benchmark has no ground-truth alpha matte, so confidence and proxy metrics must not be presented as accuracy.

On the prior 1280×960 fox benchmark, the cached CLIPSeg model reported about 603 MB of parameter storage and completed CPU inference in 0.49 seconds after a 12.66-second cold load. The new placement comparison used the same deterministic 32×32 image/prompt on the P620 with PyTorch 2.7.1+cu126 and 128 MiB reserve. CPU/full-CUDA/sequential inference took 0.272/0.178/0.307 seconds. Full CUDA peaked at 660,646,912 allocated and 687,865,856 reserved bytes; sequential offload peaked at 131,516,416 allocated and 146,800,640 reserved bytes, an 80.1% lower allocated peak. The offload run had 233 hooks, 602,990,984 meta/CPU-backed parameter bytes, zero persistent CUDA parameter bytes, and sequential_offload_verified=true. Full CUDA retained all 602,990,984 parameter bytes on CUDA. Both CUDA placements produced identical masks; compared with CPU, mean absolute difference was 0.00098 on the 8-bit mask and maximum difference was one level. Sequential RSS was higher (about 1.71 GiB) because offload trades VRAM for system RAM. These are one-machine observations, not performance guarantees.

For document navigation, start with get_current_context. It returns the open image IDs, a current-image snapshot when one can be established, the resolution source, and all selected layer IDs. GIMP 3 uses multi-layer selection, so get_selected_layers returns a list rather than inventing a single active layer. get_layer_tree recursively reports groups and children with stable IDs, parent IDs, positions, and bounded recursion/item limits. The Script-Fu server does not always expose a default-display context; when exactly one image is open, the service reports single-open-image as an explicit fallback, and it reports multiple open images as ambiguous instead of guessing.

Example requests

• “Open this image and crop it to 16:9 around the subject.”
• “Duplicate the background layer, desaturate it, and reduce its opacity to 40%.”
• “Resize this to 2048 pixels wide while preserving aspect ratio.”
• “Export a JPEG copy at this path without overwriting anything.”
• “What layers are currently in this document?”
• “Separate the red fox from the background and leave both on editable layers.”
• “Separate the fox, trees, and snowbank into layers and report any overlap.”
• “Find a GIMP procedure capable of applying Gaussian blur and describe its arguments.”

The final request is supported through PDB search and description; the current description reports procedure counts, documentation, and the explicit argument-metadata capability state. Rich argument records remain unavailable through the default Script-Fu bridge until a live, trustworthy adapter is available.

Security model

See SECURITY.md. In brief:

• the MCP server is stdio-first and connects to loopback by default;
• no shell, Python eval, raw Script-Fu MCP tool, or cloud image API exists;
• procedure names and structured values are validated before serialization;
• strings and paths are escaped as Scheme literals;
• output paths must be explicit, normalized, and non-overwriting unless requested;
• closing a dirty image requires an explicit discard choice.

Limitations

• Live validation in this iteration used GIMP 3.2.0 on the default loopback endpoint 127.0.0.1:10008. Other GIMP 3 versions still need compatibility validation.
• On the tested GIMP 3.2.0 Script-Fu environment, the legacy gimp-pdb-query and gimp-pdb-proc-exists helper bindings are unavailable. The typed PDB adapter therefore retains its explicit unavailable fallback; the filter bindings are validated independently. A future bridge adapter should use a supported GIMP-side PDB or GObject-introspection channel rather than guessing signatures.
• Export metadata behavior uses GIMP’s configured defaults in v0.1; no hidden metadata is added or removed.
• The tested GIMP 3.2.0 Script-Fu bridge does not expose gimp-file-export; export_image refuses to fall back to gimp-file-save, which could change the open document's associated file. Use GIMP's own export UI until a supported export binding is available.
• The initial adjustment tools call stable legacy PDB adjustment procedures that GIMP 3.2 marks deprecated in favor of non-destructive filters. Only brightness/contrast and Gaussian blur have non-destructive high-level slices so far.
• Native foreground extraction is unavailable through the tested GIMP 3.2.0 Script-Fu bridge. CLIPSeg is genuinely text-prompted but produces a coarse 352×352 semantic probability mask; upsampling can leave halos, soften fur, omit thin whiskers, or retain nearby regions. It does not separate instances or discover/language-label every object. Without a capable worker, auto retains the bounded high-key fallback.
• The tested GIMP 3.2.0 bridge safely supports duplicate-image PNG snapshots through gimp-image-duplicate plus gimp-file-save on the duplicate. The unsafe save fallback remains prohibited for user images. The mask import bridge preserves partial alpha observed through GIMP's selection-to-mask path, but GIMP image-space mask encoding changes the raw sampled byte values for partial alpha. This is not a learned hair/fur-matting engine.
• Independently prompted concept masks can overlap. The default and currently supported policy is report: layers retain their masks, overlap statistics are returned, and remainder background uses their soft union. No arbitrary first-concept ownership is imposed.
• The Quadro P620 probe found compute capability 6.1 and 2 GiB total VRAM, with about 1.06 GiB free under KDE/Wayland during this iteration. The official PyTorch 2.10+cu128 wheel discovered CUDA but omitted sm_61, so its real kernel probe failed with no kernel image. PyTorch 2.7.1+cu126 includes sm_60 and completed real FP32, FP16, and BF16 matmuls. Wheel architecture support must be checked for this Pascal GPU; a newer version number is not evidence of compatibility.
• Official SAM 3.1 requires gated checkpoint access. The adapter and explicit setup path are implemented, but this run had no authenticated Hugging Face account or local checkpoint, so no SAM inference or offload claim is made. Import success alone is not provider support.
• get_active_image uses GIMP’s default display when the Script-Fu context provides one. If that helper is unavailable and exactly one image is open, it returns that image with the same documented single-image fallback used by get_current_context; multiple open images remain ambiguous.
• Selected-layer control and recursive group inspection were validated against GIMP 3.2.0. The bridge rejects empty selection vectors, validates layer/image ownership, and reads selection state back after setting it.
• Multi-call layer creation and duplication are grouped into one GIMP undo step. Additional composite operations should adopt the same internal helper as they are added.

Roadmap

1. Validate the high-level vertical slices against additional GIMP 3.x releases and platforms.
2. Add a supported GIMP-side structured PDB metadata adapter for argument names, types, defaults, and enum choices; keep the explicit unavailable fallback when the bridge cannot provide it.
3. Add structured non-destructive GEGL filter operations for levels, curves, and hue/saturation.
4. Validate document context and multi-layer selection semantics against additional GIMP 3.x releases and multi-window setups.
5. Add explicit export metadata policies and more file-format option models.
6. Add safe, persistent capability caching with GIMP version invalidation.
7. Validate official SAM 3.1 text, box, multi-instance, activation-memory, and Accelerate hook behavior with an authorized local checkpoint; add provider-specific preload hooks only if the measured module access pattern requires them.
8. Add a local alpha-matting provider behind MaskRefiner when a supported lightweight runtime is available; keep semantic segmentation and matting evidence separate.
9. Implement automatic object proposals with stable unlabeled candidate IDs, boxes, areas, and previews before adding any semantic labels not produced by a model.

Development

ruff format src tests
ruff check src tests
pytest -q

Live tests are marked integration and skip when 127.0.0.1:10008 is unavailable. See CONTRIBUTING.md and AGENTS.md for repository invariants.

Forge development evidence

The committed mncs-forge.toml declares a project-owned Provider Protocol 0.1 provider in tools/forge_provider.py. It parses fixed Ruff, pytest, pip, and live-GIMP checks into explicit PASS/FAIL/UNKNOWN responses. In particular, a skipped live integration test is UNKNOWN. Forge runtime ledgers and .mncs-forge/ state remain local and are intentionally not committed. The provider disables Python bytecode writes in its subprocesses so normal checks do not mutate the Forge candidate scope with generated __pycache__ files.