MCP Synth Controller

This project implements a Python-based Model Context Protocol (MCP) server designed to allow large language models (LLMs) to control parameters of a JUCE synthesizer in real time.

The MCP server exposes structured tools that an LLM can call, and these tool calls are translated into OSC messages sent to a running JUCE plugin or application.

This repository contains the Python side only. The corresponding JUCE synthesizer must implement an OSC receiver capable of handling messages such as:

/setParameter <paramName> <value>

The JUCE implementation is being developed separately in: https://github.com/TYLERSFOSTER/MIDIControl001

---

Project Overview

This project provides:

• A working MCP server (mcp_server.py)
• Tool schemas that define how the LLM may interact (schemas.py)
• Python functions implementing those tools (tools.py)
• An OSC bridge for communicating with the JUCE synth (juce_bridge.py)
• A test suite ensuring correctness (tests/)

The goal is to enable a workflow such as:

1. A speech-to-text system turns spoken commands into text.
2. The text is fed to an LLM.
3. The LLM responds by invoking MCP tools such as setParameter.
4. The MCP server receives the tool call and executes the corresponding Python function.
5. The function sends an OSC message to the JUCE synth, modifying parameters in real time.

This architecture allows expressive, natural-language control of a synthesizer using speech or text.

---

Directory Structure

mcp_synth_controller/
├── pyproject.toml
├── README.md
├── src/
│   └── server/
│       ├── config.py
│       ├── juce_bridge.py
│       ├── mcp_server.py
│       ├── schemas.py
│       └── tools.py
├── tests/
│   ├── test_osc_client_init.py
│   ├── test_list_parameters.py
│   ├── test_set_parameter_message_format.py
│   ├── test_mcp_initialize.py
│   └── test_mcp_tool_call.py
└── examples/
    └── test_send_osc.py

---

Dependency Management

This project uses uv for Python dependency management.
To install dependencies:

uv sync

To run the MCP server:

uv run python src/server/mcp_server.py

To run the test suite:

uv run pytest

---

MCP Server

The MCP server:

1. Handles the MCP initialization handshake.
2. Advertises available tools to the LLM.
3. Receives tool calls from the LLM.
4. Dispatches these calls to the correct Python functions.
5. Returns structured tool results.

The server communicates via STDIN/STDOUT using JSON, following the MCP protocol.

---

Tools

Each tool corresponds to a callable action available to the LLM.
Tools currently implemented:

setParameter(param: str, value: float)

Sets a synthesizer parameter via OSC.

getParameter(param: str)

Placeholder implementation (returns a dummy value).
Real bidirectional communication may be added later.

listParameters()

Returns a list of known parameters.
This can be later expanded to query the JUCE synth dynamically.

---

OSC Bridge

juce_bridge.py uses python-osc to send messages to JUCE.
By default, messages follow this format:

/setParameter <paramName> <value>

The corresponding JUCE OSCReceiver must be implemented.
See the next section.

---

Required JUCE Implementation

To complete this system, the JUCE synth must:

1. Create an OSCReceiver
2. Bind to the same port specified in config.py
3. Add a listener for /setParameter
4. Parse incoming messages and map parameter names to actual JUCE parameters

Example responsibilities on the JUCE side:

• Initialize an OSCReceiver (connect(9001))
• Add a listener for /setParameter
• Extract parameter name and float value from the OSCMessage
• Apply the value using setValueNotifyingHost

The JUCE implementation belongs in the separate repository:

https://github.com/TYLERSFOSTER/MIDIControl001

---

Example OSC Test

To manually verify OSC transmission:

uv run python examples/test_send_osc.py

This sends:

/setParameter testParam 0.42

If your JUCE OSCReceiver is active, it should appear in your debug output.

---

Tests

This project includes a complete pytest suite validating:

• OSC client initialization
• OSC message format
• MCP initialization handshake
• Tool call dispatch logic
• Parameter listing behavior

Run tests with:

uv run pytest

All tests should pass.

---

Future Work

• Implement bidirectional OSC or TCP communication with JUCE
• Add dynamic parameter discovery from JUCE
• Add ramping, smoothing, and modulation utilities
• Integrate with speech-to-text pipeline
• Provide real-time LLM agent control in Claude Desktop or similar

---

License

MIT License.