macOS Control MCP Server

A Model Context Protocol (MCP) server for macOS desktop automation using FastMCP and PyAutoGUI. This MCP gives full control to your MacOS desktop. It will work with Linux, and windows but these are not tested. Needless to day this is very dangerous. Run in a container, VM, etc.

Features

Mouse Control

• Move mouse to coordinates
• Click (left, right, middle)
• Double-click
• Scroll
• Drag and drop
• Get mouse position

Keyboard Control

• Type text
• Press individual keys
• Execute hotkey combinations (Cmd+C, Cmd+V, etc.)
• Hold and release keys

Screen Capture & Analysis

• Take screenshots (full screen or regions)
• Get screen resolution
• Locate images on screen
• Get pixel colors at coordinates

Utilities

• Sleep/pause execution
• Configure failsafe and pause settings

Installation

Prerequisites

• macOS
• Python 3.10 or later
• uv package manager

Setup

1. Clone or navigate to the project directory:

   cd ~/macos-control-mcp
   

2. Install dependencies:

   uv sync
   

3. Grant necessary permissions:

   • Accessibility Access: System Settings → Privacy & Security → Accessibility
   • Screen Recording: System Settings → Privacy & Security → Screen Recording

   Add Terminal (or your terminal app) to both of these permissions.

Configure Claude Desktop For development

Add this to your Claude Desktop configuration file (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "macos-control": {
      "command": "uv",
      "args": [
        "--directory",
        "<path_to_folder>/macos-control-mcp",
        "run",
        "macos-control-mcp"
      ]
    }
  }
}

Usage Examples

Once configured in Claude Desktop, you can use natural language to control your Mac:

Mouse Control

• "Move the mouse to position 500, 300"
• "Click at coordinates 100, 200"
• "Double-click at the center of the screen"
• "Right-click at 400, 500"
• "Scroll down 5 clicks"

Keyboard Control

• "Type 'Hello World'"
• "Press the Enter key"
• "Press Cmd+C to copy"
• "Press Cmd+Shift+3 to take a screenshot"

Screen Capture

• "Take a screenshot"
• "What's the screen resolution?"
• "Get the color of the pixel at 100, 100"

Combined Actions

• "Move mouse to 500, 300 and click"
• "Type 'test@example.com' and press Tab"
• "Press Cmd+Space, type 'Safari', and press Enter"

Available Tools

Mouse Tools

• move_mouse(x, y, duration) - Move cursor to coordinates
• click_mouse(x, y, button, clicks, interval) - Click at position
• double_click(x, y) - Double-click
• right_click(x, y) - Right-click
• scroll_mouse(clicks, x, y) - Scroll wheel
• drag_mouse(x, y, duration, button) - Drag to position
• get_mouse_position() - Get current cursor position

Keyboard Tools

• type_text(text, interval) - Type text
• press_key(key, presses, interval) - Press a key
• hotkey(*keys) - Press key combination
• key_down(key) - Hold down a key
• key_up(key) - Release a key

Screen Tools

• get_screen_size() - Get screen resolution
• take_screenshot(region) - Capture screen
• locate_on_screen(image_path, confidence) - Find image
• get_pixel_color(x, y) - Get RGB color

Utility Tools

• sleep(seconds) - Pause execution
• set_failsafe(enabled) - Configure failsafe
• set_pause(duration) - Set action pause

Safety Features

Failsafe

By default, moving the mouse to any screen corner will abort PyAutoGUI operations. This can be disabled with set_failsafe(False).

Pause

A 0.1 second pause is inserted between actions by default. Adjust with set_pause(duration).

Development

Running Tests

uv run pytest

Running the Server Manually

uv run macos-control-mcp

Troubleshooting

"Permission denied" errors

Make sure you've granted Accessibility and Screen Recording permissions to your terminal application.

"Image not found" errors

When using locate_on_screen, try lowering the confidence parameter (default is 0.9).

Server not responding

1. Check Claude Desktop logs: ~/Library/Logs/Claude/
2. Verify the path in your configuration matches your installation
3. Restart Claude Desktop

License

MIT

Credits

Built with:

• FastMCP - Fast MCP server framework
• PyAutoGUI - Cross-platform GUI automation