atomic-computer-mcp

A minimal Model Context Protocol (MCP) server that exposes low-level, non-browser desktop primitives:

• Screen capture and foreground-window inspection
• Mouse and keyboard input (Windows)
• Waiting for observable changes + aborting in-flight waits

This repo is not an agent. It does not do task planning, routing, or retries. It only provides deterministic tools for an MCP client (for example: Codex / Claude Code / any MCP host) to call.

What You Can Build With It

• Automating native desktop apps (file dialogs, chat clients, installers, settings windows)
• A "desktop actuator" behind a vision-capable model that decides what to click/type based on screenshots

Tools (Stable API Surface)

Observe:

• observe_screen: captures a screenshot and returns metadata + saved file path
• observe_foreground_window: returns active window title/process/pid/hwnd

Act (Windows only):

• mouse_click: click at (x, y) (Win32 virtual screen pixel coords)
• mouse_drag: drag from start to end (Win32 virtual screen pixel coords)
• keyboard_press: press a key combo (ex: ["Ctrl","V"])
• keyboard_type: type unicode text

Wait / Abort:

• wait_until: waits for conditions such as sleep, window_title_contains, window_process_is, window_changed, screen_changed
• abort: interrupts an in-flight wait_until

Concurrency:

• Only one non-abort tool call is allowed at a time.
• If a second call arrives while one is running, the server returns JSON-RPC error -32000 with message containing tool_busy.

observe_screen Modes + Coordinates (Windows)

By default observe_screen captures the monitor that contains the current foreground window (mode="foreground_monitor"). You can override:

• mode="primary": primary monitor only
• mode="foreground_monitor": monitor containing the foreground window (default)
• mode="all_screens": full virtual desktop across monitors

The response includes capture_rect in Win32 virtual screen coordinates:

{"x":-1920,"y":0,"width":5120,"height":1440}

If your model chooses a click point (sx, sy) on the returned screenshot (pixel coords inside the image), convert it to mouse_click coords via:

• x = capture_rect.x + sx
• y = capture_rect.y + sy

Quickstart (Windows)

1) Install

From source:

git clone git@github.com:Dawdler-G/atomic-computer-mcp.git
cd atomic-computer-mcp

python -m venv .venv
# PowerShell:
#   .venv\Scripts\Activate.ps1
# cmd.exe:
#   .venv\Scripts\activate.bat

pip install -U pip
pip install .

2) Run A Local Self-Test

This prints JSON containing an observe_screen screenshot path and current foreground window info:

skillmirror-atomic-computer-mcp self-test --session-id ses_atomic_test

3) Run The MCP Server (stdio)

skillmirror-atomic-computer-mcp serve-stdio --session-id ses_atomic_a

Notes:

• --session-id is optional. If omitted, a new id is generated.
• You can also set SKILLMIRROR_ATOMIC_SESSION_ID to control the session id.

Using With An MCP Client

This server runs over stdio. Your MCP client should spawn:

skillmirror-atomic-computer-mcp serve-stdio

Most MCP hosts have a config section similar to:

{
  "mcpServers": {
    "atomic-computer": {
      "command": "skillmirror-atomic-computer-mcp",
      "args": ["serve-stdio"]
    }
  }
}

The exact config format depends on your MCP host. Use the snippet above as a shape reference.

Runtime Artifacts (Screenshots + Traces)

You can control where files are written via:

• CLI: --runtime-root <dir>
• Env: SKILLMIRROR_RUNTIME_ROOT=<dir>

By default:

• If you run inside a git checkout of this repo, artifacts go to <repo>/runtime/.
• Otherwise they go to an OS-specific state directory (Windows: %LOCALAPPDATA%\SkillMirror\runtime\).

Outputs are stored under:

• <runtime_root>/sessions/<session_id>/atomic-mcp/ (screenshots)
• <runtime_root>/logs/atomic-mcp-<session_id>.ndjson (tool call traces)

Platform Support

• Windows 10/11: full Observe/Act/Wait/Abort support.
• Non-Windows: observe tools may work depending on desktop environment; act tools return unsupported_platform.

Safety Notes

This server can click/type on the active desktop session. Treat it like a high-privilege local component:

• Run only on machines you control.
• Only connect trusted MCP clients.
• Screenshots and NDJSON traces can contain sensitive UI content (and may include window titles / process paths). Avoid sharing them, and never commit them to git (the default runtime/ folder is gitignored).