MCP 服务器

open-computer-use

Lets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.

README

open-computer-use

A local MCP server that gives AI agents eyes and hands on your desktop. Agents can see an app's interface, click, type, scroll, and drag — through the accessibility layer, without taking over your real mouse and keyboard. Runs entirely on your machine, on macOS, Windows, and Linux.

The tools

Nine core tools, identical across all three platforms:

Tool	What it does
`list_apps`	List running and recently used applications.
`get_app_state`	Read an app's accessibility tree and screenshot.
`click`	Click by `element_index` or screenshot coordinates.
`perform_secondary_action`	Invoke an element's own secondary action.
`scroll`	Scroll an element by pages, or a window by pixel deltas.
`drag`	Drag between two coordinates.
`type_text`	Type text, Unicode-safe, background-first.
`press_key`	Press a key or chord (`ctrl+s`, `return`, `page_up`…).
`set_value`	Set the value of a settable control directly.

Five additional window-level tools — list_windows, get_window, get_window_state, launch_app, activate_window — follow the newer window2 API and are currently available on Windows (macOS and Linux in progress).

Quick start

npm i -g @opensymph/open-computer-use
ocu doctor        # verify the install; macOS: prompts for permissions
ocu call list_apps

macOS 14+ needs Accessibility and Screen Recording granted once. Windows and Linux work out of the box in a signed-in desktop session (Linux desktops need AT-SPI2, which GNOME and friends ship by default).

No npm? Grab the tarball from GitHub Releases and run npm i -g <tarball> — or install it into ZCode as a plugin (below) and it will use whatever local runtime it finds.

Connect your agent

ocu install-codex-mcp      # Codex CLI & Codex App
ocu install-codex-plugin   # Codex App, plugin form
ocu install-claude-mcp     # Claude Code
ocu install-gemini-mcp     # Gemini CLI (--scope user for global)
ocu install-opencode-mcp   # opencode

Any other MCP client — add it manually:

{
  "mcpServers": {
    "open-computer-use": { "command": "open-computer-use", "args": ["mcp"] }
  }
}

ZCode

This repo ships as a ZCode plugin — one install gives you the skill and an auto-connected MCP server:

Install the runtime once (the plugin falls back to it when no local build exists):
```
npm i -g @opensymph/open-computer-use
```
In ZCode, open Settings → Plugin Management → Discover and click +.
Add this repository — the GitHub URL opensymph/open-computer-use, or a local checkout directory.
Find Open Computer Use in the list and click Get.
Start a new session. Check Settings → MCP shows open-computer-use as connected, then just ask: "list the windows on my screen".

To remove it later: Installed tab → Open Computer Use → uninstall.

Agent skill — installable guidance that teaches agents to use these tools well:

npx skills add opensymph/open-computer-use -g -a claude-code --skill open-computer-use -y

Why this one

Non-intrusive by design. Prefer the accessibility API over synthetic input; your real pointer, focus, and foreground app stay put unless you explicitly opt into global input.
Three platforms, one contract. The same tool names, arguments, and results on every OS — agents don't need per-platform branching.
A cursor you can watch. On macOS, actions drive a visible software cursor, so you can follow what the agent is doing.
Scriptable without a client. ocu call runs any tool from your shell and prints MCP-style JSON; --calls chains sequences in one process.
Guardrails built in. Password managers are always refused. Launching apps, stealing focus, and global input injection each sit behind an explicit environment-variable gate.
Signed where it matters. The macOS runtime is Developer ID signed, so granted permissions survive version upgrades.

Platform status

Platform	Runtime	Notes
macOS	Swift	Visual cursor, permission onboarding, `sky_click` background clicks.
Windows	Go, single exe	UI Automation + Win32, process-isolated operations, full window2 API.
Linux	Go, single binary	Native AT-SPI2 over D-Bus, zero runtime dependencies.