App Adapter — Control Any Software via Commands

The AI-to-App Bridge. Give any AI agent the power to control real software — and let agents contribute back to a shared ecosystem.

"zoom.send_chat|||text=Hello" → message typed into Zoom
"weibo.trends"                 → real-time trending topics  
"terminal.run|||cmd=dir"       → shell command executed

Why

Current AI agents are trapped in the browser. They can read web pages but can't do things in real apps. Codex's Record & Replay watches your screen — visual, slow, fragile.

App Adapter translates human GUI actions into machine-executable commands. No screen recording. No pixel matching. Just app_do("app.action", params).

And it's a living ecosystem. Agents don't just use adapters — they discover, learn, test, and publish them back to the community.

How It Works

┌──────────┐     ┌──────────────┐     ┌──────────┐
│ Any Agent │ ──▶ │ App Adapter  │ ──▶ │ Real App │
│ (Claude,  │     │ (MCP Server) │     │ (Zoom,   │
│  Codex,   │     │              │     │  WeChat, │
│  Cursor)  │     │ Strategies:  │     │  Excel,  │
│           │     │ • http/api   │     │  etc.)   │
│           │     │ • cdp/browser│     │          │
│           │     │ • uia/desktop│     │          │
│           │     │ • shell/cmd  │     │          │
└──────────┘     └──────────────┘     └──────────┘

Strategy Priority

Strategy	Use When	Speed	Reliability
http	App has a REST API	Fast	High
cdp	App has a web interface	Medium	Medium
uia	Desktop-only app, no API	Slow	Low
shell	CLI tool or script	Fast	High
startfile	Just need to open the app	Instant	High

Prefer http over visual strategies. Always. Visual automation is the last resort.

Quick Start

# One-line install (recommended)
pip install git+https://github.com/codenoob-jacky/app-adapter.git

# Or clone + dev install
git clone https://github.com/codenoob-jacky/app-adapter.git
cd app-adapter
pip install -e .

# With all optional deps (browser + desktop automation):
pip install "app-adapter[all] @ git+https://github.com/codenoob-jacky/app-adapter.git"

# Windows: scripts\install.bat   |   Mac/Linux: bash scripts/install.sh

CLI (install once, use everywhere)

app-adapter                          # Quick overview
app-adapter list                     # List all 26 apps
app-adapter scan                     # Find installable apps
app-adapter do "zoom.send_chat|||text=Hello"
app-adapter prompt --style brief     # Get contribution prompt for your agent
app-adapter server --port 8080       # Start HTTP server
app-adapter server --mcp             # Start MCP stdio server

Python

from app_adapter import app_do, app_list, app_scan, get_agent_prompt

print(app_list())
app_do("terminal.run|||cmd=echo Hello from Python")
print(get_agent_prompt("brief"))  # Inject this into your agent

HTTP Server

app-adapter server --port 8080
  -d '{"tool":"app_do","arguments":{"action":"zoom.send_chat|||text=Hello from AI"}}'

The Agent Ecosystem

This isn't just a tool — agents can discover, learn, and contribute:

# 1. Discover what's on this system
app_scan()                     # → "Found Zoom, Slack, VS Code, Spotify..."

# 2. Register a new app  
app_register("spotify|||Spotify|||https://open.spotify.com|||cdp|||entertainment")

# 3. Teach it actions
app_learn("spotify|||search|||Search tracks|||cdp|||input[data-testid='search-input']|||fill")
app_learn("spotify|||play|||Play/pause|||cdp|||button[data-testid='play-button']|||click")

# 4. Test before sharing
app_test("spotify.search|||query=Bohemian Rhapsody")

# 5. Publish to community
app_publish("spotify")

# Other agents can then discover and install it:
app_search("spotify")          # → Found in registry
app_install("spotify")         # → Downloaded and merged
app_do("spotify.play")

Full API

Function	Description	Example
app_do(spec)	Execute an action	app_do("zoom.send_chat|||text=Hi")
app_list(category?)	List all apps & actions	app_list("social")
app_register(spec)	Register new application	app_register("slack|||Slack|||https://slack.com|||cdp|||communication")
app_learn(spec)	Teach new action to app	app_learn("slack|||send|||Send msg|||cdp|||selector|||fill_then_click")
app_scan()	Discover installed apps	app_scan()
app_search(query?)	Search community registry	app_search("discord")
app_install(name)	Install from registry	app_install("spotify")
app_publish(name)	Publish to registry	app_publish("my_slack_adapter")
app_export(name?)	Export adapter as JSON	app_export("zoom")
app_import(json)	Import adapter from JSON	app_import('{"app":{...}}')
app_test(spec)	Test action with report	app_test("zoom.send_chat|||text=Test")

Built-in Apps (26 apps, 80+ actions)

Category	Apps
Communication	Zoom, Slack, Discord, Teams, Outlook
Social Media	WeChat MP, Weibo, Zhihu, Bilibili, Xiaohongshu, Douyin, X/Twitter
Productivity	Excel, Word, PowerPoint, PDF, Notion, GitHub
Development	VS Code, Terminal, Postman
Design	Figma, Canva
Finance	TradingView
Utility	Browser (navigate, search, screenshot)

Make Your Agent a Contributor

The killer feature: agents that voluntarily improve the ecosystem.

# Get the contribution prompt
app-adapter prompt --style full

# Inject it into your agent's system message.
# Now your agent will:
#   - Discover unregistered apps with app_scan()
#   - Check the registry with app_search()
#   - Register + teach + test + publish autonomously

Every contribution helps every AI agent worldwide. The registry grows exponentially.

Connect to Your Agent

Claude Desktop

{
  "mcpServers": {
    "app-adapter": {
      "command": "app-adapter",
      "args": ["server", "--mcp"]
    }
  }
}

Codex

# codex.yaml or .codex/config.json
mcp:
  app-adapter:
    type: stdio
    command: app-adapter
    args: [server, --mcp]

Cursor

// .cursor/mcp.json
{
  "mcpServers": {
    "app-adapter": {
      "command": "app-adapter",
      "args": ["server", "--mcp"]
    }
  }
}

LangChain / OpenAI Agents SDK

from app_adapter import app_do, get_agent_prompt
# Inject the prompt + wrap tools. See docs/INTEGRATION.md for full examples.

Full integration guide: docs/INTEGRATION.md

Community Registry

The registry/ folder in this repo is the community adapter registry. Anyone (human or agent) can contribute:

1. Create your adapter with app_register() + app_learn()
2. Test with app_test()
3. Export with app_export("your_app")
4. Save the JSON to registry/your_app.json
5. Open a PR to this repo

Agents discover these via app_search() → app_install().

Current Community Adapters

Adapter	Actions
Slack	open, send_message, search, jump_to_channel
Discord	open, send_message, mute_toggle
Spotify	open, search, play, next_track, get_current

Be the next contributor. → registry/

Design Philosophy

Humans use GUI. Machines need commands. The adapter is the translation layer. Don't make AI learn to see — teach apps to listen. Then let agents share what they've learned.

License

MIT