MCP 服务器

Yunleng

A camera vision MCP server that lets AI agents capture frames, control camera parameters, recognize gestures, detect objects, and analyze scenes via local vision models.

README

👁️ Yunleng — Give Your AI Agent Eyes

云棱 · A local MCP server that turns your cameras into tools your AI can call.

Your agent can read ten thousand files in a minute. It can browse the whole internet, write a novel, debug a kernel, beat you at chess. But right now, it has no idea what your face looks like.

Yunleng fixes that. Point your laptop camera at yourself and wave — your agent sees it. Set your phone on a tripod facing your desk and your agent watches both angles at once. No cloud, no black box, no API keys. Just a Python process on your machine that hands your AI a pair of eyes.

Built by a student at Northwestern Polytechnical University (西工大) who got tired of AI agents being blind.

Why this exists

The MCP ecosystem has servers for browsers, databases, file systems, git, Slack, you name it. A whole economy of tools that let AI touch the world.

Almost nobody built one for seeing it.

Your phone has three lenses. Your laptop has a webcam. Your AI agent can use exactly zero of them. That gap is what this project fills — a first-class, local-first vision channel for agents, with no cloud round-trip.

What it can do

👀 See multiple cameras at once. Auto-discovers every camera on your machine, and you can add your phone as a second angle over WiFi (IP Webcam / RTSP). Stereo capture with millisecond-aligned timestamps — your agent watches two sides of the room simultaneously.

🎛️ Fine-tune the shot. 20 camera properties exposed: brightness, contrast, exposure, white balance, focus, zoom, and more. Every set is read back and confirmed — no silent failures, no "trust me it worked".

🌙 Take smart photos. Dark scene? It brightens. Overexposed? It pulls it down. White balance corrected before you even finish the sentence. One call, a decent photo comes out the other end.

✋ Read your hands. MediaPipe underneath, seven gestures: open palm, fist, thumbs up, peace, OK, heart, and the one-finger "1". Rule-based and fully interpretable — no training, no black box, every decision traceable to a geometry check.

🎯 Detect objects (optional). YOLO, install-on-demand. Defaults to yolov8n and swaps to your own weights with one env var.

🧠 Understand the scene. Hand a frame to your local Ollama vision model (qwen2.5vl) and get back a plain-language description. Fully offline, fully private.

Install

git clone https://github.com/ChenLaoshiYF/yunleng.git
cd yunleng
python -m venv .venv
.venv\Scripts\activate      # Windows (macOS/Linux: source .venv/bin/activate)
pip install -e .
python scripts/download_models.py

That's it — four dependencies: mcp, opencv-python, numpy, mediapipe.

Optional YOLO object detection:

pip install -e ".[objects]"

Connect to your agent

Claude Desktop example — add to claude_desktop_config.json:

{
  "mcpServers": {
    "camera-vision": {
      "command": "/absolute/path/to/your/.venv/Scripts/python.exe",
      "args": ["-m", "camera_mcp.server"],
      "cwd": "/absolute/path/to/yunleng"
    }
  }
}

Then just talk to your agent:

"Look at the camera and tell me what you see."

"What gesture am I making?"

"Take a photo and save it."

Use your phone as a second eye

Install any IP Webcam app on your phone, join the same WiFi as your computer, start streaming, then hand the URL to your agent:

add_remote_camera url="http://192.168.x.x:8080/video"

Done. Your laptop's blind spot is now covered.

Scene understanding (optional)

analyze_scene needs a local Ollama with a vision model:

Install Ollama: https://ollama.com
Pull a vision model: ollama pull qwen2.5vl

If Ollama isn't running, that one tool returns a clear error message. Everything else keeps working.

Configuration (env vars)

Variable	What it does	Default
`CAMERA_MCP_HAND_MODEL`	Path to the hand-landmark model	project `models/` dir
`CAMERA_MCP_YOLO_MODEL`	Path to a YOLO weights file	`yolov8n.pt`
`CAMERA_MCP_OLLAMA_MODEL`	Ollama vision model name	`qwen2.5vl`
`CAMERA_MCP_OLLAMA_URL`	Ollama server address	`http://localhost:11434`

The 13 tools

Tool	What it does
`list_cameras`	List every camera (local + remote)
`capture_frame`	Grab one frame, return base64 JPEG
`capture_stereo`	Two cameras at once, timestamps aligned
`get_camera_property`	Read all 20 adjustable parameters
`set_camera_property`	Set a parameter, read back the actual value
`smart_capture`	Scene-aware photo (auto brightness + white balance)
`auto_focus`	Software autofocus — picks the sharpest frame
`set_exposure`	Auto / manual exposure control
`add_remote_camera`	Register a phone / IP camera
`remove_remote_camera`	Remove a remote camera
`detect_gestures`	Seven-gesture recognition
`detect_objects`	YOLO object detection (optional)
`analyze_scene`	Describe the frame via local Ollama

Design choices worth knowing

Cameras are opened per-call and released immediately. No lingering handles, no resource leaks on marathon sessions.
Models load once, shared process-wide. The first call is slow, everything after is fast.
Graceful degradation everywhere. No Ollama? No YOLO? Those tools tell you clearly instead of crashing the server.