gemini-vision-mcp

MCP server for describing video, image, audio, and text using the Gemini 3.1 Flash Live API.

Uses a persistent WebSocket session (Google GenAI Live API) to describe media content through FastMCP.

Requirements

• Python 3.11+
• Google API key with Gemini access

Install

uv pip install gemini-vision-mcp

Or run directly with uvx:

uvx gemini-vision-mcp

Configuration

Copy .env.example to .env and fill in your API key:

cp .env.example .env

Variable	Default	Description
GOOGLE_API_KEY	—	Required. Your Google Gemini API key
GEMINI_MODEL	gemini-3.1-flash-live-preview	Gemini model to use (More models to be tested.)
GEMINI_TPM_LIMIT	4000000	Tokens per minute limit for auto-refresh
RATE_LIMIT_WINDOW_SECONDS	60	Rate limit window in seconds
RATE_LIMIT_REQUESTS	10	Max requests per rate window

Usage

Run with uvx:

uvx --no-cache --from . gemini-vision-mcp

This project uses uvx for running and venv for development.

MCP Client Configuration

Add to your MCP client (e.g. opencode opencode.json):

{
  "mcp": {
    "gemini-vision": {
      "environment": {
        "GOOGLE_API_KEY": "your-api-key",
        "GEMINI_MODEL": "gemini-3.1-flash-live-preview",
        "RATE_LIMIT_WINDOW_SECONDS": "60",
        "RATE_LIMIT_REQUESTS": "10"
      },
      "enabled": true,
      "type": "local",
      "cwd": ".",
      "command": [
        "uvx",
        "--no-cache",
        "--from",
        ".",
        "gemini-vision-mcp"
      ]
    }
  }
}

Tools

Tool	Description
vision_from_file	Describe a single file (image, audio, video, or text)
from_file	Alias for vision_from_file
vision_from_files	Describe multiple files in one request
from_files	Alias for vision_from_files
vision_from_files_mixed	Multiple files with different media types
from_files_mixed	Alias for vision_from_files_mixed
vision_from_resource	Describe a URL resource
from_resource	Alias for vision_from_resource
vision_from_resources	Describe multiple URL resources
from_resources	Alias for vision_from_resources
vision_from_resources_mixed	Multiple URLs with different media types
from_resources_mixed	Alias for vision_from_resources_mixed
new_vision	Force a fresh Gemini Live session (loses context)
compact	Summarize history via model, reconnect with summary
gemini_usage	Report current token usage and TPM status

Parameters

• type: Media category — text, image, audio, or video
• prompt: Instructions to the model (what to describe/extract)
• path: Local file path (file tools)
• resource: URL to fetch (resource tools)
• compact: If true, auto-compact history after this request (vision tools only)

Example: describe an image

vision_from_file(type="image", path="/path/to/photo.jpg", prompt="Describe the scene in detail")

Example: extract text from a screenshot

vision_from_file(type="image", path="/path/to/screenshot.png", prompt="Extract all visible text")

Example: batch describe images

vision_from_files(type="image", paths=["/a.jpg", "/b.jpg", "/c.jpg"], prompt="Describe each image")

Example: describe a URL resource

vision_from_resource(type="image", resource="https://example.com/photo.png", prompt="What is in this image?")

Session management

The server maintains a conversation history across tool calls and uses a two-tier TPM management strategy:

• 70% TPM — Auto-compaction: the model summarizes the conversation history, then reconnects with the summary as context. History shrinks, session continues.
• 75% TPM — Hard refresh: reconnects with whatever compacted history exists. Use new_vision to reset completely (loses all context).

You can also trigger compaction manually:

compact()

Or pass compact=true to any vision tool to compact after that request:

vision_from_file(type="image", path="photo.jpg", prompt="Describe this", compact=true)

Call new_vision to force a clean reset (no history carried over). Call gemini_usage to inspect current token consumption and rate limit status.

Architecture

• src/server.py — FastMCP server, tool registration, rate limiting
• src/gemini_live.py — Persistent Gemini Live WebSocket client with auto-reconnect, history tracking, and self-compaction
• src/rate_limiter.py — Sliding window rate limiter
• src/tool/ — Tool implementations (compact, file, files, files_mixed, gemini_usage, new_vision, resource, resources, resources_mixed)
• src/event/ — Background TPM monitoring