VideoScan MCP

An MCP (Model Context Protocol) server for comprehensive video analysis — AI-powered transcription, visual frame analysis, and metadata extraction from 1000+ platforms.

Features

• Full video analysis — combines transcription, frame extraction, and metadata in a single call
• AI vision analysis — describes frames and extracts on-screen text (OCR) using GPT-4o, Claude, or Gemini
• Audio transcription — Whisper-based transcription with timestamps and language detection
• Auto-tuning — automatically adjusts frame extraction density, interval, and detail level based on video duration
• Smart frame extraction — scene-change detection, interval sampling, or combined strategy
• Deduplication — perceptual hashing removes near-duplicate frames before analysis
• Metadata extraction — title, duration, chapters, tags, view count, and more without full download
• Multi-provider — OpenAI, Anthropic, and Google vision providers with per-request override
• Caching — persistent cache for downloads, frames, and results to minimize repeat costs
• 1000+ platforms — powered by yt-dlp (YouTube, Vimeo, Twitter/X, TikTok, and more)

Installation

pip install videoscan-mcp

System dependencies

VideoScan requires ffmpeg for video processing and yt-dlp for downloading from URLs.

# macOS
brew install ffmpeg yt-dlp

# Ubuntu/Debian
apt install ffmpeg
pip install yt-dlp

# Windows — install ffmpeg from https://ffmpeg.org/download.html, then:
pip install yt-dlp

Configuration

Copy .env.example to .env and fill in at minimum one API key:

# Vision provider (frame analysis)
VISION_PROVIDER=openai          # openai | anthropic | google
VISION_MODEL=                   # optional — defaults: gpt-4o / claude-sonnet-4-20250514 / gemini-2.0-flash

# Transcription provider
TRANSCRIPTION_PROVIDER=openai   # openai only for now
TRANSCRIPTION_MODEL=whisper-1

# Concurrency
VISION_CONCURRENCY=5            # max parallel vision API calls

# API keys — only need the key for your chosen provider
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...

# Cache
CACHE_ENABLED=true
CACHE_DIR=~/.videoscan/cache
CACHE_MAX_SIZE_GB=5
CACHE_DOWNLOAD_TTL=3600         # 1 hour
CACHE_FRAMES_TTL=86400          # 24 hours
CACHE_RESULTS_TTL=604800        # 7 days

# Safety limits (set to 0 for unlimited)
MAX_VIDEO_DURATION=3600         # 60 minutes in seconds
MAX_DOWNLOAD_SIZE=2147483648    # 2 GB in bytes
MAX_ANALYZED_FRAMES=100
DOWNLOAD_TIMEOUT=300
FRAME_ANALYSIS_TIMEOUT=30

Quick Start — Claude Code

Add VideoScan to your Claude Code settings.json (usually at ~/.claude/settings.json):

{
  "mcpServers": {
    "videoscan": {
      "command": "videoscan",
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Or using uvx without a global install:

{
  "mcpServers": {
    "videoscan": {
      "command": "uvx",
      "args": ["videoscan-mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Once connected, you can ask Claude things like:

• "Analyze this YouTube video: https://youtube.com/watch?v=..."
• "Transcribe the audio from this video file"
• "What's on screen at the 2:30 mark of this video?"
• "Extract frames from this video and describe what you see"

Auto-Tuning

When max_frames and interval are not explicitly set, VideoScan automatically adjusts frame extraction parameters based on video duration to optimize cost and coverage:

Duration	Frames	Interval	Strategy	Detail
< 2 min	~1/sec (dense)	1s	combined	detailed
2–10 min	~40	3s	combined	standard
10–30 min	~30	10s	combined	standard
30–60 min	~30	20s	combined	brief
> 60 min	~20	30s	scene only	brief

Short videos get dense frame extraction for maximum detail, while longer videos use lighter sampling to keep costs down. You can always override by setting max_frames or interval explicitly.

Tool Reference

analyze_video

Full pipeline — transcription + AI frame analysis + metadata in one call. Uses auto-tuning by default.

Parameter	Type	Default	Description
source	string	required	URL or local file path
detail	string	"standard"	Vision level: "brief", "standard", "detailed"
max_frames	int	auto	Maximum frames to analyze — set to -1 (default) for auto-tuning based on duration
threshold	float	0.3	Scene change sensitivity (0.0–1.0)
strategy	string	"combined"	Frame extraction: "scene", "interval", "combined"
interval	int	auto	Seconds between frames — set to -1 (default) for auto-tuning based on duration
skip_frames	bool	false	Skip visual analysis (transcription only)
skip_audio	bool	false	Skip transcription (frames only)
language	string	"auto"	Transcription language or "auto"
provider	string	null	Override vision provider
force_refresh	bool	false	Bypass cache

---

transcribe

Transcribe video or audio to text with timestamps.

Parameter	Type	Default	Description
source	string	required	URL or local file path
language	string	"auto"	Preferred language or "auto" for detection

---

extract_frames

Extract and AI-analyze frames from a video.

Parameter	Type	Default	Description
source	string	required	URL or local file path
max_frames	int	30	Maximum frames to extract (1–100)
threshold	float	0.3	Scene change sensitivity (0.0–1.0)
strategy	string	"combined"	"scene", "interval", or "combined"
interval	int	5	Seconds between frames in interval mode
detail	string	"standard"	Vision analysis level
deduplicate	bool	true	Remove near-duplicate frames via dHash
provider	string	null	Override vision provider
force_refresh	bool	false	Bypass cache

---

analyze_moment

Deep-dive analysis on a specific time range.

Parameter	Type	Default	Description
source	string	required	URL or local file path
start	float	required	Start time in seconds
end	float	required	End time in seconds
dense	bool	true	Extract 1 frame per second in the range
detail	string	"detailed"	Vision analysis level
provider	string	null	Override vision provider
force_refresh	bool	false	Bypass cache

---

get_frame_at

Get a single frame at a specific timestamp, optionally analyzed by AI.

Parameter	Type	Default	Description
source	string	required	URL or local file path
timestamp	float	required	Time in seconds
analyze	bool	true	Run AI vision analysis
provider	string	null	Override vision provider
force_refresh	bool	false	Bypass cache

---

get_metadata

Fetch video metadata without downloading the full video.

Parameter	Type	Default	Description
source	string	required	URL or local file path
include	list	null	Specific fields to return — "title", "duration", "channel", "description", "thumbnail", "chapters", "tags", "view_count". Returns all if omitted.

---

Supported Platforms

VideoScan uses yt-dlp under the hood, which supports 1000+ video platforms including:

• YouTube, YouTube Shorts, YouTube Live
• Vimeo, Dailymotion, Twitch
• Twitter/X, Instagram, TikTok, Facebook
• Reddit, LinkedIn, Pinterest
• BBC iPlayer, CNN, NBC, CBS
• SoundCloud, Bandcamp (audio)
• And hundreds more — see the yt-dlp supported sites list

Local files in any format supported by ffmpeg (mp4, mov, avi, mkv, webm, mp3, wav, etc.) are also supported.

Cost Estimates

Costs depend on your chosen provider and usage:

Operation	Provider	Approx. Cost
Vision analysis	OpenAI GPT-4o	~$0.015 per frame
Vision analysis	Anthropic Claude	~$0.024 per frame
Vision analysis	Google Gemini	~$0.002 per frame
Transcription	OpenAI Whisper	~$0.006 per minute

A typical 10-minute video analyzed with analyze_video (30 frames + transcription) costs approximately $0.45–$0.51 with OpenAI.

Development

git clone https://github.com/guguborbh/videoscan-mcp
cd videoscan-mcp
pip install -e ".[dev]"
pytest

License

MIT License — see LICENSE for details.