SSMCP - Super Simple MCP Server

Model Context Protocol (MCP) server providing web search with content extraction.

Why Use SSMCP?

Many AI models, especially local models or certain cloud-based models, don't have built-in web browsing capabilities. SSMCP bridges that gap by providing a simple, self-hosted solution that gives your AI assistant the ability to:

• Search and Read the Web: Let your AI search for current information, read articles, documentation, or any web content
• Extract Clean Content: Automatically converts messy web pages into clean Markdown format that AI models can easily understand
• Access YouTube Transcripts: Extract subtitles from videos with timestamps for analysis or summarization
• AI-Powered Summarization: Optional LLM summarization extracts only relevant information from web pages based on your search query
• Privacy-Focused: Self-hosted solution - your searches and browsing stay on your infrastructure
• Works with Any Model: Compatible with local models (like Qwen, Llama) and cloud APIs (DeepSeek, Claude, GPT) that support MCP

Example Use Cases:

• Research recent news or developments on a topic
• Read and summarize technical documentation
• Analyze current market trends or product reviews
• Extract information from YouTube tutorials or presentations
• Get up-to-date answers that aren't in the model's training data
• Get concise, query-relevant summaries of web content (with LLM summarization enabled)

Features

• Web Search: Search the web and get results with extracted content in Markdown
• Web Fetch: Fetch and extract content from any URL as clean Markdown
• YouTube Subtitles: Extract subtitles and timestamps from YouTube videos
• LLM Summarization (Optional): Use an LLM to summarize and filter search results by relevance to your query
• Powered by: SearXNG for search, Crawl4AI for content extraction, yt-dlp for subtitles
• Simple API: Easy-to-use interface designed for compatibility with both small local models(like Qwen3 30b) and cloud models lacking web capabilities (e.g., DeepSeek 3.2)
• Container Support: Full containerized deployment with Docker Compose

Quick Start

Prerequisites

• Docker
• Docker Compose

1. Clone this repository

git clone https://github.com/antonsokolskyy/SSMCP.git

2. Create .env file

cd SSMCP/
cp .env.example .env

3. Set Up SearXNG

Start and stop the searxng container to generate settings.yml:

docker compose up searxng

Wait until you see:

"/etc/searxng/settings.yml" does not exist, creating from template...

Then press Ctrl+C to stop it.

Edit deploy/docker/searxng_data/settings.yml and add json to the formats list:

# remove format to deny access, use lower case.
# formats: [html, csv, json, rss]
formats:
  - html
  - json

4. Build the SSMCP image

docker compose build

5. Run the Full Stack

docker compose up -d

YouTube Cookies (Optional)

To access age-restricted or private YouTube videos, and to reduce the chances of hitting captchas or IP blocking, you can provide cookies from your browser.

Note: The cookies file must be in Netscape cookie format.

<details> <summary>Generating cookies.txt</summary>

Option 1: Using Browser Extension

Install an extension (like "Get cookies.txt LOCALLY") for your browser and export cookies for youtube.com in Netscape format.

Option 2: Using yt-dlp Binary

yt-dlp --cookies-from-browser chrome --cookies cookies.txt https://www.youtube.com/watch?v=VIDEO_ID

Replace chrome with your browser (firefox, edge, safari, etc.). This automatically exports in Netscape format. </details>

<details> <summary>Using cookies.txt</summary>

Place the generated cookies.txt file in:

deploy/docker/ssmcp/cookies.txt

Security Warning: The cookies file contains authentication tokens and sensitive data. Set appropriate file permissions to prevent unauthorized access:

chmod 600 deploy/docker/ssmcp/cookies.txt

The file will be automatically detected and used by Docker container. </details>

MCP URL

The server uses Streamable HTTP transport. Connect to the MCP server at:

http://{HOST}:{PORT}/mcp

Example:

http://localhost:8000/mcp

Usage with MCP Clients

LM Studio

{
  "mcpServers": {
    "ssmcp": {
      "url": "http://localhost:8000/mcp"
    }
  }
}

Web UI

SSMCP includes an optional Web UI for monitoring requests and responses. This is particularly useful for debugging and inspecting how the model uses the tools.

<details> <summary>Enabling the Web UI</summary>

1. Configure Redis: Open your .env file and uncomment the REDIS_URL line:

REDIS_URL=redis://redis:6379

3. Enable Services: Open docker-compose.yml (and docker-compose.dev.yml if using development mode) and uncomment the redis and ssmcp-ui service blocks.

4. Restart Services:

make build

5. Access the Monitor: Open http://localhost:8081 in your browser. </details>

Tools

web_search

Performs a web search and returns relevant results with extracted content.

Parameters:

• query (str): Search query or keywords to find relevant web content

Returns:

• List of results, each containing:
  • url (str): The webpage URL
  • content (str): Page content in Markdown format

web_fetch

Fetches content from a specified URL and converts it to Markdown.

Parameters:

• url (str): The URL to fetch content from

Returns:

• String containing the page content in Markdown format

youtube_get_subtitles

Gets subtitles/captions from a YouTube video and returns the text content.

Parameters:

• url (str): YouTube video URL to get subtitles from

Returns:

• String containing the subtitles with timestamps in format: [HH:MM:SS] text

How Search Works

SSMCP uses a pipeline to deliver clean content from web searches:

1. Search (SearXNG)

• Queries are sent to a local SearXNG instance
• SearXNG aggregates results from multiple search engines
• Returns a list of URLs with titles and snippets

2. Content Extraction (Crawl4AI)

• Each URL is fetched using headless Chromium browser
• Pages are fully rendered (JavaScript executed, dynamic content loaded)
• Raw content is extracted
• All URLs are processed concurrently

3. Content Filtering

Two-stage filtering extracts clean main content:

1. CSS Selector Filter - Tries selectors (article, main, etc.) to find main content area
2. Residual Junk Filter - Removes UI artifacts (tooltips, duplicate text)

If any filter produces output, the filtered HTML is re-processed through Crawl4AI for cleaner output before markdown conversion.

4. Markdown Conversion

• Converts filtered HTML to clean Markdown
• Removes images and external links

5. Optional: LLM Summarization

When enabled, SSMCP uses an LLM to summarize search results before returning them to your AI model. This reduces the token count in your context window - search results can be 40k-60k tokens, but summaries significantly reduce that.

Note: LLM summarization is disabled by default. Enable it via environment variables if needed (see .env.example).

Development

All development tasks are performed inside the Docker container. Nothing needs to be installed on the host machine except Docker and Docker Compose.

Available Make Commands

Run make help to see all available commands:

<details> <summary>Development Workflow</summary>

0. Enable Development mode:
   Open .env and uncomment the line

   COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml
   

1. Start the services:

   make build
   

2. Open a shell in the container:

   make shell
   

   Inside the shell, you can run any command:

   uv run python -m ssmcp.server
   uv run pytest -v
   

3. Edit code on your host machine - Changes are automatically reflected in the container via volume mounts

4. Run tests:

   make test
   

5. Check code quality (lint + type-check):

   make check
   

6. Restart or rebuild if needed:

   make restart
   make rebuild
   

7. Stop services:

   make down
   

</details>

Configuration

All configuration is managed through environment variables. See .env.example for available options

OpenWebUI OAuth Authentication

SSMCP supports OAuth authentication for use with OpenWebUI and any OIDC-compliant identity provider.

How OpenWebUI OAuth Works

When you select OAuth in OpenWebUI for an MCP server:

• OpenWebUI forwards the system user's OAuth access token in the Authorization: Bearer <token> header
• The token is a JWT containing user information from the identity provider
• SSMCP validates the token and extracts the user identifier from the sub claim

<details> <summary>Enabling OAuth</summary>

To enable OAuth authentication, set the following environment variables in your .env file:

# Enable OAuth authentication
OAUTH_ENABLED=true

# JWKS endpoint URL for your identity provider's public keys
# Examples:
#   Authentik: https://authentik.example.com/application/o/my-app/jwks
OAUTH_JWKS_URL=https://your-idp.example.com/path/to/jwks

# Issuer URL for token issuer verification
# Must match the 'iss' claim in JWT tokens
# Examples
#   Authentik: https://authentik.example.com/application/o/my-app
OAUTH_ISSUER=https://your-idp.example.com/application/o/my-app

# Open WebUI client ID for token audience verification
OAUTH_CLIENT_ID=your-openwebui-client-id

</details>

<details> <summary>Token Validation</summary>

When OAuth is enabled, SSMCP validates:

1. JWT Signature: Verifies the token signature using the identity provider's public keys from the JWKS endpoint
2. Issuer: Validates the iss claim matches OAUTH_ISSUER
3. Expiration: Validates the exp claim - rejects expired tokens
4. Audience: Validates the aud claim matches OAUTH_CLIENT_ID
5. Subject: Requires the sub claim (contains user identifier) </details>

<details> <summary>OpenWebUI Configuration</summary>

In OpenWebUI, configure the MCP server with:

• Type: MCP Streamable HTTP
• URL: Your SSMCP server URL (e.g., http://ssmcp:8000/mcp)
• Auth: OAuth
• The system will automatically forward the user's OAuth token </details>

<details> <summary>Supported Identity Providers</summary>

SSMCP works with any OIDC-compliant identity provider that:

• Provides a JWKS endpoint for public key distribution
• Issues JWT access tokens with RS256 signing
• Includes standard claims (sub, aud, exp, iss) </details>

License

Apache License 2.0 - see LICENSE file for details.