Scrapling MCP Server

MCP (Model Context Protocol) server that exposes Scrapling web scraping framework as tools for Claude Code and other MCP clients.

Setup Local

Prerequisites

• Python 3.10+
• pip

Install

cd scrapling-mcp
pip install -r requirements.txt
pip install "scrapling[fetchers]"
scrapling install

Run Locally

python server.py

Tools Available

1. scrape_url

Fetch and parse a single URL.

Parameters:

• url (string, required): URL to scrape
• use_stealth (boolean): Use stealth mode to bypass anti-bot (default: false)
• include_html (boolean): Include raw HTML in response (default: false)
• timeout (integer): Request timeout in seconds (default: 30)

Returns: URL content, status, and optional HTML

2. scrape_with_selector

Scrape URL and extract elements using CSS or XPath selector.

Parameters:

• url (string, required): URL to scrape
• selector (string, required): CSS selector or XPath expression
• selector_type (string): "css" or "xpath" (default: "css")
• use_stealth (boolean): Use stealth mode (default: false)

Returns: List of matching elements

3. batch_scrape

Scrape multiple URLs sequentially with rate limiting.

Parameters:

• urls (array, required): List of URLs to scrape
• use_stealth (boolean): Use stealth mode for all requests (default: false)

Returns: Results for all URLs

4. health_check

Check server health and availability.

Returns: Service status and version

Deploy to Render (Free)

Steps

1. Create GitHub repo (if not already)

   cd scrapling-mcp
   git init
   git add .
   git commit -m "Initial scrapling-mcp"
   git push origin main
   

2. Create Render service

   • Go to https://render.com
   • Sign up/login
   • Click "New +" → "Web Service"
   • Connect GitHub repo
   • Select "scrapling-mcp" directory
   • Name: scrapling-mcp
   • Environment: Docker
   • Plan: Free
   • Click "Create Web Service"

3. Wait for deployment (~5 minutes)

4. Get service URL

   • From Render dashboard: https://scrapling-mcp-xxxx.onrender.com

Configure Claude Code

1. Get server URL from Render

2. Update .mcp.json:

   {
     "mcpServers": {
       "scrapling": {
         "type": "url",
         "url": "https://scrapling-mcp-xxxx.onrender.com"
       }
     }
   }
   

3. Restart Claude Code

4. Test:

   • Use /mcp-inspect to see available tools
   • Ask Claude to scrape a URL

Environment Variables (Optional)

• PYTHONUNBUFFERED=true (default on Render)

Notes

• Free Render tier goes to sleep after 15 minutes of inactivity
• First request after sleep takes 30-60 seconds
• Stealth mode requires additional resources (more CPU/memory)
• Batch scraping includes 1-second delays between requests

Troubleshooting

Server won't start:

pip install --upgrade scrapling
scrapling install

Import errors:

pip install -r requirements.txt --upgrade

Timeout on Render:

• May be caused by free tier resource limits
• Try increasing timeout parameter
• Use use_stealth=false first