MCP Log Collector - Comprehensive Guide

A production-ready client-server implementation for remote log collection across networks. The server runs on Linux with direct file access, and clients can connect from anywhere using HTTP/HTTPS.

📋 Table of Contents

• Overview
• Quick Start (Automated)
• Architecture
• Installation
  • Automated Setup
  • Manual Setup
• Network Options
• Running the System
• Configuration
• Usage
• Troubleshooting
• Advanced Topics

---

Overview

Key Features

• ✅ No SSH Required - Direct HTTP/HTTPS communication
• ✅ Cross-Network - Works across different networks via Cloudflare Tunnel
• ✅ Production Ready - Systemd services with auto-restart
• ✅ Secure - HTTPS, API keys, DDoS protection
• ✅ Fast - HTTP keep-alive vs SSH handshake overhead
• ✅ Automated - One-command setup and start scripts
• ✅ Multiple Clients - Single server, many clients

Use Cases

• Remote log monitoring for Linux servers
• Security log analysis with Claude Desktop
• Centralized log collection from multiple systems
• Real-time log access across networks

---

Quick Start (Automated)

Want to get started in 5 minutes? Use the automated scripts!

Linux Server (5 minutes)

cd server
sudo bash setup_linux.sh
# Choose option 2 for Cloudflare Tunnel (recommended)
# Enter your domain (e.g., logs.yourdomain.com)
# Authenticate in browser
# Done!

Windows Client (2 minutes)

setup_windows.bat
# Enter server URL from Linux output
# Test connection
# Done!

Start Services

Linux:

sudo bash start_server.sh start

Windows:

start_client.bat

That's it! Jump to Running the System for daily usage.

---

Architecture

System Design

┌─────────────────────┐         HTTP/HTTPS        ┌─────────────────────┐
│  Windows Machine    │ ──────────────────────> │  Linux System       │
│  (Client)           │                            │  (Server)           │
│  - MCP Client       │ <────────────────────── │  - FastAPI Server   │
│  - Claude Desktop   │                            │  - Direct Log       │
│                     │                            │    File Access      │
└─────────────────────┘                            └─────────────────────┘

With Cloudflare Tunnel (Production)

[Windows Client] ──HTTPS──> [Cloudflare CDN] ──Tunnel──> [Linux Server]
    (Anywhere)              (DDoS Protected)   (No ports)   (localhost:8000)

Directory Structure

mcp-2/
├── server/                    # Deploy on Linux system
│   ├── server.py              # FastAPI server with MCP endpoints
│   ├── config.yaml            # Log files configuration
│   ├── .env                   # Server configuration
│   ├── requirements.txt       # Server dependencies
│   ├── log-collector.service  # Systemd service file
│   ├── setup_linux.sh         # Automated setup script
│   ├── start_server.sh        # Start/manage services
│   ├── stop_server.sh         # Stop services
│   └── restart_server.sh      # Restart services
│
├── client.py                  # MCP client (runs on Windows)
├── test_client.py             # Test script for client
├── .env                       # Client configuration (server URL)
├── requirements.txt           # Client dependencies
├── setup_windows.ps1          # Automated Windows setup
├── setup_windows.bat          # Batch wrapper for setup
├── start_client.ps1           # Start client with checks
├── start_client.bat           # Batch wrapper for starting
└── README.md                  # This file

---

Installation

Automated Setup

Recommended for production deployments.

Linux Server Setup

cd server
chmod +x setup_linux.sh
sudo bash setup_linux.sh

Interactive options:

1. Local network only - Server accessible on LAN (port 8000)
2. Production with Cloudflare Tunnel - Accessible from anywhere via HTTPS

What it does:

• ✅ Installs system dependencies (Python, curl, jq)
• ✅ Installs MCP server to /opt/mcp-log-collector
• ✅ Configures systemd service for auto-start
• ✅ Sets up Cloudflare Tunnel (if selected)
• ✅ Configures firewall (if needed)
• ✅ Starts and tests services

Time: 3-7 minutes (depending on Cloudflare authentication)

Windows Client Setup

setup_windows.bat

Or with PowerShell directly:

.\setup_windows.ps1 -ServerURL "https://logs.yourdomain.com" -ApiKey "optional-key"

What it does:

• ✅ Checks Python and pip installation
• ✅ Installs Python dependencies
• ✅ Configures .env file interactively
• ✅ Tests server connection
• ✅ Creates helper scripts

Time: 2-3 minutes

---

Manual Setup

For advanced users or custom configurations.

Linux Server (Manual)

Linux Server (Manual)

1. Copy Server Files

Transfer the server/ directory to your Linux system:

# On Linux system
sudo mkdir -p /opt/mcp-log-collector
cd /opt/mcp-log-collector

# Copy files (server.py, config.yaml, .env, requirements.txt)

2. Install Dependencies

cd /opt/mcp-log-collector
pip3 install -r requirements.txt

3. Configure Server

Edit .env to set server binding (use 0.0.0.0 for network access or 127.0.0.1 for localhost-only):

nano .env

SERVER_HOST=0.0.0.0
SERVER_PORT=8000
# Optional: API_KEY=your-secret-key

4. Configure Log Files

Edit config.yaml to specify which log files to collect:

nano config.yaml

log_files:
  - path: /var/log/audit/audit.log
    description: "Audit logs"
    requires_sudo: true
  
  - path: /var/log/secure
    description: "Security logs"
    requires_sudo: true
  
  - path: /var/log/syslog
    description: "System logs"
    requires_sudo: false

5. Setup Sudo (if needed)

For logs requiring sudo, configure passwordless sudo:

sudo visudo

# Add this line (replace 'username' with actual user)
username ALL=(ALL) NOPASSWD: /usr/bin/tail

6. Test Server Manually

python3 server.py

Server should start on http://0.0.0.0:8000

7. Setup as Systemd Service

# Copy service file
sudo cp log-collector.service /etc/systemd/system/

# Enable and start service
sudo systemctl enable log-collector.service
sudo systemctl start log-collector.service

# Check status
sudo systemctl status log-collector.service

8. Configure Firewall

Allow incoming connections on port 8000 (skip if using Cloudflare Tunnel):

# For UFW (Ubuntu/Debian)
sudo ufw allow 8000/tcp

# For firewalld (RHEL/CentOS)
sudo firewall-cmd --permanent --add-port=8000/tcp
sudo firewall-cmd --reload

Windows Client (Manual)

1. Install Dependencies

cd mcp-2
pip install -r requirements.txt

2. Configure Client

Edit .env with your Linux server URL:

Edit .env with your Linux server URL:

REMOTE_SERVER_URL=http://192.168.1.100:8000
# or for Cloudflare Tunnel:
# REMOTE_SERVER_URL=https://logs.yourdomain.com

3. Test Connection

python test_client.py

4. Run MCP Client

python client.py

---

Network Options

Choose the deployment method that fits your needs:

Option 1: Local Network Only

Best for: Same LAN or VPN connections

Setup:

• Server binds to 0.0.0.0:8000
• Open firewall port 8000
• Client uses http://<server-ip>:8000

Pros: Simple, fast setup
Cons: Only works on same network

Option 2: Cloudflare Tunnel (Recommended)

Best for: Internet access, production deployments

Setup:

# Automated (included in setup_linux.sh option 2)
sudo bash setup_linux.sh

# Or manual - see CLOUDFLARE_TUNNEL.md

Pros:

• ✅ No firewall configuration
• ✅ Free HTTPS
• ✅ DDoS protection
• ✅ Access from anywhere

Cons: Requires Cloudflare account and domain

Option 3: Port Forwarding

Best for: Quick testing without domain

Setup:

• Forward port 8000 on router to Linux server IP
• Client uses http://<public-ip>:8000

Pros: Simple
Cons: Exposes port to internet, no HTTPS

Option 4: ngrok (Testing Only)

Best for: Quick demos

# On Linux
ngrok http 8000
# Use provided URL in Windows client

---

Running the System

Daily Operations

Start Services

Linux Server:

# Automated start/stop/restart
sudo bash start_server.sh start
sudo bash start_server.sh stop
sudo bash start_server.sh restart

# Check status (no sudo needed)
bash status_server.sh

# View live logs
sudo bash start_server.sh logs

Windows Client:

start_client.bat

Or with PowerShell:

.\start_client.ps1

Check Status

Linux:

sudo bash start_server.sh status

Output:

Service Status:

  MCP Server:         ● RUNNING
  Cloudflare Tunnel:  ● RUNNING

  Health Check:       ● HEALTHY
  Server Response:    1.0.0

Access URLs:
  Local:     http://localhost:8000
  Network:   http://192.168.1.100:8000
  Public:    https://logs.yourdomain.com

Windows:

.\test_connection.ps1

Stop Services

Linux:

sudo bash stop_server.sh

Windows: Press Ctrl+C in client terminal

Auto-Start Configuration

Linux - Services auto-start on boot (configured by setup script):

# Verify auto-start is enabled
sudo systemctl is-enabled log-collector.service
sudo systemctl is-enabled cloudflared  # if using Cloudflare

# To disable auto-start
sudo systemctl disable log-collector.service

Windows - Optional auto-start:

1. Task Scheduler:

   • Create Basic Task
   • Trigger: "When I log on"
   • Action: powershell.exe -ExecutionPolicy Bypass -File "C:\path\to\start_client.ps1"

2. Startup Folder:

   • Press Win+R, type shell:startup
   • Create shortcut to start_client.bat

---

Configuration

Server Configuration

Location: /opt/mcp-log-collector/

Environment Variables (.env)

# Server binding (0.0.0.0 for network, 127.0.0.1 for localhost only)
SERVER_HOST=0.0.0.0
SERVER_PORT=8000

# Optional API authentication
API_KEY=your-secret-key-here

Log Files (config.yaml)

log_files:
  - path: /var/log/audit/audit.log
    description: "Linux audit system logs"
    requires_sudo: true
  
  - path: /var/log/secure
    description: "Security/authentication logs"
    requires_sudo: true
  
  - path: /var/log/syslog
    description: "System logs"
    requires_sudo: false
  
  - path: /var/log/apache2/access.log
    description: "Apache access logs"
    requires_sudo: false

After editing:

sudo systemctl restart log-collector.service

Client Configuration

Location: mcp-2/.env

# Remote server URL
REMOTE_SERVER_URL=https://logs.yourdomain.com

# Number of log lines to collect (default)
NUM_LINES=10

# Optional API key (must match server)
API_KEY=your-secret-key-here

---

Usage

Available MCP Tools

When running the client, these tools are available to Claude Desktop:

1. collect_logs

Collects the last N lines from all configured log files.

Parameters:

• num_lines (optional): Number of lines to collect (default: 10)

Example:

{
  "tool": "collect_logs",
  "arguments": {
    "num_lines": 50
  }
}

Response:

=== Log Collection Results ===

File: /var/log/audit/audit.log (Audit logs)
[log entries...]

File: /var/log/secure (Security logs)
[log entries...]

2. list_configured_logs

Lists all log files configured on the server.

Parameters: None

Example:

{
  "tool": "list_configured_logs",
  "arguments": {}
}

Response:

Configured Log Files:
====================

Path: /var/log/audit/audit.log
Description: Audit logs
Sudo Required: True

Path: /var/log/secure
Description: Security logs
Sudo Required: True

Testing the System

Test Connection:

python test_client.py
```bash
python test_client.py

Expected output:

MCP Log Collector Client - Test Script
================================================================================

Testing connection to remote server...
Server URL: https://logs.yourdomain.com
================================================================================
✓ Server is reachable and healthy!

Testing list_configured_logs...
================================================================================
Configured Log Files:

Path: /var/log/audit/audit.log
Description: Linux audit system logs
Sudo Required: True
--------------------------------------------------------------------------------

Testing collect_logs...
================================================================================
✓ Successfully collected logs from 3 files

Integrating with Claude Desktop

Add to your MCP configuration (claude_desktop_config.json):

{
  "mcpServers": {
    "log-collector": {
      "command": "python",
      "args": ["C:/path/to/mcp-2/client.py"]
    }
  }
}

Replace C:/path/to/mcp-2/ with your actual path.

Usage in Claude:

• "Show me the last 50 lines from the audit logs"
• "What log files are configured?"
• "Collect recent security logs"

---

Troubleshooting

Linux Server Issues

Service Won't Start

# Check status
sudo systemctl status log-collector.service

# View recent logs
sudo journalctl -u log-collector.service -n 50

# Check if port is in use
sudo netstat -tulpn | grep 8000

# Test manually
python3 /opt/mcp-log-collector/server.py

Permission Denied on Log Files

# Check file permissions
ls -l /var/log/audit/audit.log

# Test sudo access
sudo tail -n 10 /var/log/audit/audit.log

# Verify sudoers configuration
sudo visudo -c

Cloudflare Tunnel Issues

# Check tunnel status
sudo systemctl status cloudflared
sudo journalctl -u cloudflared -n 50

# Verify tunnel
cloudflared tunnel info log-collector

# Test DNS
nslookup logs.yourdomain.com

# Restart tunnel
sudo systemctl restart cloudflared

Windows Client Issues

Cannot Connect to Server

Checklist:

1. ✅ Server is running: curl http://server-ip:8000/health
2. ✅ Firewall allows connections
3. ✅ REMOTE_SERVER_URL in .env is correct
4. ✅ Network connectivity: ping server-ip

Test connection:

.\test_connection.ps1

Python Not Found

# Check Python installation
python --version

# If not found, install from:
# https://www.python.org/downloads/

# Ensure "Add to PATH" is checked during installation

Module Import Errors

# Reinstall dependencies
pip install -r requirements.txt

# Or install specific package
pip install httpx mcp

Timeout Errors

• Server may be overloaded
• Network latency too high
• Increase timeout in client code if needed

Common Issues

Port 8000 Already in Use

# Find process using port
sudo netstat -tulpn | grep 8000

# Kill process (replace PID)
sudo kill -9 <PID>

# Or change port in .env

Logs Not Updating

# Verify log files exist
ls -l /var/log/audit/audit.log

# Check log rotation
sudo logrotate -f /etc/logrotate.conf

# Restart server
sudo systemctl restart log-collector.service

---

Advanced Topics

API Key Authentication

Enable on Server:

Edit /opt/mcp-log-collector/.env:

API_KEY=your-super-secret-key-here

Update server.py to require API key (see CLOUDFLARE_TUNNEL.md for code).

Enable on Client:

Edit mcp-2/.env:

API_KEY=your-super-secret-key-here

Restart both services.

Multiple Servers

Client can connect to multiple servers:

1. Create separate client directories
2. Configure different .env files
3. Add multiple entries to Claude Desktop config:

{
  "mcpServers": {
    "logs-prod": {
      "command": "python",
      "args": ["C:/path/to/mcp-2-prod/client.py"]
    },
    "logs-staging": {
      "command": "python",
      "args": ["C:/path/to/mcp-2-staging/client.py"]
    }
  }
}

Log Filtering

Server-side: Modify server.py to add filtering by pattern, date, etc.

Client-side: Use Claude to analyze and filter logs after collection.

Performance Optimization

Server:

• Limit log file sizes with logrotate
• Use grep for pre-filtering large logs
• Add caching for frequently accessed logs

Client:

• Adjust NUM_LINES to balance detail vs performance
• Use connection pooling for multiple requests

Monitoring

Server Metrics:

# View service status
sudo bash start_server.sh status

# Monitor resource usage
htop
sudo systemctl status log-collector.service

# Watch logs in real-time
sudo bash start_server.sh logs

Cloudflare Analytics:

• Go to Cloudflare Dashboard
• Zero Trust > Access > Analytics

Backup and Disaster Recovery

Backup Configuration:

# Backup server config
sudo tar -czf mcp-backup.tar.gz /opt/mcp-log-collector

# Backup Cloudflare config
tar -czf cloudflare-backup.tar.gz ~/.cloudflared

Restore:

# Restore server
sudo tar -xzf mcp-backup.tar.gz -C /

# Restart service
sudo systemctl restart log-collector.service

---

Advantages Over Alternatives

vs SSH-Based Approach

Feature	SSH-Based	MCP Log Collector
Connection Speed	⚠️ Slower (handshake overhead)	✅ Fast (HTTP keep-alive)
File Access	⚠️ Remote execution	✅ Direct local access
Multiple Clients	⚠️ One session per client	✅ Single server, many clients
Production Ready	⚠️ Manual execution	✅ Systemd service
Real-time	❌ Not supported	✅ Can add WebSocket
Cross-Network	⚠️ VPN required	✅ Cloudflare Tunnel
Authentication	SSH keys	API keys + HTTPS

vs Manual Log Collection

Feature	Manual	Automated
Time to collect	Minutes	Seconds
Error prone	High	Low
Multiple sources	Tedious	Automatic
Integration	None	Claude Desktop

---

Security Best Practices

Essential Security

1. ✅ Use HTTPS - Cloudflare Tunnel provides free HTTPS
2. ✅ API Keys - Enable API key authentication in production
3. ✅ Firewall - Only open port 8000 if not using Cloudflare
4. ✅ Sudo - Configure passwordless sudo only for specific commands
5. ✅ Log Rotation - Prevent disk space issues

Advanced Security

6. ✅ Cloudflare Access - Add authentication layer (email/Google/GitHub)
7. ✅ Rate Limiting - Configure in Cloudflare Dashboard
8. ✅ IP Whitelist - Restrict client IPs if possible
9. ✅ Audit Logging - Log all access attempts
10. ✅ Regular Updates - Keep dependencies updated

Security Checklist

• [ ] HTTPS enabled (Cloudflare Tunnel or reverse proxy)
• [ ] API keys configured and secured
• [ ] Firewall rules configured
• [ ] Sudo access restricted to specific commands
• [ ] Log files contain no sensitive credentials
• [ ] Services run with minimal privileges
• [ ] Regular security updates applied
• [ ] Access logs monitored

---

Maintenance

Regular Tasks

Weekly:

• Check service status
• Review access logs
• Monitor disk space

Monthly:

• Update Python packages: pip install --upgrade -r requirements.txt
• Review and rotate large log files
• Check for security updates

Quarterly:

• Review log file configuration
• Update Cloudflare Tunnel if used
• Test disaster recovery procedures

Updating

Server Update:

cd /opt/mcp-log-collector
sudo systemctl stop log-collector.service

# Update files (server.py, etc.)
# Update dependencies
pip3 install --upgrade -r requirements.txt

sudo systemctl start log-collector.service

Client Update:

# Update files (client.py, etc.)
# Update dependencies
pip install --upgrade -r requirements.txt

Uninstallation

Linux Server:

# Stop and disable services
sudo systemctl stop log-collector.service cloudflared
sudo systemctl disable log-collector.service cloudflared

# Remove files
sudo rm -rf /opt/mcp-log-collector
sudo rm /etc/systemd/system/log-collector.service

# Remove Cloudflare Tunnel (if used)
cloudflared service uninstall
cloudflared tunnel delete log-collector

Windows Client:

# Just delete the directory
# No system-wide changes were made

---

Additional Resources

Documentation Files

• CLOUDFLARE_TUNNEL.md - Detailed Cloudflare Tunnel setup
• SETUP_GUIDE.md - Automated setup documentation
• QUICK_START.md - Quick reference for daily operations

External Links

• Cloudflare Tunnel Docs
• FastAPI Documentation
• MCP Protocol

Support

For issues or questions:

1. Check this README and additional documentation
2. Review service logs: sudo journalctl -u log-collector.service
3. Test with test_client.py for connection issues
4. Verify configuration files (.env, config.yaml)

---

Future Enhancements

Potential additions for future versions:

• [ ] WebSocket streaming for real-time log monitoring
• [ ] Web dashboard for log visualization
• [ ] Advanced filtering and search capabilities
• [ ] Multiple log format support (JSON, CSV, etc.)
• [ ] Log aggregation from multiple servers
• [ ] Built-in log analysis and anomaly detection
• [ ] Automated alerting for critical events
• [ ] Database storage for historical logs

---

License

This code is provided as-is for log collection purposes.

---

Quick Command Reference

Linux Server Commands

# Setup (one-time)
sudo bash setup_linux.sh

# Daily operations
sudo bash start_server.sh start      # Start services
sudo bash start_server.sh stop       # Stop services
sudo bash start_server.sh restart    # Restart services
sudo bash start_server.sh status     # Check status
sudo bash start_server.sh logs       # View logs

# Configuration
sudo nano /opt/mcp-log-collector/config.yaml  # Edit log files
sudo nano /opt/mcp-log-collector/.env         # Edit settings
sudo systemctl restart log-collector.service  # Apply changes

Windows Client Commands

REM Setup (one-time)
setup_windows.bat

REM Daily operations
start_client.bat          REM Start client
test_connection.ps1       REM Test connection

REM Configuration
notepad .env              REM Edit settings

Troubleshooting Commands

# Linux
sudo systemctl status log-collector.service
sudo journalctl -u log-collector.service -n 50
curl http://localhost:8000/health
sudo netstat -tulpn | grep 8000

# Windows
python test_client.py
python --version
pip list | grep -E "httpx|mcp"

---

🎉 You're all set! Start with the Quick Start section to get running in minutes.