AtlasForge-WebProxy

Self-hosted web search/fetch proxy plus stdio MCP server for AI agents. It provides Brave-backed search, page fetch with retries and browser-like user-agent fallback, old.reddit hardening, response caching, paper PDF fetch, image search/download, and SSRF guards.

Quickstart

./install.sh
mkdir -p "${XDG_CONFIG_HOME:-$HOME/.config}/atlasforge-webproxy"
$EDITOR "${XDG_CONFIG_HOME:-$HOME/.config}/atlasforge-webproxy/config.env"
chmod 600 "${XDG_CONFIG_HOME:-$HOME/.config}/atlasforge-webproxy/config.env"

Set BRAVE_API_KEY in that file. Get a key at https://brave.com/search/api/. Without a key, the proxy falls back to DDG/DDGS where available.

Run directly:

.venv/bin/python -m atlasforge_webproxy.service --host 127.0.0.1 --port 8765
curl http://127.0.0.1:8765/health

Wire MCP into a client:

{
  "mcpServers": {
    "atlasforge-web-proxy": {
      "command": "/path/to/AtlasForge-WebProxy/.venv/bin/python",
      "args": ["-m", "atlasforge_webproxy.mcp_server"],
      "env": {}
    }
  }
}

Config

Priority: environment variables, then ${XDG_CONFIG_HOME:-~/.config}/atlasforge-webproxy/config.env, then defaults. For migration, ~/.config/atlasforge/web-proxy.env is read only when the new file does not exist, and a deprecation warning is logged.

Setting	Default	Purpose
BRAVE_API_KEY	empty	Brave Search API key.
ATLASFORGE_WEB_PROXY_HOST	127.0.0.1	Flask bind host.
ATLASFORGE_WEB_PROXY_PORT	8765	Flask bind port.
ATLASFORGE_WEB_PROXY_URL	host/port above	MCP/client target URL.
ATLASFORGE_WEB_PROXY_CACHE_DIR	XDG cache	Search/fetch/image cache.
ATLASFORGE_WEB_PROXY_PAPER_DIR	XDG cache	Paper PDF/text artifacts.
ATLASFORGE_WEB_PROXY_PROVIDER	auto	auto, brave, duckduckgo, or ddgs.
ATLASFORGE_WEB_PROXY_TIMEOUT_S	20	Service outbound timeout.
ATLASFORGE_WEB_PROXY_CLIENT_TIMEOUT_S	30	Python client timeout.
ATLASFORGE_WEB_PROXY_MCP_TIMEOUT_S	30	MCP-to-service timeout.
ATLASFORGE_WEB_PROXY_SEARCH_TTL_S	1800	Search cache TTL.
ATLASFORGE_WEB_PROXY_FETCH_TTL_S	86400	Page fetch cache TTL.
ATLASFORGE_WEB_PROXY_PAPER_TTL_S	604800	Paper fetch cache TTL.
ATLASFORGE_WEB_PROXY_RETRY_ATTEMPTS	3	Fetch retry attempts.
ATLASFORGE_WEB_PROXY_RETRY_BASE_S	0.5	Retry base delay.
ATLASFORGE_WEB_PROXY_RETRY_AFTER_MAX_S	60	Max Retry-After delay.
ATLASFORGE_WEB_PROXY_RETRY_JITTER	1	Enable retry jitter.
ATLASFORGE_WEB_PROXY_UA_RETRY_ON_403	1	Retry 403 with browser UA.
ATLASFORGE_WEB_PROXY_REDDIT_OLD_FALLBACK	1	Try old.reddit fallback.
WEB_PROXY_ALLOW_DNS_FAIL	0	Legacy DNS fail-open mode.
ATLASFORGE_WEB_PROXY_ENABLE_INVESTIGATION_CAPTURE	0	Optional AtlasForge JSON mirroring.
ATLASFORGE_WEB_PROXY_ENABLE_ATLASFORGE_TOOLS	0	Optional AtlasForge stage tools.

See config.env.example for the full list, including user-agent and size-limit knobs.

systemd

./install.sh creates .venv, installs the package, copies config.env.example if no config exists, installs ~/.config/systemd/user/atlasforge-webproxy.service, and runs systemctl --user daemon-reload.

It does not enable or start the service unless passed --start:

./install.sh --start
systemctl --user status atlasforge-webproxy.service

Tests

.venv/bin/python -m pip install -e ".[test]"
.venv/bin/python -m pytest

Use an unused non-8765 port for live smoke tests:

ATLASFORGE_WEB_PROXY_PORT=8799 .venv/bin/python -m atlasforge_webproxy.service --port 8799