mcp-osint

MCP server for Claude Code providing access to OSINT data sources:

• Government - Data.gov, LegiScan, CourtListener, Census Bureau
• Research - OpenAlex, Semantic Scholar, PubMed, CORE
• Corporate - SEC EDGAR (10-K, 10-Q), FRED (economic data)
• Compliance - OpenSanctions (sanctions, PEPs)
• News/Knowledge - GDELT, Wikidata
• Infrastructure - crt.sh (SSL certificates, subdomains)
• Web - Firecrawl (URL scraping with HTML + markdown)

Setup

1. Get API Keys

Required keys:

• LegiScan - legiscan.com/user/register
• CourtListener - courtlistener.com/api
• FRED - fred.stlouisfed.org/docs/api
• Firecrawl - firecrawl.dev

Optional keys (higher rate limits):

• Data.gov - api.data.gov/signup
• Census - census.gov/data/developers
• CORE - core.ac.uk/services/api
• OpenSanctions - opensanctions.org/api

Free (no key): SEC EDGAR, GDELT, Wikidata, crt.sh, OpenAlex, Semantic Scholar, PubMed

2. Install & Build

cd mcp-osint
npm install
npm run build

3. Add to Claude Code

Using the CLI:

claude mcp add -s user -t stdio mcp-osint \
  -e LEGISCAN_API_KEY=your-key \
  -e COURTLISTENER_API_KEY=your-key \
  -e FRED_API_KEY=your-key \
  -e FIRECRAWL_API_KEY=your-key \
  -- node /path/to/mcp-osint/dist/index.js

Or manually add to your MCP settings (~/.claude/settings.json or VS Code settings):

{
  "mcpServers": {
    "mcp-osint": {
      "command": "node",
      "args": ["/path/to/mcp-osint/dist/index.js"],
      "env": {
        "LEGISCAN_API_KEY": "your-key",
        "COURTLISTENER_API_KEY": "your-key",
        "FRED_API_KEY": "your-key",
        "FIRECRAWL_API_KEY": "your-key",
        "POLITE_EMAIL": "you@example.com"
      }
    }
  }
}

Tools

osint_search

Search across 14 OSINT data sources. Returns results with available resources.

Parameter	Type	Required	Description
query	string	Yes	Natural language search query
source	string		Force a specific connector (see table below)
jurisdiction	string		State code (e.g., "CA") or "US"
year	number		Filter to specific year
limit	number		Max results (default: 10)

Examples:

"EPA air quality data California"       → Data.gov
"Michigan renewable energy bill 2024"   → LegiScan
"Brown v. Board of Education"           → CourtListener
"population by county Texas"            → Census
"machine learning medical diagnosis"    → OpenAlex/PubMed
"Apple 10-K filing 2024"                → SEC EDGAR
"GDP quarterly growth rate"             → FRED
"Russian sanctions oligarchs"           → OpenSanctions
"Ukraine conflict news"                 → GDELT
"microsoft.com subdomains"              → crt.sh

osint_preview

Preview a resource's schema and sample data before fetching.

Parameter	Type	Required	Description
resource_id	string	Yes	Resource ID from osint_search
row_limit	number		Sample rows for tabular data (default: 5)
max_bytes	number		Max bytes for text preview (default: 4000)

osint_get

Fetch data from a resource ID or URL. Automatically handles web pages, PDFs, and structured data.

Parameter	Type	Required	Description
target	string	Yes	URL (http/https) or resource_id from osint_search
output_path	string		Path to save binary files (required for PDFs)
question	string		What to extract (e.g., "all data", "key findings")
summarize	boolean		If true with question, returns only relevant content (default: false)
columns	string[]		Specific columns to return (resource_id only)
filters	object[]		Filter conditions (resource_id tabular data only)
limit	number		Max rows for tabular data (default: 100)

Behavior by target type:

Target	Behavior
Web URL	Returns markdown + raw HTML + SHA256 hash via Firecrawl
PDF URL	Downloads to output_path, returns file path + SHA256
Binary URL	Downloads to output_path, returns file path + SHA256
Resource ID	Extracts data via connector with optional filtering

Examples:

osint_get target="https://example.com/article"
osint_get target="https://example.com/paper.pdf" output_path="./downloads/paper.pdf"
osint_get target="pubmed:paper:12345:abstract" question="key findings"

osint_list_sources

List all data sources and their configuration status.

Environment Variables

Variable	Required	Default	Description
LEGISCAN_API_KEY	Yes	-	LegiScan legislative data
COURTLISTENER_API_KEY	Yes	-	CourtListener judicial data
FRED_API_KEY	Yes	-	FRED economic data
FIRECRAWL_API_KEY	Yes	-	Firecrawl web scraping
DATAGOV_API_KEY		-	Data.gov (higher limits)
CENSUS_API_KEY		-	Census Bureau (higher limits)
CORE_API_KEY		-	CORE open access papers
OPENSANCTIONS_API_KEY		-	OpenSanctions compliance data
POLITE_EMAIL		-	Email for polite API usage (OpenAlex, PubMed, SEC)
MCP_DEBUG		true	Debug logging; set to "false" to disable
MCP_LOG_DIR		./logs	Log directory; set to "none" to disable

Connectors

Connector	Source	Data Types	Key Required
data_gov	Data.gov	Datasets, resources	Recommended
legiscan	LegiScan	Bills, votes, sponsors	Yes
courtlistener	CourtListener	Cases, opinions, dockets	Yes
census	Census Bureau	Demographics, statistics	Recommended
openalex	OpenAlex	Papers, authors, citations	No
semantic_scholar	Semantic Scholar	Papers, authors, citations	No
pubmed	PubMed/NCBI	Medical papers, abstracts	No
core	CORE	Open access papers	Recommended
sec_edgar	SEC EDGAR	10-K, 10-Q, company filings	No
fred	FRED	Time series, economic data	Yes
opensanctions	OpenSanctions	Sanctions, PEPs	Yes
gdelt	GDELT	News, global events	No
wikidata	Wikidata	Entities, knowledge graph	No
crt_sh	crt.sh	SSL certificates, subdomains	No

Development

npm install       # Install dependencies
npm run build     # Compile TypeScript
npm run dev       # Watch mode
npm start         # Run server

Testing

# Test connector metadata and identifiers
npx tsx test/test-all-connectors.ts

# Test data retrieval (downloads files)
npx tsx test/test-data-retrieval.ts

# Test Firecrawl HTML+Markdown
npx tsx test/test-firecrawl.ts

# Run comprehensive MCP tool tests
npx tsx test/test-mcp-scenarios.ts

Architecture

src/
├── index.ts              # MCP server entry point
├── types.ts              # Shared types
├── intent.ts             # Query parsing and routing
├── router.ts             # Connector selection
├── logger.ts             # Logging utility
├── cache.ts              # SQLite + file caching
├── retry.ts              # Retry with backoff
└── connectors/
    ├── base.ts           # Base connector class
    ├── data-gov.ts       # Data.gov/CKAN
    ├── legiscan.ts       # LegiScan
    ├── courtlistener.ts  # CourtListener
    ├── census.ts         # Census Bureau
    ├── openalex.ts       # OpenAlex
    ├── semantic-scholar.ts
    ├── pubmed.ts         # PubMed/NCBI
    ├── core.ts           # CORE
    ├── sec-edgar.ts      # SEC EDGAR
    ├── fred.ts           # FRED
    ├── opensanctions.ts  # OpenSanctions
    ├── gdelt.ts          # GDELT
    ├── wikidata.ts       # Wikidata
    ├── crt-sh.ts         # crt.sh
    └── firecrawl.ts      # Firecrawl

License

MIT