Public reference fork — sanitized copy of Elephant's internal investors-mcp for X Engagement Reply Agent candidates. Operational watchlists, scraped data, and internal profiles are excluded. See docs/public-reference.md.

Investors MCP Server

A Next.js-based MCP (Model Context Protocol) server that provides semantic search over investor content using Upstash Vector and Vercel Blob storage.

Uses mcp-handler

Features

• Semantic Search: Query investor content using natural language via queryInvestorContent tool
• Content Ingestion: Add new paragraphs via addInvestorParagraph tool (authenticated)
• Author Filtering: Optionally filter results by investor/author name
• Temporal Filtering: Optionally filter by publication date (dateFrom/dateTo), original content creation datetime (contentCreatedAtFrom/contentCreatedAtTo), and indexed record creation datetime (createdAtFrom/createdAtTo)
• RAG-Ready: Returns full document content from Vercel Blob for retrieval-augmented generation
• Streamable HTTP Transport: Built for modern MCP clients

Deploy to Vercel

Quick Deploy Steps

1. Deploy the project to Vercel via the button above or by connecting your repo

2. Add Upstash Vector from Vercel Marketplace:

   • Go to your project's Storage tab in Vercel dashboard
   • Click Connect Store → Create New → Upstash Vector
   • Important: Create an index with 3072 dimensions (for text-embedding-3-large)
   • This automatically sets UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN

3. Add Vercel Blob storage:

   • In the Storage tab, click Connect Store → Create New → Blob
   • This automatically sets BLOB_READ_WRITE_TOKEN

4. Add AI Gateway API Key:

   • Go to Vercel AI Gateway and create an API key
   • In your project's Settings → Environment Variables, add:
     • AI_GATEWAY_API_KEY = your API key

5. Redeploy to pick up the new environment variables

Environment Variables

Variable	Source	Description
AI_GATEWAY_API_KEY	Vercel AI Gateway	Required for embeddings
UPSTASH_VECTOR_REST_URL	Vercel Marketplace (auto)	Upstash Vector endpoint
UPSTASH_VECTOR_REST_TOKEN	Vercel Marketplace (auto)	Upstash Vector auth token
BLOB_READ_WRITE_TOKEN	Vercel Storage (auto)	Vercel Blob access token
MCP_WRITE_TOKEN	Self-generated	Optional. Bearer token for write operations (addInvestorParagraph)
X_BEARER_TOKEN	X Developer Portal	Optional. Enables authenticated X API article/full-text enrichment in ingestion scripts
AUTOMATION_TRIGGER_TOKEN	Self-generated	Bearer token for /api/automation/monitor-x trigger endpoint
CRON_SECRET	Self-generated	Optional. If set, Vercel Cron can authenticate to trigger endpoint
ASANA_ACCESS_TOKEN	Asana	Required to create tasks from monitored posts
ASANA_PROJECT_GID	Asana	Project where monitored-post tasks are created
ASANA_ASSIGNEE_GID	Asana	Optional default assignee for created tasks
ASANA_ARTICLE_THRESHOLD_ASSIGNEE_GID	Asana	Optional assignee when a post meets the article recommendation threshold
ASANA_SECTION_GID	Asana	Optional section to place newly created tasks into
ASANA_WORKSPACE_GID	Asana	Optional workspace override for task creation
ASANA_SIMILARITY_SCORE_CUSTOM_FIELD_GID	Asana	Optional shared number custom field GID for both parent tasks and recommendation subtasks
ASANA_TASK_SIMILARITY_SCORE_CUSTOM_FIELD_GID	Asana	Optional parent-task-specific number custom field GID; overrides the shared GID for parent tasks
ASANA_SUBTASK_SIMILARITY_SCORE_CUSTOM_FIELD_GID	Asana	Optional subtask-specific number custom field GID; overrides the shared GID for recommendation subtasks
AUTOMATION_WATCH_AUTHORS	Optional list	Optional comma-separated subset of watched authors
RAG_AUTOREPLY_MODEL	Optional	Legacy draft-model override (currently unused by Asana task creation)
X_MONITOR_TWSTALKER_ORIGIN	Optional	TwStalker base URL override (default https://twstalker.com)

When the similarity-score custom field env vars are configured, the Asana create flow writes the raw vector similarity value into Asana:

• parent task: highest recommended-article raw similarity score
• recommendation subtask: that subtask's article raw similarity score

Generating MCP_WRITE_TOKEN

To enable write operations, generate a secure random token:

# Linux/macOS
openssl rand -hex 32

# Or with Node.js
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

Add the generated token as MCP_WRITE_TOKEN in your Vercel Environment Variables. Clients must include this token in the Authorization: Bearer <token> header when calling write tools.

Index Requirements

Your Upstash Vector index must be 3072 dimensions to match openai/text-embedding-3-large embeddings.

Available Tools

echo

Simple echo tool for testing connectivity.

Parameters:

• message (string, required): Message to echo back

queryInvestorContent

Query relevant investor content using semantic search.

Parameters:

• query (string, required): The search query to find relevant investor content
• author (string, optional): Author name to filter results (e.g., "Paul Graham", "Marc Andreessen")
• company (string, optional): Company name to filter results (e.g., "16z", "Y Combinator")
• dateFrom (string, optional): Publication date lower bound, inclusive (YYYY-MM-DD)
• dateTo (string, optional): Publication date upper bound, inclusive (YYYY-MM-DD)
• contentCreatedAtFrom (string, optional): Original content creation datetime lower bound, inclusive (ISO 8601 datetime with timezone)
• contentCreatedAtTo (string, optional): Original content creation datetime upper bound, inclusive (ISO 8601 datetime with timezone)
• createdAtFrom (string, optional): Record creation-time lower bound, inclusive (ISO 8601 datetime with timezone)
• createdAtTo (string, optional): Record creation-time upper bound, inclusive (ISO 8601 datetime with timezone)
• topK (number, optional): Number of results to return (1-20, default: 5)

Response Structure:

{
  "query": "what makes a good startup idea?",
  "author": "Paul Graham",
  "authorNormalized": "paulgraham",
  "company": "Y Combinator",
  "topK": 5,
  "matchCount": 3,
  "matches": [
    {
      "id": "essay-123",
      "score": 0.89,
      "key": "investors/paulgraham/essay-123.json",
      "metadata": {
        "blobKey": "investors/paulgraham/essay-123.json",
        "authorNormalized": "paulgraham",
        "company": "Y Combinator",
        "date": "2012-11-01"
      },
      "blob": {
        "title": "How to Get Startup Ideas",
        "content": "...",
        "author": "Paul Graham",
        "company": "Y Combinator",
        "date": "2012-11-01"
      }
    }
  ]
}

addInvestorParagraph

Add a new investor paragraph to the knowledge base. Requires authentication with write:investors scope.

Authentication: Include Authorization: Bearer <MCP_WRITE_TOKEN> header in your MCP client request.

Parameters:

• content (string, required): The paragraph text content (10-10000 characters)
• author (string, required): The primary author name (e.g., "Paul Graham")
• company (string, optional): Company/organization name (e.g., "16z", "Y Combinator")
• date (string, optional): Publication date in ISO format (YYYY-MM-DD)
• contentCreatedAt (string, optional): Original content creation datetime in ISO 8601 format with timezone
• title (string, optional): Title or context for the paragraph
• authors (string[], optional): List of all authors if multiple
• sourceUri (string, optional): Source URL where the content was originally published

Request Example:

{
  "content": "The best startup ideas are often ones that the founders themselves want...",
  "author": "Paul Graham",
  "company": "Y Combinator",
  "date": "2012-11-01",
  "contentCreatedAt": "2012-11-01T00:00:00.000Z",
  "title": "How to Get Startup Ideas",
  "sourceUri": "https://paulgraham.com/startupideas.html"
}

Response Structure:

{
  "success": true,
  "vectorId": "investors/paulgraham/paragraph-abc123.json",
  "blobKey": "investors/paulgraham/paragraph-abc123.json",
  "blobUrl": "https://xyz.public.blob.vercel-storage.com/investors/paulgraham/paragraph-abc123.json",
  "authorNormalized": "paulgraham",
  "company": "Y Combinator",
  "date": "2012-11-01",
  "contentCreatedAt": "2012-11-01T00:00:00.000Z",
  "createdAt": "2024-01-15T10:30:00.000Z"
}

Error Response (Unauthorized):

{
  "error": "Unauthorized",
  "message": "This tool requires authentication with write:investors scope. Please provide a valid Authorization header with Bearer token."
}

Data Schema

Upstash Vector Metadata

Each vector record should have metadata with:

• blobKey (string, required): Key/URL to retrieve full content from Vercel Blob
• authorNormalized (string, optional): Normalized author name for filtering (lowercase, no spaces)
• company (string, optional): Company/organization name for filtering
• date (string, optional): Publication date in ISO format (YYYY-MM-DD)
• title (string, optional): Title or context
• sourceUri (string, optional): Source URL
• dateTs (number, optional): Publication date as UTC epoch milliseconds (used for range filtering)
• contentCreatedAt (string, optional): Original content creation datetime in ISO 8601 format
• contentCreatedAtTs (number, optional): Original content creation datetime as UTC epoch milliseconds (used for range filtering)
• createdAt (string, optional): When the record was added to the system
• createdAtTs (number, optional): Record creation datetime as UTC epoch milliseconds (used for range filtering)
• kind (string, optional): Type of content (e.g., "paragraph")

Vercel Blob Content

Store your content as JSON in Vercel Blob. The full JSON object is returned in the blob field of each match.

Local Development

# Install dependencies
pnpm install

# Copy environment variables
cp .env.example .env.local
# Then fill in your values in .env.local

# Start development server
pnpm dev

Sample MCP Client

# Test with Streamable HTTP transport
node scripts/test-streamable-http-client.mjs http://localhost:3000

# Test with SSE transport (requires Redis - see below)
node scripts/test-client.mjs http://localhost:3000

Data Output Structure

Generated artifacts are stored outside the repository root:

• data/scraping/: scraper and enrichment outputs (for example scraping-output-*.json)
• data/reports/: ingestion and dedupe reports

Ingesting X Article Text (Authenticated)

For x.com/i/article/... stubs and long-form post content, use authenticated X API enrichment:

export X_BEARER_TOKEN=<your-x-api-bearer-token>

# Option A: fetch + enrich in one step (TwStalker + X API + syndication fallback)
node scripts/fetch-twstalker-profile.mjs \
  --handle ssafavi \
  --author "Soofi Safavi" \
  --company elephant-xyz \
  --output data/scraping/scraping-output-soofi.json

# Option B: enrich an existing scraping-output file
node scripts/enrich-x-article-posts.mjs \
  --input data/scraping/scraping-output-soofi.json \
  --output data/scraping/scraping-output-soofi-enriched.json

Then upload to MCP:

MCP_WRITE_TOKEN=<token> node scripts/bulk-upload-paragraphs.mjs \
  --input data/scraping/scraping-output-soofi-enriched.json \
  --origin https://investors-mcp.vercel.app

Reporting Ingestion Coverage

Use the reporting script to audit who is in the RAG and each author's date coverage:

# Table output
BLOB_READ_WRITE_TOKEN=<token> node scripts/report-rag-ingestion.mjs

# JSON output (with full publication date list per author)
BLOB_READ_WRITE_TOKEN=<token> node scripts/report-rag-ingestion.mjs \
  --json \
  --include-date-list \
  --output data/reports/ingestion-report.json

Tracking Author Replies (e.g., Soofi on X)

Use this script to detect who an author replied to, based on reply-style posts that begin with one or more @handle mentions. The JSON output also includes daily post and reply counts for the author:

BLOB_READ_WRITE_TOKEN=<token> node scripts/report-author-replies.mjs \
  --author "Soofi Safavi" \
  --output data/reports/soofi-replies-report.json

Optional JSON output:

BLOB_READ_WRITE_TOKEN=<token> node scripts/report-author-replies.mjs \
  --author "Soofi Safavi" \
  --json

Reporting X Polling Runs

Use this script to summarize /api/automation/monitor-x runs from Postgres-backed run records, including when polling ran and how many posts were fetched/new:

POSTGRES_URL_NON_POOLING=<url> node scripts/report-x-monitor-runs.mjs \
  --output data/reports/x-monitor-runs-report.json

Optional JSON output:

 POSTGRES_URL_NON_POOLING=<url> node scripts/report-x-monitor-runs.mjs \
  --json

Real-Time X Trigger Automation

GET/POST /api/automation/monitor-x runs the real-time polling pipeline:

1. polls watched X authors for new posts
2. ingests new posts into RAG
3. contextually compares each post against Soofi Safavi's RAG content
4. drafts a response
5. creates an Asana task with source post + draft + context matches

Watchlist source:

• managed_authors in Postgres (author/handle/company registry and polling source of truth)
• data/reports/ingestion-report.json (author/company universe used to fill company context)

Trigger Parameters

• dryRun=true|false (default false): run pipeline without RAG writes or Asana task creation
• backfill=true|false (default false): process historical unseen posts on first run
• ingestToRag=true|false (default true): control RAG ingestion step
• batchSize=<n> (default 5): number of authors processed per trigger
• maxPostsPerAuthor=<n> (default 20): cap per-author post scan
• topK=<n> (default 6): Soofi context matches used for drafting
• author=<author-or-handle>: run only one author

Example (manual dry-run)

curl -H "Authorization: Bearer $AUTOMATION_TRIGGER_TOKEN" \
  "https://investors-mcp.vercel.app/api/automation/monitor-x?dryRun=true&batchSize=2"

Deduplicating Records

Remove duplicate ingested records by (author + sourceUri) while keeping one canonical record:

# Preview only
BLOB_READ_WRITE_TOKEN=<token> \
UPSTASH_VECTOR_REST_URL=<url> \
UPSTASH_VECTOR_REST_TOKEN=<token> \
node scripts/dedupe-rag-records.mjs --report data/reports/dedupe-report.json

# Apply deletions in vector + blob
BLOB_READ_WRITE_TOKEN=<token> \
UPSTASH_VECTOR_REST_URL=<url> \
UPSTASH_VECTOR_REST_TOKEN=<token> \
node scripts/dedupe-rag-records.mjs --apply --report data/reports/dedupe-report.json

Reporting UI

A simple reporting dashboard is available at:

• https://investors-mcp.vercel.app/reporting (production)
• http://localhost:3000/reporting (local)

The page reads data/reports/ingestion-report.json and can also read:

• data/reports/soofi-replies-report.json
• Postgres-backed monitor_runs records for the Polling Runs tab

It provides:

• per-author coverage table
• search and sorting
• CSV export
• Soofi replies reporting

Dashboard Auth

The reporting routes (/authors, /polling, /replies, /reporting) are protected by Google sign-in.

Required environment variables:

• NEXTAUTH_SECRET
• GOOGLE_CLIENT_ID
• GOOGLE_CLIENT_SECRET
• AUTH_ALLOWED_GOOGLE_DOMAIN

Recommended value for AUTH_ALLOWED_GOOGLE_DOMAIN:

• elephant-labs.xyz

For local development, add Google OAuth callback URLs for:

• http://localhost:3000/api/auth/callback/google
• https://investors-mcp.vercel.app/api/auth/callback/google
• polling run reporting (run timestamp, duration, fetched/new posts, and task counts)

Notes for Production on Vercel

• Fluid Compute: Enable Fluid compute for efficient execution
• Max Duration: With Vercel Pro/Enterprise, you can increase maxDuration to 800 in vercel.json
• SSE Transport: Requires Redis. Add Upstash Redis from Vercel Marketplace and set disableSse: false in app/mcp/route.ts

Architecture

┌─────────────┐     ┌──────────────────┐     ┌─────────────────┐
│ MCP Client  │────▶│ app/mcp/route.ts │────▶│ Vercel AI       │
│             │     │                  │     │ Gateway         │
└─────────────┘     └──────────────────┘     │ (embeddings)    │
                            │                └─────────────────┘
                            │
                            ▼
                    ┌──────────────────┐
                    │ Upstash Vector   │
                    │ (semantic search)│
                    └──────────────────┘
                            │
                            ▼
                    ┌──────────────────┐
                    │ Vercel Blob      │
                    │ (full content)   │
                    └──────────────────┘

License

See LICENSE file.