Phony

AI-Powered Voice Call & SMS MCP Server

A Model Context Protocol (MCP) server that enables Claude and other AI assistants to initiate and manage voice calls and SMS messages using Twilio and OpenAI (GPT-4o Realtime model).

Use this as a base to kick-start your AI-powered voice calling and messaging explorations, save time and develop additional functionality on top of it.

Sequence Diagram

sequenceDiagram
    participant AI as AI Assistant (e.g., Claude)
    participant MCP as MCP Server
    participant Twilio as Twilio
    participant Phone as Destination Phone
    participant OpenAI as OpenAI
    
    AI->>MCP: 1) Initiate outbound call request <br>(POST /calls)
    MCP->>Twilio: 2) Place outbound call via Twilio API
    Twilio->>Phone: 3) Ring the destination phone
    Twilio->>MCP: 4) Call status updates & audio callbacks (webhooks)
    MCP->>OpenAI: 5) Forward real-time audio to OpenaAI's realtime model
    OpenAI->>MCP: 6) Return voice stream
    MCP->>Twilio: 7) Send voice stream
    Twilio->>Phone: 8) Forward voice stream
    Note over Phone: Two-way conversation continues <br>until the call ends

Features

Voice

• Make outbound phone calls via Twilio 📞
• Real-time AI-powered conversations with ElevenLabs Conversational AI or OpenAI GPT-4o Realtime 🎙️
• Real-time language switching during calls 🌐
• In-band DTMF support — AI can autonomously navigate IVR menus
• Pre-built prompts for common calling scenarios (restaurant reservations, etc.) 🍽️

SMS (1-on-1)

• Send/receive SMS and MMS (media via URLs or base64) 💬
• Persistent MongoDB storage with status tracking (queued → sent → delivered → failed)
• Per-contact slug proxy system: incoming messages are mirrored to a configurable list of proxy target numbers (e.g. family members). Those targets can reply with {slug}: msg to route back to the external contact.
• Reconciliation loop (5-min poll, 24-hour lookback) — replays anything the webhook missed

Group MMS (Twilio Conversations API) ⭐

• True native group MMS threading on participants' phones — not fan-out
• Autocreate-driven: groups a user starts with Phony in them show up automatically
• projectedAddress pattern — Phony joins each group as an avatar, externals join as native SMS
• Slug-based reply routing ({9630-grp}: msg from proxy targets posts into the group)
• Proxy targets receive group activity as 1-on-1 SMS when not in the group; skipped when in the group (no native-thread duplication)
• Full participant lifecycle handled (add, remove, message, conversation-removed events)
• Conversation reconciliation and one-shot historical backfill

Infrastructure

• Public URL support via nginx reverse proxy 🔄
• Secure handling of credentials 🔒
• Docker Compose deployment; MongoDB exposed on loopback for host-side tooling

Why MCP?

The Model Context Protocol (MCP) bridges the gap between AI assistants and real-world actions. By implementing MCP, this server allows AI models like Claude to:

1. Initiate actual phone calls on behalf of users
2. Process and respond to real-time audio conversations
3. Execute complex tasks requiring voice communication

This open-source implementation provides transparency and customizability, allowing developers to extend functionality while maintaining control over their data and privacy.

Requirements

• Node.js >= 22
  • If you need to update Node.js, we recommend using nvm (Node Version Manager):

    nvm install 22
    nvm use 22
    

• Twilio account with API credentials
• OpenAI API key
• Public URL with nginx reverse proxy (or similar) for Twilio webhooks

Installation

Manual Installation

1. Clone the repository

   git clone https://github.com/sackio/phony.git
   cd phony
   

2. Install dependencies and build

   npm install
   npm run build
   

Configuration

The server requires several environment variables. Copy .env.example to .env and configure:

cp .env.example .env

Required Environment Variables

• PUBLIC_URL: Your public URL for Twilio callbacks (e.g., https://your-domain.com)
• TWILIO_ACCOUNT_SID: Your Twilio account SID
• TWILIO_AUTH_TOKEN: Your Twilio auth token
• TWILIO_NUMBER: Your Twilio number (in E.164 format)
• OPENAI_API_KEY: Your OpenAI API key

MongoDB Security

MongoDB is configured with authentication enabled. Generate a secure password:

openssl rand -base64 32

Then configure in .env:

• MONGODB_USERNAME: Database username (default: voicecalls_admin)
• MONGODB_PASSWORD: Randomly generated password (use command above)
• MONGODB_DATABASE: Database name (default: phony)
• MONGODB_URI: Full connection string with authentication

⚠️ Important: Never commit .env to version control. The .env.example file contains placeholder values only.

Optional Variables

• RECORD_CALLS: Set to "true" to record calls (optional)
• PORT: Server port (optional, defaults to 3004)
• API_SECRET: Webhook security secret (randomly generated if not set)

Nginx Configuration

You need to configure nginx (or similar reverse proxy) to forward requests to the server. Example nginx location block:

# Phony - Twilio webhooks and WebSocket
location /call/ {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 600s;
    proxy_connect_timeout 600s;
    proxy_send_timeout 600s;
    proxy_pass http://localhost:3004;
}

Note: The /call/ location should not require authentication as Twilio needs to access these endpoints.

Claude Desktop Configuration

To use this server with Claude Desktop, add the following to your configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "phony": {
      "command": "node",
      "args": ["/path/to/your/phony/dist/start-all.cjs"],
      "env": {
        "PUBLIC_URL": "https://your-domain.com",
        "TWILIO_ACCOUNT_SID": "your_account_sid",
        "TWILIO_AUTH_TOKEN": "your_auth_token",
        "TWILIO_NUMBER": "your_e.164_format_number",
        "OPENAI_API_KEY": "your_openai_api_key"
      }
    }
  }
}

After that, restart Claude Desktop to reload the configuration. If connected, you should see Voice Call under the 🔨 menu.

Example Interactions with Claude

Here are some natural ways to interact with the server through Claude:

Voice calls

1. Simple call:

Can you call +1-123-456-7890 and let them know I'll be 15 minutes late for our meeting?

2. Restaurant reservation:

Please call Delicious Restaurant at +1-123-456-7890 and make a reservation for 4 people tonight at 7:30 PM. Please speak in German.

3. Appointment scheduling:

Please call Expert Dental NYC (+1-123-456-7899) and reschedule my Monday appointment to next Friday between 4–6pm.

SMS and group MMS

4. Send an SMS:

Text +1-978-569-9797 that the door dimensions are 30x80.

5. Start a group MMS:

Create a group with +1-978-569-9797 (Murilo) and +1-978-786-2110 (Junior) and ask them to confirm they're coming tomorrow morning.

The tool (phony_create_group_conversation) creates a Twilio Conversation with Phony as the projected address host and sends the first message as a native group MMS to all externals.

6. Post into an existing group:

Reply to group {9630-grp}: confirmed, see you at 8am.

7. Read the group history:

Show me all messages in group {9630-grp} since yesterday.

See docs/GROUP_MMS_ARCHITECTURE.md for the full data flow, Twilio Console configuration, and the participant pattern that makes Group MMS actually render as a single native thread on recipient phones.

Important Notes

1. Phone Number Format: All phone numbers must be in E.164 format (e.g., +11234567890)
2. Rate Limits: Be aware of your Twilio and OpenAI account's rate limits and pricing
3. Voice Conversations: The AI will handle natural conversations in real-time
4. Call Duration: Be mindful of call durations as they affect OpenAI API and Twilio costs
5. Public Exposure: Ensure your nginx configuration properly secures endpoints while allowing Twilio webhook access to /call/ routes

Troubleshooting

Common error messages and solutions:

1. "Phone number must be in E.164 format"

   • Make sure the phone number starts with "+" and the country code

2. "Invalid credentials"

   • Double-check your TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN. You can copy them from the Twilio Console

3. "OpenAI API error"

   • Verify your OPENAI_API_KEY is correct and has sufficient credits

4. "PUBLIC_URL environment variable is required"

   • Make sure you've set the PUBLIC_URL environment variable to your public domain (e.g., https://your-domain.com)
   • Ensure your nginx reverse proxy is properly configured to forward /call/ requests to the server

5. "OpenAI Realtime does not detect the end of voice input, or is lagging."

   • Sometimes, there might be voice encoding issues between Twilio and the receiver's network operator. Try using a different receiver.

Contributing

Contributions are welcome! Here are some areas we're looking to improve:

• Implement support for multiple AI models beyond the current implementation
• Add database integration to store conversation history locally and make it accessible for AI context
• Improve latency and response times to enhance call experiences
• Enhance error handling and recovery mechanisms
• Add more pre-built conversation templates for common scenarios
• Implement improved call monitoring and analytics

If you'd like to contribute, please open an issue to discuss your ideas before submitting a pull request.

Credits

Phony is a fork of the Voice Call MCP Server originally created by the Popcorn team.

We've extended it with:

• SMS messaging capabilities (send, receive, conversation tracking)
• Incoming call handling with proper greeting protocols
• Human-in-the-loop call management with real-time context injection
• React frontend UI for call and SMS management
• Call hold/resume functionality with TTS hold messages
• Enhanced features and comprehensive documentation

License

This project is licensed under the MIT License - see the LICENSE file for details.

Security

Please do not include any sensitive information (like phone numbers or API credentials) in GitHub issues or pull requests. This server handles sensitive communications; deploy it responsibly and ensure all credentials are kept secure.