Databricks MCP Server

A FastAPI-based MCP (Model Context Protocol) server that provides tools for local file management and Databricks operations.

Features

Local File Management

• Create folders and directories
• Create Python files with content
• Edit existing files

Databricks Operations

• Submit code to Databricks clusters
• Create and run Databricks jobs
• Create Delta Live Tables (DLT) pipelines
• Get job errors and status information

Setup

1. Install Dependencies

pip install -r requirements.txt

2. Environment Configuration

Create a .env file in the project root:

DATABRICKS_HOST=https://e2-demo-west.cloud.databricks.com/
DATABRICKS_TOKEN=YOUR_DAPI_TOKEN

3. Run the Server

uvicorn main:app --reload --host 0.0.0.0 --port 8000

API Endpoints

File Management

Create Folder

POST /file/create_folder
{
  "path": "/path/to/folder"
}

Create Python File

POST /file/create_py_file
{
  "path": "/path/to/file.py",
  "content": "print('Hello, World!')"
}

Edit File

POST /file/edit_file
{
  "path": "/path/to/file.py",
  "content": "print('Updated content')"
}

Databricks Operations

Submit Code

POST /databricks/submit_code
{
  "code": "print('Hello from Databricks!')",
  "cluster_id": "your-cluster-id"
}

Create Job

POST /databricks/create_job
{
  "job_config": {
    "name": "My Job",
    "new_cluster": {
      "spark_version": "11.3.x-scala2.12",
      "node_type_id": "i3.xlarge",
      "num_workers": 1
    },
    "notebook_task": {
      "notebook_path": "/Users/your.email@databricks.com/your_notebook"
    }
  }
}

Run Job

POST /databricks/run_job
{
  "job_id": "your-job-id"
}

Create DLT Pipeline

POST /databricks/create_dlt_pipeline
{
  "pipeline_config": {
    "name": "My DLT Pipeline",
    "storage": "dbfs:/pipelines/storage",
    "clusters": [{"label": "default", "num_workers": 1}],
    "libraries": [{"notebook": {"path": "/Users/your.email@databricks.com/your_dlt_notebook"}}]
  }
}

Get Job Error

POST /databricks/get_job_error
{
  "run_id": "your-run-id"
}

Check Job Status

POST /databricks/check_job_status
{
  "job_id": "your-job-id",
  "run_id": "your-run-id"
}

Claude Desktop Integration

1. Copy Configuration Files

Copy the MCP configuration files to your Claude Desktop configuration directory:

macOS:

cp mcp_server_config.json ~/Library/Application\ Support/Claude/claude_desktop_config.json
cp mcp_tools.json ~/Library/Application\ Support/Claude/mcp_tools.json

Windows:

copy mcp_server_config.json %APPDATA%\Claude\claude_desktop_config.json
copy mcp_tools.json %APPDATA%\Claude\mcp_tools.json

Linux:

cp mcp_server_config.json ~/.config/Claude/claude_desktop_config.json
cp mcp_tools.json ~/.config/Claude/mcp_tools.json

2. Start Your MCP Server

Make sure your Databricks MCP server is running:

uvicorn main:app --host 0.0.0.0 --port 8000

3. Restart Claude Desktop

Restart Claude Desktop to load the new MCP configuration.

4. Use the Tools

Claude Desktop will now have access to all the Databricks and file management tools. You can ask Claude to:

• "Create a folder called 'my_project'"
• "Create a Python file with some Databricks code"
• "Submit this code to my Databricks cluster"
• "Create a DLT pipeline for data processing"
• "Check the status of my job"

Testing

Run the test suite:

pytest test_main.py -v

Error Handling

The server provides detailed error messages and logging. All operations return a consistent response format:

{
  "status": "success|error",
  "message": "Description of what happened",
  "detail": "Error details (if status is error)"
}

Security Notes

• Store your Databricks token securely
• Use environment variables for sensitive configuration
• Consider using Databricks workspace-specific tokens with limited permissions
• The server runs on 0.0.0.0:8000 by default - adjust for your security requirements

Troubleshooting

Common Issues

1. Environment Variables Not Loaded: Make sure you have python-dotenv installed and a .env file in the project root.

2. Databricks Connection Issues: Verify your host URL and token are correct. Test with a simple API call first.

3. Permission Errors: Ensure the server has write permissions for file operations.

4. Port Already in Use: Change the port in the uvicorn command or kill the existing process.

Logs

The server provides detailed logging. Check the console output for debugging information.

---

1. Create the Subfolder and Minimal Server

Directory structure:

mcp_test_server/
  ├── main.py
  └── requirements.txt

main.py:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def root():
    return {"status": "ok", "message": "MCP test server is running"}

requirements.txt:

fastapi
uvicorn

---

2. Install Dependencies

cd mcp_test_server
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

---

3. Test the Server Manually

venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000

• Visit http://localhost:9000/ in your browser. You should see:

  {"status": "ok", "message": "MCP test server is running"}
  

---

4. Update Claude Config

Edit your claude_desktop_config.json to point to this test server:

{
  "mcpServers": {
    "test-mcp": {
      "command": "/full/path/to/mcp_test_server/venv/bin/python",
      "args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"]
    }
  }
}

• Replace /full/path/to/ with the actual path on your system.

---

5. Restart Claude Desktop

• Fully quit and restart Claude Desktop.
• See if you get any connection or server errors.

---

6. What to Look For

• If Claude Desktop can connect, you should see requests in your test server’s terminal.
• If not, and you still see no requests, the issue is with Claude’s config, file location, or local network.

---

7. If You Want, I Can Generate the Files

Let me know if you want me to generate the exact code for main.py and requirements.txt for you!

---

This minimal test will help you isolate whether the problem is with your main server, your config, or Claude Desktop’s ability to reach any local MCP server.
Let me know when you’ve tried this and what you see!

---

Why This Happens

• Claude Desktop starts the server in the directory:
  /Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure
• But your test server code is in:
  /Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/main.py

So, when it runs:

venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000

from the parent directory, there is no main.py in the current directory, so import fails.

---

How to Fix

1. Update the "args" in your config to include the subfolder:

Change from:

"args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"]

to:

{
  "mcpServers": {
    "test-mcp": {
      "command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python",
      "args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"]
    }
  }
}

This tells Uvicorn to look for the main module inside the mcp_test_server package.

---

2. Make mcp_test_server a Python package

Add an empty __init__.py file to mcp_test_server:

touch mcp_test_server/__init__.py

---

3. Update Your Config

Your claude_desktop_config.json should look like:

{
  "mcpServers": {
    "test-mcp": {
      "command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python",
      "args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"]
    }
  }
}

---

4. Restart Claude Desktop

• Save the config.
• Restart Claude Desktop.

---

Why This Works

• uvicorn mcp_test_server.main:app tells Python to look for main.py inside the mcp_test_server package, no matter what the current working directory is.
• The __init__.py file makes mcp_test_server a valid Python package.

---

Summary

• Update the "args" to use mcp_test_server.main:app
• Add __init__.py to mcp_test_server
• Restart Claude Desktop

This should allow Claude Desktop to start and connect to your minimal MCP test server!

Let me know if you see requests in your test server terminal after this change, or if you get a new error.