NexaCorp Agentic RAG with MCP & LangSmith

This project demonstrates a production-ready Agentic AI architecture. It features an advanced Hybrid RAG pipeline exposed as a standard Model Context Protocol (MCP) tool, an autonomous agent powered by Google's Gemini model, and full observability using LangSmith.

Built as part of an Agentic AI roadmap, this project moves beyond simple vector search by separating the retrieval engine (MCP Server) from the reasoning engine (Gemini Client) and tracking the entire flow with enterprise-grade tracing.

Architecture Highlights

• Model Context Protocol (MCP): The RAG pipeline is wrapped in an MCP server. This means the knowledge base acts as an independent tool that any MCP-compliant client (like Claude Desktop, Cursor, or custom agents) can securely query.
• Advanced Hybrid RAG:
  • Dense Retrieval: FAISS with all-MiniLM-L6-v2 embeddings for semantic similarity.
  • Sparse Retrieval: BM25 for precise keyword matching.
  • Reranking: Cross-Encoder (ms-marco-TinyBERT-L-2-v2) to re-score and surface the most relevant chunks.
• Agentic Reasoning Engine: A Python client using LangChain and gemini-2.5-flash that autonomously decides when to query the MCP server and synthesizes the retrieved context into a natural language response.
• Observability (LangSmith): Integrated LangSmith tracing to monitor LLM latency, token usage, tool-call success rates, and the exact context payloads sent between the agent and the MCP server.

System Components

1. server.py: The local MCP Tool Server. Initializes the Hybrid RAG pipeline, ingests sample_data.txt, and exposes a search_knowledge_base tool via stdio.
2. agent_client.py: The agent runtime. Connects to the local MCP server, sets up the Gemini LLM with tool-calling capabilities, and handles the chat loop.
3. rag_pipeline.py: The core Hybrid Search logic.
4. sample_data.txt: The mock enterprise knowledge base (NexaCorp policies, secrets, and IT rules).

Prerequisites

• Python 3.10+
• Google Gemini API Key
• LangSmith API Key (for observability)

Setup & Installation

1. Clone the repository:

   git clone <your-repo-url>
   cd rag_project
   

2. Install dependencies:

   pip install langchain-google-genai mcp langchain langchain-community sentence-transformers faiss-cpu rank_bm25 python-dotenv
   

3. Configure Environment Variables: Create a .env file in the root directory:

   GOOGLE_API_KEY=your_gemini_api_key_here
   
   # LangSmith Observability
   LANGCHAIN_TRACING_V2=true
   LANGCHAIN_PROJECT=nexacorp-rag-agent
   LANGCHAIN_API_KEY=your_langsmith_key_here
   LANGCHAIN_ENDPOINT=https://apac.api.smith.langchain.com # Only if using APAC region
   

Usage

Start the agent client. The client will automatically boot up the MCP server in the background and establish a connection.

python agent_client.py

Wait for the FAISS and SentenceTransformer models to load into memory (~20 seconds). Once the [Agent] Ready! prompt appears, you can ask questions like:

• "What is the guest wifi password?"
• "Who is the lead engineer for Project Crimson Falcon?"
• "Can I deploy code on Friday afternoon?"

Observability & Tracing

<img width="1917" height="767" alt="image" src="https://github.com/user-attachments/assets/c8062628-acb8-4f94-8544-e8986d7bf9f6" />

With LangSmith enabled, every interaction is logged. You can view the traces in your LangSmith dashboard to inspect:

• The raw prompt sent to Gemini.
• The exact JSON-RPC payload of the MCP tool call.
• The text chunks returned by the Hybrid RAG pipeline.
• Latency and token cost per query.