Nectar AI Facility Agent

An autonomous AI facility operations assistant built for the Nectar Intelligent Facilities Platform challenge.

The system allows facility operators to interact naturally through voice and enables the agent to understand requests, route them to the appropriate workflow, retrieve facility knowledge using RAG, access live facility information through MCP tools, reason over multiple sources, safely perform operational actions, and respond through voice.

---

1. Problem Statement

Facility operators often need to investigate issues by checking multiple sources of information such as:

• Building temperature
• HVAC asset status
• Sensor readings
• Energy consumption
• Active alerts
• Equipment relationships
• Maintenance procedures
• Troubleshooting documentation

A conventional chatbot can answer questions but cannot reliably investigate operational problems using live facility data and internal documentation.

This project addresses that problem by combining:

Voice AI + LLM Routing + RAG + MCP + Agentic Reasoning + Tool Calling + Controlled Actions + Text-to-Speech

The goal is to provide an autonomous facility operations assistant rather than a simple question-answer chatbot.

---

2. Objectives

The system is designed to:

1. Receive natural language or voice input.
2. Convert speech to text.
3. Understand user intent.
4. Route the request to the appropriate agent workflow.
5. Retrieve information from facility documentation.
6. Query live facility data through MCP tools.
7. Combine multiple sources of information.
8. Reason about facility conditions.
9. Safely execute operational actions after confirmation.
10. Convert the final response back to speech.
11. Maintain conversational interaction.
12. Provide grounded responses and avoid unsupported claims.

---

3. High-Level Architecture

                         USER
                          |
                    Voice / Text
                          |
                          v
                 Speech-to-Text
                          |
                          v
                    FastAPI API
                          |
                          v
                 Agent / LLM Router
                          |
          +---------------+----------------+
          |               |                |
          v               v                v
        RAG             MCP             General
       Agent            Tools             LLM
          |               |
          v               v
   Facility Docs      Live Facility
   Knowledge Base         Data
          |               |
          +-------+-------+
                  |
                  v
            Reasoning Layer
                  |
                  v
          Decision / Response
                  |
          +-------+--------+
          |                |
          v                v
     MCP Action           Answer
          |                |
          v                v
 Maintenance Request    Text-to-Speech
                              |
                              v
                         Voice Response

Technology Stack
Backend
Python
FastAPI
Uvicorn
Agent Orchestration
LangGraph
LLM-based routing
Agentic workflow
LLM
Google Gemini
RAG
LangChain
ChromaDB
HuggingFace Embeddings
Semantic retrieval
MCP
Model Context Protocol
MCP Server
MCP Client
Facility operation tools
Voice
SpeechRecognition
Browser Speech Recognition / Speech-to-Text
Browser Text-to-Speech
Frontend
HTML
CSS
JavaScript
Testing
Pytest

Conclusion

The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.

The system combines:

Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing

Conclusion

The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.

The system combines:

Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing

Conclusion

The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.

The system combines:

Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing

The key objective is to demonstrate how an AI agent can move beyond simple question answering and autonomously investigate facility problems using both organizational knowledge and live operational data while maintaining safety around operational actions.