NOEMA-AI

An end-to-end Retrieval-Augmented Generation pipeline that ingests documents and answers queries grounded in their content. It supports three retrieval paths side by side — vector-based (embeddings + vector store), vector-less (keyword/BM25 structured retrieval), and hybrid (Reciprocal Rank Fusion of both) — orchestrated with LangChain and LangGraph, traced with LangSmith, and exposed to external clients via MCP (Model Context Protocol). It also supports multi-turn chat with session memory and streaming answers over Server-Sent Events.

Document  ─▶  Ingestion  ─▶  Chunking  ─▶  ┬─▶  Embeddings ─▶ Vector Store  ─┐
                                            └─▶  BM25 Index (vector-less)    ─┤─▶  LangGraph  ─▶  LLM  ─▶  Answer
                                                                              │      (retrieve node)
Query     ───────────────────────────────────────────────────────────────────┘

Screenshots

<!-- Drop your screenshot images into docs/screenshots/ using these exact filenames and they'll render automatically on GitHub — no other changes needed. Recommended: 1280px+ wide PNGs, dark mode (matches the UI theme). -->

Home page — landing view of the NOEMA UI

Console — ask a question, watch retrieval + generation trace live

Hybrid retrieval mode — fused vector + BM25 results with match source

Chat — multi-turn conversation with session memory

API reference — full endpoint interface view

Streaming answer — tokens arriving live over SSE

Features

• Vector RAG — chunking → embeddings → persistent vector store (Chroma) → similarity search. Ships with a zero-dependency offline hash embedding by default, and a real local semantic embedding option (EMBEDDING_PROVIDER=sentence-transformers) for actual semantic search
• Vector-less RAG — BM25 keyword ranking over raw chunks, no embedding model required
• Hybrid RAG — fuses the vector and BM25 result lists with Reciprocal Rank Fusion (RRF), so a chunk that ranks well in either retriever surfaces, and one that ranks well in both surfaces first
• Multi-turn chat with memory — POST /chat keeps a rolling window of prior turns per session, so follow-up questions ("what about the second one?") resolve against earlier turns instead of being answered in a vacuum
• Streaming answers — GET /query/stream (Server-Sent Events) streams the answer token-by-token instead of waiting for the full response
• Document management — GET /documents lists every indexed source; DELETE /documents/{source} removes it from every retrieval path at once
• Query result caching — a short-lived in-memory TTL cache serves repeated identical questions without re-running retrieval + the LLM; automatically invalidated on ingest/delete
• Persistence across restarts — the BM25 corpus and the ingested-document manifest are saved to PERSIST_DIR and reloaded on startup, so a restart no longer silently wipes what was ingested
• LangGraph pipeline — ingestion, retrieval (vector / vectorless / hybrid), and generation modeled as composable graph nodes, with automatic fallback from vector to vector-less
• LangSmith tracing — every LLM call and retrieval step is traceable/evaluatable when configured
• LLM Gateway / Middleware — a single layer that routes requests, logs, and rate-limits before calls reach the LLM
• Guardrails — input/output checks with normalized (whitespace/zero-width-character-resistant) prompt-injection pattern matching; still pattern-based, not a full classifier — see the note below
• API key auth — optional X-API-Key header enforcement on /ingest, /query/*, /documents/*, and /chat/*, off by default so the app still runs with zero setup
• MCP server — exposes ingest_document, delete_document, list_documents, query_vector_rag, query_vectorless_rag, query_hybrid_rag, and chat_with_memory as MCP tools for external LLM clients
• Docker support — a Dockerfile and docker-compose.yml for one-command containerized deployment with a persistent volume
• 52 unit/API tests covering the chunker, guardrails (including evasion attempts), the BM25 store, hybrid RRF fusion, the query cache, chat memory, and the FastAPI endpoints (including auth enforcement, document deletion, and chat sessions)

Project layout

noema/
├── app/
│   ├── config.py              # environment / settings
│   ├── main.py                 # FastAPI entrypoint (REST API)
│   ├── middleware.py           # LLM gateway / logging / rate-limit middleware
│   ├── guardrails.py           # input & output guardrails
│   ├── cache.py                 # TTL query-result cache
│   ├── chat.py                  # session-based conversational memory
│   ├── ingestion/
│   │   ├── loader.py           # load .txt / .pdf / .docx into raw text
│   │   └── chunker.py          # sliding-window chunking with overlap
│   ├── retrieval/
│   │   ├── embeddings.py       # embedding provider (API or local fallback)
│   │   ├── vector_store.py     # Chroma-backed Vector RAG
│   │   ├── vectorless_store.py # BM25-backed Vector-less RAG (+ persistence)
│   │   └── hybrid.py            # Reciprocal Rank Fusion of vector + BM25
│   ├── llm/
│   │   └── client.py           # LLM client (Groq / LLaMA 3.3 70B, mockable, streaming + history-aware)
│   ├── graph/
│   │   └── pipeline.py         # LangGraph state machine wiring it all together
│   ├── mcp/
│   │   └── server.py           # MCP server exposing the pipeline as tools
│   └── static/                 # web UI (served at "/" by app/main.py)
│       ├── index.html
│       └── assets/
│           ├── style.css
│           └── app.js
├── docs/screenshots/            # drop screenshot images here (see Screenshots above)
├── data/sample_docs/sample.txt # example document to ingest
├── scripts/ingest.py            # CLI: ingest a document from the command line
├── tests/                       # 52 tests: chunker, guardrails, stores, hybrid, cache, chat, API
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
└── .env.example

Setup

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env                # then fill in your keys

Environment variables (.env)

Variable	Purpose
GROQ_API_KEY	Key for Groq-hosted LLaMA 3.3 70B calls (leave blank to use the built-in mock LLM for local testing)
LANGCHAIN_TRACING_V2	true to enable LangSmith tracing
LANGCHAIN_API_KEY	LangSmith API key
LANGCHAIN_PROJECT	LangSmith project name, e.g. noema
VECTOR_DB_PATH	Local path for the persistent Chroma store, default ./chroma_db
EMBEDDING_PROVIDER	hash (default, deterministic offline bag-of-words embedding, zero deps) · sentence-transformers (real local semantic embeddings — pip install sentence-transformers, downloads a small model on first use) · remote (placeholder for a hosted embedding API you wire in)
EMBEDDING_MODEL_NAME	Model name used when EMBEDDING_PROVIDER=sentence-transformers, default all-MiniLM-L6-v2
API_KEY	Leave blank to run the API in open dev mode. Set a value to require a matching X-API-Key header on /ingest and /query/*
PERSIST_DIR	Where the BM25 corpus and ingested-document manifest are saved, default ./storage. Restoring on startup means a restart no longer wipes what was ingested
QUERY_CACHE_TTL_SECONDS / QUERY_CACHE_MAX_ENTRIES	How long identical (mode, query, top_k) requests are served from cache, and how many entries the cache holds before evicting
CHAT_HISTORY_MAX_TURNS / CHAT_SESSION_TTL_SECONDS	How many turns of conversation /chat keeps per session, and how long an idle session is kept before being swept
HYBRID_RRF_K	Reciprocal Rank Fusion constant for /query/hybrid; higher values reduce how much rank position 1 vs 2 matters

Running it

1. Ingest a document

python scripts/ingest.py data/sample_docs/sample.txt

2. Start the API (and the UI)

uvicorn app.main:app --reload

Open http://127.0.0.1:8000 — this now serves the NOEMA web UI directly from the FastAPI app (app/static/). Drop a file on the ingest panel, pick a retrieval mode, and ask a question; the pipeline trace strip lights up as the request moves through guardrails → retrieval → generation, and results show which lane (vector / vector-less) answered along with the retrieved chunks and their scores.

Endpoints:

• GET /health — liveness + uptime
• GET /stats — indexed document list, BM25 chunk count, vector store availability, active chat sessions, query cache stats
• GET /documents — list every indexed source with its chunk count
• DELETE /documents/{source} — remove a document from every retrieval path
• POST /ingest — upload/ingest a document
• POST /query/vector — answer a query using Vector RAG
• POST /query/vectorless — answer a query using Vector-less (BM25) RAG
• POST /query/hybrid — answer a query using Reciprocal Rank Fusion of both
• POST /query/graph — run the full LangGraph pipeline (vector / vectorless / hybrid, with fallback)
• GET /query/stream — Server-Sent Events: streams the answer token-by-token
• POST /chat — multi-turn conversational endpoint with session memory
• GET /chat/{session_id}/history — read back a session's turns
• DELETE /chat/{session_id} — clear a session

The UI is plain HTML/CSS/JS (no build step) so it runs from the same uvicorn process with no extra tooling. If you'd rather host it separately, set window.__DOCUMIND_API_BASE__ in app/static/index.html before app.js loads, and point it at wherever the API is running — CORS is already open on the API side.

3. Run the MCP server (so an MCP-compatible client can call this pipeline as tools)

python -m app.mcp.server

4. Or run it in Docker

docker compose up --build

This builds the image, mounts a named volume at /data for the BM25 corpus/Chroma store to persist across container restarts, and serves the API + UI on http://localhost:8000. Configuration still comes from .env (referenced via env_file in docker-compose.yml).

Notes on the current state

This is a working scaffold, not a black-box demo: the chunker, guardrails, BM25 retrieval, hybrid fusion, query cache, chat memory, and the FastAPI endpoints (52 tests total) run fully offline and are unit-tested. The vector store, LLM client, and MCP server are wired up with real integrations but need a GROQ_API_KEY (and, optionally, chromadb/LangSmith credentials) to hit real APIs. Without keys, the LLM client falls back to a mock responder so the pipeline still runs end-to-end for development and demos.

Known, honestly-stated limitations:

• Embeddings: the default EMBEDDING_PROVIDER=hash is a bag-of-words hash, not real semantic search — good for wiring/testing the pipeline offline, not for retrieval quality. Set EMBEDDING_PROVIDER=sentence-transformers for real semantic embeddings (needs an install + a one-time model download).
• Guardrails are regex/pattern-based prompt-injection checks, not a trained classifier — they catch common phrasings and simple spacing/unicode evasions, not a determined adversary. Swap in a managed guardrails service for anything beyond a demo.
• Persistence is file-based, not a real database: the BM25 corpus and ingested-document manifest are now saved to PERSIST_DIR and reloaded on startup (previously they were in-memory only and a restart silently wiped them), but this is still a JSON file on local disk, not a proper store — it won't behave correctly behind multiple worker processes or hosts. Chat sessions and the rate limiter are still in-memory-only by design (short-lived, session-scoped state that doesn't need to survive a restart).
• API auth is a single shared X-API-Key, not per-user auth — fine for a demo or an internal tool, not a substitute for real auth in a multi-tenant deployment.
• The query cache is a simple in-memory TTL dict, not a distributed cache — fine for a single-process deployment, and it's invalidated on every ingest/delete so it can't serve a stale answer past a document change, but it won't be shared across multiple worker processes.

A dependency-free web UI sits in front of all of this (app/static/, served at /), and several functional bugs were fixed while wiring it up so the demo actually works out of the box: python-multipart was missing from requirements.txt (FastAPI needs it for file uploads — /ingest would 500 on boot without it), the BM25 vector-less path was silently dropping every hit on small corpora (with only one or two chunks indexed, BM25's IDF term can go negative for common words, so the old score <= 0 filter discarded true matches — it now ranks by score but only discards chunks that share no terms with the query), and the frontend was reading stats.vectorless_chunk_count / stats.ingested_documents while /stats returned differently-named fields, so the document list and chunk count in the UI always silently showed zero regardless of what had actually been ingested.