ShopSense — AI Shopping Assistant (RAG + MCP Agent)

A grounded shopping assistant: hybrid (RAG-style) product search, stock-aware recommendations, and honest reviews/pricing — exposed as an MCP server so any agent can use it without guessing.

Why this project exists

I wanted a project in a domain I actually care about (e-commerce), built with the same rigor real AI infrastructure needs: retrieval, grounding, and honest refusal when the data doesn't support an answer.

A shopping agent that confidently recommends an out-of-stock product, states a stale price as current, or invents a star rating is worse than useless — it actively misleads a buyer. This project treats "can I trust this recommendation" as a first-class concern, not an afterthought:

• Hybrid retrieval — BM25 + Vector Search + HNSW — every search runs two independent retrieval signals: BM25 (lexical/keyword ranking) and a TF-IDF vector looked up via an HNSW approximate-nearest-neighbour index (the same indexing approach real vector databases use), then combines and normalizes both into one ranked result. Not a single keyword match — a genuine hybrid RAG retrieval pipeline.
• Structured tracing (Query → Retrieval → Ranking → Response) — every search is logged as a 4-stage trace, Langfuse-style: what was asked, every candidate considered with its BM25/vector scores, how they were ranked, and what was finally returned. Viewable live in the dashboard's Transaction Log.
• Product lineage — "why was this recommended" — a dedicated explainability tool that traces a result back to the actual signals that surfaced it: matched keywords, stock status, and price freshness. Never a generic justification — grounded in the same data the search itself used.
• Stock-aware grounding — every recommendation path checks real stock; an out-of-stock product is flagged, never silently recommended.
• Price freshness — every product tracks when its price was last updated; stale prices (>30 days) are explicitly flagged, not presented as current.
• Honest reviews — ratings are computed only from real review data. Zero reviews means zero reviews, never a fabricated rating.
• Refuses to hallucinate — unknown product ids, empty search results, and missing data all return an explicit "not found" instead of an invented answer. Tested directly in the eval harness.

Why each layer exists (and what breaks without it)

Layer	Why it exists	What breaks without it
BM25 + Vector/HNSW hybrid search	Catches both exact keywords and semantically-similar phrasing	Keyword-only search misses loosely-worded queries; vector-only search misses precise matches
Structured tracing	Makes every ranking decision auditable stage by stage	No way to tell whether a bad result came from retrieval or ranking
Lineage / explainability	Lets an agent justify why it recommended something	A recommendation becomes an unexplainable black box
Stock check	Separates "relevant" from "actually buyable right now"	Agent recommends something the buyer can't actually purchase
Price freshness	Tells the agent whether a price is safe to quote	A 4-month-old price looks identical to today's price
Review grounding	Prevents inventing social proof	A fabricated "4.5 stars" is worse than saying "no reviews yet"
Eval harness	Proves the refusal/grounding behavior actually works	"Looks fine in the demo" with no evidence it holds up

How to run it

Requires Python 3.10+.

pip install -r requirements.txt
./run_demo.sh

This seeds the catalog, runs the eval harness (prints a pass/fail report), and runs the demo client showing 5 realistic shopping questions answered from real catalog data.

Run the web dashboard

python3 app.py

Then open http://localhost:5001 — a live "trust receipt" UI where you can search the catalog and see each result stamped in real time (VERIFIED · IN STOCK / VOID · OUT OF STOCK), click any product for its full trust breakdown (price freshness, honest reviews, and a "Why this result?" lineage explanation), plus "Run Eval Harness" and "View Recent Tool Calls" buttons that execute the real eval suite and structured trace log live in the browser. This is the same catalog_core logic underneath — the UI is just a window into it, nothing is reimplemented or faked.

Run the real MCP server

python3 catalog_server.py

Connect it to Claude Desktop by adding to your MCP config:

{
  "mcpServers": {
    "shopsense": {
      "command": "python3",
      "args": ["/absolute/path/to/catalog_server.py"]
    }
  }
}

Try the live LLM agent (optional)

export ANTHROPIC_API_KEY=sk-ant-...
python3 agent_demo.py "recommend a lightweight gaming laptop under 50000"

Eval results

12/12 passed  (100.0% accuracy)   — eval_harness.py (live demo harness)
17/17 passed                       — pytest tests/ (isolated CI-style suite)

Covers correct search ranking and price filtering, AND explicit refusal behavior: unknown products, out-of-stock recommendations, stale prices, missing descriptions, and zero-review products are all handled honestly instead of guessed. tests/ uses pytest fixtures with a fresh isolated database per test — no shared state between tests, unlike the flat demo-harness script.

Honest trade-offs

This project makes deliberate simplifications (TF-IDF instead of neural embeddings, HNSW at a tiny catalog scale, custom tracing instead of a real observability vendor) to stay fully offline and reviewable in minutes. Every one of those trade-offs, and what a production version would do differently, is written up in DESIGN_DECISIONS.md.

Tech stack

Python, SQLite, rank-bm25 + scikit-learn (TF-IDF) + hnswlib (hybrid retrieval), Flask (web dashboard), the official mcp SDK, Anthropic API (optional, for the live agent demo). Runs fully offline except for the optional agent demo and web fonts.