rag-mcp

Hybrid RAG over Claude Code and Hermes session history on this box, plus an SFT export + LoRA fine-tuning pipeline for distilling frontier-model sessions into small local models.

One persistent service (port 8004, systemd user unit rag-mcp) provides:

• MCP (streamable HTTP, https://rag.mcp.tyrel.cloud/mcp): rag_search, rag_ingest_text, rag_status — registered in both Claude Code (~/.claude.json) and Hermes (~/.hermes/config.yaml → rag-mcp).
• REST for hooks:
  • POST /api/ingest — enqueue a session transcript (202, background worker)
  • POST /api/context — hybrid search, returns a provenance-tagged context block
  • GET /health — store stats + queue depth

How data flows

Claude Code SessionEnd  ─┐
Hermes on_session_end   ─┼─► POST /api/ingest ─► queue ─► parse ─► junk filter
                         │      (pending_jobs table survives restarts)
                         │   ─► distill (llama.cpp :9090, Qwen3.6-35b-1M) ─► chunk
                         │   ─► scrub secrets ─► embed (llama.cpp :9090, qwen3-embedding-0.6b)
                         │   ─► SQLite: chunks + FTS5 + sqlite-vec
Claude Code UserPromptSubmit ─► POST /api/context ─► vec KNN + BM25 → RRF → boosts
                                 → dedupe (per-session `injected` cache) → inject

Store: ~/.local/share/rag-mcp/rag.db (WAL). Chunk kinds: summary, fact, error_fix, user_prompt, assistant_answer, code, manual.

Hooks (all fail-open — service down means silence, never a blocked prompt):

• ~/.claude/hooks/claude-rag-context.sh (UserPromptSubmit), claude-rag-ingest.sh (SessionEnd) — wired in ~/.claude/settings.json.
• ~/.hermes/agent-hooks/hermes-rag-ingest.sh — wired in ~/.hermes/config.yaml hooks: block, allowlisted in ~/.hermes/shell-hooks-allowlist.json.

CLI

rag-mcp serve                     # what systemd runs
rag-mcp status
rag-mcp backfill --source all     # seed from existing history (--no-distill for speed)
rag-mcp ingest <path> --source claude
rag-mcp export --out data/sft-$(date +%Y%m%d) --min-turns 3
rag-mcp reembed --model <router-model-id> --dim <n>   # switch embedding models

Fine-tuning (training/)

1. rag-mcp export --out data/sft-YYYYMMDD → train_tools.jsonl (full tool trajectories), train_chat.jsonl (text-only), val splits, stats.json. Quality gates: frontier (claude*) model, ≥N user turns, <30% tool errors, no failure endings, dedup; secrets redacted.
2. training/run_container.sh python train_lora.py --base Qwen/Qwen3.5-9B \ --data /ws/data/sft-YYYYMMDD/train_tools.jsonl --run-name my-run — bf16 LoRA via TRL/PEFT inside the NGC pytorch container (aarch64; no bitsandbytes). Smoke: --base Qwen/Qwen3-0.6B --max-steps 2.
3. training/merge_and_export.sh runs/my-run Qwen/Qwen3.5-9B tyrel-tuned-qwen — merge → GGUF (~/llama.cpp) → Q4_K_M → preset in ~/models/presets.ini, served by the llama.cpp router on :9090.

Config (env)

PORT (8004) · RAG_DB · RAG_EMBED_URL/RAG_EMBED_MODEL (llama.cpp router /v1/embeddings, qwen3-embedding-0.6b via ~/models/presets.ini, dim recorded in meta; mismatch refuses startup) · RAG_DISTILL_URL/RAG_DISTILL_MODEL (llama.cpp :9090, Qwen3.6-35b-1M-P1-MTP-NGRAM) · RAG_CONTEXT_TOKENS (1500) · RAG_MIN_SESSION_CHARS (700).

Dev

uv sync && .venv/bin/python -m pytest tests/ -q