Historical Search Engineering KB MCP

Stores and retrieves completed defect-triage analyses so the orchestrator can find similar past defects when triaging a new ticket. This is the third MCP in the Rapid7 SI Triage pipeline — the persistent memory of the system.

---

What it does

1. Store — after the triage agent completes its analysis, it calls store_analysis with the full triage result: root cause, verdict, fix, affected class/method, error signature, PR link, confidence, and 30+ other metadata fields. If the same ticket is stored again, fields are merged (upsert).
2. Search — at the start of a new triage, search_similar runs hybrid retrieval (dense + BM25 via Reciprocal Rank Fusion) to find historical defects with similar error signatures, root causes, or components.
3. Retrieve — get_analysis returns the full record for a specific ticket.
4. Update — update_analysis patches fields (e.g. adding the PR link after the fix is merged, or upgrading the verdict after investigation).
5. List — list_analyses with optional filters for browsing/dashboarding.
6. Delete — delete_analysis removes a record.
7. Stats — get_kb_stats returns counts by verdict, defect type, component, product, and average resolution time.

Tools

Tool	Purpose
store_analysis(data)	Store or merge a triage result (ticket_id required)
get_analysis(ticket_id)	Retrieve full record by Jira key
search_similar(query, top_k, component?, verdict?, product?, error_type?)	Hybrid search with optional filters
update_analysis(ticket_id, updates)	Patch specific fields
list_analyses(component?, verdict?, product?, limit?)	Filtered listing
delete_analysis(ticket_id)	Remove from KB
get_kb_stats()	Aggregate statistics

---

Metadata schema (AnalysisRecord)

Every analysis stores 30+ fields across these categories:

• Identity — ticket_id, record_id
• Ticket context — summary, description, product, component, version, environment, reporter, priority, labels
• Triage analysis — root_cause, verdict, defect_type, severity_assessed, confidence
• Code location — affected_class, affected_method, affected_file, affected_line
• Error signature — error_type, error_message, error_code, stack_trace_signature
• Fix / resolution — fix_description, fix_type, pr_link, pr_status, commit_hash, branch, files_changed
• Linked artefacts — confluence_page_ids, related_tickets, log_query_used, log_chunks_referenced
• Timestamps — created_at, resolved_at, resolution_time_hours, updated_at
• Agent metadata — analyst, model_used, pipeline_version
• Extension — tags, custom_fields

Verdict taxonomy

code_fix · config_change · infra · dependency · data_issue · user_error · duplicate · wontfix · needs_info · not_a_bug

Defect type taxonomy

null_pointer · class_cast · arithmetic · concurrent_modification · number_format · parse_error · resource_leak · timeout · connection_failure · auth_failure · permission_denied · missing_table · schema_mismatch · config_invalid · thread_pool · memory · other

---

Install & run

cd historical-kb-mcp
python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install -e .
cp .env.example .env    # adjust SI_DATA_DIR if needed

# stdio (for Claude Desktop / local MCP client):
python -m historical_kb_mcp --transport stdio

# HTTP (for MCP Inspector / remote clients):
python -m historical_kb_mcp --transport http
# Serves at http://127.0.0.1:8082/mcp

This server is one of four processes in the SI Triage POC (this + the Jira/Confluence and Log Intelligence MCPs + the orchestrator). For the full multi-service manual startup sequence, .env layout across all four repos, and end-to-end test steps, see orchestrator-agent/si-triage-automation/README.md → "Running the full system manually".

Seed the KB with the 10 Nexpose defects

python -m historical_kb_mcp.seed

This pre-populates the KB with completed triage analyses for all 10 planted defects (root causes, verdicts, fixes, PR links, error signatures), so search_similar returns meaningful results immediately.

MCP Inspector

npx @modelcontextprotocol/inspector
# URL: http://127.0.0.1:8082/mcp

Register with an MCP client (stdio)

{
  "mcpServers": {
    "historical-kb": {
      "command": "python",
      "args": ["-m", "historical_kb_mcp", "--transport", "stdio"],
      "env": { "SI_DATA_DIR": "/absolute/path/to/si_data" }
    }
  }
}

---

How it fits in the pipeline

 Orchestrator (Claude)
   │
   ├──1─▶ jira-confluence-mcp       (get ticket, download logs, search Confluence)
   │      └─writes→ si_data/logs/<ticket_id>/
   │
   ├──2─▶ log-intelligence-mcp      (ingest logs, hybrid query)
   │      └─reads←  si_data/logs/<ticket_id>/
   │      └─writes→ si_data/vector_store/
   │
   ├──3─▶ historical-kb-mcp         ◀── THIS MCP
   │      ├─ search_similar(error_signature)   ← find past defects
   │      └─ store_analysis(triage_result)     ← save when done
   │         └─writes→ si_data/kb/records/ + si_data/kb/vectors/
   │
   └──4─▶ jira-confluence-mcp       (post comment, update ticket)

All three MCPs share SI_DATA_DIR — set it to the same absolute path.

---

Hybrid search

Same approach as the Log Intelligence MCP:

• Dense — sentence-transformers all-mpnet-base-v2 (768-dim) for semantic similarity ("payment failed" ≈ "authorization error")
• BM25 — keyword matching for exact identifiers (class names, error codes, CVEs)
• Reciprocal Rank Fusion — merges the two rankings without fragile score normalisation

The text embedded is the concatenation of error signature fields, root cause, fix, summary, description, and class/method — ordered so the most semantically distinctive fields dominate.

---

Backends (same as log MCP)

Concern	Production	Offline fallback
Embeddings	sentence-transformers all-mpnet-base-v2	Hashed n-gram TF-IDF (numpy)
Vector store	Chroma (persistent)	Numpy .npz + JSON
Sparse search	BM25 (always)	same

EMBED_BACKEND=auto / VECTOR_BACKEND=auto use production backends when importable and fall back otherwise. The offline backends are real (genuine vectors, real cosine search) — not mocks.

---

Tests

pytest                        # in the POC environment
python tests/_runner.py       # offline (when pytest isn't installed)

12 tests: model serialisation, searchable text ordering, BM25 keyword ranking, store/retrieve, update (field merge), search (hybrid + filters), delete, list (with filters), stats aggregation, upsert merge (no duplicates), and the full seed of all 10 defects.

Configuration

See .env.example. Key variables: SI_DATA_DIR, EMBED_BACKEND, VECTOR_BACKEND, RRF_K, DEFAULT_TOP_K, MCP_HTTP_PORT (default 8082).