Extended RLM

extended-rlm is a self-learning knowledge layer for local LLM chat, built as a Model Context Protocol (MCP) server. It has two faces:

1. The engine (this repository): a Python MCP server that answers questions from plain Markdown files in a workspace, writes new knowledge back every turn, and stamps every answer with an explicit grounding verdict. Retrieval follows recursive language model (RLM) research (arXiv:2512.24601): cheap grep-like pre-filtering plus distillation by a small local "reader" model beats heavyweight RAG for project-scale corpora — no vector store on the default path.
2. A multi-agent knowledge stack: the same engine started N times, once per knowledge domain (a specialised knowledge agent, e.g. extended-rlm-marketing, extended-rlm-lawyer), each with its own workspace/KB, all serving one LM Studio chat. The chat model routes each question to the agent whose domain matches — or answers directly when no domain matches.

• Version: 0.2.0 (see CHANGELOG.md)
• Licence: MIT
• Transport: stdio (FastMCP) · Package: src/extended_rlm/ · Platform: Windows-first (PowerShell host scripts), engine itself is cross-platform Python 3.10+

---

Repository map (what precisely is inside)

extended-rlm\
├── src\extended_rlm\           the engine (Python package)
│   ├── server.py               MCP server: 20 tools, agent identity/suffixing, bootstrap
│   ├── config.py               layered config: defaults < config.yaml < env vars
│   ├── search.py               ripgrep/grep search, keyword extraction/expansion, triggers
│   ├── chunker.py              header-aware ~500-token chunking
│   ├── ranker.py               TF-IDF-like ranking (INDEX 1.5x / trigger 2.0x boosts)
│   ├── reader_client.py        OpenAI-compatible reader client, grounded distillation,
│   │                           response cache, no-think prefill, session token counters
│   ├── kb_writer.py            atomic topic/memory writes, INDEX maintenance, dedup,
│   │                           supersedes, procedural-rule mirroring (F1)
│   ├── rules_digest.py         [MAIN]-tagged rules -> MAIN-RULES.md digest (~1000-token cap)
│   ├── enforce.py              action-time gates from topic frontmatter (F4)
│   ├── health.py               KB drift diagnosis, claim verification, write log
│   ├── stats_log.py            per-agent usage JSONL + stack-wide per-agent/TOTAL summary
│   ├── files.py                workspace-scoped file tools + soft delete (.deleted\)
│   └── fileserver.py           standalone localhost download server (one per agent)
├── scripts\
│   ├── start-chat.ps1          one-command stack launcher (agents, reader, LM Studio,
│   │                           system-prompt pinning, stats markers)
│   ├── stop-chat.ps1           session token summary (chat vs readers, per agent) + shutdown
│   ├── new-agent.ps1           create + register a new knowledge agent (scaffolds workspace)
│   └── watch-chat-speed.ps1    live tok/s monitor
├── host\lmstudio\              LM Studio host integration
│   ├── system-prompt.md        condition-neutral bootstrap prompt (routing mandate +
│   │                           get_session_rules_* bootstrap order)
│   ├── enable-layer.ps1        register one agent in ~\.lmstudio\mcp.json (env wiring)
│   ├── disable-layer.ps1       remove all extended-rlm* entries (OFF condition)
│   ├── setup-filesystem.ps1    filesystem MCP server (present in both conditions)
│   ├── chat-model-load.config.json  pinned chat-model load config (context/seed/KV cache)
│   └── README.md               step-by-step Windows host setup
├── COMMON-RULES.md             engine-wide behaviour rules served to EVERY agent's chat
│                               session (routing, query-first, capture, stats duties)
├── extended-rlm.json           stack config: chat model, reader backend, agent registry
├── docs\
│   ├── creating-a-new-agent.md           agent lifecycle guide (scaffold -> seed -> verify)
│   ├── authoring-erlm-knowledge.md       FULL spec for authoring a new agent's knowledge
│   │                                     data (for frontier-LLM KB generation)
│   ├── msc-experiment-environment.md     experiment configuration record (pre-freeze)
│   ├── EVALUATION-RUN-GUIDE.md           ON/OFF evaluation procedure
│   ├── GROUNDING_AND_HIERARCHY.md        grounding-verdict contract
│   ├── RECONCILIATION-recorded-retrievable-enforced.md   F1-F4 design note
│   ├── KNOWLEDGE_BASE_SETUP_GUIDE.md     turning any workspace into a structured KB
│   └── WEBSITE_GUIDE.md                  optional docs website from the KB
├── tests\                      182 tests, no network / no live reader needed
├── examples\                   RULES.example.md, .env.example
├── config.example.yaml         per-workspace tuning template
└── start_server.bat            single-server launcher (legacy/bare-engine mode)

A knowledge workspace (one per agent) lives OUTSIDE this repo and contains only data: AGENT.md (domain + routing lists), RULES.md (strict rules), PRINCIPLES.md (behaviour principles), auto-generated MAIN-RULES.md (session digest), and .kb\ (INDEX.md, topics, code_examples, memory). The complete authoring specification is docs/authoring-erlm-knowledge.md.

---

MCP tool surface (20 tools per agent)

In multi-agent mode every tool name gets the agent suffix (ask_knowledge_base_lawyer, …) and its description is prefixed with [agent: <name> | domain: <domain>] — the routing signal.

Session bootstrap

Tool	Purpose
get_session_rules()	Binding session rules: the agent's AGENT.md + engine COMMON-RULES.md + the MAIN-RULES.md digest. Called once per agent at chat start. Marks the session as bootstrapped (see safety net below).

Retrieval

Tool	Purpose
ask_knowledge_base(query, context?)	MANDATORY first step for any in-domain question: grounded answer with Source N citations, Grounding: verdict, reader token stats line. Carries a ⚠ SESSION RULES NOT LOADED banner if called before get_session_rules.
list_knowledge_base()	KB structure (files, sizes, Tier-1 memory, reader status).
clear_knowledge_cache()	Clear the reader response cache.
get_kb_session_stats()	Session token usage for ALL connected agents + TOTAL (computed from the shared stats dir — one call covers the whole stack).
reset_kb_session_stats()	Reset this agent's in-memory counters.

Writing (self-learning)

Tool	Purpose
update_knowledge_base(topic, content, ...)	Atomic topic write + INDEX row, dedup, supersedes, triggers, enforce-gate frontmatter, verify-on-write.
record_memory(kind, name, content, ...)	Tier-1 behavioural memory (feedback/project/user) + MEMORY.md index; procedural rules auto-mirrored into retrievable topics (F1).
record_rule(kind, text, main)	Append a strict rule (→ RULES.md) or behaviour principle (→ PRINCIPLES.md); main=True tags it [MAIN]; regenerates the session digest and reports budget %.

Enforcement, health, accuracy

Tool	Purpose
applicable_rules(file_path?, change_kind?, query?)	Action-time gates from topic enforce_globs/enforce_kinds frontmatter, with a grounded semantic fallback.
kb_health_check()	KB drift: oversized topics, INDEX orphans/broken rows, dead links, staleness.
reader_health()	Live-ping the reader endpoint; concrete failure reason on error.
verify_kb_claim(file, snippet, line_hint?)	Confirm a cited file+snippet still exists (stale-KB detector).
get_recent_sessions_summary(days)	Recent KB-related git activity.
kb_update_digest()	KB writes since last digest + the stack-wide token block (end-of-turn summary).

Workspace files

Tool	Purpose
list_workspace_files(subdir?) / read_workspace_file(path) / write_workspace_file(path, content, mode)	Workspace-scoped file access; writes to RULES.md/PRINCIPLES.md auto-regenerate the MAIN digest.
delete_workspace_file(path)	Soft delete — moved to .deleted\<timestamp>\, always undoable.
get_download_link(path)	Clickable localhost download link served by the per-agent fileserver.

---

Architecture

LM Studio chat (one local model, e.g. Qwen3.5-9B)
  │  system prompt (pinned into model defaults by start-chat.ps1):
  │  bootstrap order + routing mandate
  │
  ├── MCP: extended-rlm-marketing ──► workspace A (own RULES/PRINCIPLES/.kb)
  ├── MCP: extended-rlm-lawyer    ──► workspace B         │
  ├── MCP: <any further agents>   ──► workspace ...       │  all agents share
  └── MCP: filesystem (workspace file access)             ▼  ONE reader:
                                              small local LLM (Ollama or the
                                              LM Studio model itself) distills
                                              top-ranked chunks into grounded,
                                              cited answers

Per query: keyword extraction/expansion → ripgrep across the knowledge tiers → header-aware chunking (+ trigger-matched topics) → ranking (INDEX/trigger boosts) → reader distillation → answer + Grounding: GROUNDED | PARTIAL | NOT-FOUND + token stats.

Session bootstrap & the [MAIN] rules digest

• RULES.md (strict rules) and PRINCIPLES.md (behaviour principles) are canonical and unlimited; the agent updates them at runtime (record_rule, direct edits).
• Entries/sections tagged [MAIN] are compiled — deterministically, no LLM — into MAIN-RULES.md, hard-capped at ~1000 tokens. Only this digest (plus AGENT.md and COMMON-RULES.md) is loaded into chat sessions; everything untagged stays retrievable on demand. At ≥90% budget the engine demands consolidation: move a cohesive rule set into a KB guideline topic, leave a one-line [MAIN] summary + pointer.
• Routing failure modes are defended on three channels: the system prompt is pinned into the chat model's LM Studio defaults (every new chat gets it), the ask_knowledge_base description itself declares the call mandatory for in-domain questions, and KB answers produced before the bootstrap carry a corrective banner.

Token accounting

Every reader call is appended to logs\reader-usage-<agent>.jsonl (shared stats dir, session marker from start-chat.ps1). Any single agent's get_kb_session_stats therefore reports every agent + TOTAL — one call per turn, no arithmetic for the chat model, robust to MCP process restarts. stop-chat.ps1 prints the full session summary (chat model vs readers, per agent, cache hits) from LM Studio engine logs + the JSONL files.

The grounding verdict contract

Verdict	Meaning	Calling model's action
GROUNDED	Fully answered from the KB	Rely on it; do not re-derive
PARTIAL	Some parts NOT IN KB	Escalate only the missing parts: files, then cited web
NOT-FOUND	Nothing relevant	Escalate whole question, then capture the result into the KB

The self-learning loop (F1–F4): recorded, retrievable, enforced

#	Guarantee	Mechanism
F1	Procedural rules become retrievable	procedural record_memory auto-mirrored into a topic with seeded triggers
F2	Retrieval survives re-phrasing	triggers: frontmatter indexed + boosted (2.0×)
F3	Recorded ⇒ confirmed retrievable	verify-on-write re-runs the real pipeline for a paraphrase; warns RECORDED-BUT-NOT-RETRIEVABLE
F4	Action-time enforcement	applicable_rules fires topics whose enforce_globs/enforce_kinds match the edit

---

Quick start (multi-agent stack, Windows + LM Studio)

conda create -n GenAI_FA python=3.12; conda activate GenAI_FA
git clone <this-repository> extended-rlm; cd extended-rlm
pip install -e .[dev]

# 1. Declare agents in extended-rlm.json (or create one interactively):
.\scripts\new-agent.ps1 -Name finance -Domain "Corporate finance for ...: budgeting, cash-flow, ..."

# 2. Launch everything (mcp.json, workspaces, reader, LM Studio, system prompt):
.\scripts\start-chat.ps1

# 3. In LM Studio: enable the agents + 'filesystem' in the chat's Program panel. Chat.

# 4. End the session with the token summary:
.\scripts\stop-chat.ps1

Single-workspace (bare engine, canonical tool names): start_server.bat --path C:\path\to\ws or .\scripts\start-chat.ps1 -Workspace C:\path\to\ws.

---

Configuration

Resolution order: built-in defaults → workspace config.yaml (template) → environment variables. Secrets only via environment.

Key environment variables (full list in src/extended_rlm/config.py):

Variable	Default	Purpose
KNOWLEDGE_BASE_PATH	cwd	Workspace root (equivalent to --path)
RLM_AGENT_NAME / RLM_AGENT_DOMAIN	unset	Agent identity; tool-name suffix + description tag (AGENT.md Domain: overrides the env domain)
RLM_COMMON_RULES	unset	Path to the engine-wide COMMON-RULES.md served by get_session_rules
RLM_STATS_LOG	unset	Per-agent usage JSONL; its parent dir is the shared stack-stats dir
RLM_FILE_PORT	auto	Stable download-server port for this agent
RLM_READER_MODEL / READER_BASE_URL / READER_API_KEY	qwen3:4b / Ollama / unset	Reader endpoint (any OpenAI-compatible /chat/completions)
RLM_READER_ENABLED	1	0 → raw ranked chunks (no distillation)
RLM_READER_NOTHINK	unset	Suppress Qwen3/3.5 reader thinking via assistant prefill (fast path)
RLM_READER_THINK_ALLOWANCE / RLM_READER_TIMEOUT	0 / 120	Reasoning-reader headroom / HTTP timeout (s)
RLM_RETRIEVAL_MODE	grep	grep or hybrid (opt-in embeddings: pip install -e .[embeddings])
RLM_GROUNDING_ENABLED / RLM_GROUNDING_STRICTNESS	1 / strict	Verdict line behaviour
RLM_MEMORY_SEARCH / MEMORY_DIR	1 / <ws>\.kb\memory	Tier-1 memory search
RLM_ENFORCEMENT_ENABLED / RLM_ENFORCE_SEMANTIC	1 / 1	Action-time gates
RLM_CONFIG	unset	Explicit config YAML path

Stack-level settings (chat model + pinned load config, reader backend ollama/lmstudio, agent registry) live in extended-rlm.json and are applied by start-chat.ps1.

---

Evaluation: the ON/OFF switch

Designed for a controlled experiment (MSc project): same base model, same workspaces, same prompts and sampler settings; the single manipulated factor is the set of extended-rlm* entries in mcp.json.

• ON: start-chat.ps1 (default) registers the configured agents.
• OFF: start-chat.ps1 -Condition OFF removes them; the model runs stock (filesystem MCP stays in both conditions).
• The system prompt is condition-neutral and pinned into the model defaults; the pinned load config freezes context length, seed, and KV-cache quantisation.
• Measures: task correctness, cross-session consistency, rule-following, factual accuracy, token cost (per-answer stats line, get_kb_session_stats, stop-chat.ps1 summary).
• Full procedure: docs/EVALUATION-RUN-GUIDE.md; environment record: docs/msc-experiment-environment.md.

---

Tests

pytest            # 182 tests, no network, no live reader needed
ruff check src/

Distillation, grounding, verify-on-write, enforcement, the [MAIN] digest, stats aggregation, bootstrap banner, and file tools are all exercised with fake clients and pure functions.

Security notes

• Secrets (READER_API_KEY) come from the environment only; never from config files.
• The engine writes only inside the knowledge tiers (RULES.md, PRINCIPLES.md, .kb\, memory dir) and the workspace file tools are path-jailed to the workspace (soft delete only).
• The download fileserver binds to 127.0.0.1 and serves only workspace files.

Licence

MIT, see LICENSE.

Credits

• Recursive language model (RLM) research: arXiv:2512.24601
• MCP Python SDK / FastMCP