D-Knowledge Graph

<p align="center"> <img src="assets/brand/logo.png" alt="D-Knowledge Graph" width="360"> </p>

<p align="center"> <b>Ask your documents, media, and code hard questions, and get answers you can trace to the source, entirely on your own machine.</b> </p>

<p align="center"> A local-first, LLM-agnostic knowledge graph for researchers, engineers, and teams who need an answer they can audit, offline. </p>

[!IMPORTANT] Local-first and air-gapped by default. No cloud call, no telemetry, and no mandatory runtime dependency beyond the Python standard library. Every capability status in this repository is backed by an executed, on-disk test, never a green checkmark by assertion.

<div align="center">

</div>

<p align="center"> <a href="#overview">Overview</a>  ·  <a href="#how-it-works">How it works</a>  ·  <a href="#capabilities">Capabilities</a>  ·  <a href="#use-cases">Use cases</a>  ·  <a href="#install">Install</a>  ·  <a href="#security">Security</a>  ·  <a href="#benchmarks">Benchmarks</a>  ·  <a href="#faq">FAQ</a>  ·  <a href="#troubleshooting">Troubleshooting</a>  ·  <a href="#licence">Licence</a> </p>

---

Overview

D-Knowledge Graph is one shared knowledge-graph core with two pluggable analysis planes on top. The core is a schema-migrated SQLite store with full-text search, deterministic content-derived IDs, an append-only hash-chained audit log, a provenance envelope on every record, and a read-only MCP surface. The two planes share that substrate and a single honest-evidence standard:

• A document-and-media plane that ingests text, structured data, web content, images, video, and audio, extracts entities and claims deterministically, and grades every claim with an explainable confidence.
• A source-code plane that parses seven languages with Tree-sitter, builds a code graph, and answers structural questions such as blast-radius, execution-flow, architectural chokepoints, unexpected coupling, and knowledge gaps, with optional type-aware resolution.

Retrieval runs keyword search, FTS5, and a hybrid path that fuses both, with an optional real local embedding model and an optional cross-encoder reranker that load pre-staged weights with no runtime download. Graph structure is summarized by two community detectors that both run by default: Mnemosyne builds the base partition and Ariadne refines it, and whichever scores higher modularity is the one returned. A delivery surface adds a multi-repo watch daemon, offline graph visualization, interop exports, and a consumer GitHub Action, and a one-command benchmark harness regenerates every measured number in this document from a single seeded run. Nothing here reaches the network unless you explicitly opt in.

The problem, and how the platform answers it

Problem	How D-Knowledge Graph answers it
Knowledge tools ship your data to a cloud service you cannot audit.	Everything runs on your machine against a local SQLite file. Outbound network is off by default and every egress path requires an explicit flag.
Answers from a graph or an assistant cannot be traced back to a source.	Every record carries a provenance envelope, every claim has an evidence packet with an explainable confidence, and an append-only hash-chained audit log detects tampering with one verify command.
Status dashboards are green because someone typed green.	A row is production ready only when its implementation files resolve on disk, its acceptance is an executed test that passes, and its evidence is real. The status is generated from a validated matrix, not hand-typed.
Retrieval quality claims are marketing, not measurement.	Retrieval, community detection, code resolution, execution-flow, and media accuracy are measured on documented corpora with a fixed seed and published in docs/BENCHMARKS.md.
Code impact review needs a hosted service or a heavy toolchain.	The source-code plane parses in-process with permissive grammars, computes structural blast-radius over the local code graph, and runs with no network and no cloud account.
An assistant reaching your data can be told what to do by the data.	The MCP surface is read-only, content fetched from the web is labelled untrusted evidence and never instructions, and security decisions run deterministically outside the model.

How it works

One shared core, two pluggable planes. The core owns storage, search, evidence, provenance, audit, and the read-only MCP surface. Each plane brings its own parsers and extractors and writes into the same graph, so a question can cross from a document to the code it describes.

flowchart TB
    subgraph inputs["Your data, never leaves the machine"]
        docs["Documents<br/>text, markdown, json, csv, docx, pdf"]
        media["Media<br/>images, video, audio"]
        code["Source code<br/>7 languages"]
    end

    subgraph planes["Analysis planes"]
        dmp["Document and media plane<br/>readers, OCR, ASR, keyframes, detection<br/>entity and claim extraction"]
        scp["Source-code plane<br/>Tree-sitter parsers, code graph<br/>impact, flow, centrality, coupling"]
    end

    subgraph core["Shared knowledge-graph core"]
        store["SQLite store<br/>entities, relationships, chunks"]
        search["Search<br/>keyword, FTS5, hybrid, rerank"]
        evid["Evidence ledger<br/>provenance, confidence, audit chain"]
    end

    surfaces["Surfaces<br/>CLI, read-only MCP, exports, offline viewer"]

    docs --> dmp
    media --> dmp
    code --> scp
    dmp --> store
    scp --> store
    store <--> search
    store <--> evid
    search --> surfaces
    evid --> surfaces

The two planes stay separate on purpose. They share the substrate and one honest-evidence standard, but never each other's parsers.

flowchart LR
    subgraph dm["Document and media plane"]
        direction TB
        r1["Readers<br/>stdlib formats, html, pdf, rss"]
        r2["Media<br/>EXIF, OCR, ffprobe, keyframes, ASR"]
        r3["Extraction<br/>entities, claims, relations, dedupe"]
        r1 --> r3
        r2 --> r3
    end

    subgraph sc["Source-code plane"]
        direction TB
        c1["Tree-sitter parse<br/>symbols and references"]
        c2["Edge resolution<br/>name-based, optional type-aware"]
        c3["Analysis<br/>blast-radius, flow, hubs, coupling, gaps"]
        c1 --> c2 --> c3
    end

    shared[("Shared core<br/>entities, relationships, chunks,<br/>provenance, evidence, audit")]
    r3 --> shared
    c3 --> shared

A query never guesses. It fans out across the retrieval paths, fuses the results, and returns evidence with every hit.

flowchart LR
    q["Question<br/>CLI or MCP tool"] --> plan{"Which surface?"}
    plan -->|search| kw["Keyword"]
    plan -->|search| fts["FTS5"]
    plan -->|search| vec["Vector similarity<br/>optional embeddings"]
    kw --> fuse["Rank fusion"]
    fts --> fuse
    vec --> fuse
    fuse --> rr["Cross-encoder rerank<br/>optional, degrades cleanly"]
    rr --> ev["Attach evidence<br/>provenance, confidence, source"]
    plan -->|graph| trav["Bounded traversal<br/>neighbourhood, impact, flow"]
    trav --> ev
    ev --> ans["Answer with citations<br/>every hit traceable to a document"]

One measured before and after

Type-aware resolution is the clearest measured improvement in the project, and it is also the clearest illustration of why the default is labelled advisory. On the ambiguity corpus, structural name-matching resolves a call to every same-named candidate, so blast-radius over-flags badly. With a staged language server, the same query resolves to one target.

flowchart LR
    subgraph before["Before: structural name matching (default)"]
        b1["Blast-radius precision<br/><b>0.108</b>"]
        b2["Recall<br/>1.0"]
        b3["Every same-named<br/>candidate flagged"]
    end
    subgraph after["After: type-aware resolution (--resolve)"]
        a1["Blast-radius precision<br/><b>1.0</b>"]
        a2["Recall<br/>1.0"]
        a3["One resolved target<br/>per call site"]
    end
    before --> after

Measured on 42 evaluation nodes and 24 true edges per language, for Python and JavaScript. Go stays structural because no language server is staged for it. Recall is 1.0 in both configurations, so the gain is entirely in precision: the structural path was never missing real impact, it was reporting far too much. Regenerate with python scripts/benchmark.py.

Capabilities

Everything in the table below runs on the standard library alone unless the last column names an optional extra. Extras are opt-in and install with pip install -e ".[name]".

Capability	What it does	Built-in or extra
Knowledge-graph store	Schema-migrated SQLite with FTS5, content-derived IDs, provenance, and an append-only hash-chained audit log.	Built-in
Deterministic extraction	Entity, claim, and co-occurrence relationship extraction with no model required.	Built-in
Search	Keyword, FTS5, and a hybrid path that fuses both and explains which engines contributed.	Built-in
Real embeddings	Local vector similarity with a numpy-only model, persisted per model so backends never mix. Falls back to a hashing adapter when absent.	embeddings
Reranking	A local cross-encoder that reranks hybrid results over ONNX. Degrades to keyword-plus-FTS fusion when absent.	reranker
Evidence and confidence	Claim-level evidence packets, an explainable confidence formula, and a contradiction scanner that groups claims about the same subject even when the two documents phrase it differently, then tests them for unit-aware numeric, negation, and antonym conflict. Lexical and over-approximate, so its output is advisory: measured recall 6 of 9 and precision 0.75 on a held-out corpus.	Built-in
Community detection	Both detectors run by default: Mnemosyne builds the base partition with no third-party dependency, Ariadne refines it with semantic edge weighting and an auto-tuned resolution, and the higher-modularity partition wins. Selection is by measured modularity, never preference.	Built-in (both)
Read-only MCP server	Eighteen read-only tools over stdio JSON-RPC 2.0, plus a loopback-bound HTTP surface with bearer auth, size limits, and rate limiting.	Built-in
Graph analysis	Hub and bridge detection (Brandes betweenness, articulation points, chokepoints), unexpected-coupling scoring, knowledge-gap analysis, auto-generated review questions, an architecture map with coupling warnings, and graph diffing over time.	Built-in
Editor integration	Write the read-only MCP server entry for Claude Code, Cursor, or Windsurf, with a dry run and an uninstall that removes only what it wrote.	Built-in
Multi-agent workflows	A deterministic coordinator that runs research, validation, contradiction, and security-review agents with budgets and timeouts, with no model provider connected.	Built-in
Source-code plane	Tree-sitter parsing for Python, JavaScript, Go, TypeScript, Java, Ruby, and Rust, a code graph, structural blast-radius, execution-flow tracing, and optional type-aware resolution.	code, code-extended
Image detection	Zero-shot CLIP tagging over ONNX, torch-free and pre-staged.	media-detect
Media enrichment	Image decode and EXIF, OCR, video metadata, keyframe and scene detection, and pre-staged ASR.	media-image, external binaries
Delivery surface	Multi-repo watch daemon, offline HTML visualization, DOT / Cypher / SVG / Obsidian exports, and a consumer GitHub Action.	Built-in (watch optional)
Reproducible benchmarks	One seeded command regenerates every accuracy and quality number across both planes.	Built-in

Supported inputs

The core ingests these formats with the Python standard library only, no extra and no external binary:

Built-in input	Formats
Text and Markdown	.txt, .md, .markdown, .rst, .log
Structured data	.json, .csv, .tsv
Word documents	.docx (parsed with zipfile and the standard-library XML parser, entity expansion disabled)
RSS and Atom	feed parsing with the standard-library XML parser (fetching a remote feed needs network opt-in and the web extra)

These inputs are capability-detected and enabled by an optional extra or an external binary. When the extra or tool is absent, the input degrades cleanly with an honest reason rather than failing:

Optional input	Needs
HTML	html extra (beautifulsoup4, lxml)
PDF	pdf extra (pypdf)
Web fetch	web extra (httpx), plus an explicit --allow-network
Images and EXIF, OCR	media-image extra (Pillow), OCR via the external tesseract binary
Video metadata, keyframes, scene detection	external ffprobe and ffmpeg binaries
Speech-to-text	pre-staged whisper.cpp or the asr-faster-whisper extra, with a local model referenced by DKG_ASR_MODEL
Source code	code extra (Tree-sitter plus MIT grammars for Python, JavaScript, and Go)

Use cases

Functional use cases

The platform is a private, auditable research and knowledge substrate. Because it runs offline and records provenance for every record, it fits work where the source of an answer matters as much as the answer.

Team or role	What they use it for	Outcome
Research and analysis	Ingest a corpus of notes, reports, and feeds, then search, traverse, and cross-check claims against their sources.	A searchable graph where every claim links to the document it came from.
Compliance and legal review	Keep sensitive material on a local or air-gapped machine and produce evidence packets with an explainable confidence.	A defensible, offline record of what was found and where.
Knowledge management	Turn scattered files into a connected graph, group related material with community detection, and export to Obsidian or Graphviz.	A maintained map of an organization's own knowledge, with no vendor lock-in.
Investigative and due-diligence work	Surface contradictions across sources and follow bounded neighbourhood queries between entities.	A reviewable map of where sources agree and disagree.

Run a deterministic multi-agent workflow over the graph with no model connected, then register a local model behind the adapter interface when you want higher recall:

dkg agent research        --input '{"query":"knowledge graph"}'
dkg agent contradiction   --input '{}'
dkg agent security-review --input '{"limit":500}'

Technical use cases

Engineering use case	How the platform serves it
Code intelligence	Parse a repository into a code graph and query symbols, call structure, and structural execution-flow from an entry point.
Architecture review	Surface the most connected symbols, the cut vertices whose removal splits the graph, dependency cycles between components, and edges that cross a cluster or a language boundary.
Reviewing an unfamiliar change	Generate review questions from the graph, each naming a symbol and carrying the measurement that prompted it, then diff two graph snapshots to see what actually moved.
Change-impact review	Compute an advisory, over-approximate changed-file blast-radius, with an opt-in gate for CI, through the consumer GitHub Action or dkg code-report.
Offline knowledge graphs	Build and browse a graph with a self-contained HTML viewer that loads no CDN, script, font, or remote asset.
Evidence and provenance	Attach a provenance envelope to every record and verify an append-only hash-chained audit log with one command.
Air-gapped deployments	Install from source with zero runtime dependencies and run every core capability with the network off.

Structural blast-radius and execution-flow are over-approximate by design in the default path; the optional --resolve path upgrades ambiguous call edges with type-aware resolution where a language server is staged.

Install

Requirements. Python 3.10 or newer. The core install pulls zero runtime dependencies. macOS and Linux are the tested targets.

# from a clone of the repository
python3 -m venv .venv
./.venv/bin/python -m pip install --upgrade pip
./.venv/bin/python -m pip install -e ".[dev]"
./.venv/bin/dkg --version        # dkg 0.1.0

Add optional extras only when you need them, for example pip install -e ".[embeddings,reranker,code]". Every optional adapter reports the exact reason when it is unavailable.

Quick start

dkg init                                   # create a project-local .dkg home
dkg ingest ./my-notes --recursive          # ingest text, markdown, json, csv, docx
dkg status                                 # print counts and configuration
dkg search "confidence formula"            # keyword, fts, or hybrid (default hybrid)
dkg graph "beta" --depth 2                 # bounded graph neighbourhood
dkg evidence <claim-id>                    # evidence packet for a claim
dkg community --detector mnemosyne         # summarize graph structure
dkg code-ingest ./my-repo                  # parse a repository into the code graph
dkg code-hubs                              # most connected symbols and chokepoints
dkg code-gaps                              # isolated symbols and untested hotspots
dkg code-questions                         # review questions generated from the graph
dkg code-architecture                      # component overview with coupling warnings
dkg graph-snapshot before.json             # snapshot now, diff later with graph-diff
dkg export --format html --out graph.html  # offline viewer, or json / csv / dot / cypher / obsidian
dkg audit --verify                         # verify the hash chain
dkg mcp-stdio                              # start the read-only stdio MCP server

If you are new to the command line

1. Install Python 3.10 or newer from python.org, then open a terminal in the project folder.
2. Copy the four install commands above one line at a time. The last line should print dkg 0.1.0.
3. Run dkg init, then dkg ingest pointed at a folder of your notes, then dkg search "a phrase you expect to find".

Every command prints human-readable text by default and machine-readable JSON with --json. No step contacts the network unless you pass --allow-network.

Security

The platform is secure by default. Each control below is implemented in the tracked source and covered by a test.

Control	Default	Detail
Outbound network	Off	Egress requires an explicit --allow-network flag and a config allowance; the default configuration refuses outbound requests.
Telemetry	None	There is nothing to disable; telemetry is opt-in only through an explicit environment variable.
MCP surface	Read-only	Only query tools are registered; write tools are intentionally never exposed. The HTTP server binds loopback, requires a bearer token, and caps request size and rate.
SSRF and DNS-rebind	Blocked	A post-resolution check rejects private, loopback, link-local, multicast, reserved, and cloud-metadata addresses before any fetch.
Secret redaction	On	Audit lines, logs, and exported packets pass through a credential redactor that masks keys, tokens, and private-key blocks.
Untrusted content	Enforced	Fetched web content is labelled untrusted evidence, never instructions, and is scored for prompt-injection attempts.
Storage	Parameter-bound	Every SQL path is parameterized; the database layer rejects string interpolation of query parameters.
Provenance and evidence	Always on	Every record carries a provenance envelope, and the audit log is append-only with a per-row hash chain.
Supply chain	Hardened	GitHub Actions are SHA-pinned, dependencies are pinned with a generated lockfile and SBOM, and a licence audit and dependency vulnerability scan run in CI.

See docs/SECURITY_MODEL.md and docs/THREAT_MODEL.md for the full model.

Benchmarks

Every number below is measured on a documented corpus with a fixed seed (PYTHONHASHSEED=0) and regenerated by python scripts/benchmark.py. The authoritative published results, with corpus sizes and the staged environment, live in docs/BENCHMARKS.md. Comparisons are internal only, and a benchmark whose tool or model is not staged is reported not run in this environment, never failed.

Benchmark	Corpus	Result
Retrieval quality	30 documents, 40 queries	Keyword baseline MRR 0.9375 / nDCG 0.9473; embeddings plus cross-encoder rerank MRR 1.0 / nDCG 1.0. The new system beats the keyword baseline; the margin is smaller on this larger, cleaner corpus than on the original smaller corpus.
Community detection	80-node 16-clique structural corpus; 40-entity 5-topic semantic corpus	The base and refinement passes tie on the structural corpus (Rand 1.0); the refinement pass is better on the semantic corpus (Rand 0.7641 versus 0.641), which is when the default returns it.
Code resolution	42 eval nodes, 24 true edges per resolved language	Blast-radius precision 0.108 structural to 1.0 resolved for Python and JavaScript (recall 1.0). Go remains structural this wave.
Execution-flow	Hand-labelled per-language call graphs	Edge precision and recall 1.0 for Python, JavaScript, and Go.
Code parse accuracy	One labelled file per language, 4 to 8 symbols each	Symbol precision and recall 1.0 for all seven languages. A language whose optional grammar is absent is reported not measured, never scored zero.
Token cost, four tasks	414 code files and 20 documents, 2927 nodes, 7431 edges; cl100k_base on both sides	Against a competent grep-and-read baseline (rarity-weighted ranking, top 12 files read whole) the graph route costs about twice the tokens overall, 71,088 against 34,744, and is more correct and more complete: mean correctness 1.0 against 0.6206. The baseline is cheap because it is incomplete, not because it is efficient, so the two columns only mean anything read together. It wins on tokens with correctness held in 2 of 4 tasks: contradiction surfacing, 82.0% fewer tokens, and question answering, 59.7% fewer. It loses on impact analysis (171% more tokens) and code review (466% more), in both cases to a baseline scoring under 0.25. Two levers cut cost on their own: model-free exact answers 52.21% on impact analysis, delta-only session context 40.98% across a four-turn code review.
Contradiction detection	15 held-out cases, 9 of them real disagreements, in domains absent from the token-cost corpus; 6 cases contributed by an adversarial review trying to break it	Recall 6 of 9 (0.6667) and precision 0.75: it misses three real disagreements and raises two false alarms, and all five stay in the score. The misses are a verb outside the claim extractor's patterns and two paraphrases where one differing word breaks topic containment. The false alarms are a qualifier that scopes a claim, and an incidental "no" in an unrelated clause. It is a lexical scanner, not an entailment model, so its output is advisory.
Media OCR and enrichment	Synthetic rendered corpora	OCR character and word error rate 0.0 on the sample; zero-shot image detection top-1 0.9375. Natural-photo accuracy is not represented by the synthetic corpus.

Licence

Source-available and free for personal and non-commercial use. This is not an open-source licence: commercial use is not permitted, and neither is modification or distributing a modified version.

Component	Licence	Terms
The entire repository, Ariadne included	D-Knowledge Graph Source-Available Non-Commercial Licence (PolyForm Noncommercial 1.0.0 plus a no-modification term)	Read, run, and use the output for any non-commercial purpose. Redistribute verbatim copies with LICENSE and NOTICE. No commercial use. No modification and no modified redistribution.
Optional third-party dependencies	Their own permissive licences (Apache-2.0, MIT, BSD, ISC, HPND)	Unaffected by the terms above. Full inventory in THIRD_PARTY_NOTICES.md.

One licence covers everything. There is no separately licensed module and no component excluded from the build. The default runtime uses only the Python standard library and copies no source from any other project.

Versions distributed before 2026-08-05 were released under Apache-2.0. That grant remains in force for those versions and for anyone who received a copy under it; these terms govern this version onward. See LICENSE and NOTICE.