<div align="center"> <img src="assets/thumbnail.png" alt="Perseus Vault x AMD Instinct - encrypted memory for AI agents, kept off the GPU" width="100%"> </div>

Perseus Vault × AMD Instinct

Encrypted, local-first, persistent memory for AI agents — kept off the GPU so every byte of MI300X HBM serves tokens.

AMD Developer Hackathon: Act II — Unicorn (Open) Track. Built on Perseus Vault, a production, MIT-licensed memory engine (10★, v2.19.1). lablab project: lablab.ai/…/perseus.

▶ Try the live demo — amd-demo.perseus.observer

Teach the agent a fact, open a brand-new session, recall it, then run a decay tick — the real SQLite + FTS5 engine (webdemo/) running on the host CPU (0 bytes of GPU HBM), alongside the projected MI300X economics table. No login, per-visitor sandbox.

⚠️ Honesty banner (please read)

Cloud credits did not arrive before the July 11 deadline, so we do not have MI300X measurements yet. Every number in this repo is tagged with a data_source: measured (timed live, reproducible now), published-spec (vendor datasheet / cloud price list, cited below), or projection (derived from published-spec inputs with stated assumptions). No projected number is presented as measured. The demo and benchmark scripts print this warning on every run.

---

Problem

An AI agent is only as smart as what it remembers — but its "memory" is usually just the LLM context window, which dies when the session ends. The common fix is to bolt on a vector database: embed everything, store the vectors, do nearest-neighbour search at recall. That buys persistence at a steep price:

• a second system of record (Postgres/pgvector, Pinecone, a Docker sidecar) that drifts out of sync with the agent's state;
• an embedding model on the hot path for every write and query — latency and GPU cycles spent before the actual model runs;
• HBM pressure — if the index or embedder shares the accelerator, it eats the very memory you wanted for weights and KV cache.

On an AMD Instinct MI300X, that last point is the whole game. Its 192 GB of HBM3 is the scarce resource. Every gigabyte spent storing what the agent knows is a gigabyte not serving tokens.

Solution

Perseus Vault is a single Rust binary that gives an agent durable memory over the Model Context Protocol (MCP). Its recall path is SQLite + FTS5 hybrid search (BM25 lexical ranking blended with a recency/decay prior) — no embedding model, no external vector database, no GPU.

That's the design, not a limitation. The memory layer runs on the host CPU beside the accelerator, so:

• 100% of the MI300X's 192 GB HBM3 stays available for weights + KV cache.
• Recall adds zero GPU work and never competes with inference for the accelerator.
• One GPU backs many concurrent agents — each with its own AES-256-GCM-encrypted memory file (~85 MB RAM + ~45 MB disk per 100K memories, measured), because those files live in host RAM/disk, not HBM.

Architecture

                     AMD Instinct MI300X (192 GB HBM3)
                  +------------------------------------+
 user turn ─────► | LLM weights + KV cache (inference) |
     ▲            | via Fireworks AI / vLLM / ROCm     |
     │            +------------------------------------+
     │ recall THEN infer     ▲ grounding │ tokens
     │                    ┌───────────────┐
     └────────────────────┤  Agent loop   │
                          └───────────────┘
                             ▲ remember() / recall() / decay()  (MCP, CPU only)
                    ┌───────────────────────────────┐
                    │ Perseus Vault (Rust binary)    │
                    │ SQLite+FTS5 · AES-256-GCM      │  ── 0 bytes HBM
                    │ one portable .db file / agent  │
                    └───────────────────────────────┘
                          host CPU + RAM + disk

Full write-up: docs/ARCHITECTURE.md.

Benchmarks

Full tables, sources, and reproduction steps: docs/BENCHMARKS.md. Reproduce §1–§2 with python3 src/benchmark.py.

Recall latency scales flat with store size — measured

Reference implementation (src/benchmark.py, AMD-CPU laptop, Python 3.14):

Entries	Recall p50 (ms)	Recall p99 (ms)	Insert ops/s
1,000	0.20	0.39	72,917
10,000	1.14	1.44	72,217
100,000	11.87	15.67	68,276

Shipping engine (Perseus Vault v2.19.x, measured, AMD CPU — see PERF.md): FTS5 recall 17.0 ms p50 / 19.4 ms p99 @100K; bulk insert 98,732 entities/s.

Footprint stays tiny — measured

Entries	DB file (MB)	RSS (MB, shipping engine)
1,000	0.31	—
10,000	2.61	—
100,000	25.95	~85

One accelerator serves N agents — projection (from published-spec inputs)

Serving Llama-3.1-70B (FP16, ~141 GB) with 8K-token KV cache (~2.5 GB/seq):

Accelerator	Cards for weights	Concurrent agents	GPU $/hr	GPU $/agent-hr
AMD Instinct MI300X	1	~20	$2.72	$0.133
NVIDIA H100 SXM	2	~8	$7.86	$1.034
NVIDIA A100 80GB SXM	2	~8	$3.60	$0.474

The MI300X fits a 70B model on one card and has the most HBM left for concurrent sessions → ~7.8× lower GPU $/agent-hour than H100 for this workload. Perseus Vault memory runs on the CPU (~$0.0004/agent-hr ≈ 0.3% of the agent's cost) and uses 0 bytes of HBM. Reproduce: python3 src/economics.py.

Quick start

git clone https://github.com/tcconnally/perseus-amd-act-ii.git
cd perseus-amd-act-ii

# 1) Run the agent end-to-end (stdlib only, no GPU, no network needed):
python3 src/agent_memory_demo.py

# 2) Reproduce the measured benchmark tables + economics:
python3 src/benchmark.py            # add --quick to skip the 100K row

# 3) (optional) Real inference on AMD Instinct via Fireworks AI:
cp .env.example .env                # then set FIREWORKS_API_KEY
python3 src/agent_memory_demo.py

# 4) (optional) Run against the real Perseus Vault Rust binary:
curl -sSf https://raw.githubusercontent.com/Perseus-Computing-LLC/perseus-vault/main/scripts/install.sh | sh
PERSEUS_VAULT_BIN=~/.local/bin/perseus-vault python3 src/agent_memory_demo.py

Docker (ROCm base — GPU-ready)

docker build -t perseus-amd-act-ii .          # FROM rocm/dev-ubuntu-22.04:6.2
docker run --rm perseus-amd-act-ii            # runs the demo
docker run --rm perseus-amd-act-ii python3 src/benchmark.py --quick

# On an AMD GPU host, expose the accelerator:
docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \
  -e FIREWORKS_API_KEY=... perseus-amd-act-ii

Published-Spec Estimates

GPU rows above are not measured. They come from vendor datasheets and 2026 cloud price lists:

• AMD Instinct MI300X — 192 GB HBM3, 5.325 TB/s bandwidth, 1,307.4 TFLOPS FP16 (peak), 750 W TDP. AMD MI300X datasheet (PDF) · product page · ROCm software: rocm.docs.amd.com.
• NVIDIA H100 SXM — 80 GB HBM3, 3.35 TB/s, ~989 TFLOPS FP16, 700 W.
• NVIDIA A100 80GB SXM — 80 GB HBM2e, 2.039 TB/s, 312 TFLOPS FP16, 400 W.
• Cloud pricing (2026, per-GPU-hour): MI300X median ~$2.72 (from ~$1.99); H100 ~$3.93; A100 80GB ~$1.80. Sources: cloud-GPU price trackers (getdeploying, thundercompute, gpucost.org), July 2026.
• Model assumption: Llama-3.1-70B, FP16 weights ~141 GB; KV cache per 8K-token sequence ~2.5 GB (80 layers, 8 GQA KV heads, head_dim 128, fp16). Derivation lives in src/economics.py.

What We Would Measure on Real AMD Hardware

Given an MI300X node on AMD Developer Cloud we would replace every projection with a measurement (details in docs/BENCHMARKS.md §4):

1. Recall p50/p99 on the host EPYC CPU while the MI300X is saturated serving Llama-3.1-70B via ROCm/vLLM — proving the CPU memory layer steals no inference cycles.
2. The true concurrent-agent ceiling on one MI300X (HBM vs host-RAM bound) vs the ~20 projection.
3. End-to-end agent-turn latency (CPU recall + MI300X generation) vs a vector-DB baseline.
4. Measured $/agent-hour from real Fireworks/ROCm throughput × real cloud price.
5. A ROCm/HIP prototype offloading Perseus Vault's dense re-rank to an idle GPU slice — an open question we'd answer with data, not claims.

What's in this repo

Path
src/agent_memory_demo.py	End-to-end stateful agent (learn → recall → infer → decay).
src/perseus_vault_store.py	Memory interface: CPU reference store + real-binary bridge.
src/benchmark.py	Measured throughput/footprint tables + economics.
src/economics.py	The "one MI300X serves N agents" model.
docs/ARCHITECTURE.md	Design + the off-the-GPU thesis.
docs/BENCHMARKS.md	All tables, sources, reproduction.
docs/SUBMISSION.md	Every lablab form field, pre-filled.
Dockerfile	ROCm-based, GPU-ready container.

About

Built by Perseus Computing LLC (Wyoming). Perseus Vault is MIT-licensed and production-deployed. This submission is original and MIT-compliant.

License

MIT © 2026 Perseus Computing LLC.