TokenFluid

Let AI work on your code safely. It gets the right context, works inside a controlled local workspace, validates changes, and only approved patches touch your real repository.

TokenFluid is a local-first AI development runtime. It gives AI coding assistants:

1. Precise repository context — token-budgeted context packs, FTS5 search, symbol extraction for 32 languages.
2. Safe isolated workspaces — a persistent local copy of your repo where the AI can experiment.
3. Deterministic Git control — all Git operations go through a safe GitManager that blocks destructive commands.
4. Build/test validation — run commands inside the workspace with a blocklist of destructive operations.
5. Patch approval — the AI prepares patches; only a human can apply them to the real repo.
6. Full rollback history — every attempt is a commit; roll back to any point.

No cloud. No telemetry. No Docker. No remote execution. Your code stays on your machine.

Quick start

# Install
pip install -e .

# Index your repo (incremental, fast)
tokenfluid index /path/to/your/repo

# Build a context pack for a task
tokenfluid pack /path/to/your/repo "add forgot password feature" --max-tokens 8000

# Initialize a safe workspace (one-time per repo)
tokenfluid workspace init /path/to/your/repo

# Start a task
tokenfluid task start /path/to/your/repo "rename hello to greet"

# (The AI or you modifies files in the workspace copy)
# Workspace: /path/to/your/repo/.tokenfluid/workspaces/<id>/working-copy/

# Snapshot the changes
tokenfluid task snapshot /path/to/your/repo 1 --summary "renamed"

# Generate a patch (checks if it applies cleanly to the real repo)
tokenfluid task patch /path/to/your/repo 1

# Review the diff
tokenfluid task diff /path/to/your/repo 1

# Approve and apply to the real repo (HUMAN-CONTROLLED)
tokenfluid task approve /path/to/your/repo 1 --commit --commit-message "rename hello to greet"

CLI commands

Command	Description
tokenfluid index <repo>	Index a repo. --full, --stats.
tokenfluid status <repo>	Show repo index stats + workspace + tasks.
tokenfluid search <query> --repo <repo>	Full-text search with ranked, explainable results.
tokenfluid pack <repo> <task>	Build a token-budgeted context pack.
tokenfluid related <repo> <file>	Find files related to a given file.
tokenfluid memory add|search|list|delete	Manage saved memories.
tokenfluid eval <repo> <cases.json>	Run an evaluation benchmark.
tokenfluid doctor <repo>	Check environment + setup.
tokenfluid plugins	List all 32 language plugins.
tokenfluid graph <repo> --kind imports|calls|inheritance	Show dependency graphs.
tokenfluid config <repo>	Show or modify layered config.
tokenfluid init <repo>	Create .tokenfluid/ with default config.
tokenfluid export <repo> / tokenfluid import <repo>	Export/import metadata.
tokenfluid workspace init|status|list|refresh|reset|remove <repo>	Manage safe workspaces.
tokenfluid task start|status|list|snapshot|diff|patch|rollback|approve|discard|run|build|test|lint	Manage AI coding tasks.
tokenfluid benchmark run|report	Run local benchmarks.
tokenfluid serve --repo <repo>	Run as a stdio MCP server.

MCP tools

TokenFluid exposes 17 tools over stdio MCP:

Search & context: search_code, get_file, get_symbol, get_related_files, build_context_pack, recall_memory, save_memory, repo_status

Workspace & tasks (safe subset): workspace_status, task_start, task_status, task_snapshot, task_diff, task_patch, task_run, task_rollback, task_discard

NOT exposed via MCP: task_approve — approval is human-controlled (CLI only).

See docs/mcp-setup.md and docs/mcp-workspace-tools.md.

Safety model

• AI does not control Git. All Git operations go through GitManager which blocks reset, clean, push, pull, rebase, merge, stash, etc.
• Real repo is untouched until approval. The AI works in a workspace copy. Only tokenfluid task approve (CLI, human-run) applies patches to the real repo.
• Approval is human-controlled. task_approve is NOT exposed as an MCP tool.
• Workspace is local. No Docker, no containers, no remote execution.
• Path traversal protection on every MCP tool that accepts a rel_path.
• Secret files (.env, *.pem, id_rsa) never have content indexed.
• Build/test runner blocks destructive commands (rm -rf, git push, format, etc.).

See docs/git-safety.md, docs/workspace.md, docs/patch-approval.md, and docs/security.md.

Privacy statement

TokenFluid indexes repositories locally. By default, code and memory are stored only on your machine in .tokenfluid/. This package does not call external LLM APIs or upload source code. No telemetry, no analytics, no auto-update pings.

Limitations

• Lexical only. No embeddings or semantic search. Ranking is BM25 + token overlap + synonyms.
• No multi-repo federation. One SQLite DB per repo.
• Workspace is a full copy. For very large repos (>100k files), workspace init takes time and disk.
• No web UI. CLI + MCP only, by design.
• Patch approval is manual. The AI cannot auto-apply patches — that's the safety model.

Documentation

• docs/architecture.md — module map, data flow, schema.
• docs/workspace.md — workspace engine.
• docs/git-safety.md — GitManager safety model.
• docs/task-workflow.md — task lifecycle.
• docs/patch-approval.md — patch approval safety.
• docs/benchmarks.md — local benchmark suite.
• docs/mcp-workspace-tools.md — MCP workspace tools.
• docs/mcp-setup.md — generic MCP client config.
• docs/security.md — threat model and controls.
• CHANGELOG.md — versioned changes.

License

MIT. See LICENSE. "# tokenfluid-mcp"