<div align="center">

openlocal

Open tools. Local brains. Zero cloud.

A tool belt that small local models can actually use — and the receipts to prove it.

</div>

---

pip install openlocal
openlocal quickstart

That's it. quickstart finds the model server you already run — ollama, LM Studio, llama.cpp, vLLM — writes your config, measures which tool-calling protocol your model can actually drive, and runs a real task so you see it work:

$ openlocal quickstart
openlocal quickstart - looking for a model server...
found ollama at http://127.0.0.1:11434/v1 with 6 model(s)
wrote ~/.openlocal/models.toml
default model: qwen3-4b  (served as qwen3:4b)
measuring which tool protocol this model can drive...
  -> native tool calling

demo: Read notes.md and tell me how many lines it has.
  tool     file_read
  final

  The file notes.md has 3 lines.

You are set up. Try:
  openlocal run "list the files here and summarise what this project is"
  openlocal eval          # score this model on 15 deterministic tool tasks
  openlocal mcp           # serve these tools to Claude Code over MCP

Why openlocal

🪶 Zero dependencies	Pure standard library. Installs in seconds, runs on a Raspberry Pi, and can't be broken by somebody else's release.
🔌 Works with what you have	Anything that speaks the OpenAI /v1 API. One URL, no adapters, no accounts.
📊 Measured, not claimed	10 models from 9 vendors, scored on 17 deterministic tasks. No LLM judge — every check is a file diff or a regex.
🧰 One registry, two surfaces	The same 8 tools power an MCP server (Claude Code, any MCP client) and a local-model agent loop.
🩹 Tuned on real failures	Every ergonomic rule here was paid for by a model breaking. The lessons table is the part you can't get anywhere else.
🧩 Add a vendor by editing TOML	New model family? Edit families.toml, run openlocal probe, send a PR. No Python needed.

The leaderboard

Same tasks, same tools, one model at a time, two runs each. tools is the protocol each model was measured driving — not the one its docs claim.

model	vendor	tools	passed	pass rate	bad json	tok/s
qwen3.8-4b	Alibaba	native	17/17	100%	0	40.4
glm-4.7-flash	z.ai	native	17/17	100%	0	39.9
qwen3.8-27b	Alibaba	native	17/17	100%	0	16.6
nemotron-nano-4b	NVIDIA	native	16/17	94%	0	54.9
gemma-4-e4b	Google	native	16/17	94%	0	50.3
gpt-oss-20b	OpenAI	native	16/17	94%	2¹	45.3
granite-4.1-3b	IBM	native	15/17	88%	4	66.0
llama-3.1-8b	Meta	native	14/17	85%	2	33.4
mistral-small-3.2	Mistral	native	14/17	82%	0	11.1
phi-4-mini	Microsoft	native	13/17	79%	0	61.5

_{¹ gpt-oss's harmony parser rejects some of its own tool calls (llama.cpp answers 500). The client's retry ladder recovers every one — the retries column in RESULTS.md keeps that honest.}

Reproduce it: openlocal leaderboard --repeat 2

Why your small model fails at tools

A 3B model doesn't fail because it's stupid. It fails because tool output is a prompt, and most tool output is written for humans. Everything below was measured — kept when the score went up, reverted when it didn't.

what broke	the fix	measured effect
"What is line 300?" → model answered line 311	number every line: 300| row 300	fixed in one turn, across models
Model walked a 500-line file one line per turn	footers state facts, never commands ("file continues" ≠ "call again with…")	phi: 12 wasted steps → 2
Prose instead of a tool call	prose with no tool = the final answer (native only)	phi: 51 malformed replies → 5
…but "Sure, I'll use file_grep" is a plan	intent detection: plans get nudged, answers get accepted	runs stopped ending on intentions
file_read(notes.md) typed as text	prose call-syntax is parsed and executed	wasted turns eliminated
{"limit": None}, trailing junk, split objects	JSON repair: brace scan, tail closing, Python literals, fragment merge	phi bad_json 5 → 0
Model wrote the file before reading the source	stale-write detection at finish time	llama: 13 → 15 / 15
Model guessed "44" and repeated it when nudged	one turn of tool_choice: "required"	granite recovered the task
Read a 4-line CSV perfectly, said the sum was 40	a calc tool — and a guard so it can't be used before reading	granite: +2 tasks
Wrote the output file without ever reading the source	thin-air-write detection at finish time	qwen-4b over MCP recovered
Copied the N| reference prefixes into edit targets	file_edit strips them when the raw text does not match	edit_code: 10/10 models pass
Same call, forever (A→B→A→B)	repeat guard replays the cached result	llama pagination loop gone
Server 500s on its own tool-call syntax	retry ladder: as-asked → warmer → same turn without tools	gpt-oss: 11/15 → 14/15
Reverted: one more system-prompt rule	—	granite 12/15 → 11/15. Fewer rules win.

MCP is a first-class citizen, not a wrapper

openlocal eval --via-mcp reruns the whole eval suite with the tools served over a real stdio MCP server — schemas crossing the wire, results as content blocks, a subprocess per task. The scores match the in-process numbers, which is the point: what Claude Code experiences is what was measured.

Use it from Claude Code (or any MCP client)

pip install "openlocal[mcp]"
claude mcp add openlocal -- openlocal mcp

The same 10 tools your local model uses, now in Claude Code. finish stays behind — it's loop control, not a capability.

Commands

openlocal quickstart              # find a server, configure, probe, demo
openlocal run "goal"              # agent loop over your tools
openlocal run "goal" --url http://host:1234/v1     # no config at all
openlocal chat "hello"            # one plain turn, no tools
openlocal tools                   # what the model can call
openlocal probe                   # measure native vs JSON tool calling
openlocal eval                    # 17 deterministic tasks, pass/fail
openlocal eval --via-mcp          # same tasks, tools served over a REAL MCP server
openlocal leaderboard --repeat 2  # every configured model, one table
openlocal mcp                     # stdio MCP server
openlocal models / serve / stop / status / pull    # local llama-server management

smol is a shorter alias for the same CLI.

Configuration

Your workspace is ~/.openlocal (or any directory containing models.toml, or $OPENLOCAL_HOME). It holds models.toml, state.json, logs, downloads and eval results — never the installed package.

[engine.ollama]
type = "external"
base_url = "http://127.0.0.1:11434/v1"

[[model]]
id = "qwen"
engine = "ollama"
family = "qwen"              # tells openlocal this vendor's tool-calling quirks
served_model = "qwen3:4b"    # the exact name the backend knows
default = true

env	default	meaning
OPENLOCAL_HOME	~/.openlocal	workspace directory
OPENLOCAL_BASE_URL	–	point every command at one /v1 server
SMOL_ROOT	cwd	sandbox root for the file and shell tools
SMOL_MAX_CHARS	8000	hard cap on one tool result
SMOL_SHELL_ALLOW	dev commands	comma list, or *
SMOL_TOOL_TIMEOUT	120	seconds before an MCP tool call is abandoned

The tools

file_list(path=".")                       list files and folders
file_read(path, start_line=1, limit=200)  read a file, one page at a time
file_write(path, content)                 write a file
file_edit(path, find, replace)            replace text exactly, leave the rest alone
file_append(path, content)                add to the end without touching what exists
file_grep(pattern, path=".")              search files
web_search(query, limit=5)                search the web (DuckDuckGo, or Tavily via key)
web_read(url, offset=0, max_chars=4000)   read a page as text
shell_run(command, timeout=60)            run one allowlisted command
calc(expression)                          exact arithmetic - models cannot count
finish(answer)                            agent loop only: end the task

House rules, enforced by tests: flat arguments with defaults, an example call in every description, one-line result headers, hard-capped output with a truthful continuation hint, and errors that name the next move — no such file: x - call file_list(".") to see the files that exist.

Architecture

tools/          one registry  →  mcp_server.py   (Claude Code speaks MCP to it)
                              →  agent.py        (a local model calls the same tools)
_http.py        the entire network layer, on urllib (this is why deps = 0)
runner.py       external servers, or llama-server processes it starts and owns
protocol.py     native tool_calls → loose JSON → repair → symptom-specific nudge
families.toml   per-vendor quirks as DATA - the file contributors edit
evals/          17 deterministic tasks + a leaderboard across models

Contributing

Adding a model family takes no Python: edit families.toml, add your model to models.toml, then

openlocal probe --model your-model
openlocal eval  --model your-model

and paste the resulting row into the PR. See CONTRIBUTING.md.

Running local GGUF models on AMD hardware (ROCmFP4 + MTP speculative decoding) is documented in docs/ENGINE.md — optional, and not needed for anything above.

License

MIT © DevXV3