AVO — Agentic Variation Operators

An open reproduction of AVO: Agentic Variation Operators for Autonomous Evolutionary Search (Chen, Ye, Xu et al., NVIDIA, 2026), runnable on a laptop.

Classical evolutionary search, and the LLM-augmented systems that followed it, decompose the variation operator into a fixed pipeline:

Vary(P_t) = Generate(Sample(P_t))

The framework samples parents; the model produces one candidate from them. AVO replaces that whole decomposition with a single autonomous agent run:

Vary(P_t) = Agent(P_t, K, f)

The agent sees the full lineage P_t, a domain knowledge base K, and the scoring function f — and decides for itself what to read, what to change, and when to measure. It stops being a candidate generator and becomes the variation operator.

This repo implements that framework, plus the surrounding machinery the paper describes: a git-backed lineage, a correctness-gated score vector, the matches-or-improves commit policy, a supervisor that intervenes on stagnation, and trajectory plots. Two optimisation targets ship with it.

---

The part that matters: it runs on the session you already have

The default driver does not spawn an agent and does not call an API. It hands the variation prompt to the Claude Code session you are already talking to, and that session does the work. Nothing extra is billed, no ANTHROPIC_API_KEY is needed, and the agent doing the optimising is a real general-purpose coding agent — which is exactly what the paper used.

Unattended mode (spawn an agent per step and let it run for days, like the paper's 7-day experiment) is available too, and is opt-in precisely because it spends quota.

---

Install

git clone https://github.com/gatordevin/avo
cd avo
pip install -e ".[all]"     # or: pip install -e .  for the core only
avo doctor

On a system with an externally-managed Python (Homebrew, most Linux distros), use a virtualenv — the --system-site-packages flag reuses a NumPy and Matplotlib you already have:

python3 -m venv --system-site-packages .venv
.venv/bin/pip install -e ".[all]"
.venv/bin/avo doctor

Requirements: Python 3.10+, git, and a C compiler if you want the attention_c target. numpy is needed by the bundled targets, matplotlib for plots. The core framework depends only on PyYAML.

---

Quickstart — drive it from the agent you already have

Full protocol, including Codex and plain-CLI use, in docs/DRIVING.md.

Claude Code

Register the MCP server once, at user scope so it is available in every folder:

claude mcp add avo -s user -- python3 -m avo.mcp_server
# from a virtualenv, point at its interpreter:
claude mcp add avo -s user -- /path/to/avo/.venv/bin/python -m avo.mcp_server

claude mcp list should show avo — ✔ Connected. Optionally install the bundled skill so /avo works anywhere:

cp -r .claude/skills/avo ~/.claude/skills/avo

Then, in a Claude Code session in any directory:

Use the avo tools to evolve the game2048 target for 10 steps. Call avo_start_run, then loop: avo_next_step, do the work it asks for, avo_evaluate until you're happy, then avo_submit. If it reports a stall, call avo_supervisor_brief, answer it, and file it with avo_record_supervisor.

The eleven tools are the whole loop:

tool	what it does
avo_start_run	seed x_0, score it, measure baselines, open the lineage
avo_next_step	the variation prompt: P_t, the index of K, the contract for f
avo_evaluate	run f on the work tree — free, call it as often as you like
avo_submit	end the step: score, then commit or revert per the policy
avo_revert	abandon an experiment without spending the step
avo_status / avo_lineage	where the run is
avo_supervisor_brief / avo_record_supervisor	the stagnation intervention
avo_plot	render the trajectory
avo_list_targets	what can be evolved

Codex

Codex CLI speaks MCP and reads AGENTS.md, so both halves work:

codex mcp add avo -- python3 -m avo.mcp_server

AGENTS.md at the repo root documents the loop and the rules that keep a run honest; Codex picks it up automatically when working in this directory.

Without MCP

Every tool has a CLI twin, so a plain shell works just as well — this is the most portable option and works with any agent, or by hand:

avo start --target game2048          # seeds x0 and prints the first prompt
# ... edit runs/<id>/work/, run runs/<id>/avo-eval as often as you like ...
avo submit -m "expectimax depth 2 with a positional weight matrix"
avo prompt                           # the next step's prompt
avo status
avo plot -o trajectory.png

---

Worked runs

Two complete runs ship with the repo, both driven in session mode by a Claude Code session, both including their dead ends.

attention_decode — beating the vendor kernel

examples/attention-decode-run/ evolves the decode step of attention: one query token against a long KV cache, the computation an LLM runs for every generated token. Scored against mx.fast.scaled_dot_product_attention — Apple's own fused Metal kernel.

0.05 → 1.14× MLX in three steps. This is the one where the evolved kernel actually beats the vendor implementation, and the interesting part is how:

• Step 1 was implementation — split-K flash-decoding took the kernel from 1.6 GB/s to 106 GB/s, about 95% of the machine's streaming limit. That reached 0.95× MLX and exhausted the lever: you cannot read bytes faster than the memory controller delivers them.
• Step 2 was mathematics. The target's gate is an output-error budget rather than exact equality, so the search could change the computation. Measurement showed 99.9% of the softmax mass sits in ~11% of keys, so the kernel now scores every key but reads V only above a threshold derived so the discarded mass is provably under 0.3%. That crossed 1.0, spending 2% of the error budget.

The lesson generalises: once a bandwidth-bound kernel is at the roofline, the only remaining lever is to read fewer bytes, and that is an algorithmic change.

Read the full write-up →

attention_c — the paper's own domain

examples/attention-c-run/ evolves a forward attention kernel in C, reaching 2.2× a straightforward NumPy/BLAS implementation and close to the NEON roofline. Note the honest framing: that baseline is not a tuned attention library, and this kernel is slower than torch's CPU SDPA and MLX — Apple's AMX matrix units are unreachable from portable C. The write-up gives the full comparison.

Three findings from it are worth the click:

• The paper's own algorithm was the wrong answer here. A FlashAttention-style tiled kernel with a streaming online softmax measured worse, twice. At these sizes a whole head fits in L2, so blocking for locality buys nothing while the per-block rescale is pure added work. The cost is arithmetic, not memory.
• -ffast-math silently breaks the standard fast-exp, by algebraically cancelling the add-magic-constant rounding trick it depends on. The correctness gate caught it on an N=3 shape; the throughput number never would have.
• The run forced a target fix. Scoring raw GFLOP/s on a laptop doing other work is not a measurement — identical code ranged 44–76 GFLOP/s in twenty minutes. eval.py now times a NumPy/BLAS reference in the same process, interleaved with the candidate, and scores the ratio.

Read the full write-up →

game2048 — evolving a game-playing policy

examples/game2048-run/ is a complete 8-step run of the game2048 target, driven in session mode by a Claude Code session. The directory holds the unedited output: the evolved policy, the operator's working notes, the full trajectory, the screening tools it built, and its dead ends.

876 → 43 826 — 50× the seed, 14× the strongest baseline. Games reaching 2048: 0% → 77%. Best tile: 512 → 8192. Apple M5, single-threaded, standard library only.

Improvement arrives in discrete jumps separated by plateaus, matching the paper's Figure 5. The two flat versions are pure throughput work that bought the budget the next step spent — the same role the paper's v19→v20 branchless-rescale change plays.

The largest single gain (+50.5%) was not an optimisation. The benchmark scores accumulated game points; the heuristic only measured how survivable a board looked, so nothing in the search knew that merging two 256s banks 512 points. Four steps of throughput work were worth +27% combined; one step of checking what was actually being optimised was worth +50%.

Read the full write-up →

What ships with it

game2048 — evolve a game-playing policy

Evolve agent.py into the strongest 2048 player you can, under a hard thinking-time budget. Scored as the geometric mean of mean game score across four banks of twelve deterministic seeds. Blowing the 120 s budget scores zero, not "slightly less" — so search depth, evaluation-function cost, and pruning all trade against each other, and that trade-off is the problem.

Measured on an Apple M5:

policy	score
seed x_0 (first legal move)	876
random baseline	1 076
corner heuristic baseline	2 565
greedy one-ply baseline	3 132

Strong expectimax players score in the tens of thousands. The worked run above reached 43 826.

attention_c — evolve a kernel, the paper's own domain

Evolve a single-precision forward attention kernel in C: O = softmax(QKᵀ/√D)V, causal and non-causal, D = 64. Gated on agreement with a float64 reference over eighteen shapes — including prime and off-by-one sequence lengths, so a kernel that mishandles its tail fails rather than quietly scoring well.

Scored as speedup over a NumPy/BLAS reference timed in the same process, geometric mean across four sequence lengths × two masking modes. 1.0 is parity with the library. Scoring a ratio rather than raw GFLOP/s makes the benchmark immune to whatever else the machine is doing — absolute throughput on a shared laptop moves by more than most optimisations are worth.

kernel	score
seed x_0 (naive, materialises the full N×N score matrix)	0.19×
NumPy/BLAS baseline — the "cuDNN" of this setup	1.00×
evolved in 3 steps (write-up)	2.11×

The knowledge base covers the online-softmax formulation, tiling and block-size selection, CPU vectorisation, threading, and how to interrogate the host machine rather than assuming an ISA. Beating BLAS needs most of them.

---

How it works

The run directory

runs/<run-id>/
  work/              the candidate x_t — a standalone git repo whose history IS the lineage
    .avo/scores.jsonl    every committed version's full score vector
  kb/                the knowledge base K, copied in so paths are stable
  avo-eval           f, as a zero-argument shim the agent can call at will
  NOTES.md           scratch space that survives across steps
  trajectory.jsonl   every step, accepted or rejected
  rejected/          the diff of each rejected candidate, kept for the record
  logs/              evaluator and agent logs

Making the lineage a git repo means the agent inspects P_t with tools it already knows — git log, git show v7:attention.c, git diff v6 v7 — instead of a bespoke API. Each accepted version is a commit tagged vN whose message carries the score vector.

The commit policy

Paper §3.2: a candidate is committed only if it passes the correctness gate and matches or improves the best committed score so far. Anything else is reverted and its diff archived — it stays part of the agent's internal search trajectory, but never enters the lineage.

Correctness is a gate, not a dimension. A candidate that fails it scores zero regardless of what it measured (§3.1). In attention_c that means a kernel that is 10× faster and numerically wrong is worth exactly as much as one that does not compile.

The score vector

f(x) = (f_1(x), …, f_n(x)) — one number per benchmark configuration, with the geometric mean as the scalar being maximised. This is what makes per-config movement diagnostic: a change that helps n1024 and hurts n128 is a blocking problem, not a win, and the aggregate alone would hide it.

The supervisor

Paper §3.3: long autonomous runs fail in two ways — the agent stalls when it exhausts its current line of attack, or enters unproductive cycles of edits that keep failing. After N steps without a new best (default 3), AVO stops and asks for a redirect: a review of the whole trajectory that proposes several concrete, different optimisation directions. The redirect is injected into the next variation prompt as a strong prior, and consumed by exactly one step.

In session mode the supervisor is the same session wearing a different hat, which is cheap enough to actually use. In unattended mode it is a separate agent run with read-only intent.

The trajectory

avo plot renders the paper's Figure 5/6: running-best geometric mean as a step function, filled circles at each new best, dotted per-configuration curves, and the baselines as horizontal lines. Same caveat as the paper — it shows the committed sequence, not the internal search tree explored between commits.

---

Unattended mode

To reproduce the paper's setup, where the operator is a spawned agent and nobody is watching:

avo run --target attention_c --backend claude_cli --max-steps 40 --time 12h
avo run --resume runs/attention_c-20260321-091500 --time 24h

Backends: claude_cli (Claude Code headless — the closest analogue to the paper's agent), api (a self-contained agent loop on the Messages API, for people with only an API key), agent_sdk (in-process via claude-agent-sdk), and mock (a shell command, for testing the machinery without a model).

This spends quota or credits on every step. Session mode does not.

---

Adding your own target

A target is a directory with a target.yaml, a seed program, a knowledge base, and an evaluator. The evaluator is any executable in any language; the whole contract is one JSON object on stdout:

{"correct": true,
 "metrics": {"config_a": 1520.3, "config_b": 1477.0},
 "error": null,
 "notes": "shown to the agent"}

correct is the gate. metrics is the score vector. The scalar being optimised is their geometric mean unless you supply an explicit primary.

name: my_target
description: One line, shown in `avo targets`.
seed: seed                 # copied to work/ as x_0
knowledge_base: kb         # copied to the run dir as K
entrypoint: kernel.c       # informational, used in prompts

evaluate:
  command: ["python3", "{target}/eval.py", "--workdir", "{workdir}"]
  timeout: 30m

baselines:                 # optional, measured once before evolution starts
  command: ["python3", "{target}/eval.py", "--baselines"]

score:
  direction: maximize

agent:
  goal: |
    What the agent is actually trying to do, and what the trade-offs are.

See docs/TARGETS.md for the full contract and tests/fixtures/toy/ for a minimal working example.

The knowledge base is worth real effort. It is the K in Agent(P_t, K, f), and the difference between an agent that rediscovers tiling from first principles over ten steps and one that gets there in two.

---

What is faithful, and what is not

Faithful:

• the operator formulation Vary(P_t) = Agent(P_t, K, f) — a real coding agent with file editing, shell access and persistent memory, given no task-specific modifications
• single-lineage continuous evolution with git-backed state (§3.3)
• the correctness gate and the n-dimensional score vector (§3.1)
• the matches-or-improves commit policy, with failed attempts excluded from the lineage (§3.2)
• supervisor intervention on stagnation and unproductive cycles (§3.3)
• geometric-mean aggregation across benchmark configurations, and Figure 5/6 trajectory plots

Not faithful, and deliberately so:

• The hardware. The paper evolves attention kernels on B200 GPUs against cuDNN and FlashAttention-4. attention_c is the same problem on a CPU against NumPy/BLAS. The optimisations transfer in kind (tiling, online softmax, vectorisation, scheduling), not in magnitude.
• The scale. The paper ran 7 days, 40 committed versions, 500+ explored directions. A session-mode run of 10–20 steps is a demonstration, not a replication.
• Population structure. Like the paper, this implements the single-lineage case to isolate the operator. Archive- and island-based regimes are compatible with the formulation but not implemented.

---

Repo layout

src/avo/
  types.py        Score, LineageEntry, the correctness gate, geomean
  config.py       target specs and run configuration
  lineage.py      P_t as git history
  scoring.py      f as an external process
  knowledge.py    K
  prompts.py      the variation and supervisor prompts — the whole framework/agent interface
  run.py          run state: seed, evaluate, commit policy, trajectory
  session.py      driver: the session you already have is the operator
  loop.py         driver: unattended, spawns an agent per step
  mcp_server.py   the same operations as MCP tools (no dependencies)
  cli.py          the same operations as subcommands
  plot.py         Figure 5/6
  agents/         backends for unattended mode
targets/
  game2048/       policy evolution under a time budget
  attention_c/    kernel evolution — the paper's domain, on a CPU
examples/
  attention-decode-run/  beats Apple's own fused kernel by changing the maths
  attention-c-run/       CPU kernel evolution, with an honest baseline caveat
  attention-metal-run/   GPU prefill — every CUDA instinct measured worse
  game2048-run/          policy evolution — 50x the seed
docs/
  PAPER_MAP.md    every section of the paper, and where it lives in the code
  TARGETS.md      the evaluator contract
  DRIVING.md      how to drive a run from Claude Code, Codex, or a shell
AGENTS.md         cross-agent instructions (read automatically by Codex)
.claude/skills/   the `/avo` skill for Claude Code

---

Citing

This is an independent reproduction. Cite the original work:

@article{chen2026avo,
  title  = {AVO: Agentic Variation Operators for Autonomous Evolutionary Search},
  author = {Chen, Terry and Ye, Zhifan and Xu, Bing and Ye, Zihao and Liu, Timmy
            and Hassani, Ali and Chen, Tianqi and Kerr, Andrew and Wu, Haicheng
            and Xu, Yang and Chen, Yu-Jung and Chen, Hanfeng and Kane, Aditya
            and Krashinsky, Ronny and Liu, Ming-Yu and Grover, Vinod and Ceze, Luis
            and Bringmann, Roger and Tran, John and Liu, Wei and Xie, Fung
            and Lightstone, Michael and Shi, Humphrey},
  journal = {arXiv preprint arXiv:2603.24517},
  year    = {2026}
}

Licensed under Apache-2.0. Not affiliated with or endorsed by NVIDIA.