Agent Farm v3.4 - Chunked Write Edition

AI organism evolution and parallel task execution with tool-enabled agents. Now with Chunked Write Pattern for generating large documents and code files!

What's New in v3.4

• Chunked Write Pattern: Bugs write sections in parallel, Python assembles directly
• chunked_write: Generate large markdown/text documents (unlimited size)
• chunked_code_gen: Generate multi-function code files in parallel
• chunked_analysis: Multi-perspective analysis with synthesis
• Bypasses 500-char limit: Each bug writes small chunks, combined output is unlimited

Performance

• 8.6x faster than v3.0 (103s -> 12s for 4-task swarm)
• 1 iteration per task (was 3-5)
• 100% success rate with real tool data
• Local synthesis - qwen2.5:14b synthesizes results (no cloud tokens!)

Models

Role	Model	VRAM	Purpose
Scout	qwen3:4b	2.5GB	Reconnaissance
Worker	qwen3:4b	2.5GB	Task execution
Memory	qwen3:4b	2.5GB	Context retention
Guardian	qwen3:4b	2.5GB	System monitoring
Learner	qwen3:4b	2.5GB	Pattern acquisition
Synthesizer	qwen2.5:14b	8.99GB	Result synthesis

MCP Tools (30)

Colony Management

• spawn_colony - Create bug colony (standard/fast/heavy/hybrid)
• list_colonies - List active colonies
• colony_status - Detailed colony info
• quick_colony - Quick health check
• dissolve_colony - Remove colony
• cleanup_idle - Remove idle colonies
• farm_stats - Comprehensive statistics

Swarm Deployment

• deploy_swarm - Deploy tasks to colony
• quick_swarm - One-shot spawn + deploy

Specialized Swarms

• code_review_swarm - 4-perspective code review
• code_gen_swarm - Generate code + tests + docs
• file_swarm - Parallel file operations
• exec_swarm - Parallel shell commands
• api_swarm - Parallel HTTP requests
• kmkb_swarm - Multi-angle knowledge queries

Tool-Enabled Agents

• tool_swarm - Deploy bugs with real system tools
• system_health_swarm - Quick system health check
• recon_swarm - Directory/codebase reconnaissance
• deep_analysis_swarm - Deep disk/file analysis
• worker_task - Single worker with full tools

Direct Operations

• heavy_write - Direct file write (bypasses LLM for large content)
• synthesize - Standalone synthesis of any JSON results

Chunked Write Pattern (NEW)

• chunked_write - Generate large documents via parallel section writing
• chunked_code_gen - Generate code files with functions written in parallel
• chunked_analysis - Multi-perspective analysis with synthesis

Bug Tool Permissions

Role	Tools
Scout	read_file, list_dir, file_exists, system_status, process_list, disk_usage, check_service, exec_cmd
Worker	read_file, write_file, list_dir, exec_cmd, http_get, http_post, system_status, disk_usage, check_service
Memory	read_file, kmkb_search, kmkb_ask, list_dir, system_status, process_list, disk_usage, check_service, exec_cmd
Guardian	system_status, process_list, disk_usage, check_service, read_file, list_dir, exec_cmd
Learner	read_file, analyze_code, list_dir, kmkb_search, system_status, process_list, disk_usage, check_service, exec_cmd

Structured Output Details

Agent Farm v3.3 uses Ollama's structured output feature to enforce JSON schemas on model responses:

# Bug responds with guaranteed-valid JSON:
{"tool": "system_status", "arg": ""}
{"tool": "exec_cmd", "arg": "df -h"}
{"tool": "check_service", "arg": "ollama"}

The constrained decoding (GBNF grammar) masks invalid tokens during generation, ensuring:

• Always valid JSON
• Correct tool names
• Proper argument structure
• No parsing failures

Results now include a mode field showing which method was used:

• structured - JSON schema enforced
• structured+autoformat - JSON + simple result formatting
• structured+deep - JSON with multi-step reasoning
• regex - Fallback regex parsing
• regex+autoformat - Regex + simple result formatting

Chunked Write Pattern

The chunked write pattern solves the ~500 char output limitation of small models by decomposing large tasks:

1. PLANNER BUG (qwen2.5:14b)
   |-- Creates structured JSON outline
   |-- {"sections": [{"title": "...", "description": "..."}]}

2. WORKER BUGS (qwen3:4b) - IN PARALLEL
   |-- Each writes one section (~300-500 chars)
   |-- 4 workers = 4 sections simultaneously

3. PYTHON CONCATENATION (NO LLM)
   |-- header + separator.join(sections)
   |-- Zero token cost, instant assembly

4. DIRECT FILE WRITE (NO LLM)
   |-- tool_write_file() saves result
   |-- Bypasses any output corruption

Performance

Tool	Output Size	Sections	Time
chunked_write	9.6 KB	5	78s
chunked_code_gen	1.9 KB	4 functions	88s
chunked_analysis	Varies	4 perspectives	~60s

Why It Works

• Small models excel at focused, short outputs
• Each section is within the "safe zone" (<500 chars)
• Python handles assembly (no LLM token cost)
• Parallel execution via ThreadPoolExecutor
• Structured output ensures reliable planning

Usage Examples

System Health Check

agent-farm:system_health_swarm

Custom Task Swarm

agent-farm:tool_swarm
  colony_type: "heavy"
  tasks: [
    {"prompt": "Check CPU temperature"},
    {"prompt": "List top 5 memory processes"},
    {"prompt": "Check if docker is running"}
  ]

Large File Write (Direct)

agent-farm:heavy_write
  path: "/tmp/large_output.txt"
  content: "... large content ..."

Codebase Reconnaissance

agent-farm:recon_swarm
  target_path: "/home/kyle/repos/my-project"

Generate Large Document (Chunked)

agent-farm:chunked_write
  output_path: "/tmp/security_guide.md"
  spec: "Linux server security hardening guide"
  num_sections: 5
  doc_type: "markdown"

Output: 9KB+ document with 5 coherent sections

Generate Code File (Chunked)

agent-farm:chunked_code_gen
  output_path: "/tmp/utils.py"
  spec: "File utilities: read, write, copy, delete"
  language: "python"
  num_functions: 4

Output: Complete Python module with 4 functions

Multi-Perspective Analysis

agent-farm:chunked_analysis
  target: "/home/kyle/repos/project"
  question: "What are the architectural patterns?"
  num_perspectives: 4

Output: Analysis from Structure, Patterns, Quality, Performance perspectives

Installation

cd ~/repos/agent-farm
uv venv
uv pip install -e .

Claude Desktop Config

{
  "mcpServers": {
    "agent-farm": {
      "command": "/home/kyle/repos/agent-farm/.venv/bin/python",
      "args": ["-m", "agent_farm.server"]
    }
  }
}

Changelog

v3.4.0 (2026-01-23)

• Chunked Write Pattern - Bugs write sections in parallel, Python assembles
• chunked_write - Generate unlimited-size documents (tested: 9.6KB in 78s)
• chunked_code_gen - Generate multi-function code files in parallel
• chunked_analysis - Multi-perspective analysis with synthesis
• Bypasses 500-char bug limitation via task decomposition
• Planner uses structured JSON output for reliable outlines

v3.3.0 (2026-01-23)

• Ollama Structured Output - JSON schema enforcement via constrained decoding
• Reliable tool parsing - No more regex failures
• Mode tracking - Results show parsing method used
• Regex fallback - Legacy parsing still available as backup
• All roles get exec_cmd for complex shell queries

v3.2.0 (2026-01-22)

• Synthesizer role - qwen2.5:14b for accurate result synthesis
• synthesize parameter - Added to tool_swarm, system_health_swarm, recon_swarm, deep_analysis_swarm
• synthesize tool - Standalone synthesis of any JSON results
• No more Claude synthesis tax - bugs do ALL the work locally

v3.1.0 (2026-01-20)

• 8.6x speed improvement (103s -> 12s)
• Auto-format results skip redundant LLM calls
• Reject invalid tools instantly
• Force tool usage before answers
• Complex shell command support fixed
• All roles upgraded to qwen3:4b minimum

v3.0.0 (2026-01-19)

• Linux rebuild from Windows version
• Tool-enabled agents with role permissions
• System health, recon, worker swarms
• TRUE PARALLEL via ThreadPoolExecutor