JSON2TOON v2.0 🚀

Advanced Token-Optimized Object Notation - The most powerful JSON compression system for AI context management.

JSON2TOON is a next-generation MCP server that revolutionizes JSON compression with AI-powered pattern detection, achieving 75-85% token reduction while maintaining perfect data integrity.

---

✨ Key Features

🎯 4 Compression Levels

• MINIMAL (30-40% savings): Lightning-fast key abbreviations
• STANDARD (40-60% savings): Balanced performance + compression
• AGGRESSIVE (60-75% savings): Advanced pattern optimization
• EXTREME (75-85% savings): Maximum compression with zlib

🤖 AI-Powered Pattern Detection

• 17+ Pattern Types: API responses, databases, time series, graphs, trees, and more
• Smart Strategy Selection: Automatic optimization based on data structure
• Confidence Scoring: Each pattern comes with accuracy metrics
• Compression Potential: Estimates savings before conversion

🔧 12 Advanced MCP Tools

1. convert_to_toon - Multi-level JSON compression
2. convert_to_json - Lossless decompression
3. analyze_patterns - Deep pattern analysis with AI
4. get_optimal_strategy - AI-recommended compression plan
5. calculate_metrics - Detailed compression statistics
6. batch_convert - High-performance batch processing
7. smart_optimize - Auto-detect and apply best compression
8. compare_levels - Side-by-side level comparison
9. validate_toon - Format validation + round-trip testing
10. suggest_abbreviations - Custom abbreviation generation
11. estimate_savings - Pre-conversion savings estimation
12. get_server_stats - Real-time performance metrics

💡 Advanced Capabilities

• 150+ Key Abbreviations (vs 68 in TOON v1.0)
• String Dictionary: De-duplication for repeated values
• Partial Schema Compression: Works with inconsistent data
• Value Pattern Compression: Optimizes timestamps, UUIDs, URLs, emails
• Reference System: Eliminates duplicate structures
• zlib Integration: Optional extreme compression

---

📊 Performance Benchmarks

Data Type	Compression	Speed	Round-Trip
API Responses	50-65%	0.3ms/KB	✅ Perfect
Database Results	60-70%	0.3ms/KB	✅ Perfect
Time Series	65-75%	0.5ms/KB	✅ Perfect
User Profiles	45-55%	0.3ms/KB	✅ Perfect
Config Files	40-55%	0.1ms/KB	✅ Perfect

---

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/muhammedehab35/JSON2TOON-MCP.git
cd json2toon

# Install with pip
pip install -e .

# Or use Docker
docker-compose up -d

MCP Configuration

Add to your Claude Desktop config (~/.config/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "json2toon": {
      "command": "python",
      "args": ["-m", "src.mcp_server"],
      "cwd": "/path/to/json2toon"
    }
  }
}

Docker Configuration:

{
  "mcpServers": {
    "json2toon": {
      "command": "docker",
      "args": ["run", "-i", "json2toon:2.0.0"]
    }
  }
}

---

💻 Usage Examples

Basic Conversion

from src.advanced_converter import convert_json_to_toon, convert_toon_to_json, CompressionLevel

# Simple conversion with STANDARD level
data = {
    "id": 12345,
    "name": "John Doe",
    "email": "john@example.com",
    "created_at": "2025-01-01T00:00:00Z"
}

# Convert to TOON
toon = convert_json_to_toon(data, level=CompressionLevel.STANDARD)
print(f"Compressed: {toon}")

# Convert back to JSON
original = convert_toon_to_json(toon)
print(f"Restored: {original}")

Advanced Pattern Analysis

from src.pattern_analyzer import AdvancedPatternAnalyzer

analyzer = AdvancedPatternAnalyzer()

# Analyze your data
patterns = analyzer.analyze(large_json_data)

# Get compression strategy
strategy = analyzer.get_compression_strategy(large_json_data)

print(f"Detected {len(patterns)} patterns")
print(f"Expected savings: {strategy.expected_savings * 100:.1f}%")
print(f"Recommended level: {strategy.recommended_level}")
print(f"Reasoning: {strategy.reasoning}")

Smart Optimization

from src.optimizer import SmartOptimizer

optimizer = SmartOptimizer()

# Automatic optimization with profile
result = optimizer.optimize(data, profile="balanced")
# Profiles: "speed", "balanced", "size"

print(f"Used profile: {result['profile_used']}")
print(f"Selected level: {result['level_selected']}")
print(f"Savings: {result['metrics']['savings_percent']:.1f}%")

Batch Processing

from src.advanced_converter import AdvancedTOONConverter, CompressionLevel

converter = AdvancedTOONConverter(level=CompressionLevel.AGGRESSIVE)

# Process multiple items
items = [
    {"id": i, "data": f"Item {i}"}
    for i in range(1000)
]

for item in items:
    toon = converter.json_to_toon(item)
    # Process compressed data

---

🔬 MCP Tools Examples

In Claude Code

1. Convert with Custom Level

Use the convert_to_toon tool with:
- json_data: <your JSON>
- level: 3 (AGGRESSIVE)

2. Analyze Patterns

Use the analyze_patterns tool to detect:
- Pattern types
- Compression potential
- Optimization recommendations

3. Compare All Levels

Use the compare_levels tool to see:
- Side-by-side comparison
- Savings per level
- Best recommendation

4. Smart Auto-Optimize

Use the smart_optimize tool with:
- json_data: <your JSON>
- profile: "size" (for maximum compression)

---

📖 Format Specification

TOON v2.0 Structure

{
  "_toon": "2.0",           // Version identifier
  "_lvl": 2,                // Compression level used
  "d": {...},               // Compressed data
  "_refs": {...},           // Optional: structure references
  "_dict": {...}            // Optional: string dictionary
}

Key Abbreviations (Sample)

Original	TOON	Original	TOON	Original	TOON
id	i	email	eml	status	s
name	n	phone	ph	created_at	ca
type	t	address	addr	updated_at	ua
value	v	username	unm	timestamp	ts

150+ abbreviations covering common API, database, and application fields.

Value Optimizations

• null → ~
• true → T, false → F
• Timestamps: $ts:2025-01-01T00:00:00Z
• UUIDs: $uid:550e8400-e29b-41d4-a716-446655440000
• String refs: @s0, @s1 (from dictionary)

Schema Compression

Before:

[
  {"id": 1, "name": "Alice", "email": "alice@test.com"},
  {"id": 2, "name": "Bob", "email": "bob@test.com"},
  {"id": 3, "name": "Carol", "email": "carol@test.com"}
]

After (TOON):

{
  "_sch": ["i", "n", "eml"],
  "_dat": [
    [1, "Alice", "alice@test.com"],
    [2, "Bob", "bob@test.com"],
    [3, "Carol", "carol@test.com"]
  ]
}

Savings: ~55-60% for arrays with consistent schemas

---

🧪 Testing

# Run all tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=src --cov-report=html

# Specific test file
pytest tests/test_converter.py -v

# Run tests in Docker
docker-compose run json2toon-server pytest tests/ -v

Test Coverage

• ✅ Converter: 100+ test cases covering all compression levels
• ✅ Pattern Analyzer: 30+ tests for all 17 pattern types
• ✅ Round-trip: Perfect data integrity verification
• ✅ Edge cases: Unicode, large numbers, special characters
• ✅ Performance: Benchmarks for all levels

---

🐳 Docker Deployment

Build Image

docker build -t json2toon:2.0.0 .

Run with Docker Compose

# Production mode
docker-compose up -d json2toon-server

# Development mode
docker-compose --profile dev up json2toon-dev

Docker Features

• ✅ Python 3.11 optimized image
• ✅ Non-root user for security
• ✅ Health checks
• ✅ Resource limits (2 CPU, 1GB RAM)
• ✅ Logging configuration
• ✅ Development mode with live reload

---

📐 Architecture

┌─────────────────────────────────────────┐
│         JSON2TOON MCP Server            │
│              (v2.0)                     │
└─────────────┬───────────────────────────┘
              │
    ┌─────────┼─────────┐
    │         │         │
    ▼         ▼         ▼
┌─────────┐ ┌──────────┐ ┌──────────┐
│Advanced │ │Pattern   │ │Smart     │
│Converter│ │Analyzer  │ │Optimizer │
└─────────┘ └──────────┘ └──────────┘
    │         │              │
    └─────────┴──────────────┘
              │
    ┌─────────┼─────────┐
    ▼         ▼         ▼
┌──────┐  ┌──────┐  ┌──────┐
│Schema│  │String│  │Value │
│Comp  │  │ Dict │  │ Comp │
└──────┘  └──────┘  └──────┘

---

🎯 Pattern Types Detected

1. API Response - REST, GraphQL, JSON-RPC
2. Database Record - CRUD, audit logs, versioned
3. User Data - Profiles, auth, preferences
4. Pagination - Page-based, offset-based
5. Nested Address - Street, city, state, country
6. Nested Coordinates - Lat/lng/alt
7. Nested Dimensions - Width/height/depth
8. Nested Metadata - Created/updated by, tags
9. Homogeneous Array - Same-type elements
10. Consistent Schema Array - Similar object structures
11. Repeated Structure - Duplicate patterns
12. Time Series - Temporal data sequences
13. Graph Node - Network/graph structures
14. Tree Structure - Hierarchical data
15. Enum Values - Limited value sets
16. Sparse Array - Many null/empty values
17. Deep Nesting - Complex nested levels

---

🔧 Development

Setup Development Environment

# Install dev dependencies
pip install -e ".[dev]"

# Format code
black src/ tests/

# Lint
ruff src/ tests/

# Type check
mypy src/

Code Quality Tools

• black: Code formatting (line length: 100)
• ruff: Fast Python linter
• mypy: Static type checking (strict mode)
• pytest: Testing framework with async support

---

📊 Comparison with TOON v1.0

Feature	TOON v1.0	JSON2TOON v2.0
Compression Levels	2	4
Key Abbreviations	68	150+
Pattern Types	8	17+
MCP Tools	6	12
Max Savings	60%	85%
String Dictionary	❌	✅
Value Compression	❌	✅
Partial Schema	❌	✅
zlib Support	❌	✅
AI Analysis	Basic	Advanced
Custom Abbreviations	❌	✅
Savings Estimation	❌	✅

---

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Run tests (pytest tests/ -v)
4. Format code (black src/ tests/)
5. Commit changes (git commit -m 'Add amazing feature')
6. Push to branch (git push origin feature/amazing-feature)
7. Open a Pull Request

---

🌟 Use Cases

1. Large API Responses

Save 50-65% tokens when storing API responses in Claude conversations.

2. Database Query Results

Compress database results by 60-70% for efficient context usage.

3. Time Series Data

Achieve 65-75% compression on temporal datasets.

4. Configuration Files

Store configs in compact format with 40-55% savings.

5. Codebase Analysis

Fit more file contents in token limits when analyzing code.

6. Log Processing

Compress structured logs by 50-60% for pattern analysis.

---

🚦 Quick Tips

When to Use Each Level

• MINIMAL: Quick conversions, need high speed
• STANDARD: General purpose (best balance)
• AGGRESSIVE: Large datasets, high savings needed
• EXTREME: Maximum compression, archival use

Optimization Profiles

• speed: Prefer MINIMAL/STANDARD levels
• balanced: Auto-select based on data (recommended)
• size: Prefer AGGRESSIVE/EXTREME levels

Best Practices

1. ✅ Analyze patterns first with analyze_patterns
2. ✅ Use smart_optimize for automatic best results
3. ✅ Validate with validate_toon after conversion
4. ✅ Use estimate_savings before large batch jobs
5. ✅ Monitor with get_server_stats for metrics

pip install -e .
python -m src.mcp_server