SpatialLens

USD/USDZ/USDA scene graph parser and spatial analysis tool for AI agents.

SpatialLens gives AI coding agents (Claude, Codex, Gemini, etc.) structured access to 3D scene data from Apple's USD ecosystem. It parses binary .usdz and text .usda files into structured JSON that any AI can reason about — entity hierarchies, material shader graphs, spatial positions, transforms, and relationships between objects.

Status: Work in Progress. Actively developed and tested against real visionOS game assets. Core parsing and spatial analysis are functional. Contributions and feedback welcome.

---

Why This Exists

AI agents working on visionOS, RealityKit, or any USD-based 3D project are blind to what's actually in the asset files. They can read Swift code that references entity names and positions, but they can't see the USDZ contents — the hierarchy, the materials, the transforms, the shader graphs. This creates a gap where the AI guesses instead of knowing.

SpatialLens closes that gap by translating binary 3D assets into structured text that AI agents natively understand.

---

Two Ways to Use It

1. CLI (any AI agent, any environment)

python cli.py <command> <file> [options]

No MCP required. Runs anywhere with Python 3.12+ and Apple's USD tools. Output is JSON to stdout — pipe it, save it, paste it into a prompt.

2. MCP Server (Claude Desktop, or any MCP-compatible client)

python server.py

Exposes all tools over MCP for real-time use during coding sessions.

---

Prerequisites

• macOS (required for Apple USD tools)
• Python 3.12+
• Apple USD CLI tools — usdcat and usdtree at /usr/bin/ (ships with Xcode Command Line Tools)

Verify your setup:

usdcat --version    # Should print "Apple USD Tools (x.x.x)"
usdtree --version   # Same
python3 --version   # 3.12+

---

Setup

git clone https://github.com/RevivrStudios/spatiallens-mcp.git
cd spatiallens-mcp
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

---

CLI Commands

All commands output structured JSON to stdout unless noted.

scene — Full Scene Graph

python cli.py scene Player_Ships.usdz

Returns the complete prim hierarchy with types, paths, properties, and children. Use this for deep analysis.

tree — Quick Hierarchy (text output)

python cli.py tree Player_Ships.usdz

Returns an indented text tree. Fast overview of what's in a file.

materials — Materials and Shader Graphs

python cli.py materials TerrainGlass.usda

Returns all materials with shader node IDs, input parameters, output connections, and the full shader graph. Use this for debugging visual issues.

transforms — Positioned Entities

python cli.py transforms Player_Ships.usdz

Returns every entity that has translate, rotate, or scale data.

entities — List All Entities

python cli.py entities Enemy_Ships.usdz
python cli.py entities Enemy_Ships.usdz --type Mesh

Flat list of all entities. Filter by type: Mesh, Material, Shader, Xform, Sphere, Scope.

details — Single Entity Deep Dive

python cli.py details Player_Ships.usdz --path "/__spaceships_set/Meshes"

Full properties and children for one specific entity. If the path doesn't exist, returns all available paths.

layout — Spatial Layout Analysis

python cli.py layout Player_Ships.usdz

Returns:

• All positioned entities with coordinates
• Bounding box (min, max, center, size in meters)
• 5 nearest entity pairs with distances
• Plain English spread description (compact / room-scale / large-scale)
• Coordinate convention reference

diff — Compare Two Files

python cli.py diff Player_Ships.usdz Enemy_Ships.usdz

Structural diff: added prims, removed prims, changed properties. Use after editing a USDZ in Reality Composer Pro.

overlaps — Find Overlapping Entities

python cli.py overlaps Player_Ships.usdz --threshold 0.05

Flags entity pairs closer than the threshold (default 1cm). Catches z-fighting, stacked geometry, accidental overlaps.

relationship — Spatial Relationship Between Two Entities

python cli.py relationship Player_Ships.usdz \
    --a "/__spaceships_set/.../SpaceShip1_" \
    --b "/__spaceships_set/.../SpaceShip2_"

Returns distance and plain English description: "SpaceShip2_ is 1.665m to the right and 0.022m toward viewer from SpaceShip1_"

move — Compute New Position

python cli.py move Player_Ships.usdz \
    --path "/__spaceships_set/.../SpaceShip1_" \
    --direction right \
    --meters 0.5

Returns current position, direction, and computed new position. Directions: left, right, up, down, forward, back, toward viewer, away from viewer.

---

Coordinate Convention

SpatialLens uses the RealityKit / visionOS coordinate system:

Direction	Axis	Sign
Right	X	+
Left	X	-
Up	Y	+
Down	Y	-
Toward viewer	Z	+
Away from viewer	Z	-

Units: 1 unit = metersPerUnit meters (read from the USD file metadata). Common values:

• metersPerUnit = 1 — units are meters (RealityKit default)
• metersPerUnit = 0.01 — units are centimeters (common in Sketchfab/Blender exports)

SpatialLens reads metersPerUnit from each file and converts all spatial output to meters automatically.

---

MCP Server Setup

For Claude Desktop, add to your claude_desktop_config.json:

{
  "mcpServers": {
    "spatiallens": {
      "command": "/path/to/spatiallens-mcp/venv/bin/python",
      "args": ["/path/to/spatiallens-mcp/server.py"]
    }
  }
}

The MCP server exposes 11 tools:

• parse_scene_graph — full scene graph
• hierarchy_tree — quick text tree
• get_materials — materials and shaders
• get_transforms — positioned entities
• list_entities — flat entity list with type filter
• get_entity_details — single entity deep dive
• diff_scenes — structural diff
• spatial_layout — spatial analysis with bounding box
• spatial_relationship — relationship between two entities
• check_spatial_overlaps — overlap detection
• move_suggestion — compute new position from direction

---

Project Structure

spatiallens-mcp/
├── cli.py              # Standalone CLI (no MCP needed)
├── server.py           # MCP server entry point
├── usda_parser.py      # USDA text → structured dict parser
├── spatial_math.py     # Coordinate geometry and spatial analysis
├── requirements.txt    # Python dependencies (fastmcp, mcp)
└── tests/
    ├── test_parser.py  # Unit tests for USDA parser
    └── test_tools.py   # Integration tests against real USDZ files

Core modules:

• usda_parser.py — Line-by-line state machine that parses the text output of usdcat --flatten. Extracts prim hierarchy, types, properties, material bindings, shader connections. Handles nested metadata blocks and customData annotations.

• spatial_math.py — Pure math operations on parsed prim data. Position extraction (from xformOp:translate or 4x4 transform matrices), distance calculation, relative position descriptions, bounding box analysis, overlap detection, move suggestions. No app-specific logic.

• server.py — FastMCP wrapper that shells out to usdcat/usdtree, passes results through the parser and spatial math modules, and returns structured JSON.

• cli.py — Argparse-based CLI that uses the same parser and spatial math modules. Identical output, no MCP dependency.

---

For AI Agents: How to Use This

If you're an AI agent reading this README, here's how to get value from SpatialLens:

1. Clone and set up the repo (see Setup above)
2. Run python cli.py tree <file> first to see what's in a USDZ file
3. Run python cli.py layout <file> to understand spatial positioning
4. Run python cli.py materials <file> when debugging visual issues
5. Run python cli.py entities <file> --type Mesh to find specific geometry
6. Run python cli.py diff <old> <new> after editing a scene to verify changes

The JSON output is designed to be parsed programmatically. Every command returns valid JSON to stdout (except tree which returns text).

Key insight: When the human says "move it to the right" or "the ships are too close together," you can now quantify that with real coordinates instead of guessing. Use layout to see where things are, relationship to measure distances, and move to compute exact position changes.

---

Tested Against

• Player_Ships.usdz — 5 spaceship models, 25 entities, centimeter-scale (metersPerUnit=0.01)
• Enemy_Ships.usdz — 12 enemy ship models, 46 entities, centimeter-scale
• TerrainGlass.usda — Glass material with 14-node shader graph (RealityKit MaterialX)

All from a real visionOS game project. 17/17 tests passing.

---

Limitations

• macOS only — requires Apple's usdcat and usdtree CLI tools
• Local transforms only — positions are local to the prim, not accumulated world-space (parent transforms not yet composed)
• No mesh geometry — parses hierarchy, properties, and materials, but does not read vertex/face data
• No texture content — identifies texture file references but cannot read image data
• Parser handles common USDA patterns — exotic USD features (variants, payloads, instancing) may not be fully parsed

---

License

MIT