sf-symbols-mcp

An MCP server that helps AI agents pick the right SF Symbol for a UI function — "download the invoice", "archive this message", "show account settings" — without hallucinating symbol names.

Every result is a verified catalog entry (SF Symbols 7.2, 7,781 symbols, 7,006 with vision-derived annotations), found through layered retrieval: query decomposition → lexical search (BM25/FTS5) → local semantic embeddings → curated UI-convention priors → visual-description matching → explainable ranking with availability and restriction filtering.

No API keys. No network at query time. No native build steps. The prebuilt catalog and a small embedding model ship in the data package; queries run in ~100 ms on the built-in node:sqlite plus a 34 MB ONNX model.

On the project's golden-query benchmark the right symbol family is in the top 3 for 98.8% of decomposed queries (top 5: 100%).

Installation

Requires Node ≥ 22.13. The install pulls ~110 MB (catalog database + embedding model) once; everything afterwards is offline.

Claude Code:

claude mcp add sf-symbols -- npx -y sf-symbols-mcp

Claude Desktop / Cursor / any MCP client — add to your MCP config:

{
  "mcpServers": {
    "sf-symbols": {
      "command": "npx",
      "args": ["-y", "sf-symbols-mcp"]
    }
  }
}

Usage

Ask your agent things like:

"Which SF Symbol should I use for the download button?" "Give me an icon for muting a conversation, must work on iOS 15." "What's the filled variant of the bell icon for a selected tab?"

The agent calls search_sf_symbols and gets verified names with reasons:

// search_sf_symbols { query: "download the invoice", primaryAction: "download", object: "invoice" }
{
  "results": [
    {
      "name": "tray.and.arrow.down",
      "score": 0.61,
      "reason": "standard download icon; semantic similarity",
      "description": "A rectangular tray, open at the top, has a solid down arrow pointing into its opening…",
      "family": { "baseName": "tray.and.arrow.down", "variants": [{ "name": "tray.and.arrow.down.fill", "modifiers": ["fill"] }] },
      "availability": { "iOS": "13.0", "macOS": "10.15", "…": "…" },
      "warnings": []
    }
    // …
  ]
}

Tools

Tool	Purpose
search_sf_symbols	Find symbols for a natural-language UI function. Optional structured fields (primaryAction, object, direction, state, excludedMetaphors, platforms, …) let the calling LLM decompose the query for sharply better ranking. explain: true returns the score breakdown.
get_sf_symbol_info	Full metadata: availability, categories, family and variants, restriction status, semantic/visual annotations with provenance.
compare_sf_symbols	Structured diff of 2–6 candidates: semantics, pairwise visual similarity, availability deltas, when-to-use-which guidance.
resolve_sf_symbol_variant	Pick fill/slash/badge/enclosure variants by UI state and platform conventions (selected tab-bar → .fill, off → .slash, watchOS prefers fill, …). Never invents names.
find_visually_similar_symbols	Symbols that look alike (CLIP + perceptual hash): confusable glyphs, simpler alternatives.
update_local_catalog	Refresh from your locally installed SF Symbols app (macOS). New symbols become searchable immediately; annotations are preserved.

Results carry scores, one-line reasons, family variants, availability, and typed warnings (restricted symbols, renamed inputs, close calls, ambiguous glyphs).

How it works

SF Symbols.app plists ──extract──▶ normalized catalog (names, availability,
        (macOS, plutil)             categories, aliases, restrictions)
renders (Swift, public APIs) ──▶ vision passes (literal → semantic → reconcile)
                                  + family analysis + deterministic features
                        ──build──▶ catalog.db (SQLite FTS5 + embedding BLOBs)
                                   + bge-small ONNX model, shipped via npm

• Never fabricates: retrieval happens only over catalog rows; curated mappings are CI-validated against the catalog.
• Sees the glyphs: each symbol was rendered locally and described by a vision model in three passes (literal content → UI semantics → reconciliation against the name, with contradictions stored, not hidden), plus per-family variant analysis and cross-model consensus checks.
• Family-aware: bell/bell.fill/bell.badge/bell.slash collapse to one result with variants attached; the plain base symbol fronts the family unless your query implies a variant ("muted", "line through it").
• Explainable: weighted linear scoring (lexical, semantic, action/object match, curated prior, visual) with explicit penalties (direction conflicts, antonym actions, excluded metaphors, restrictions, deprecations).

Versioning

The data package minor-tracks SF Symbols releases; the server takes any compatible data version. A new SF Symbols release means a data-package update only — npx picks it up automatically.

Maintainer pipeline (not needed by users)

All Apple-derived intermediates live in gitignored generated-local/.

pnpm extract        # read the local SF Symbols app metadata (macOS, plutil)
pnpm render         # deterministic 256px monochrome PNGs (Swift, public APIs)
pnpm features       # pHash, fill-score, symmetry, family grammar validation
pnpm annotate all --yes   # 3-pass vision annotation + family + consensus
pnpm embed          # bge-small text vectors + CLIP image vectors (local)
pnpm build-data --profile=default   # assemble catalog.db
pnpm eval           # golden-query ranking regression (81 queries)
pnpm pack-data      # stage packages/data for publishing

Annotation providers (pick one; ~22,000 requests per full catalog):

• OpenRouter (cheap, default google/gemini-2.5-flash, ≈ $15–20 total): export OPENROUTER_API_KEY=…, then start with pnpm annotate pass1 --pilot=50 --yes and review before the full run. Useful flags: --concurrency=N, --route=<provider> to pin the upstream, --shard=i/n to split a pass across parallel processes.
• Anthropic Batches: export ANTHROPIC_API_KEY=… — claude-sonnet-5 with structured outputs, ≈ $130 per full catalog.

Every annotation stores provenance (model, prompt version, batch id) and resumes from per-symbol checkpoints. Releasing: pnpm publish --access public in packages/data, then packages/server.

Data profiles & licensing posture

Profile	Contents
default (published)	Symbol names, availability, categories (facts), plus independently authored annotations, embeddings, curated lexicons. No Apple-authored keyword lists or restriction sentences.
local (your machine only)	Everything, including Apple's search keywords — created by update_local_catalog from your own SF Symbols installation.
safe (fallback)	Only independently authored data + bare names.

Apple IP notice

This project does not include, embed, or redistribute any Apple artwork, fonts, symbol images, SVG templates, or Apple-authored metadata files. Symbol names are used as factual identifiers for interoperability; all symbol descriptions in the published data are independently authored. Some symbols may be used only as-is to refer to the Apple technology they represent, and no symbol may be used in app icons, logos, or any trademark-related way.

To use the local-extraction features you must install the SF Symbols app yourself and accept Apple's license. Locally rendered previews stay on your machine and must not be redistributed. See NOTICE for the full statement.

SF Symbols is a trademark of Apple Inc. This project is not affiliated with, endorsed, or sponsored by Apple Inc.

Development

pnpm install && pnpm test        # unit + e2e (e2e needs a local catalog build)
pnpm typecheck
pnpm --filter sf-symbols-mcp build   # bundle the server (single-file dist)

License

MIT for all original code and independently authored data. See LICENSE and NOTICE.