NVIDIA-NeMo/Switchyard
A router that swaps the model behind the scenes while your coding agent keeps talking the same way
Switchyard is a Rust-based proxy that lets coding agents like Claude Code or Codex keep using their native API format while their requests actually get served by other models such as vLLM, NVIDIA NIM, or Ollama. It translates between OpenAI and Anthropic API formats, so you can switch backend models or split traffic across several for testing without touching the agent's code. It's currently pre-alpha software, not meant for production use.
What it does
- Apps and agents keep using whatever format they already speak - OpenAI Chat, Anthropic Messages, or OpenAI Responses - while Switchyard translates the request into the backend model's native format behind the scenes.
- It supports multiple routing strategies: random traffic splitting, LLM-based classification to pick a route, a stage router that reads signals already in the conversation (like tool results or errors), and an escalation router that tries a weaker model first and lets a judge decide whether to escalate to a stronger one.
- It can be used three ways: as a standalone server (switchyard-server), through a command-line launcher (switchyard CLI) for tools like Claude Code, or embedded directly into your own Rust app as a library (switchyard-libsy).
- It exposes Prometheus metrics covering requests, errors, latency, tokens, and routing overhead for operational monitoring.
- The Rust project has 1927 GitHub stars and is released under the Apache 2.0 license, copyrighted by NVIDIA.
Why it matters
Teams wanting to cut costs or benchmark open-source models against proprietary ones can swap or A/B test backend models without changing a single line of their coding agent's code. That said, the maintainers themselves describe it as experimental, pre-alpha software whose API and algorithms are still expected to change significantly.
Terms in this repo
- proxy · an intermediary server that relays requests between a client and the real backend service
- OpenAI Chat / Anthropic Messages / OpenAI Responses · the distinct request-response API formats defined by OpenAI and Anthropic for LLM interactions
- vLLM, NVIDIA NIM, Ollama · serving engines used to run open-source or self-hosted LLMs
- Prometheus metrics · standardized measurements (request counts, latency, etc.) that monitoring tools can collect and display
- pre-alpha · an early development stage before official release, where features and APIs are still expected to change significantly
Repository description (English)
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Open on GitHubTrending repos
- cathrynlavery/diagram-designA skill that makes AI coding tools draw magazine-quality diagrams instead of generic rounded boxes
- public-apis/public-apisA giant crowd-curated directory of free APIs for developers
- semantica-agi/semanticaAn open-source graph infrastructure that lets AI agents show their work, not just their answers
- cactus-compute/needleA 14MB AI model small enough to run tool-calling on a phone or watch, without internet
- unslothai/unslothA desktop app that lets you run and train AI models on your own computer, no coding required
- macro-inc/macroAn all-in-one workspace where email, chat, docs, tasks, and CRM are cross-linked and share one AI memory
- harry0703/MoneyPrinterTurboAn open-source tool that turns a single topic or keyword into a finished short video, complete with script, footage, subtitles, and music
- basecamp/omarchyA ready-made, opinionated Linux setup built by DHH
Latest from METAL LAB
- Grok leaks conversations, location data via commands hidden in ciphertext
- Cursor lets agents handle long-running tasks with new "/goal" command
- NVIDIA releases Switchyard, an LLM routing proxy for coding agents
- Caveman Cuts Claude Code Token Usage by 33% Using Caveman-Speak
- Apple Keeps 2027 Target for Camera AirPods, Qwen3.8-27B Makes 80 Tool Calls Locally