이미지: METAL LAB 생성
Summary
- Reddit user Atretador has released Unswarm, an open-source tool that manages multiple local LLM runtimes through a single proxy.
- Once containers or bash scripts are registered, the tool automatically spins up the needed runtime when a request comes in, queues it, and streams back the response.
- It's aimed at people running several older GPUs like the MI50, P100, and P40, and it can also be deployed on a VPS for remote access or spread across multiple machines to run in parallel.
- 프로젝트명
- Unswarm(언스웜)
- 개발자
- 레딧 이용자 u/Atretador (개인 프로젝트)
- 공개 시점·채널
- 2026년 8월 23일, r/LocalLLaMA
- 저장소
- github.com/atretador/unswarm
- 핵심 기능
- 런타임 등록, 동시 실행 규칙, 요청 큐잉, API 프록시
- 겨냥 하드웨어
- MI50·P100·MI25·P40 등 구형 GPU
- 알려진 한계
- 세션 중 모델 전환 시 캐시 히트율 손실
Multiple containers, multiple scripts — the everyday headache of running local LLMs
Anyone who's tried running several LLMs locally knows the scene. Each model has its own execution engine — some need to run in a Docker container, others just need a bash script. Reddit user Atretador ran into exactly this problem and ended up building an open-source tool called Unswarm, hosted at github.com/atretador/unswarm and announced on r/LocalLLaMA. In a post from August 23, 2026, he wrote that "forks, containers, and engines all differ from model to model, so I had to manage everything by hand."
He specifically pointed to setups running several generations-old GPUs at once — cards like the MI50, P100, MI25, and P40. These older cards tend to clash with modern OS packages, he explained, which makes isolating them inside containers the more practical option.
What Unswarm actually does
As the name suggests, Unswarm is a runtime manager and proxy that herds multiple models together like a swarm. Users register the containers or bash runtime scripts they're already using with Unswarm. From there, they define rules for which runtimes are allowed to run at the same time, and Unswarm queues incoming requests according to those rules.
Set up an API key, register Unswarm as a single provider in a coding harness like Claude Code or Codex, and the proxy makes it look like every model is available at once — even though the runtimes are actually running separately behind the scenes. Pick a model in the harness and send a message, and the request goes into a queue; if that runtime isn't currently running, Unswarm automatically boots it up and streams the response back.
Atretador was clear that Unswarm is meant to manage things that already work well — it's not a platform for tuning models themselves. He also noted that switching models mid-session inevitably hurts cache hit rates, calling it an unavoidable tradeoff.
Getting started
To use Unswarm, you pull the code from github.com/atretador/unswarm and self-host it.
- Register the containers or bash runtime scripts you're already using with Unswarm, one by one.
- Set rules for which runtimes can run simultaneously. In VRAM-constrained setups, you can limit things so only specific combinations run together.
- Issue an API key and register Unswarm as a provider in the coding harness you use — Claude Code or Codex, for example.
- Pick a model in the harness and send a message. The request goes into a queue, and if the needed runtime is off, it automatically starts up and streams the response.
Beyond a personal machine, the tool can also be deployed on a VPS so you can reach your models from anywhere, and it supports spreading different agents across separate machines to run them in parallel.
A sample multi-agent setup
Atretador offered an example using a mix of 24GB and 16GB graphics cards to show what kind of setup Unswarm makes possible.
| Group | Role | Example model | Execution mode |
|---|---|---|---|
| Group 1 | Orchestrator | Qwen 3.8 27B A3B | Always running |
| Group 2 | Codebase exploration | Qwen 3.5 9B | Switched in as needed |
| Group 2 | Executor | Qwen 3.6 35B A3B | Switched in as needed |
| Group 2 | Designer | Custom fine-tuned model | Switched in as needed |
The orchestrator that directs the overall task stays on continuously, while sub-agents for things like code exploration, execution, and design are only loaded when needed. It's a useful reference layout for anyone trying to split multiple roles across a setup without much VRAM to spare.
Editor's take
Watching Unswarm come together, it's clear the conversation in the local LLM world is shifting — from "which model should I use" to "how do I run several models together." Runners like Ollama and vLLM have each focused on serving a single model well, in their own particular way. But the orchestration layer that people actually need when juggling multiple runners and containers at once often doesn't come from a company at all — it gets built by an individual developer scratching their own itch, as happened here.
Anyone who's run local LLMs on a handful of secondhand server GPUs in the 8GB–24GB range knows this frustration firsthand. Bringing a container down and back up every time you want to switch models, over and over by hand, eventually eats up more time than the model's actual performance ever does. Unswarm essentially adds a thin layer that hands that repetitive work off to rules and a queue instead.
For teams testing multiple LLMs on their own internal servers, a proxy layer like this can function much like adding a single API gateway. That said, the hit to cache rates might look minor on paper, but in real workloads it shows up immediately as added latency. So it's worth designing the split between frequently-switched sub-agents and an always-on orchestrator before bolting a tool like this on top.
In the weeks ahead, it wouldn't be surprising to see personal projects like this either get absorbed as official features into mainstream runners like Ollama and vLLM, or see community forks multiply and expand support for specific hardware — older AMD cards being one likely candidate.
Comments