이미지: METAL LAB 생성
Summary
- A Reddit r/LocalLLaMA user raised the issue that new open-source models in the 8B-12B range have become rare
- LM Studio's latest recommended model is Gemma4 12B QAT, released 66 days ago, while the rest dated back 147-161 days
- The discussion pointed to a recent trend of open-source releases concentrating on 27B and above
- 게시판
- r/LocalLLaMA
- 제기 시점
- 2026-08-11
- 사용자 환경
- MacBook Pro M4, 16GB 통합메모리
- 언급된 최신 모델
- Gemma4 12B QAT (66일 전 출시)
- 언급된 구형 모델
- Nemotron 3 Nano 4B, Qwen3.5 9B (147~161일 전)
- 언급된 신규 대형 모델
- Bonsai 27B (10일 전 출시)
No usable models in sight for a 16GB MacBook
A post on Reddit's r/LocalLLaMA struck a chord with local LLM users. A MacBook Pro M4 user with 16GB of unified memory raised the issue, saying "I can't find any new 8B-12B models worth running in LM Studio." The model the poster is currently using is Gemma4 12B QAT, released 66 days ago. The next models recommended by LM Studio, Nemotron 3 Nano 4B and Qwen3.5 9B, were released 147 and 161 days ago respectively — effectively outdated options. Bonsai 27B, released 10 days ago, also caught attention, but the poster wasn't even sure whether 16GB of memory could handle it. The core of the post was the question: "Everything coming out these days is 27B or larger — why is that?"
Why have mid-sized models become scarce
The 8B-12B range has long been called the "sweet spot" for local inference. After 4-bit quantization, models in this range run near real-time on around 16GB of unified memory while still delivering response quality good enough for practical use. Recently, however, competition in the open-source space appears concentrated on the 27B-and-above range, where it's easier to claim benchmark superiority. The larger the parameter count, the easier it is to boost scores on reasoning and coding benchmarks, and companies also have a strong incentive to showcase flagship-level performance. Comments on the post pointed out that, in contrast, the 8B-12B range tends to get deprioritized in favor of large-model development rather than receiving resources to refine already-proven architectures.
Indeed, Google's Gemma 4 12B QAT, released on August 9 — a checkpoint pretrained with quantization-aware training (QAT) to reduce compression loss — was nearly the only model in the 12B class that could be considered up to date. It's a multimodal model that accepts text and image input, supports up to 256K tokens of context and more than 140 languages, and is distributed under the Apache 2.0 license. This aligns with why the poster singled it out as "still usable."
Google releases Gemma 4, a 12B model with QAT-quantized checkpoint
So what does this change?
This debate is notable not because of any single company's announcement, but because it reflects the community's lived experience. For users without a high-end Mac with 32GB or more, or a desktop GPU — that is, people trying to run local LLMs on entry-level laptops — the release cadence of new models directly determines the quality of their experience. When new releases in the 8B-12B range become scarce, users are left either sticking with models more than six months old or forcing themselves onto oversized models they can barely handle. If developers continue racing to compete on benchmarks with ever-larger models, demand for on-device, lightweight inference is likely to remain dependent on a handful of recent exceptions for the time being.
