Mid-size model tier gets crowded again — is 70-80B next

A string of mid-size models has landed over the past few weeks. DSV4 Flash 0731, Inkling Small, Laguna S 2.1, and Step 3.7 Flash all launched within a short span, noticeably widening the field of options. The community is now asking who will fill the 70-80B slot next.

Why mid-size, specifically

For huge models, using the API is usually cheaper. Small models often fall short on quality. The zone where self-hosting actually makes sense sits in between — the sweet spot where a single server, or two to three cards, can run something with production-grade quality. That's why models are converging on this mid-size tier right now.

Organizations typically choose self-hosting for one of three reasons.

1. Data can't leave the premises — regulated industries, contractual restrictions, internal policy
2. Call volume is high — past a certain scale, API costs exceed hardware costs
3. Latency matters — real-time use cases that need to eliminate network round-trips

If none of the three apply, the API is usually the better choice. Operational overhead is frequently underestimated.

What's out right now

Recent release	Tier
DSV4 Flash 0731	Mid-size
Inkling Small	Mid-size (upper end of small)
Laguna S 2.1	Mid-size
Step 3.7 Flash	Mid-size
Empty slot	70-80B

Exact parameter counts and benchmarks vary in disclosure across distributors, so they aren't listed here. For accurate comparisons, check the model cards on the original repositories directly.

Why 70-80B is a special slot

This tier sits right on a hardware boundary. It barely fits on a single 80GB card at 4-bit quantization, and two cards give it room to breathe. In other words, it's close to the last point where you can bump up quality without bumping up your hardware tier. That's why the community keeps calling out this specific number.

Tier	Approx. size at 4-bit	Realistic hardware
7-9B	4-6 GB	Laptop · 1 consumer GPU
24-32B	14-20 GB	1 consumer GPU (high-end)
70-80B	40-48 GB	1 workstation GPU or 2 consumer GPUs
200B+	110 GB+	Server · multi-GPU

Each step up the ladder multiplies the hardware budget several times over. That's why models sitting just below the boundary see the most real-world use.

A checklist for evaluating adoption

1. Narrow down the use case first. For summarization, classification, and extraction, mid-size is often enough. Long-form reasoning and code generation are different matters.
2. Set the VRAM budget first. Picking the model before matching the hardware usually blows the budget. Go the other way around.
3. Measure quantization loss against your own data. Comparing Q4 and Q6 on 100 real work prompts speeds up the decision considerably.
4. Factor concurrent users into the math. Hardware needs differ wildly between solo use and ten simultaneous users. The KV cache scales with user count.
5. Revisit the license last. Commercial-use terms vary by model, and sometimes even by version within the same company's lineup.

Worth building a comparison process, not just picking one

When candidates pile up all at once like this, what pays off isn't picking a single winner — it's building a process you can use to pick again next time. Because there will be a next model.

At minimum, that process needs: (1) a set of 100-200 evaluation prompts representative of your own workload, (2) reference answers or grading criteria, (3) an abstraction layer that lets you swap models in and out, and (4) logs that track cost-per-token and latency together. With these four in place, a new model release can be resolved in half a day.

Source: r/LocalLLaMA community discussion. Refer to each distributor's model card for individual specifications.