Liquid AI's 300M Draft Model Speeds Up Decoding by Up to 3.18x

Liquid AI announced this via a public post on X on August 20, uploading DSpark draft model checkpoints for its three LFM2.5-series models — LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. The company had previously uploaded LFM2.5-VL-3B, a lightweight vision-language model that reads screens and documents, to Hugging Face on August 12, and our monitoring also caught a 4-bit checkpoint update for the smaller LFM2.5 models on August 19. This latest release leaves the series' performance untouched while focusing purely on speed.

The draft model writes nine tokens ahead, and the main model grades them all at once

The technique is called speculative decoding. Instead of the large model generating tokens one by one, a small draft model first writes out several likely next tokens, and the large model grades the whole batch in a single pass. Correct tokens pass through as-is; generation restarts from the point of the first mistake. Because multiple tokens get confirmed in the time it takes the large model to run once, perceived speed goes up.

DSpark's draft model proposes 9 tokens at a time. Its size is 295.7M parameters for the 1.2B-Instruct pairing and 327.7M for the 2.6B and 8B-A1B pairings. It uses a GQA structure with 5 full-attention layers, a hidden size of 2048, an intermediate size of 6144, and 32 query heads sharing 8 KV heads. It carries no vocabulary weights at all, instead loading the embedding and LM head from the target model at load time. The actual added memory footprint is 655MB in BF16 for the 2.6B pairing.

According to MarkTechPost's technical writeup, DSpark combines three components: a DFlash-style parallel backbone that takes the target model's context features as a condition and extracts hidden states for all draft tokens at once; a lightweight sequential head that reconstructs dependencies between neighboring tokens using a rank-256 Markov chain to boost acceptance rates later in the block; and a confidence-based verifier that predicts each token's survival probability and truncates the tail end when verification cost is likely to outweigh the benefit.

The critical point is that quality is unchanged. Under greedy decoding at temperature 0, the resulting output is literally identical to running the original model alone. Since this is built into the structure by design, benchmark scores don't change either.

Measured results: 3.18x on H100, 2.87x on MacBook

Testing was done using a single H100 with BF16 and SGLang, and separately on an M4 Max MacBook Pro with llama.cpp, Metal, and FP16 GGUF. Both used a block size of 9, batch size of 1, and temperature 0, run across five benchmarks: MATH500, HumanEval, MBPP, GSM8K, and MT-Bench.

Target Model	H100 Average	H100 Best	M4 Max Average	M4 Max Best
LFM2.5-1.2B-Instruct	2.10x (656→1384 tok/s)	2.56x (MATH500)	2.54x (138→350 tok/s)	2.87x (HumanEval, 136→389)
LFM2.5-2.6B	2.67x (323→864 tok/s)	3.06x (MATH500)	2.27x (61→139 tok/s)	2.63x (HumanEval)
LFM2.5-8B-A1B	2.54x (418→1074 tok/s)	3.18x (MATH500, 428→1362)	1.18x (90→106 tok/s)	1.44x (GSM8K)

The batch size of 1 condition is worth noting. This technique's home ground isn't a server batching requests from many users at once — it's a single person running the model alone on their own laptop or GPU.

Speed tracks "how often it guesses right"

The gain varies significantly by workload, because the speed itself is essentially the acceptance rate — the proportion of the draft model's proposed tokens that actually get accepted. On MATH500, 8B-A1B accepted 8.27 out of 10 tokens per step, yielding 3.18x, but on GSM8K that dropped to 4.02, bringing the same GPU's speedup down to 1.29x. The 1.2B model also saw acceptance fall to 3.90 on MT-Bench, limiting its H100 gain to just 1.66x. Overall, measurements ranged from 1.04x to 3.18x.

Condition	Accepted Tokens per Step	Speed Multiplier
8B-A1B · MATH500 · H100	8.27 / 10	3.18x
8B-A1B · GSM8K · H100	4.02 / 10	1.29x
1.2B · MT-Bench · H100	3.90 / 10	1.66x

This means the more predictable the output, the faster the speedup. Structured mathematical derivations or code let even a small model guess the next token accurately, but free-form conversation is harder to predict.

The most notable weak point is MoE models on Apple silicon. 8B-A1B, which only activates a subset of its experts, averaged just 1.18x on M4 Max. Liquid AI attributed this to how llama.cpp's Metal backend currently handles MoE, and to the fact that verifying k tokens at once wakes up more experts than single-token decoding does, increasing the volume of weights that need to be transferred.

The biggest gains show up in tool-calling agents

The effect concentrates in one particular spot: agent workflows where the model thinks for a long time before every tool call, and the user waits, staring at the screen, in the meantime. In multi-tool function-calling scenarios, latency for LFM2.5-2.6B dropped substantially — though the two sources cite different reduction figures. The X post says the average reduction on BFCL multi-tool scenarios was nearly 50%, while MarkTechPost put the average at 57%. Either way, agents that plan, call, and re-plan pay the decoding cost multiple times within a single user turn, so the absolute time savings are large.

How to try it

Where to start. It's not available via hosted API. No inference provider currently serves the draft model checkpoints on Hugging Face, so you'll need to run it yourself. Weights are available in both Safetensors and GGUF formats, and you'll need an SGLang or llama.cpp build that supports DSpark for LFM2-family target models. Both support it from day one of release.

Step by step.

1. Download the target model (e.g., LiquidAI/LFM2.5-2.6B) together with its paired draft model (LiquidAI/LFM2.5-2.6B-DSpark). The draft model alone won't run standalone since it has no vocabulary weights.
2. With SGLang, launch the server with the draft model attached when starting up the target model.

python -m sglang.launch_server
--model-path LiquidAI/LFM2.5-2.6B
--speculative-algorithm DSPARK
--speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark
--speculative-draft-attention-backend flashinfer
--disable-radix-cache --mem-fraction-static 0.75 --port 30000

3. There's no need to specify block size separately — it's read from the draft model's config.json.
4. To compare speeds, run the same command without the three --speculative- prefixed lines. That's your baseline.
5. For laptop use, point llama.cpp at the FP16 GGUF weights. On Mac, it runs on the Metal backend.

Who can use it. The LFM Open License v1.0 only permits free commercial use for organizations with under $10 million in annual revenue. Individual developers, startups, and SMBs fall within that bound; larger companies need to contact Liquid AI separately for a commercial license.

What you can try. For local coding assistants running on a laptop, autocomplete wait times drop by more than half. On-premises chatbots for healthcare, finance, or defense work where data can't leave the internal network are another good fit. Attaching it to on-device agents that make repeated tool calls yields the biggest cut in the time users spend staring at the screen waiting.

Editor's take

The number worth watching in this announcement isn't 3.18x — it's 655MB. Anyone who's put on-device models into real use hits the same bottleneck every time: accuracy is decent enough, but responses are slow enough that users close the window. Until now, that problem has been solved by shrinking the model further or tightening quantization, and each time, quality took a small hit. DSpark goes the other direction. It costs a bit more memory and doesn't touch quality at all. For someone squeezing by on 8GB of RAM, 655MB is a real burden, but for anyone with 16GB or more, it's a manageable size.

The design choice to have the draft model itself carry no vocabulary weights is also practically clever. Vocabulary embeddings eat up a substantial share of parameters in small models, and by borrowing them from the target model at load time, a 300M-class model actually only computes 300M-worth of work. This structure ties one draft model to one target model, so there's no general-purpose flexibility, but for a company only using models within its own family, that's not really a downside.

That said, capacity planning shouldn't be based on the benchmark's best-case numbers when adopting this. The same 8B-A1B goes from 3.18x on MATH500 to 1.29x on GSM8K — more than double the difference. That means the outcome depends heavily on whether your actual conversation logs look more like mathematical derivations or more like free-form dialogue. Anyone evaluating adoption should skip the benchmarks and instead replay a few hundred samples of their own actual traffic to measure acceptance rate directly. If the acceptance rate falls below 6, it's safer to expect around 1.5x rather than 2x.

The Apple silicon MoE results are less a flaw than an indicator of the current state of llama.cpp's Metal backend. There's substantial room for this to be fixed at the framework level, and until then, anyone running locally on Mac will get more practical value out of the dense 2.6B model than the MoE-based 8B-A1B. The batch size of 1 condition is also worth remembering — on a server handling multiple concurrent users, where the GPU is already busy, gains of this magnitude are unlikely to materialize.

In the coming weeks, expect other players in the small-model space to follow with their own family-specific draft models. The axis of competition for on-device models has already shifted from parameter count to "how many tokens per second on the same hardware" — and if there's a way to buy speed without touching quality, there's little reason not to take it.