Pipette

Pipette is a tool that precisely measures how fast an AI model generates answers and how much memory it consumes on personal devices such as phones or laptops. The name comes from the lab instrument used to transfer exact amounts of liquid, reflecting the idea of testing multiple models one by one under identical conditions with precision.

Most performance comparison charts so far have been built around measuring large models hosted on servers. But even the same model can vary widely in speed and accuracy depending on how much it's been compressed, what runtime it's executed on, and what device it's deployed to. Pipette bundles four factors—model, quantization level, runtime, and device—into a single unit of comparison, letting you line up multiple combinations side by side under the same conditions.

It was released by Liquid AI together with Artificial Analysis, and it splits the measurement work: latency and memory usage are measured directly on real devices like iPhones or Galaxy phones, while answer quality is scored separately on servers.

Articles describe it like this: "In Pipette, a single comparison unit is defined as model + quantization + runtime + device." One common misunderstanding is that Pipette isn't a single model—it's a bundle of tools combining a dashboard, dataset, benchmark client, and scoring infrastructure.

On the Pipette dashboard, pick a device you're interested in (say, an iPhone or a Galaxy-tier smartphone), select two models of different sizes, and compare their response latency and memory usage side by side. It also helps to see how speed and memory change for the same model as you vary the quantization level.