State Space Model

A State Space Model reads through a sentence or sound from start to finish, one piece at a time, continually updating a single 'state' that compresses everything heard so far. It's a bit like reading a book and summarizing the plot in one sentence at the end of each page before moving on. Instead of having to re-examine every sentence read so far, you only need to carry the latest summary forward, so the processing burden doesn't grow much even as the input gets very long.

By contrast, the architecture most widely used today re-scans every previous word each time it processes a new one, so computation snowballs as sentences get longer. State Space Models trade that off by having to process things strictly in order, which makes them especially well suited to situations with continuous, ongoing data — like real-time speech synthesis or streaming services.

Cartesia's speech synthesis model Sonic-3.6 was introduced as running on a State Space Model (SSM) architecture instead of the widely used one. The article explains that thanks to this architecture, the time to first audio output is under 90ms, while noting that this figure comes from the company's own measurements and may differ from the round-trip latency actually experienced in live service.