Continuous Thought Machines

Continuous Thought Machines are built so that the internal flow of time a neural network takes before producing an answer becomes part of the computation itself. A typical neural network takes an input, runs through a fixed sequence of calculations once, and immediately outputs an answer. This architecture takes a different approach: it lets the model keep thinking internally over multiple steps, gradually refining its answer along the way.

A useful comparison is how a person tackles a hard problem: rather than the answer popping into their head instantly, they pause, mull it over, and revise their thinking. Inside this model, the computational units are synchronized with each other in terms of timing — when they send signals to one another — and the key idea is that the pattern of that timing itself carries information. In other words, when a signal is sent matters just as much as what is being computed.

This is a different approach from having a model write out its reasoning in human-readable sentences before giving an answer. Where that approach exposes the thinking process as text, Continuous Thought Machines build the passage of thinking time directly into the architecture itself. It emerged as one of the experiments searching for alternatives to the Transformer architecture that dominates today.