sensor data processing

Sensor data processing is the process by which a computer reads, in real time, the flood of information pouring out of sensing devices like cameras, lidar, and radar, and turns it into useful material for making decisions. It's similar to how, when a person drives, the brain instantly combines what the eyes see, what the ears hear, and what the hands feel through the steering wheel — in a self-driving car, a computer takes over that job.

The problem is that the sheer volume of information these sensing devices produce is enormous. Camera footage, distance data generated by lidar, and radar signals all come streaming in many times per second, and all of it has to be organized without delay so it can be passed on to the next stage: AI decision-making. If processing slows down even slightly, the car ends up learning about a situation after it has already passed, which means speed is directly tied to safety.

For this reason, dedicated hardware built solely for this task is often used separately. It's possible to do this with general-purpose computer components too, but in a field like self-driving where real-time performance is critical, how fast and how cheaply this processing can be done has become a key point of technological competition among companies.

As it appears in the article: "Robotaxis process the sensor data pouring in from cameras, lidar, and radar in real time, then use an AI model trained on that data to understand their surroundings and decide their next move."

One point that's easy to confuse: sensor data processing and the AI's situational judgment are not the same thing — they are separate stages that happen in sequence. Sensor data processing is the earlier stage, where the raw signals sent from cameras and lidar are organized. Based on that, making decisions like "there's a person ahead" or "I need to change lanes" is the job of the next stage, the AI model. What makes Waymo's new chip special is that it's designed to handle both of these stages quickly within a single chip.