Meta Shows Off AI Hardware Inside Its Menlo Park Infrastructure Lab

At the start of the video, Tom Shaw says, "Size doesn't matter in AI. Efficiency matters." He says the common assumption is that doubling a new model's power just means doubling the hardware, and that he only realized that wasn't true after coming to this lab. He explains that the hardware handling a like on an Instagram photo is a completely different animal from the hardware that trains and runs inference on AI models, which is why data center infrastructure built for AI looks different from a conventional data center.

His analogy is building a PC. You decide what you need, then buy parts from various sites — AI companies go through the same process, but past a certain scale, designing your own hardware becomes more efficient than buying off-the-shelf. The video explains that in a room sized to fit 1,000 GPUs' worth of off-the-shelf servers, custom-designed hardware can pack twice the compute capacity into that same space, because the system gets denser and each individual chip gets more powerful.

The physical comparison is the heart of the video. The first tray Tom Shaw shows holds eight NVIDIA H100 chips; the tray right next to it holds four NVIDIA GB300 chips. He says he won't get into a detailed comparison of the two chips, but that four GB300s outperform eight H100s. They're much smaller, so more chips fit into a single rack, and each chip is also more powerful, so the compute packed into one rack jumps sharply. But you can't just drop the GB300 into an H100 tray. The biggest reason is cooling.

The H100 tray was air-cooled; the GB300 tray is liquid-cooled. An unnamed engineer from Meta's hardware engineering team explains in the video that coolant runs through cold plates inside the tray to chill the chips touching those plates, and that the coolant circulates in a closed loop through the tray, the cold plate, and the rack, dumping heat outside through heat exchange inside the data center. He adds that today's GPUs generate so much heat that past a certain point, air cooling alone isn't just less efficient — it's simply impossible. According to a cooling explainer Tom Shaw wrote for Meta's Newsroom in August, this coolant is a water-glycol mix that can go up to ten years without being changed, and buildings without liquid-cooling infrastructure use an air-assisted liquid-cooling setup with pumps and heat exchangers built into the rack. In that piece, he wrote, "A typical AI data center using closed-loop liquid cooling and dry coolers uses less water in a year than a couple of full-service restaurants." Cooling the same servers with air would require roughly doubling the server tray size, while liquid cooling that touches the chip directly lets you pack more GPUs into the same rack, according to that piece.

After cooling comes networking. The new chips process more data per second, but existing networking gear can't keep up with that throughput. The rack in the video holds 72 GPUs, and the approach used to link 16 H100s in an earlier data center video no longer works at this scale. According to the engineer, the 72 GPUs are tied together by a central switch network inside the rack, and a copper cable backplane at the back of the rack forms a GPU mesh within a 72-chip scale-up domain. On top of that sits another layer, a scale-out interconnect that links multiple racks across a single building and even across multiple buildings. He says training jobs aren't a simple picture of a manager handing work to racks and racks handing it down to trays — it's closer to organized chaos, scattering work across 72 GPUs or a much larger cluster. He says the innovation isn't the sheer amount of networking but doing it at low latency, adding, "Tying all the pieces together is the real challenge." In other words, buying and deploying powerful chips doesn't help if there's no network to move data between chips during training.

The idea that innovation never stops is literal in this video. The rack with 72 GB300s was deployed earlier this year, and Meta is already building the next prototype. The video doesn't give specs or a deployment timeline, but the company says it's not ready for mass production yet — though close. The new rack is considerably bigger than before, and engineers put a lot of work into structural validation: whether it survives the shocks and vibration cycles of truck shipping, and whether the rear connectors seat properly. If a rack sags or falls out of alignment, it causes interference that hurts reliability, especially when servicing compute trays or network trays. Back to the PC-building analogy, this isn't swapping in the latest parts — it's redesigning the case itself to fit the requirements, sometimes even changing the form factor.

Racks built this way get deployed by the hundreds of thousands. According to the company, the goal of hardware engineering is to ship fully integrated systems from the factory to the data center in good condition. The ideal picture: crates come off the truck, dedicated material-handling equipment moves the racks into a data row, a few connections get made, the power comes on, and the rack boots and provisions on the first try, becoming part of the fleet. The direction is to minimize hands-on work at the data center site, the way an appliance works the moment you open the shipping box.

All of the hardware innovation shown in the video is built on chips made by AMD and NVIDIA, but one corner of the lab also holds a prototype of Meta's own in-house silicon. Tom Shaw explains that at this scale, you need an approach that's very tightly tailored, and he made clear that this chip, too, is a prototype not yet ready for mass production. METAL has reported on how Waymo is building its own chip for its robotaxis to cut its reliance on NVIDIA — the same pattern, of companies that have grown large enough to design their own cooling, racks, and chips, is now playing out at Meta too.

Cooling choices translate directly into water use. According to Tom Shaw's August cooling explainer, Meta used reinforcement learning in a pilot at one data center to cut air-cooling supply fan energy by an average of 20% and water use by 4%, then rolled that approach out across its air-cooled data centers more broadly. In 2025, it open-sourced its liquid-cooled networking rack platform, Icepack, through the Open Compute Project. METAL has reported on protests outside a G20 venue over the water and electricity data centers consume, and this video's point — that the cooling method you choose determines how much water you use — sits right at the center of that debate.

What this lab says about real competitiveness in AI infrastructure isn't how many GPUs you bought — it's how fast you can redesign cooling, networking, and racks every time a new chip arrives, and then stamp them out of the factory by the hundreds of thousands. Meta is looking for that answer not in a PC assembled from other people's parts, but in a design where it redraws the case itself.