Walden Yan Appears in Video on Devin Testing With GPT-6 Astra

In the video METAL reviewed, Yan introduces himself as a Cognition co-founder and says the company is building the future of software engineering. Its main product, Devin, is an AI software engineer, and the company works with some of the largest companies in the world, from big banks to tech-native startups, he explains. Recently the team has been testing Astra inside Devin and using it to make improvements across the product, a scope that includes not only the core cloud agent that is Devin but also the CLI and desktop products.

The heart of the video is how the problem gets defined. "Us and other engineering teams now write so much code, the real issue now has become how do you review all that code, how do you get confident in it before it ships," Yan said. The diagnosis is that the bottleneck is not the speed of writing code but the speed of reading it. His answer is Astra's testing ability, and he added, "One of the big pieces that Astra improves on is its ability to test and prove that its work actually functions the way you expect."

The customer story page on OpenAI's site tells the same story in text. According to the page dated September 11, Devin uses Astra to test an iPhone game called Otter Run and returns a recording of the game running in a simulator, alongside a report listing the checks that passed and the areas left untested. The page also describes a round trip in which a customer sends a screenshot of a bug, the team passes it to Devin using Astra, and Devin fixes the issue and sends back a screenshot of the result. Yan said this lets the team get back to customers much quicker. METAL reported on that page the day after it went up, and this time the founder speaks on camera.

The numbers are in Cognition's own announcements. On September 3, Cognition said on X that GPT-6 Astra was coming to Devin, adding that on its FrontierCode 1.1 benchmark Astra performs within 0.4 points of Fable 5 at a 64% lower cost. That post passed 851,000 views. The company blog added one more line: FrontierCode 1.1 is Cognition's proprietary benchmark that grades models on real-world engineering tasks by quality and mergeability, and on it GPT-6 Astra surpasses Fable 5.1, second only to Fable 5. When powering Devin's testing capabilities, the company said, Astra achieves state-of-the-art results on its internal testing benchmark, driving tests that are more comprehensive, with clearer reports and more user-friendly video evidence.

The deployment is described there too. GPT-6 Astra is available now in Devin Desktop and Devin CLI, and in Devin Cloud it is part of the model mixture that blends several models. Rather than one company picking one model, the structure assigns different models to different tasks. METAL reported that Cognition brought Fable 5.1 into Devin and cut the cost of coding tasks by 54%. Two weeks later, another company's model entered the same product as the tester.

Seen through an engineering manager's eyes, this video is about the accounting of review cost. The more code an agent writes, the more code people have to read, and a team's throughput ends up tied to review speed. Cognition's direction is not to eliminate review but to change what gets reviewed: instead of reading code line by line, engineers look at recordings and reports, and dig into only the untested areas listed in the report. Yan put it this way: "We expect over time to mean that we have to manually look at less code and end up shipping more at the end of the day." He closed the video by saying this is one of the things he is most excited about when it comes to GPT-6.

The video is one installment in OpenAI's run of short customer videos for GPT-6 Astra. METAL reported on the 68-second video of Perplexity co-founder Johnny Ho posted a day earlier, on September 14. The two videos say the same sentence through different companies: the model does not stop at writing code, it also builds the apparatus that checks the code is right, and people look at the results less. OpenAI is selling the model with customers' faces and voices instead of benchmark tables, and Cognition has added its own benchmark numbers to that approach.