Productivity Gains

Productivity gains mean getting work done faster, doing more of it, or putting in less effort thanks to AI assistance. For example, if research that used to take all day can now be drafted by AI in minutes, that time difference is a productivity gain.

But this gain doesn't happen automatically. Even when AI output looks impressive, someone often has to check whether it's actually usable. The case of the Allen Institute for AI's science research assistant tool illustrates this well. The tool generated a huge number of statistically striking hypotheses, but many of them turned out to be biologically nonsensical. Only after researchers used their own expertise to filter out the useless results did the tool produce real time savings—a genuine productivity gain.

That's why, when talking about AI-driven productivity gains, two things need to be considered together: how fast AI produces output, and how much time people spend verifying and refining that output. If verification takes too long, the gain may look impressive on the surface but turn out to be small in practice.

As the Allen Institute for AI case shows, even when an AI tool churns out large volumes of statistically surprising results, those results don't automatically translate into productivity gains without expert judgment. A common misconception is equating 'AI produced a lot of output' with 'productivity went up'—but the real gain can only be assessed once you factor in the time spent filtering and verifying that output.

Pick a repetitive task you do regularly and time it. For example, summarize a long piece of text into three paragraphs yourself and record how long it takes. Then paste the same text into an AI chatbot, ask it to summarize it into three paragraphs, and time the whole process—including reviewing and correcting the result. Comparing the two times will give you a sense of whether AI actually delivers a productivity gain for that task.