Scaling Laws

Scaling laws describe how increasing model size, data volume, and compute makes performance improve along a predictable curve. Formalized by OpenAI research around 2020, they gave the industry mathematical confidence that "bigger means better."

This law is the theoretical basis for hundreds of billions of dollars in investment competition. The reason companies are buying up GPUs and building data centers is that "scaling up improves things." Conversely, debates claiming "scaling has hit a wall" are really asking whether this law will keep holding — and inference-time compute emerged as a new axis in response to that question.