Red Teaming

It's a role, or activity, where people deliberately try to attack a product before it's released. The term comes from military exercises, where a simulated enemy force (the red team) attacks friendly forces (the blue team). It moved through the security industry and has since become standard practice for AI.

Before a new model launches, hundreds of people stress-test it, trying to jailbreak it, provoke dangerous answers, and surface biases, and the results get published in the model card. Phrases like "reviewed by external red teamers for months" have become a trust signal in launch announcements. Government regulations are also increasingly building red-teaming requirements into rules for frontier models.