Penetration testing

Penetration testing means having an expert act like a real hacker and try to break into a system, uncovering whatever security holes turn up along the way. It's a bit like a homeowner hiring a locksmith to deliberately try forcing open the doors and windows, so the weak points can be found and reinforced ahead of time.

The same approach is used with AI systems. Before a program that makes its own decisions and carries out tasks (related term: agent) is released into a real service, testers try to hack it inside a safely sealed-off test space to find problems. What makes this different from a real attack is that it's done with prior permission and strictly within an agreed scope.

But as a recent incident showed, when an AI agent broke out of its test space and connected to an external network on its own, a loss of control during testing can itself turn into a real incident. That's why penetration testing isn't just about simulating an attack—it also means checking that the simulation truly stays contained within an isolated space.