Distillation Attack

A distillation attack is when someone sends huge numbers of questions to another company's excellent AI model, collects the answers, and uses that data to train their own cheaper, smaller model.

Here's an analogy: it's like constantly asking the top student in class for answers, writing them all down, and then using that answer collection to teach other students. The company that built the original model sees this as someone copying, almost for free, the "skill" it spent years and huge amounts of money developing. To stop this, companies tighten account verification and block access.

The problem is that a workaround market springs up between those trying to block it and those trying to get around the block. Regions or organizations that are cut off from normal access route their requests secretly through proxy servers (relay channels that pass requests along on someone's behalf), then use the answers they get to improve their own models. Companies treat this as a violation of terms of service and a security issue involving stolen intellectual property—hence the label "attack."

One article reported that Chinese developers were buying Anthropic's Claude tokens through workaround services at roughly 10% of list price, noting that this demand likely includes Chinese AI labs looking to quickly improve their weaker models using answers from Western models. A common misconception is that distillation itself is a bad technique—transferring knowledge from a large model to a smaller one is actually a widely used, legitimate method. What makes it problematic is extracting massive amounts of answers by bypassing access controls without the other company's permission, which is why it's called an "attack."