TamperBench

TamperBench is a checklist that tests how easily the "don't do this" safeguards built into an AI model disappear once the model is in a user's hands. Think of it like handing a locked box to many different people and testing, in various ways, how easily each of them can pick that lock.

An international research team led by the University of Waterloo in Canada and the AI safety research group FAR.AI created this test and ran it on 21 publicly downloadable open-weight models that anyone can tweak on their own. The results were alarming: for most of the models, just one additional round of fine-tuning aimed at reshaping the model's behavior was enough to strip away its safeguards.

This matters because of a core feature of open-weight models: you can download the entire finished AI file and modify it freely on your own computer. That same freedom that makes these models cheap and useful as defensive tools also means anyone can remove the restrictions the maker originally built in. TamperBench is a yardstick that puts concrete numbers on this risk.

Articles introduce the TamperBench research by noting that "an international research team led by the University of Waterloo in Canada and the AI safety research group FAR.AI found that the safeguards of 21 widely used open-weight models collapse after just one round of fine-tuning." A common misunderstanding is that this isn't a story criticizing any single model—it's a test result confirming a structural weakness across multiple models that is inherent to the open-weight approach itself.