dual-use query

A dual-use query is a question or request that can be used for both good and bad purposes. Think of a kitchen knife: it's used for cooking, but it could also hurt someone. Neither the seller nor the buyer can know in advance how it will be used. The same goes for questions asked of AI. For example, a question about how a certain pathogen affects the body could come from a scientist researching vaccines, or from someone with dangerous intentions. Looking at the wording of the question alone, it's hard to tell the intent apart.

In reality, making a drug to treat a disease often requires first handling the very dangerous substance that causes it. Making a vaccine requires cultivating the pathogen you're trying to prevent, and some blood pressure medications came from the process of isolating venom from venomous snakes. There are genuinely areas where dangerous knowledge and beneficial knowledge are inseparably intertwined like this.

This is why AI companies maintain gatekeeper systems to filter out such questions, and early on, they often broadly blocked entire categories of related questions for safety's sake. The problem is that this also blocks questions from students, patients, and medical professionals who are asking out of genuine curiosity or need. Recent improvement efforts have focused on refining this gatekeeper more precisely, so it can better distinguish between truly dangerous questions and those that reflect ordinary, legitimate curiosity.

News articles describe safety systems being redesigned to "better distinguish the subtle difference between benign queries and dual-use queries." A common misunderstanding: a dual-use query isn't inherently a bad question. Rather, the same question could be safe or dangerous depending on context, which is exactly why it's classified as a case that's tricky for AI to judge.