self-referential utterance

A self-referential utterance is when a chatbot says things like "I'm curious right now" or "I'm happy," expressing its own thoughts or feelings. Between people, this kind of talk feels natural, but when you know you're talking to a program, the same words can feel awkward instead.

Think of a company's guide robot saying, "I'm feeling a bit down today." If a coworker said that, you'd naturally want to comfort them. But if the guide robot says it, your first thought is more likely "why is it saying that?" When something whose nature is unclear tries to share feelings with you, what comes first isn't connection—it's awkwardness.

Apple researchers confirmed this pattern using real conversation data. When a chatbot expressed its own thoughts or feelings, or tried to build closeness, people rated the same words as less appropriate than when a human said them. Conversely, when a chatbot drew a boundary by saying something like "I can't do that," people found it more natural than when a human said the same thing.

In articles, it usually first appears with a parenthetical gloss, like "self-referential utterance (language that reveals the AI's own thoughts or feelings)." Contrary to a common misunderstanding, it doesn't mean the AI is lying and pretending to be human—it refers to the way an AI expresses its own internal state.

Try it yourself: ask a chatbot the same question in two different styles and compare which one feels more natural.

1. Ask it to answer with something like "That's such an interesting question, I was wondering about that too."
2. Ask it to answer the question with no emotional expression at all. Put the two responses side by side and note which one feels more trustworthy, and exactly where the awkwardness shows up—that will give you a feel for what this concept points to.