In-context Learning

In-context learning is the ability of an AI model to imitate a new trick right on the spot, just from the conversation or examples it's currently given, without being retrained separately.

A useful comparison is a new employee who, instead of going through a months-long training program, is shown a task once and immediately copies it. The employee's actual skill hasn't changed — they're just mimicking what they saw a moment ago, and if they forget the demonstration, they have to learn it from scratch again. AI works the same way. The knowledge values stored inside the model (called weights) aren't touched at all; the model simply skims the examples or instructions currently on screen and produces an answer for that moment.

A striking example of this showed up in robotics. A robot arm was shown a single 12-second video of sweeping blocks into a bowl, and without any additional training, it immediately copied the motion. The developer said this ability wasn't deliberately designed in — it simply emerged on its own after continued training on massive amounts of data. It's the same principle at work when you type a few examples into a chat AI's conversation window and it immediately picks up the pattern to answer.

The article explained a robot's ability to perform a new motion right after watching just one demonstration video through in-context learning, noting that "there was no architectural change to encourage in-context learning, nor any meta-learning loop." Contrary to common assumption, this ability isn't a feature that was deliberately built in — it's a phenomenon that emerged naturally after prolonged training on large amounts of data.

Try typing two or three rules like below into a chatbot's conversation window, then ask a new question — you'll see it pick up the pattern and answer accordingly, without any retraining.

Apple -> 사과 Banana -> 바나나 Grape ->

When you type this, the model doesn't just pull out some random Korean word it knows — it figures out on the spot the rule from the two examples given (translating fruit names into Korean) and answers 포도. Change the examples to a completely different rule, and the model instantly follows the new rule too — that's in-context learning.