shadow testing

Shadow testing means quietly standing a newly built system or version behind the actual live service, letting real user requests flow through it as well, and just recording and comparing the results.

Think of it like an understudy in theater. The understudy recites the lines backstage at the exact same moments as the lead actor, but the audience only sees the lead on stage and notices nothing unusual. The lead still delivers the real lines, and only people from the production crew watch the understudy's performance to judge how well they're doing. Shadow testing works the same way. The answer that actually goes out to the user is still produced by the existing system, but the new system also receives the same requests behind the scenes and generates its own answers — answers that are never shown to the user, just quietly saved and compared.

Why go through this trouble? Because if a new version is released directly to users and suddenly gives wrong answers or slows down, it can shake the entire service. By catching problems in advance under conditions identical to real traffic, the risk at the moment of the official switch can be greatly reduced. However, if the new system performs actions that leave real traces — like processing a payment or sending a message — that part needs to be specifically blocked from actually executing, so it only simulates the action instead.

If you want to safely verify a new version before adopting it, you could ask an AI coding tool something like this:

Write code that runs the newly built model (or function) version alongside the existing version, but only returns the existing version's response to the user while logging the new version's response to a log file only. Also include a request ID with each entry so the two responses can be compared.