LLMOps

LLMOps is the management system for taking large language models out of the lab and running them in real services every day. Think of it like a car: training the model is building the car, while LLMOps is more like getting that car onto the road, then maintaining it, refueling it, and responding when accidents happen. A model isn't something you build once and forget — it's something you keep tending to.

Concretely, this includes comparing performance against previous versions when prompts change, checking that response quality doesn't drop when swapping in a new model, and monitoring that costs don't suddenly spike when call volume surges. There used to be a term for the practice of putting predictive models into production, but with the arrival of language models, new challenges around managing prompts and model versions emerged, and this term carved out its own space.

More recently, as language models have evolved to autonomously choose tools and carry out multi-step tasks on their own, a separate discipline for managing these autonomous execution units has begun to be discussed as the stage that comes after LLMOps.

The article explains that "the old practice of putting predictive models into production used to be called MLOps, and with the arrival of language models like ChatGPT, LLMOps followed, adding prompt and model version management." Here, LLMOps refers to the stage of reliably serving the language model itself that generates responses, distinct from the next stage (AgentOps) of managing agents that choose tools and autonomously execute multiple steps.