Delta Lake

Delta Lake is a storage technology that turns a warehouse full of scattered files into an organized filing cabinet. A regular data lake is like a warehouse where photos, logs, and tables all get dumped in without any distinction by format, so if multiple people edit the same file at once, the contents can easily get corrupted or jumbled. Delta Lake adds a table of contents and a locking mechanism on top of that warehouse, so that even if several people write data at the same time, any failed operation is blocked from being applied at all, and it keeps a record of who changed what and when.

Thanks to this, companies no longer have to choose between a warehouse for storing raw, unstructured data and a warehouse for handling only neatly organized tables. Delta Lake's role is to let a single storage system hold both the raw originals and a reliable, up-to-date table at the same time. Databricks released this concept as open source, so other companies' systems can manage data the same way too.

News articles often use the term "data lake," which refers to the storage system itself where files pile up — it shouldn't be confused with Delta Lake, which is the technology that keeps that storage from turning into a mess. The reason Databricks calls its platform a "lakehouse" is that technologies like Delta Lake add the organized structure of a filing cabinet ("house") to the warehouse ("lake").