Data Flow Space

Data Flow Space is a conceptual search space used to figure out in what order to connect layers taken from different AI models.

Think of it like arranging dishes prepared by different chefs into a course order. Each dish is already finished, but the order in which they're served changes how satisfying the whole meal feels. There are far too many possible orderings for a person to check by hand, so a computer generates many arrangements automatically, keeps only the ones that perform well, and repeats this process to find the best order.

The key feature of this approach is that it doesn't train anything from scratch. Since it uses the layers of already-trained models as-is and only changes the order in which they're connected, it can greatly cut the computational cost of building a new model. In the evolutionary model merging developed by Sakana AI, Data Flow Space was used alongside another search space that mixes the weights of multiple models.

In articles, you'll see it used like: "In Data Flow Space (DFS), the order in which layers from different models are connected is evolved." It's important not to confuse this with a storage space where data is kept or moved around — it refers to a conceptual space that a computer explores through trial and error to determine the order in which model layers should be arranged.