topology-aware parallelization

Topology-aware parallelization is a strategy for training huge AI models by dividing the work among hundreds or thousands of chips, while first checking how closely or loosely those chips are connected to one another before handing out the tasks.

Think of a large factory assembling parts. Workstations linked directly by a conveyor belt can exchange parts back and forth constantly without any problem, but shuttling parts between a workstation and one on the opposite side of the building wastes a lot of time if it happens too often. AI chips work the same way. Chips tightly wired together can exchange information frequently and still stay fast, but chips that are far apart slow down dramatically if they have to communicate often. Topology-aware parallelization means mapping out this connection structure (the topology) in advance, then grouping tasks that need to talk to each other a lot onto nearby chips, while assigning tasks that need less communication to chips that are farther apart.

This kind of placement strategy matters especially for models with enormous parameter counts, particularly those built from many smaller expert models working together as one (see the mixture of experts entry for this concept). As the number of chips grows, failing to account for the connection structure properly causes communication delays to snowball.

Articles use it in phrasing like: "researching topology-aware parallelization strategies for mixture-of-experts models with up to a trillion parameters across as many as 1,024 Trainium chips." This doesn't refer to a specific product name but to a research methodology for designing large-scale parallel training. Note that it isn't Amazon itself that developed this strategy — rather, it's a university research team (UIUC) using Amazon's computing resources that is studying this topic.