How GPUs Actually Talk: The Two Networks Inside Every AI Datacenter
Two networks carry a model's data between GPUs. Scale-up connects the GPUs inside one rack. Scale-out connects racks to each other. In NVIDIA's 72-GPU rack, scale-up carries 900 GB/s per GPU in each direction and scale-out carries 100 GB/s, a 9x difference.
Scrollypedia