"NVLink is a high-speed interconnect that allows multiple GPUs spanning multiple systems to pool resources and behave like a single accelerator. Previously, this technology has been limited to Nvidia CPUs and GPUs, but back in May, the AI infrastructure giant announced it was opening the tech to others with the introduction of NVLink Fusion at Computex. Amazon claims that the technology will allow its Trainium4 accelerators, Graviton CPUs, and EFA networking tech to communicate seamlessly across Nvidia's MGX racks."
"In its current form, Nvidia's fifth-gen NVLink fabrics support up to 1.8 TB/s of bandwidth (900 GB/s in each direction) per GPU, but the company is on track to double that to 3.6 TB/s by next year. Beyond Nvidia's interconnect tech, details are somewhat vague. We're told that the new chips will deliver 3x more FLOPS at FP8, 6x the performance at FP4, and 4x the memory bandwidth. Whether those claims pertain to the individual chips or its UltraServer rack systems, Amazon hasn't said."
Trainium4 accelerators will adopt Nvidia's NVLink Fusion interconnect to enable chip-to-chip communications across MGX racks. NVLink allows multiple accelerators across systems to pool resources and behave like a single unit. Nvidia's fifth-generation NVLink currently supports up to 1.8 TB/s per GPU and is expected to double to 3.6 TB/s next year. AWS states the new silicon will deliver 3x more FP8 FLOPS, 6x FP4 performance, and 4x memory bandwidth. Uncertainty remains whether the performance claims apply to individual chips or to UltraServer rack systems. Trainium continues to target both training and inference workloads.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]