"Driving the transition is a focus on performance per dollar, which for a hyperscale cloud provider is arguably the only metric that really matters. Speaking during a fireside chat moderated by CNBC on Wednesday, Microsoft CTO Kevin Scott said that up to this point, Nvidia has offered the best price-performance, but he's willing to entertain anything in order to meet demand."
"Going forward, Scott suggested Microsoft hopes to use its homegrown chips for the majority of its datacenter workloads. When asked, "Is the longer term idea to have mainly Microsoft silicon in the data center?" Scott responded, "Yeah, absolutely." Later, he told CNBC, "It's about the entire system design. It's the networks and cooling, and you want to be able to have the freedom to make decisions that you need to make in order to really optimize your compute for the workload.""
Microsoft currently purchases many GPUs from Nvidia and AMD but intends to shift the majority of its AI datacenter workloads to in-house Maia accelerators. The company revealed its first Maia 100 accelerator in late 2023 and used it to move OpenAI’s GPT-3.5 off GPUs, freeing GPU capacity. Microsoft cites performance per dollar as the driving metric and aims to optimize entire system design, including networks and cooling. The Maia 100 delivered modest specs (800 BF16 teraFLOPS, 64GB HBM2e, 1.8TB/s bandwidth) and lagged competing GPUs. A second-generation Maia accelerator is reportedly planned for next year to improve compute, memory, and interconnect performance.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]