Microsoft announces powerful new chip for AI inference | TechCrunch

"Microsoft has announced the launch of its latest chip, the Maia 200, which the company describes as a silicon workhorse designed for scaling AI inference. The 200, which follows the company's Maia 100 released in 2023, has been technically outfitted to run powerful AI models at faster speeds and with more efficiency, the company has said. Maia comes equipped with over 100 billion transistors, delivering over 10 petaflops in 4-bit precision and approximately 5 petaflops of 8-bit performance-a substantial increase over its predecessor."

"As AI companies mature, inference costs have become an increasingly important part of their overall operating cost, leading to renewed interest in ways to optimize the process. Microsoft is hoping that the Maia 200 can be part of that optimization, making AI businesses run with less disruption and lower power use. "In practical terms, one Maia 200 node can effortlessly run today's largest models, with plenty of headroom for even bigger models in the future," the company said."

Microsoft launched the Maia 200, a chip designed to scale AI inference with over 100 billion transistors. The Maia 200 delivers over 10 petaflops in 4-bit precision and about 5 petaflops in 8-bit performance, improving on the Maia 100. Inference is defined as running models rather than training them, and inference costs are becoming a larger share of AI operating expenses. Microsoft aims for Maia 200 to lower power use and disruption while running the largest models on a single node. The launch follows a trend of hyperscalers building custom accelerators such as Google's TPU and Amazon's Trainium to reduce reliance on NVIDIA GPUs.

Read at TechCrunch