Cloud-native computing is poised to explode, thanks to AI inference work

"AI inference is the process by which a trained large language model (LLM) applies what it has learned to new data to make predictions, decisions, or classifications. In practical terms, the process goes like this. After a model is trained, say the new GPT 5.1, we use it during the inference phase, where it analyzes data (like a new image) and produces an output (identifying what's in the image) without being explicitly programmed for each fresh image. These inference workloads bridge the gap between LLMs and AI chatbots and agents."

"Also: Kubernetes, cloud-native computing's engine, is getting turbocharged for AI CNCF Executive Director Jonathan Bryce explained in a KubeCon press conference that AI inference is "a stage where you take that model, you serve the model, and you answer questions, you make predictions, you feed it into systems to take that intelligence and connect it out to the world." He emphasized that inference involves transforming a trained AI model into a service that can respond to new questions or situations."

"Making an LLM is mind-bogglingly expensive. According to Bryce, Sam Altman, OpenAI's CEO, has said that GPT-5 training runs may cost up to a billion dollars. Fortunately, most companies, said Bryce, don't need, nor should they even try, to build massive LLMs. Instead, they should use "hundreds of smaller, fine-tuned,""

The Cloud Native Computing Foundation (CNCF) projects a large surge in cloud-native computing driven by rapidly growing AI inference workloads, with hundreds of billions in expected spending over 18 months. AI inference turns trained LLMs into services that analyze new data and produce outputs, linking models to chatbots, agents, and production systems. Kubernetes and cloud-native platforms are being optimized to serve inference workloads at scale. LLM training costs can be extremely high, so most organizations are advised to deploy many smaller, fine-tuned models rather than build massive foundational models. New AI-first cloud types, like neoclouds, are emerging.

Read at ZDNET