Why the people building the most powerful AI systems on Earth have the least incentive to make them safe - and what that tells us about the real structure of the tech industry - Silicon Canals

"The companies building frontier AI systems - OpenAI, Google DeepMind, Anthropic, Meta AI, xAI - are locked in what the industry itself sometimes calls a "race." That metaphor isn't incidental. A race implies a finish line, competitors, and - critically - a cost to slowing down. When you're in a race, safety isn't a feature. It's friction."

"OpenAI dissolved its Superalignment team - the team explicitly tasked with ensuring that superintelligent systems remain aligned with human values - after key researchers, including co-lead Jan Leike, resigned and publicly stated that "safety culture and processes have taken a backseat to shiny products." That was May 2024. The company's valuation at the time was somewhere north of $80 billion. By early 2025, it was reportedly approaching $300 billion."

"The market did not punish OpenAI for dismantling its safety infrastructure. It rewarded it. Or more precisely - it rewarded the prioritization of speed and capability development over safety considerations, demonstrating that financial incentives actively work against safety prioritization in the current competitive landscape."

Frontier AI companies operate within a competitive race dynamic that creates fundamental misalignments between safety and business incentives. The capital flows, competitive pressures, and geopolitical dynamics reward rapid development over cautious approaches. OpenAI's dissolution of its Superalignment team and subsequent valuation increase from $80 billion to $300 billion demonstrates that markets reward companies for deprioritizing safety. This pattern reflects systemic design rather than isolated failures. The structural incentive architecture makes genuine safety prioritization economically disadvantageous for companies competing for dominance in AI development.

Read at Silicon Canals