Why everyone is talking about Andrej Karpathy's autonomous AI research agent | Fortune

"Andrej Karpathy put an AI coding agent to work running a series of experiments to figure out how to improve the training of a small language model. He let the AI agent run continuously for two days, during which time it conducted 700 different experiments. Over the course of those experiments, it discovered 20 optimizations that improved the training time."

"What caught many people's attention was that the autoresearch is close to the idea of self-improving AI systems that were originally broached in science fiction and that some AI researchers fervently desire and others deeply fear. The concern is that "recursive self-improvement," where an AI continually optimizes its own code and training in a kind of loop, could lead to what AI safety researchers sometimes call a "hard takeoff" or an "intelligence explosion.""

"Tobias Lütke, the cofounder and CEO of Shopify, posted on X that he tried autoresearch to optimize an AI model on internal company data, giving the agent instructions to improve the model's quality and speed. Lütke reported that after letting autoresearch run overnight, it ran 37 experiments and delivered a 19% performance gain."

Andrej Karpathy demonstrated an AI coding agent that conducted 700 experiments over two days to optimize language model training, discovering 20 improvements that yielded an 11% speed increase. Shopify CEO Tobias Lütke replicated the approach with similar results, achieving a 19% performance gain. Karpathy termed this system "autoresearch." These demonstrations sparked discussion about self-improving AI systems and recursive self-improvement, where AI continually optimizes its own code and training. This capability raises concerns among AI safety researchers about potential "hard takeoff" scenarios or "intelligence explosions," where rapid autonomous improvement could lead to unpredictable outcomes.

Read at Fortune