"If you aren't ten times faster today than you were in 2023, Karpathy implies that the problem isn't the tools. The problem is you. Which seems both right...and very wrong. After all, the raw potential for leverage in the current generation of LLM tools is staggering. But his entire argument hinges on a single adverb that does an awful lot of heavy lifting: "Properly.""
"In reality, AI speed only seems to be free. Earlier this year, for example, METR (Model Evaluation and Threat Research) ran a randomized controlled trial that gave experienced open source developers tasks to complete. Half used AI tools; half didn't. The developers using AI were convinced the LLMs had accelerated their development speed by 20%. But reality bites: The AI-assisted group was, on average, 19% slower."
"In the enterprise, where code lives for decades, not days, that word "properly" is easy to say but very hard to achieve. The reality on the ground, backed by a growing mountain of data, suggests that for most developers, the "skill issue" isn't a failure to prompt effectively. It's a failure to verify rigorously. AI speed is free, but trust is incredibly expensive."
Proper integration of recent LLM tools could theoretically multiply developer productivity substantially, but achieving that integration is difficult in enterprise settings where code endures for decades. Randomized controlled trial results show experienced open-source developers using AI believed they were 20% faster yet were on average 19% slower. AI-assisted outputs frequently introduce significant security vulnerabilities, with nearly half introducing Top-10-level threats. Perceived speed gains are thus offset by verification and remediation costs. Speed appears free while trust is costly. Organizations must prioritize rigorous verification, governance, and dedicated tooling to realize AI benefits safely and effectively.
Read at InfoWorld
Unable to calculate read time
Collection
[
|
...
]