In a blog published this week, Manifold Security showed how an AI-powered code reviewer built on Claude accepted changes that appeared to come from a legitimate maintainer. By setting a fake author name and email in Git, the team made a commit appear to originate from a trusted source, then passed it through an automated review flow where the model approved it.
The most dangerous assumption in quality engineering right now is that you can validate an autonomous testing agent the same way you validated a deterministic application. When your systems can reason, adapt, and make decisions on their own, that linear validation model collapses.
This extends to the software development community, which is seeing a near-ubiquitous presence of AI-coding assistants as teams face pressures to generate more output in less time. While the huge spike in efficiencies greatly helps them, these teams too often fail to incorporate adequate safety controls and practices into AI deployments. The resulting risks leave their organizations exposed, and developers will struggle to backtrack in tracing and identifying where - and how - a security gap occurred.
Dependabot sounded the alarm on a large scale. Thousands of repositories automatically received pull requests and warnings, including a high vulnerability score and signals about possible compatibility issues. According to Valsorda, this shows that the tool mainly checks whether a dependency is present, without analyzing whether the vulnerable code is actually accessible within a project.
One thing I always do when I prompt a coding agent is to tell it to ask me any questions that it might have about what I've asked it to do. (I need to add this to my default system prompt...) And, holy mackerel, if it doesn't ask good questions. It almost always asks me things that I should have thought of myself.
Hast mentioned that they trust their unit tests and integration tests individually, and all of them together as a whole. They have no end-to-end tests: We achieved this by using good separation of concerns, modularity, abstraction, low coupling, and high cohesion. These mechanisms go hand in hand with TDD and pair programming. The result is a better domain-driven design with high code quality. Previously, they had more HTTP application integration tests that tested the whole app, but they have moved away from this (or just have some happy cases) to more focused tests that have shorter feedback loops, Hast mentioned.
Your coding apprentice can build, at your direction, pretty much anything now. The task becomes more like conducting an orchestra than playing in it. Not all members of the orchestra want to conduct, but given that is where things are headed, I think we all need to consider it at least.
To find the typical example, just observe an average stand-up meeting. The ones who talk more get all the attention. In her article, software engineer Priyanka Jain tells the story of two colleagues assigned the same task. One posted updates, asked questions, and collaborated loudly. The other stayed silent and shipped clean code. Both delivered. Yet only one was praised as a "great team player."
