"In the blog post introducing the report, the company said the results were, "Clear, measurable, and consistent with what many developers have been feeling intuitively: AI accelerates output, but it also amplifies certain categories of mistakes." The report also found security issues increasing consistently in AI co-authored pull requests. While none of the noted vulnerabilities were unique to AI-generated code, they appeared significantly more often, increasing the overall risk profile of AI-assisted development."
"AI makes dangerous security mistakes that development teams must get better at catching, advised the report. There were, however, some advantages with AI, said the report. Spelling errors were almost twice as common in human-authored code (18.92 vs. 10.77). This might be because human coders write far more inline prose and comments, or it could just be that developers were "bad at spelling," the report speculated. Testability issues also appeared more frequently in human code (23.65 vs. 17.85)."
"Project-specific context should be provided up-front, with models accessing constraints, such as invariants, config patterns, and architectural rules. To reduce issues with readability, formatting, and naming, strict CI rules should be applied. For correctness, developers should require pre-merge tests for any non-trivial control flow. Security defaults should be codified. Also, developers should encourage idiomatic data structures, batched I/O, and pagination."
Analysis of 470 open source GitHub pull requests, including 320 AI-co-authored and 150 human-only, shows AI accelerates output while increasing certain mistake categories. Security issues rise consistently in AI co-authored pull requests; vulnerabilities are not unique to AI but occur more frequently, raising the risk profile of AI-assisted development. Human-authored code had almost twice the spelling errors and more testability issues. Recommended mitigations include project-specific context for models, strict CI formatting and naming rules, pre-merge tests for non-trivial control flow, codified security defaults, idiomatic data structures, batched I/O, pagination, smoke tests for I/O-heavy paths, and AI-aware pull-request checklists.
Read at InfoWorld
Unable to calculate read time
Collection
[
|
...
]