"AI agents need skills - specific procedural knowledge - to perform tasks well, but they can't teach themselves, a new research suggests. The authors of the research have developed a new benchmark, SkillsBench, which evaluates agentic AI performance on 84 tasks across 11 domains including healthcare, manufacturing, cybersecurity and software engineering. The researchers looked at each task under three conditions:"
"no skills (agent receives instructions only), with curated skills (provided with a directory, code snippets and resources to help it) and self-generated skills (agent has no skills but is prompted to develop them). Typical tasks included conducting a security audit of npm dependencies for vulnerabilities, or analyzing differential protein expression in cancer cell line data."
A benchmark named SkillsBench measures agentic AI performance on 84 tasks across 11 domains, including healthcare, manufacturing, cybersecurity, and software engineering. Tasks are evaluated under three conditions: no skills (instructions only), curated skills (provided directory, code snippets, and resources), and self-generated skills (agents prompted to develop skills). AI agents require specific procedural knowledge to complete complex tasks accurately. Performance improves when agents are provided curated skills compared with no skills or self-generated skills. Example tasks include auditing npm dependencies for vulnerabilities and analyzing differential protein expression in cancer cell line data.
Read at InfoWorld
Unable to calculate read time
Collection
[
|
...
]