"Automation is often involved in counter-intuitive ways in software incidents (e.g., impeding resolution or complicating human response). Separating tasks entirely by those that automation handles and those that humans handle influences the design of systems in ways that make resolving incidents more difficult. Automation can degrade human knowledge and skill acquisition/retention in unanticipated ways due to humans having less experience with the systems intended to be run by automation."
"On August 1, 2012, Knight Capital Group, a financial services company, released an update that ended up losing the company $460 million in twenty minutes, severely impacting the valuations of numerous other companies. By the next day, Knight Capital's value plummeted by seventy-five percent leading to its acquisition at a fraction of its original value and the eventual dissolution of the company."
"Our research has shed light on these assumptions and limitations, and how they continue to wreak havoc on software systems that are increasingly reliant on automation (and increasingly, AI). Below I lay out some of the common assumptions and misconceptions about automation and its role in software (and software incidents), what our research has found regarding how automation shows up in software incidents, and some ideas around how"
Automation frequently appears in counter-intuitive ways during software incidents, sometimes impeding resolution or complicating human response. Strictly separating tasks into those handled by automation and those handled by humans can shape system designs that make incident resolution more difficult. Heavy reliance on automation can erode human knowledge, skill acquisition, and retention because operators gain less hands-on experience with automated systems. Cognitive science principles, including Joint Cognitive Systems, can inform better design of tools and procedures that integrate automation. Designers should implement automation to augment and improve human work rather than aiming to replace human operators. High-profile automated failures reveal rapid, catastrophic consequences.
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]