"Enterprise IT execs know well the dangers of relying too much on third-parties, how automated decision systems need to always have a human in the loop, and the dangers of telling customers too much/too little when policy violations require an account shutdown. But a saga that played out Tuesday between Anthropic and the CEO of a Swiss cybersecurity company brings it all into a new and disturbing context."
"Within a day, the account was restored - sort of. The new alert told him: "Earlier this week, your account was disabled by an automated system for being in violation of our Terms of Service or Acceptable Use Policy. Upon further investigation, we believe this was an error and your account has been reinstated. We apologize for the inconvenience and for your patience." Hoffman briefly celebrated, logged in and found that most of the account - 80% of company projects and data, he said - was missing."
Tom Hoffman, CEO of Swiss cybersecurity company Wicked Design, received an automated alert that Anthropic had cancelled the company's entire account for unspecified policy violations. Hoffman posted on LinkedIn and alerted Anthropic contacts, including the head of product legal, who said the issue was flagged for internal review. The account was reinstated within a day with a notification admitting error, but approximately 80% of company projects and data were missing. Anthropic's automated support response offered empathy but no clear recovery. The incident demonstrates risks of excessive dependence on third-party automated enforcement and inadequate human oversight.
Read at Computerworld
Unable to calculate read time
Collection
[
|
...
]