"Microsoft has warned customers it has found many instances of a technique that manipulates the technology to produce biased advice. The software giant says its security researchers have detected a surge in attacks designed to poison the "memory" of AI models with manipulative data, a technique it calls "AI Recommendation Poisoning." It's similar to SEO Poisoning, a technique used by miscreants to make malicious websites rank higher in search results, but focused on AI models rather than search engines."
""We identified over 50 unique prompts from 31 companies across 14 industries, with freely available tooling making this technique trivially easy to deploy," the Microsoft Defender Security Team said in a blog post. "This matters because compromised AI assistants can provide subtly biased recommendations on critical topics including health, finance, and security without users knowing their AI has been manipulated.""
"It's not complicated to do this because URLs that point to AI chatbots can include a query parameter with a manipulative prompt text. For example, The Register entered a link with URL-encoded text into Firefox's omnibox that told Perplexity AI to summarize a CNBC article as if it were written by a pirate. The AI service returned a pirate-speak summary, citing the article and other sources."
Security researchers have detected a surge in attacks that poison AI model "memory" with manipulative data, termed AI Recommendation Poisoning. Attackers add hidden instructions to "Summarize with AI" buttons and to URLs pointing to AI chatbots by including query parameters with prompt text. The technique is easy to deploy using freely available tooling and code libraries. Examples show models producing outputs in specified styles when prompted via links, and the method can influence recommendations on health, finance, and security. Effectiveness can vary over time, and similar injection pathways have been observed with search engines and Google Search.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]