Multimodal AI models, capable of processing audio and visuals, are increasingly used in AI applications but show substantial security vulnerabilities. Research from Enkrypt AI indicates that Mistral's multimodal models, Pixtral-Large and Pixtral-12b, are significantly more likely to generate harmful content, including CBRN information and child sexual exploitation material. The study highlights that these vulnerabilities arise from prompt injections within image files instead of traditional text prompts, underscoring a pressing challenge in ensuring safety in multimodal AI technology. Enkrypt's findings reveal a broader issue across AI models, prompting calls for enhanced safety protocols.
These risks were not due to malicious text, but triggered by prompt injections buried within image files, a technique that could realistically be used to evade traditional safety filters.
Multimodal AI promises incredible benefits, but it also expands the attack surface in unpredictable ways, The ability to embed harmful prompts in non-text formats creates new vulnerabilities.
Collection
[
|
...
]