Google has enhanced its AI Mode chatbot with multimodal capabilities, allowing it to interpret images by integrating its Gemini AI and Lens recognition technology. Users can now upload or take a photo and receive comprehensive answers about the image’s content, complete with relevant links. This update, available in the Google app for Android and iOS, represents a significant leap in visual search technology. AI Mode responds to inquiries using detailed summaries from Google's search index, making it a competitive solution against other AI chatbots.
"AI Mode builds on our years of work on visual search and takes it a step further," says Robby Stein, VP of product for Google Search. "With Gemini's multimodal capabilities, AI Mode can understand the entire scene in an image, including the context of how objects relate to one another and their unique materials, colors, shapes, and arrangements."
The update combines a custom version of Gemini AI with the company's Lens image recognition tech, allowing AI Mode Search users to take or upload a picture and receive a "rich, comprehensive response with links" about its contents.
Google says the update uses a "fan-out technique" that issues multiple queries about the image it sees, and any objects within it, to provide responses that are "incredibly nuanced and contextually relevant."
AI Mode for Search serves as Google's answer to Perplexity and ChatGPT Search, a chatbot-like experience that responds to inquiries with AI-generated summaries pulled from everything in Google's search index.
Collection
[
|
...
]