Google is about to scroll through all your photos for its ‘Ask Photos’ feature
Google’s AI-powered Ask Photos feature, previewed at Google I/O 2024, has entered initial testing, first shared by 9to5 Google. Ask Photos uses Google’s Gemini AI to search a user’s photo library using natural language, without first organizing and tagging the image. Google hasn’t shared a release date, but it appears that a select group of users will be asked to try it out and provide feedback before it rolls out more broadly.
Ask Photos essentially extends Gemini’s ability to parse visual information from a user’s photos. The AI can then comb through images, presumably those in an album or on a device that the AI has permission to access and search. It’s not just a search for keywords, either; the AI can also answer questions the user poses. In its first demonstration at Google I/O, CEO Sundar Pichai showed how he could use it to ask his phone for his license plate number, with the answer provided by a photo of the license plate in his album.
The version highlighted by 9to5 Google was discovered by a user on their Pixel 8 Pro. Unlike the official demo, where Ask Photos had its own tab, the feature now appears to be available within the Search tab via an “Ask” button next to the search bar. Tapping the button reveals Ask Photos as a new interface, complete with sample prompts and a space to write your own response to the “What do you want to see?” question. Some aspects of the feature are still being tested, as the sample prompt “Photos of me over time” returned a technical error response and a suggestion to “Use classic search.” Regardless, the feature apparently went away shortly after the user first saw it.
Google Ask Photos reads license plate numbers from the photo library#GoogleIO pic.twitter.com/ElSEQHrvVwMay 14, 2024
Visual search
Using Gemini to browse photos is clearly beneficial to users, especially those with a lot of poorly organized images. Google Photos has some ability to find photos of specific events and people, but applying Gemini’s AI models would vastly improve the platform’s capabilities. The AI would be much better at distinguishing which photos were in the same place, identifying people from different time periods, and otherwise organizing images based on granular criteria.
Ultimately, Ask Photos is just another aspect of Google’s work to integrate Gemini into every product and service it offers. Of course, the feature’s apparent limitations and technical issues at this point highlight its complexity. But when combined with related rollouts, like making Gemini available from Android phone lock screens, the bigger picture becomes clear, even without asking Gemini for help.