Nearly everyone in the tech sector is investing heavily in artificial intelligence right now, and Google is among those most committed to an AI future. Project Astra, unveiled at Google I/O 2024, is a big part of that – and it could end up being one of Google’s most important AI tools.
Astra is billed as “a universal AI agent useful in everyday life.” It’s essentially like a combination of Google Assistant and Google Gemini, with extra features and powerful capabilities for a natural, conversational experience.
Here we’re going to explain everything you need to know about Project Astra: how it works, what it can do, when you can get it, and how it could shape the future.
What is Project Astra?
In some ways, Project Astra is no different from the AI chatbots we already have: you ask a question about what’s in a photo, or how to do something, or ask for creative text to be generated, and Astra gets to work on it.
What takes this particular AI project to the next level is its multimodal functionality (the way text, images, video and audio can all be combined), the speed at which the bot works and how conversational it is. Google’s goal, as we have already mentioned, is to create “a universal AI agent” that can do everything and understand everything.
Consider the Hal 9000 bot Kubrick’s 2001: A Space Odysseyor the Samantha assistant in the movie Her: Talking to them is like talking to a human, and there isn’t much they can’t do. (Both AIs eventually became too big for their creators to control, but let’s ignore that for now.)
Project Astra is built to understand context and take action, work in real time, and remember past conversations. From the demos we’ve seen so far, it appears to work on phones as well as smart glasses, and is powered by the Google Gemini AI models – so it could end up being part of the Gemini app, rather than something that separate and self-contained.
When will Project Astra be released?
Project Astra is in its early stages: this isn’t something that will be available to the masses for at least a few months. That said, Google says that “some of these agent capabilities will be coming to Google products like the Gemini app later this year,” so it looks like elements of Astra will gradually appear in Google’s apps over the course of 2024.
When we got some hands-on time with Project Astra at I/O 2024, these sessions were limited to four minutes each – so that gives you an idea of how far this is from something that anyone, anywhere can use. Furthermore, the Astra kit didn’t look particularly portable, and Google representatives were careful to call it a prototype.
All things considered, we get the impression that some of the Project Astra tricks we saw in the demo could appear in the Google Gemini app sooner or later. At the same time, the full Astra experience – perhaps with special hardware – will probably not be rolled out until 2025 at the earliest.
Now that Google has shared what Project Astra is and what it’s capable of, it’s likely we’ll hear a lot more about it in the coming months. Keep in mind that ChatGPT and Dall-E developer OpenAI is busy releasing major upgrades of its own, and Google doesn’t want to be left behind.
What can I do with Project Astra?
One of Google’s demos can be seen Astra runs on a phone, using the camera input and talking naturally to a user: asking to highlight something on screen that can play sounds and correctly identify a speaker. When an arrow is drawn on the screen, Astra recognizes the speaker part highlighted by the arrow and speaks about it.
In another demowe see that Astra correctly identifies world landmarks from drawings in a sketchbook. It is also able to remember the order of objects in a list, identify a neighborhood from an image, understand the purpose of parts of the code presented to it, and solve mathematical problems written out.
Great emphasis is placed on recognizing objects, drawings, text and more through a camera system, while simultaneously understanding human speech and generating appropriate responses. This is the multimodal part of Project Astra in action, taking it a step further than what we already have – with improvements in caching, recording and processing that are key to real-time responsiveness.
In our hands-on time with Project Astra, we were able to make it tell a story based on objects we showed to the camera – and adjust the story as we went along. Further down the line, it’s not hard to imagine Astra applying these smart features while exploring a city on holiday, solving a physics problem on a whiteboard, or providing detailed information about what’s being shown in a sports game.
What devices include Project Astra?
In Google’s Project Astra demonstrations so far, the AI runs on an unidentified smartphone and unidentified smart glasses – suggesting we may not have heard the last of Google Glass.
Google has also hinted that Project Astra will be coming to devices with other form factors. We mentioned the Her movie, and it’s entirely possible that we’ll eventually see the Astra bot built into wireless earbuds (assuming they have a strong enough Wi-Fi connection).
In the hands-on room set up during Google I/O 2024, the Astra was powered by a large camera and could only work with a specific set of objects as props. Clearly, any device with Astra’s impressive features will need a lot of built-in processing power, or a very fast connection to the cloud, to continue the real-time conversation that is at the heart of the AI.
As time goes on and technology improves, these limitations should slowly be overcome. The next time we hear anything major about Project Astra could be around the time of the Google Pixel 9’s launch in the last few months of 2024; Google will undoubtedly want to make this the most AI-enabled smartphone yet.