Ever since it was first introduced, Google Gemini has been getting smarter. The dedicated app brings the powerful chatbot to your phones, and Google just brought two new features that will make it even better. Google launched the Ask about this screen and Ask about this video feature for Gemini.
The Gemini app is free to download, and it’s making its way over to tablets. With it, you have the power of Gemini right on your phone. If you want to use Gemini as your voice assistant, you can set it as your assistant. Be sure to download the app if you’re interested in what it has to offer.
Google brings the Ask about this screen feature
Let’s rewind back to the year 2015. Google unveiled a revolutionary new feature called Google Now On Tap. If you remember it, then you were probably as excited about it as we all were. This was the feature that would allow Google to look at your screen and read the context of what’s on it.
Of course, that feature has come and gone, but the mentality lived on with Google Lens. Well, it looks like, after nine years of hard work, Google has finally realized this dream feature, but its name isn’t any more catchy. Today, Google started rolling out the feature called Ask about this screen.
While the name doesn’t quite roll off the tongue, it’s pretty easy and intuitive to use. As the name implies, when you summon Gemini, you’ll be able to ask it what’s currently on your screen. It basically makes it easier to use one of Google Lens’ most useful features. You’re able to take screenshots and import them into Google Lens and have it read what’s in it. Using Ask about this screen, Gemini will take the screenshot and analyze it for you, cutting out the middleman.
After summoning Gemini, you’ll see the action button pop up above the text field, so you won’t have to type it or say it. That makes the feature that much more convenient. The button will appear for you no matter what screen or app you’re using. The only exception is if you’re on YouTube.
In classic AI chatbot fashion, Gemini will give you an in-depth explanation of what’s on the screen. Say, you’re seeing an image of a certain flower on social media, and the post doesn’t give you any context as to what it is. You can summon Gemini to scan what’s on your screen and tell you what it is.
There’s also Ask about this video
The other feature that Google unveiled is specialized for YouTube. Now, Google won’t somehow watch the entire video in the blink of an eye and give you a play-by-play of what it’s about. It will extract information about the video like the captions. If the video doesn’t have captions, the feature actually won’t work.
These two features are making their way to users, and it doesn’t seem that Google is holding them hostage in the Pixel phones. They were discovered in version 15.33.38.28 of the app.
Using the feature
The feature works well in my testing. The thing is that you have to be using an app for it to work, so you won’t see it pop up when you’re on your home screen. However, it works with system apps like the settings.
As for the speed, it’s very speedy. This also goes for analyzing videos. I was able to get a full rundown of the video in just a matter of seconds, and I think that it’s really impressive. Also, I like how you can ask follow-up questions about what you’re looking at.
I only have one minor complaint about the feature. Firstly, after you press the button, you have to press the microphone button to start speaking. It would be nice to have the microphone active right off the bat to start speaking right away. This way, people who are speaking don’t have to do an extra step. People who are typing are going to be interacting with the screen anyway, so an extra tap doesn’t really impact the experience.
The post These new features will let Gemini look at your screen appeared first on Android Headlines.