Key Takeaways
- Gemini’s floating panels allow for a seamless user experience without disrupting your current app.
- This has paved the way for a new context-based feature that lets you ask about what’s on your screen without leaving the app, rolling out now in beta.
- YouTube video summaries are also now possible with Gemini, parsing through captions for quick answers.
Back at Google I/O in May, Google showed off some neat Gemini tricks, some of which have already been spotted in the wild, some that have rolled out, and some that hadn’t been seen until now.
A small-yet-necessary change that has already been implemented to an extent is floating Gemini panels. Previously, when you asked the AI assistant to perform a complex task, it would take you to a separate awkward window altogether, away from the app that you were originally on when you triggered the assistant. Now, instead of opening up a new instance, Gemini remains in its crisp floating form (in beta), regardless of your query.
While this may seem like a small change, it was important to implement it first to allow an upcoming feature to work seamlessly.
At I/O, the tech giant showed off Gemini with on-screen contextual abilities, which also appear to be rolling out now. The feature, called “Ask about this screen,” essentially allows users to put forward queries about whatever is on the screen.
The tool shows up right above the floating Gemini panel when you first trigger it. For reference, in case you trigger it over a news article, you’d be able to ask something like “What is this article about,” or something more casual like “TL;DR this,” and the AI assistant will be able to offer a summary without leaving the news article you were on.
Easy AI-generated YouTube video summaries
In the case of YouTube, the button placement will remain the same, though instead of “Ask about this screen,” the tool icon will say “Ask about this video,” as seen in the screenshots above. You’ll then be able to type or speak in your query, and the tool will parse through the video’s captions/transcripts.
We tried replicating the feature with other caption-enabled video apps, like Vimeo, ESPN, and even with a video playing in Chrome, but the tool resorts to the “Ask about this screen” tool instead of “Ask about this video.”
Fun fact: Considering that the tool parses through video captions on YouTube to answer your queries, you don’t even need to wait for the pre-roll ads to finish, provided you don’t subscribe to YouTube Premium. If you’re not interested in watching the video, and just need a brief summary of it, you can pause the pre-roll ad the moment it begins, trigger the AI assistant and have your query answered.
We’re seeing the feature widely across devices at Android Police on Gemini beta app version 1.0.662093464 and the Google beta app version 15.34.32. Running the two betas should automatically give you access to the floating Gemini panel as well.
Source link
