This Gemini feature in YouTube convinced me to ditch the Assistant

Ever since Google announced that it would begin phasing out its Assistant for Gemini as the default voice assistant, I have found myself struggling to pick between the two. On one hand, I like the familiarity and reliability of the Assistant but on the other, Gemini is the only one still getting new features. The latter is exactly what finally pushed me to start using Gemini more seriously — despite the occasional hiccup and hallucination.

In particular, I’ve found one Gemini feature that’s quietly worked its way into my day-to-day life: the Ask about this video button that only pops up when you’re watching a video within the YouTube app. It’s surprisingly easy to overlook this feature, but it has proven far more useful than I expected and completely changed how I consume certain types of videos.

Gemini Apps: Google’s ecosystem integration pays dividends

Like many others, I often rely on YouTube as a knowledge resource to fix a need or problem in my life. From changing my car’s air filters to recreating a particular fast food item at home, I find it useful for practically everything. But there’s usually one problem: once I’ve watched a video, going back to it after some time usually isn’t as helpful. Scrubbing through to find a specific fact can be frustrating, especially in longer videos where the structure is loose and the creator has not provided timestamps.

Sometimes, I just want to revisit a single point in a YouTube video — to recall what a phone reviewer said about battery life, or the name of a setting they showed in passing, or the amounts of flour and water I need for a specific pizza dough. It gets worse in longer videos too — stock market analysis pieces, college lectures, and documentaries can run over an hour. I rarely want to commit to a full rewatch just to find one idea I didn’t write down.

Luckily, I no longer have to do that. The other day, I noticed YouTube in the list of Gemini Apps. This feature — formerly called Gemini Extensions — is Google’s way of connecting the chatbot to services like Gmail, Drive, and even the Assistant on Android. But in the case of YouTube, that means you can ask Gemini questions about a specific video. On the web, this can be done by pasting in a YouTube video URL into a Gemini prompt.

More importantly, though, the YouTube integration presents an Ask about this video button in the YouTube app on Android (pictured below). In practical terms, this allows me to simply summon Gemini by long pressing the phone’s power button, attach the video I’m watching, and enter a prompt.

It’s worth noting that Gemini does not stop at generating a text-based summary for YouTube videos; you can steer it in just about any direction with your own prompts. For example, I have used it for countless recipe videos where the creator did not add a written version in the description. Another time, I needed to remember the name of a research paper a YouTuber cited midway through a video. I asked Gemini directly what studies were mentioned, and it responded within seconds, even providing a timestamp alongside the answer.

The way this feature works is pretty ingenious: Gemini accesses the YouTube video’s closed captions and adds it to your prompt alongside any other information you provide or ask for. You can think of it like uploading a PDF to ChatGPT, except with fewer steps. On Android with the YouTube app open, Gemini can “see” the video you’re watching so you don’t have to manually share the URL.

Most videos have automatically generated closed captions and transcripts these days — thanks to improving AI. As a result, I have found that the feature works across most types of content and even with videos from smaller creators who don’t typically add subtitles.

Of course, Gemini’s reliance on captions is also its Achilles’ heel. If something isn’t said out loud, it doesn’t exist to Gemini. That means it won’t recognize background visuals, on-screen text, or actions shown without narration. For example, in a music theory video, I can’t actually ask what chord progression someone played unless they explicitly described it. If they just demonstrated it on a keyboard without saying anything, Gemini has nothing to go on. The same applies to non-verbal tutorials, language-learning content, or any video in a language without proper caption support. That said, Gemini now supports analysis of short videos, so it probably won’t be long before the AI can fully understand long-form YouTube content too.

Gemini can also stumble on tone and context. Jokes, satire, or casual sarcasm tend to be taken literally, which can make the responses misleading for certain creators. It’s not a dealbreaker if you’re using AI as a shortcut to revisit information, but it does mean you can’t always trust it. In the end, it’s still a tool and not a substitute for understanding the source content.

So while I won’t be using Gemini’s YouTube integration to substitute my binge of technology reviews, it has quietly become a feature I reach for more often than not. Say what you will about the state of AI on smartphones, but Gemini has solved a real-world problem, and I did not expect a singular feature to save hours of my life. The only tradeoff is that you need to download the Gemini app and ditch the Assistant for good. But even if you don’t right now, Google is taking away that choice from all of us by the end of the year.