Gemini agent-based video analysis cuts token use by up to 88 percent

By NewSunSEO | September 17, 2026

Gemini Flash models can now decide on their own which parts of a video to look at, which means developers get up to 88 percent lower token usage, 66 percent lower cost, and slightly higher accuracy when finding specific scenes in long footage. The capability is live today in the Gemini API and is rolling out to the Gemini app and YouTube over time.

Until now, Gemini sampled video at a fixed rate, one frame per second by default. That made it expensive on long files and easy to miss sub-second moments like cuts or state changes. The new mode lets the model loop through the file, pull only the sections it needs, and resample suspicious windows at higher rates when it wants to confirm something.

How the new video analysis works

Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite can pick up events shorter than one second, including rapid state changes and cuts that the one-frame-per-second approach would skip. The model chooses on its own which sections to examine, at what speed, and through which channel, whether frames, audio, or transcript, and it only retrieves the signals a task actually needs.

Internally, the model calls a tool that grabs just the relevant slice of the video file. Developers could build that kind of selective approach by hand before, but now Gemini handles it directly. For tasks like spotting anomalies, the system resamples the relevant time window at a higher frame rate. For counting repeated motions or individual objects, it can sustain accurate counts across long clips.

Where the savings show up

The efficiency gains land hardest on long videos, from 10-minute tutorials and 90-minute lectures to multi-hour recordings. With the old static processing, developers had to choose between high token costs and lower-resolution sampling that dropped important details.

On Google’s 1H-VideoQA and LVBench benchmarks, token usage drops by 88 percent while accuracy edges up slightly. On LongVideoBench, Gemini 3.7 Flash with agent-based analysis posts the highest overall quality score and the best balance of accuracy and cost. In Google’s own 1H-VideoQA evaluation, Gemini 3.7 Flash hits the highest accuracy at the lowest cost per query.

What it builds on

The new mode extends “agentic vision,” which shipped for Gemini 3 Flash in January. That earlier feature let the model write and run Python code to zoom, crop, and annotate images, checking each result in a think-act-observe loop before answering. It did not run automatically in every case at launch, but the plumbing was already in place. When Google announced Gemini 3 Flash in December, visual and spatial reasoning for video was flagged as a coming capability.

How developers turn it on

The feature is available now for video uploads and YouTube videos through the Gemini API in Google AI Studio and on the Gemini Enterprise Agent Platform. Developers set the processing mode to “agentic” in the API config and pay standard Gemini API token rates with no added fee. Full instructions are in the Developer Guide.

Google also plans to bring the same gains to consumer-facing products. The agent-based video analysis should reach Gemini app users on Flash and Flash Lite devices soon. Over the coming months, it will be folded into the “Ask YouTube” feature on the playback page, so answers can be tied more closely to what is actually visible in the video rather than to a sparse sampling of frames.

FAQ

What is Gemini’s new agent-based video analysis?

It is a processing mode for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite that lets the model decide on its own which sections of a video to examine and at what speed, instead of sampling at a fixed one frame per second.

How much does it cut token usage and cost?

Google reports up to 88 percent lower token usage and 66 percent lower cost on its 1H-VideoQA and LVBench benchmarks, with slightly higher accuracy. On LongVideoBench, Gemini 3.7 Flash with agent-based analysis scores the highest overall quality and the best accuracy-versus-cost balance.

Where is agent-based video analysis available?

It is live now through the Gemini API in Google AI Studio and on the Gemini Enterprise Agent Platform, with no extra fee beyond standard Gemini API token rates. Google plans to bring it to the Gemini app on Flash and Flash Lite devices and to the “Ask YouTube” playback page in the coming months.

Related coverage

• NVIDIA’s AVO Coding Agent Hits 100 Percent on ARC-AGI-3 Public Set