Dream-RSI: Google's method for replaying AI agent searches

Free Instant SEO Audit

Dream-RSI concept shown as an AI orb replaying glowing search path arrows

By NewSunSEO September 22, 2026

AI agents can now test thousands of alternative strategies against the results of past searches before spending compute on a live run, thanks to a method called Dream-RSI from researchers at Google and DeepMind. The approach cuts the number of attempts needed to find better solutions on hard coding and math problems, and it lets agents keep improving as they accumulate search history.

What Dream-RSI does

Self-improving AI agents work by proposing a solution, evaluating it, learning from the result, and trying again. Over many attempts, they gradually move toward a good outcome. For complex tasks, the space of possible approaches grows enormous, and the agent has to keep deciding which ideas to pursue, which to try in parallel, and which to abandon. That exploration step often determines whether a search succeeds or wastes compute chasing the wrong ideas.

Dream-RSI changes how the agent searches, not the underlying AI model. Existing approaches handle exploration in one of two ways. A fixed search strategy cannot learn from experience, so the agent may hit the same dead ends repeatedly. Adapting the strategy during a live search avoids that rigidity, but each test costs compute, and trying countless alternatives means repeating long, expensive runs.

The new method reuses data from a completed search to test alternative strategies within the space the agent has already explored. The agent records its attempts and their results as it searches, producing a search tree that can be replayed later.

How “dreaming” works

The researchers compare the idea to finding your way through an unfamiliar area. On a first visit, you hit dead ends and double back. Once you have a mental map, you can plan another route without visiting every spot again.

Dream-RSI applies that principle to recorded search histories. Rather than testing a new strategy in a live run, the agent runs it against stored results. It can check what would have happened if it had pursued other approaches first or abandoned some earlier. Because all results are already stored, thousands of alternative strategies can be tested without calling the model or evaluator again.

The researchers call this process dreaming. The agent plays through thousands of variations and picks the best one before putting it to work in a live search. After each search, the agent uses the recorded results to test better strategies, then applies the improved version to its next live run. Throughout the cycle, only the search strategy changes. The model generating the solutions stays untouched.

How much compute Dream-RSI saves

The team tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across three areas. Each comparison used a baseline with the same starting conditions but a fixed search strategy.

One task asked the system to write the fastest possible program for a statistical calculation commonly used in genomics and finance. Dream-RSI’s program ran faster than the established libraries sklearn and glmnet on all six test datasets. With Gemini 3.1 Pro, average runtime fell from 3,587 to 2,931 milliseconds, while the number of attempts dropped from 550 to 317.

Dream-RSI also outperformed a competing system called SimpleTES, which needed 51,200 runs compared with Dream-RSI’s 317 attempts on the same task.

The same pattern held for math optimization tasks and for efforts to write efficient GPU kernels, with comparable or better results at much lower computational cost. On two GPU tasks, Dream-RSI matched performance while cutting the number of generations by a factor of up to 2.43. On two others, it delivered up to 2.09 times the performance within the same budget.

Why the learned strategy adjusts over time

The analysis showed how the learned strategy shifted its effort as performance improved. It first reduced the number of attempts. When progress stalled, it increased the search effort again, which coincided with further gains. In other words, the agent learns to spend less when things are going well and to spend more when it needs to break out of a rut.

Explicit instructions can limit exploration

In a follow-up analysis, the researchers tested another way to use search histories. Instead of replaying them to test strategies, they condensed them into instructions telling the agent where to search. On one GPU task, the version with these instructions performed worse than the version without them. The researchers suggest that overly specific directions can narrow the search space too much, keeping the agent from exploring a broader range of approaches.

That finding matters for related work. Google Research recently presented a different way to reuse past runs with WikiSkill, a system that records failures and successes in a wiki and turns them into reusable instructions. Dream-RSI’s follow-up analysis suggests that explicit instructions like these can restrict exploration on open-ended search tasks.

Where Dream-RSI fits in a broader trend

Recursive self-improvement has drawn growing attention. Google DeepMind introduced AlphaEvolve in 2025, using the same basic principle. Gemini Flash generates code proposals, Gemini Pro analyzes them, and an evolutionary algorithm selects the best versions. Dream-RSI works one level above that process by optimizing the search strategy itself. AutoTTS takes a related approach, using a coding agent to search for algorithms in a simulated environment; these algorithms decide when a language model should start, expand, or abandon reasoning paths. Meta’s Hyperagents go further, allowing agents to rewrite the mechanism that controls how they improve.

The researchers have shared code and more details on GitHub.

FAQ

What is Dream-RSI?

Dream-RSI is a method from Google and DeepMind that lets AI agents test new search strategies against stored results from past searches instead of running them live. It changes how the agent searches, not the underlying AI model.

How much compute does Dream-RSI save?

On a task that asked the system to write the fastest possible program for a common statistical calculation, Dream-RSI cut average runtime with Gemini 3.1 Pro from 3,587 to 2,931 milliseconds and reduced attempts from 550 to 317. It reached the same outcome as a competing system called SimpleTES, which needed 51,200 runs, using only 317 attempts. On GPU tasks, it matched performance with up to 2.43 times fewer generations or reached up to 2.09 times the performance within the same budget.

Why can explicit instructions hurt performance?

In a follow-up test, the researchers condensed search histories into instructions telling the agent where to search. On one GPU task, that version performed worse than Dream-RSI without instructions, suggesting that overly specific directions can narrow the search space and keep the agent from exploring a broader range of approaches.