Google researchers have unveiled the Dream-RSI approach, a cutting-edge method that refines strategies by enabling AI to engage in a form of ‘dreaming’ within its historical exploration data. Specifically, Dream-RSI converts past data into a simulated setting and constructs a ‘discovery tree’ to mimic the implementation of novel strategies. This innovative process significantly cuts down on computational expenses while boosting the efficiency of exploration. The method unfolds in three distinct phases: initially, a discovery tree is generated based on genuine exploration data; subsequently, large language models are harnessed to devise and filter new strategies; and ultimately, the refined strategies are deployed into real-world environments to broaden the scope of simulation. Experimental outcomes reveal that, in comparison to baseline methods like fixed strategies, Dream-RSI exhibits marked superiority in terms of call frequency, time utilization, and overall performance, though it shows a slight dip in performance compared to other methods in select tasks. Furthermore, the research uncovered that explicitly feeding historical ‘lessons learned’ into the system can, paradoxically, lead to a downturn in performance. The related paper, project webpage, and code repository are now accessible to the public.