Title

Cognitive maps in brains and AI: Memory and planning at multiple scales

Bio

Abstract

Memory and planning are interconnected, they both rely on learning the relational structures of experience. A century after Tolman’s ‘latent learning’ experiments, the larger puzzle of cognitive maps remains elusive: how does the brain learn compact representations of relational structures to guide flexible behavior? How can this research contribute to building and evaluating AI? I will first show how I use reinforcement learning (RL), behavioral experiments, and neural measurements to study how humans learn predictive representations in memory and planning. The results suggest that multi-scale predictive representations in hippocampal and prefrontal hierarchies, learned and updated via online experience and offline replay, underlie memory and planning. This approach advances the century old notion of cognitive maps and can inform biologically inspired artificial agents as well as computational psychiatry. In the second part, I will show how this framework empowers evaluating and building generative AI and large language models. I will show recent work on the evaluation of cognitive maps and planning in eight LLMs, analysis of embedding layers in LLMs using neuroscience methods, and a multi-agent GPT-4 based planning architecture inspired by the human prefrontal cortex.