Memory, Space, and Planning: Multiscale Predictive Representations
(2401.09491)Abstract
Memory is inherently entangled with prediction and planning. Flexible behavior in biological and artificial agents depends on the interplay of learning from the past and predicting the future in ever-changing environments. This chapter reviews computational, behavioral, and neural evidence suggesting these processes rely on learning the relational structure of experiences, known as cognitive maps, and draws two key takeaways. First, that these memory structures are organized as multiscale, compact predictive representations in hippocampal and prefrontal cortex, or PFC, hierarchies. Second, we argue that such predictive memory structures are crucial to the complementary functions of the hippocampus and PFC, both for enabling a recall of detailed and coherent past episodes as well as generalizing experiences at varying scales for efficient prediction and planning. These insights advance our understanding of memory and planning mechanisms in the brain and hold significant implications for advancing artificial intelligence systems.
Overview
-
The paper explores the role of memory in prediction and planning, showing how cognitive maps in the brain aid in these processes.
-
Findings reveal that cognitive maps are predictive, multiscale representations within the hippocampus and prefrontal cortex.
-
These memory structures support adaptive behavior by encoding memories at scales that match task complexity.
-
Successor representation (SR) in reinforcement learning provides a computational framework understanding of memory organization.
-
Neuroscientific studies demonstrate that multiscale predictive representations are active during navigation and planning tasks, and are updated during rest.
Overview of Multiscale Predictive Representations
The Integral Role of Memory in Planning
In an effort to understand how biological and artificial systems utilize memory for prediction and planning, research explore the concept of cognitive maps—spatial mental models aiding in navigation and problem-solving. Findings suggest that cognitive maps manifest as multiscale, predictive representations within the hippocampus and prefrontal cortex. This arrangement allows agents to recall past experiences vividly and generalize across different situations for effective forecasting and strategizing.
Cognitive Maps and Predictive Learning
Cognitive maps enable agents to navigate and infer relationships within their environment, drawing on the analogy of spatial navigation. These cognitive constructs are not merely reflections of past interactions but are structured to predict future events. The evidence suggests that memories are encoded at varying scales within the brain, corresponding to the complexity and abstraction level of tasks at hand. This multifaceted representation is critical for adaptive behavior, allowing for choices that range from detailed to broadly conceptual based on the context.
Computational Perspectives on Memory Organization
The discussion transitions into how these memory structures, particularly cognitive maps, can be understood through computational frameworks like reinforcement learning (RL). One method, the successor representation (SR), is notable for capturing multi-step, predictive representations, unlike traditional RL models that learn associations step by step. SR emphasizes a predictive aspect, where each memory activation cues the potential of future states. This aligns with recent behavioral and neural studies, positing that such representations could underlie complex memory tasks and decision-making processes.
Memory in Action: Behavioral and Neural Corroboration
Neuroscience research corroborates this theory. For instance, studies show multiscale predictive representations in hippocampal and prefrontal hierarchies during tasks that mimic real-life conditions like navigating a virtual city. Such evidence highlights the brain's ability to use these predictive memory structures to influence goal-directed behavior. Additionally, investigations into offline replay during rest times tie into the brain's process of updating plans based on past experiences, further supporting the theory. The successor representation potentially unites spatial navigation with episodic recall, suggesting a universal principle for memory organization across biological and artificial cognizers.
Create an account to read this summary for free: