Echo: Constant-Memory Associative Recall Without the KV Cache
This presentation explores Echo, a breakthrough architecture that eliminates the memory cliff problem in state space models while avoiding the exploding memory costs of transformer KV caches. By introducing Spectral Koopman Attention, Echo achieves perfect associative recall with constant memory overhead, maintaining robust performance even as context grows to 64 times the training length. We examine the core mechanism, empirical results on synthetic and language modeling benchmarks, and the implications for scalable long-context processing.Script
Transformers store every past token in a growing KV cache, while state space models compress everything into fixed memory but forget distant facts. Echo solves both problems at once.
State space models like Mamba suffer from a memory cliff: their exponential decay means retrieval accuracy collapses as the gap between a stored fact and the query grows. This isn't a capacity problem, it's structural, baked into the multiplicative recurrence itself.
Spectral Koopman Attention reframes retrieval as kernel ridge regression. Instead of storing tokens, it accumulates three fixed-size covariance matrices: key Gram, lag-one key covariance, and value-key covariance. These sufficient statistics never grow with sequence length.
The secret is a power spectral filter. By applying K iterations of the whitened Koopman operator, Echo amplifies modes with eigenvalues near one and suppresses transient noise. Persistent content survives; distractors decay.
On the MQAR benchmark, pure Mamba collapses to 3 percent accuracy. Hybrid attention helps, but only Echo achieves 100 percent exact recall across every configuration, including 4,096 distractors and gaps 64 times longer than training length.
Echo proves that constant-memory, high-fidelity retrieval is possible without the KV cache. This opens the door to long-context agentic systems and multi-turn reasoning on standard hardware. Explore the full architecture and results at EmergentMind.com, where you can create your own research videos.
