Empty Shelves or Lost Keys? Google Research Explains Why AI Models "Forget" Facts They've Learned

Researchers at Google Research have published an analysis that sheds light on one of the most common and frustrating phenomena associated with large language models: factual hallucinations. The study offers a simple yet remarkably useful metaphor to explain why these systems get things wrong when asked about concrete facts.

The metaphor of empty shelves and lost keys

According to Google Research, the problem of parametric factuality — that is, how a model stores and accesses information learned during training — can be understood through two distinct scenarios. The first is the "empty shelves" scenario: the information simply doesn't exist in the model's memory, because it wasn't encountered often enough or clearly enough in the training data. The second scenario, "lost keys," is more subtle: the information was learned correctly and does exist somewhere within the model's parameters, but the retrieval mechanism fails to locate and use it at the right moment.

This distinction isn't merely theoretical. It has direct implications for how developers should approach solutions aimed at reducing factual errors. If the problem stems from empty shelves, the logical fix would be to improve and expand the training data. But if it's a matter of lost keys, the focus should instead be on optimizing the internal mechanisms for retrieving information — not necessarily on adding new data.

Why this distinction matters

The main conclusion of the research, according to Google Research, is that recall — the process of retrieving already-stored information — is often the primary bottleneck for factual accuracy, rather than an actual gap in knowledge. In other words, many of the errors we observe in AI models don't stem from ignorance, but from a breakdown in the "remembering" process itself.

This perspective significantly shifts the direction research efforts should take to reduce hallucinations. Rather than focusing exclusively on the volume and quality of training data, development teams might achieve better results by investing in architectures and techniques that improve models' ability to correctly "recall" what they have already learned.

Implications for the future of generative AI

The Google Research study contributes to a more nuanced understanding of how the internal memory of language models works, offering a useful conceptual framework for researchers and engineers working to reduce factual errors in generative AI systems.