Spaced Repetition for Knowledge Creation

Thanks to Henrik Karlsson, Paul Raymond Robichaud, Alex Giorev and Moritz Wallawitsch for their feedback on drafts of this essay. Find me on Twitter and YouTube.

Spaced Repetition Systems (SRS) are mostly seen as study-aids to help students cram for their exams or to learn a foreign language, but I believe they also have unexplored potential to accelerate knowledge creation. 

They are generally overlooked by mathematicians, philosophers, and scientists who see flashcard reviews as mindless rote learning rather than as a process that can help them to develop new mathematics, philosophical theories and ground-breaking research. The creativity-enhancing potential of these systems is hidden behind some simple misconceptions.

SRSs allow us to build up and maintain a huge body of knowledge about existing theories. By maintaining this knowledge in memory, they make us more creative and even allow us to make progress solving problems while we sleep! And by increasing the retrievability of the components of our theories, it makes it more likely that we are able to recognise experiences which contradict them.

Eliminate Forgetting

As humans we know that we will always face an endless stream of difficult problems, but since we don't know in advance exactly what those problems will be, we can't say for sure which concepts and explanations will be helpful to us in solving them. Our best hope is to create and internalize as many highly applicable abstract concepts and explanations as possible to maximize our ability to deal with unforeseen problems.

SRSs are well-optimized for this daunting task. As we learn new concepts and relationships, we can create flashcards to encode them as memories. By calculating optimal review dates and exponentially increasing the review intervals, a single flashcard may be repeated as little as 6-20 times across your entire lifetime, meaning that you may be able to remember as many as 250-300 thousand flashcards by the time you retire.

By reviewing the information we learn at expanding intervals, we make sure we don't forget it, eliminating the churn effect caused by forgetting. Without SRS, our knowledge grows asymptotically towards a saturation level as the rate of forgetting old knowledge balances the rate of learning new knowledge. 

SRS linearizes the acquisition of knowledge.¹

This allows us to learn and retain a huge number of meaningful concepts and relationships. New knowledge slots into your prior knowledge making learning effortless and enjoyable. And if you take a break from working on a particular problem or learning about some topic and come back to it 6 months later, you won't find that the knowledge you built up has evaporated (as long as you keep up with your reviews!).

It's true that SRSs are often used for rote learning because there's nothing stopping users from adding flashcards containing content that they aren't able to meaningfully associate with what they already know, forcing them to rely on cramming or memorizing word-for-word to maintain a shallow, superficial memory without deeper conceptual understanding. However, this is not an issue with spaced repetition itself but rather the user's poor selection of learning material. It's totally possible to create flashcards from scientific, philosophical, or mathematical material in a way where new concepts are integrated into prior knowledge by understanding the semantic relationships between concepts. These memories gain stability through their connections to other long-term memories, requiring fewer repetitions to maintain, making the learning process effortless and enjoyable.

But the value of an SRS goes deeper than just raising the natural limit on how much we can know. Flashcards and the memories they encode are not isolated chunks but composable Lego bricks which can be glued together using rules of inference inside the universe's ultimate connection machine - the human brain.

Enhance Creativity

Until AGI arrives, the creation of new explanatory knowledge is a process that must take place within the human mind. It's easy to forget this fundamental point. For instance, lots of people argue that memorization is pointless in the digital age and that we can rely on our "second brains" - Google and note-taking apps - to store our memories for us. But while there is no chance that two ideas in a digital notebook or on Wikipedia could spontaneously combine into a new theory or idea, this is the foundational ability of the brain which has powered all of human creativity and innovation.

Each additional concept added to your memory creates another pathway for the brain's creative search algorithm to explore. As we grow our knowledge, we create a combinatorial explosion in the number of possible ways we can combine bits of what we already know to create new knowledge and solve problems.

Of course, it's true that not everything needs to be actively memorized; some things can just be Googled in the moment or left in a note-taking tool, but relying entirely on Google or Wikipedia to store knowledge relevant to the problems you want to solve starves your brain of the resources it needs to create new knowledge.

Remember that creativity has both conscious and unconscious components. There is the small proportion of the time when you are consciously trying to solve a problem, and the large proportion of the time when you are either asleep or not actively directing your attention towards the problem. Even when you aren’t consciously problem-solving, the brain's creative search algorithm is still churning as a background process, searching for novel associations. Sleep also plays a huge role in creative thought since it's the time when the memories you built during the day undergo neural optimization and consolidation. How often do we go to bed with a problem to solve and wake up with the solution for free? When you stop seeing creativity and problem-solving as purely conscious processes, you realize the importance of memory to take advantage of all of that additional processing time and maximize the chance of creating new ideas.

Improve Theories

The final ideas I would like to leave you with relate to memory consistency and noticing contradictions. Since observation is theory-laden, we cannot ensure that we will actually notice findings that are incompatible with our theories. I think this is partially a memory problem that can be alleviated through using an SRS.

Without using a memorization strategy, it would be very difficult to maintain every branch of a complex theory or problem at a high level of retrievability and recognise when the universe presents you with evidence that something you know is wrong, especially when problem-solving or theory-building proceeds over particularly long time frames.

Thankfully spaced repetition systems can take care of this for you for very little cost. Depending on the algorithm used, spaced repetition systems can be calibrated to ensure a 95%² probability of recall across hundreds of thousands of flashcards. This ensures that we make the most of the continual experimental testing going on in the mind as we compare our experiences to what we know and improves the odds of correcting errors in our theories.

1. I received a great comment on this point from Alex Giorev: “Maybe the volume of knowledge at some point will hit a plateau, but that doesn't mean that the explanatory power of our knowledge will hit a plateau. More powerful ideas could be learned and less powerful ones forgotten. I think it was Woz (Piotr Wozniak) who said that ‘knowledge is power by its applicability, not volume.’”

2. The default forgetting index in SuperMemo is 10%. This means that the probability of forgetting at repetition time is 10%. Since the repetition time is, at least in theory, calculated to be the point at which you are most likely to forget, the average recall across your entire collection of flashcards will be 94.91%. See this article on the forgetting index for more.