QS Primer: Spaced Repetition and Learning

Many people think the Quantified Self mostly involves physical metrics: heart rate, sleep, diet, etc. but what about what goes on in our brains? Can we quantify that? There have been several inspiring Quantified Self talks about tracking learning and memory. This post will collect all them into one place, along with good resources for further exploration.

Memorization is only a small part of learning, but it in many circumstances it is unavoidable. There is an ideal moment to practice what you want to memorize. Practice too soon and you waste your time. Practice too late and you’ve forgotten the material and have to relearn it. The right time to practice is just at the moment you’re about to forget. If you are using a computer to practice, a spaced repetition program can predict when you are likely to forget an item, and schedule it on the right day.

In this graph, you can see how successive reminders change the shape of the forgetting curve, a pattern in our mental life that was first discovered by one of the great modern self-trackers, Hermann Ebbinghaus. With each well-timed practice, you extend the time before your next practice. Spaced repetition software tracks your practice history, and schedules each review at the right time.

Graph Illustrated Effect of Spaced Practice

Convenient tools to take advantage of fast memorization techniques have been around since Piotr Wozniak began developing his Supermemo software in the early 1980s. (I wrote a profile of Wozniak for Wired in 2008, which is cited in some of these talks.) Many of us in the Quantified Self use spaced repetition. We’ve put together this page to list resources, share experiences, and invite comments and questions. We hope you find it useful. If you do, please contribute some knowledge or questions to the comments.

Below you’ll find the following sections:

  • Learning Chinese with Spaced Repetition
  • Winning Jeopardy with Spaced Repetition
  • Using Spaced Repetition to Build General Knowledge and Curiosity
  • Using Spaced Repetition as a Cognitive Assay
  • The Value and Limits of “Well-Formulated Knowledge” (i.e. good flashcards)
  • List of Spaced Repetition Software Tools
  • Academic References on Spaced Repetition

Learning Chinese With Spaced Repetition

Spaced repetition is frequently used to quickly gain vocabulary in a foreign language. In this talk from a 2012 Bay Area QS Show&Tell, Jeremy Howard describes his experience learning Chinese.

Winning Jeopardy With Spaced Repetition

Roger Craig holds the all time record for single day winnings in Jeopardy. In this video, he describes his training method, which included the spaced repetition software Anki.

Using Spaced Repetition to Build General Knowledge and Curiosity

At the Portland Quantified Self Show&Tell, Steven Jonas presented this fascinating talk about using spaced repetition to build general knowledge and stimulate curiosity. He concludes hat the value of memorization is not in the mere knowledge on the cards, but in the establishment of a mental structure upon which new thoughts and knowledge can develop.

Using Spaced Repetition as a Cognitive Assay

Nick Winter is the author of a popular Chinese and Japanese learning program called Skritter, which includes spaced repetition as a key feature. Here are the slides from a Quantified Self Show&Tell talk by Nick in which he discusses how spaced repetition practice can be used as a reliable cognitive assay; that is, a test of how well the brain is functioning. Because spaced repetition practice tends to be of consistent difficulty, patterns in our performance can offer clues about what makes us more or less able to learn. (Nick has some results from these experiments that he promised to share in comments.)


The Value and Limits of “Well-Formulated Knowledge” (i.e. good flashcards)

This is a bit of an advanced topic, but I mention here to stimulate interest and comments; do add your thoughts if you have something interesting on this. Piotr Wozniak has a long section on his web site called Effective learning: Twenty rules of formulating knowledge. There are many tricks here. But note the first rule: Do not learn if you do not understand. He offers the rather hilarious example that it is possible, using Supermemo, to memorize a textbook on German history in German, even if you know no German. Possible, yes, but also stupid and useless. Spaced repetition helps directly with one facet of learning only: memorization. Learning involves other things too. Steven Jonas talk about the value of “trivial” information demonstrates; we’re still discovering different ways to make spaced repetition valuable in the larger context of learning. We’re interested in your experiences.

Animated Simulation of Learning with Spaced Repetition

Andrew Marrero created a simulation of the effect of memorizing using spaced repetition vs. other kinds of flashcard practice. It is a great visual explanation of the power of this technique.

[Found via the spaced repetition resource page at gwern.net]

Spaced Repetition Software Tools in Common Use

This is a shorter list than you will see in the Wikipedia entry on spaced repetition. That’s because it includes only tools that we know to be in use by people who have mentioned it to us. If you are using something actively, and are willing to share your experience, please let us know.

  • Anki – Widely used and in active development. Open source. Desktop and Android versions are free. iPhone version is $24.99
  • Mnemosyne – Also open source; runs on Windows/Mac OS, with card review possible on Android and Blackberry mobiles. All versions are free.
  • Skritter – Chinese and Japanese only. Looks great and is written by a good QS friend (see Nick Winter’s talk, above), but I don’t learn these languages so I await your reports. Subscription: $9.95/month.
  • SuperMemo – The original; still widely used. There are some licensed versions on other platforms, but the main line of development is Windows only. ($60)

Academic References on Spaced Repetition

A good place to begin is with Piotr Wozniak’s General principles of SuperMemo, which has many links to deeper explanations of his software, experience, and research. Here are some additional sources. (You are welcome to contribute, too. Add references in comments for now, and I’ll go back and incorporate them into this list.) Most of these are readable without specialized knowledge.

  • Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108(3), 296.
  • Bahrick, H. P. (1984). Semantic memory content in permastore: fifty years of memory for Spanish learned in school. Journal of Experimental Psychology: General, 113(1), 1.
  • Bahrick, H. P. (1985). Associationism and the Ebbinghaus legacy. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11(3), 439.
  • Bahrick, H. P., & Phelphs, E. (1987). Retention of Spanish vocabulary over 8 years. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(2), 344.
  • Bjork, R. A. (2001). Recency and Recovery in Human Memory. In The nature of remembering: Essays in honor of Robert G. Crowder. American Psychological Association. doi:10.1037/10394-011
  • Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus fluctuation. From Learning Processes to Cognitive processes: Essays in honor of William K. Estes, 2, 35–67.
  • Bjork, R. A., & Bjork, E. L. (2006). Optimizing treatment and instruction: Implications of a new theory of disuse. Memory and society: Psychological perspectives, 116–140.
  • Bjork, R., & Richardson-Klavehn, A. (2002). Memory, Long-term. In Encyclopedia of cognitive science (Vol. 2, pp. 1096–1105). London: Nature Publishing.
  • Dudukovic, N. M., & Wagner, A. D. (2006). Attending to Remember and Remembering to Attend. Neuron, 49(6), 784–787. doi:10.1016/j.neuron.2006.03.008
  • Grote, M. G. (1995). The Effect of Massed Versus Spaced Practice on Retention and Problem-Solving in High School Physics. Ohio Journal of Science, 95(3), 243–247.
  • Landauer, T., & Bjork, R. (1978). Optimal Rehearsal Patterns and Name Learning. In M. Brunebert, P. Morris, & R. Sykes (Eds.), Practical Aspects of Memory (pp. 625–632). London: Academic Press.
  • Mozer, M. C., Howe, M., & Pashler, H. (2004). Using testing to enhance learning: A comparison of two hypotheses. Proceedings of the 26th Annual Conference of the Cognitive Science Society, 975–980.
  • Pyle, W. (1913). Economical learning. Journal of Educational Psychology, 4(3), 148.
  • Traxel, W. (1985). Hermann Ebbinghaus: In memoriam. History of Psychology Newsletter, 17, 37–41.
  • Wagner, A. D., & Davachi, L. (2001). Cognitive neuroscience: forgetting of things past. Current Biology, 11(23), R964–R967.
This entry was posted in QS Resource and tagged , , . Bookmark the permalink.

23 Responses to QS Primer: Spaced Repetition and Learning

    • Gary Wolf says:

      Gwern: Thank you for that excellent reference page, which I hadn’t seen before. I’ve added link to main post.

  1. Amy Robinson says:

    Thanks for posting this – great resource. If you ever want contributors, I run the corporate blog for healthsterling.com and am our creative director as well as an avid QSer who is very into the cognitive side of health — would be happy to sput my mind on your blog!
    a

  2. Pingback: QS Primer: Spaced Repetition and Learning | Quantified Self « Education and Technology for Future

  3. Fantastic summary of the spaced repetition/flashcard method! I’ve found flashcards massively helpful for me in learning Mandarin. Even better, I’m increasingly finding them useful for learning pronunciation and listening (e.g. using Anki’s multimedia flashcards, mentioned above). Do you know if there is any material on how to best use spaced repetition for learning non-written material? For example, with the “cards” that I listen to, I’m not sure if being able to repeat the sound file would help my learning or not? It is analogous to reading a written card again, but seems something like cheating when it comes to understanding spoken language…

  4. Devon says:

    I love Spaced Repetition. It’s not only useful for students but also great for everybody. I use this method for my lifelong study.

  5. John Jensen Ph.D. says:

    Delighted to find you all. Rowman and Littlefield, educational publishers, are issuing my 3-volume Practice Makes Permanent series. A significant aspect of how to turn classrooms around quickly is connecting an objective point of score to every point of knowledge mastered, making effort constantly rewarded and motivating. I believe I’ve also found the most efficient way to use spaced repetition that I call Time Capsule–moving a capsule of knowledge forward in time the fastest way possible. Many other uses of quantified effort are also easy to employ that stimulate students. I’m glad to email any or all of the three manuscripts to anyone interested. The first title, out now in print, is Teaching Students to Work Harder and Enjoy It: Practice Makes Permanent. Contact me directly at jjensen@gci.net. Best regards, John Jensen

  6. Pingback: Weekend Reading: After the 4th Edition - ProfHacker - The Chronicle of Higher Education

  7. Billy says:

    You need to add Memrise (www.memrise.com) to your list of spaced repitition. It’s better than Anki and Supermemo for learning languages.

  8. Nick Winter says:

    I’ll add that my experiments with using spaced repetition learning data as a measure of cognition didn’t ultimately pan out. I expected some correlation between my SRS performance and my subjective energy/focus ratings, but there was no correlation. Measuring cognitive performance is a hard problem.

    • gwern says:

      Interesting, I was thinking of using my Mnemosyne statistics as another measure too, although I didn’t yet know whether to go with ‘average grade of cards reviewed that day X’ (memory retrieval) or something more sophisticated like ‘average grade on next review day of all cards reviewed that day X’ (memory encoding?).

      • Nick Winter says:

        I tried both the grade-based analysis and combining it with the time it took to answer, and the thinking time. Actually, I ran some models through WEKA combining those things with the estimated toughness of each card, too. But nothing came up significant.

        I only did this analysis after I realized I had all this data, and my subjective measure of energy is not very reliable, so it’s possible that another, better measure (like Quantified Mind test scores) would correlate with SRS performance.

  9. Marukun says:

    I just wanted to share with readers of this post that after failing my German “O Level” (in the UK school system at the time) with a U grade (standing for Unclassified and well below an F Grade for straightforwardly failing the exam) and then forever after having a complex about language learning I was recently able to pass the most advanced Grade 1 of the Japanese Language Proficiency Test – 日本語能力試験 – at my first attempt. This is the only Japanese exam I have ever taken and a significant contribution to my passing was using Anki spaced repetition. I am now learning Mandarin.

    Don’t give up! Learning French or German would be a breeze now with spaced repetition and a little study of grammar and linguistics.

  10. Pingback: Show is better than Tell | make a powerful point

  11. Pingback: Thesis Journal #6 | Memrise, Bloggen und mein erstes virtuelles Community-High-Five | vermessen leben

  12. Pingback: Thesis Journal #6 | Memrise: Vokabel-Harikiri fürs Spaced Repetition and Learning-Programm | vermessen leben

  13. Pingback: » Spaced Repetition: How to Learn Anything and Never Forget it j2jenkins

  14. Pingback: Memorizing a programming language using spaced repetition software - OBRA MAESTRA | OBRA MAESTRA

  15. Luca Rannucci from Italy says:

    Dear all,

    I do have a question for those who actually tried to learn something using a spaced repetition pattern without the use of a software – in other words people who haven’t use a software which calculates by itself how much time you should wait after the next repetition.

    The point is, I study law and as you might imagine, I cannot broke the subject I’m studying into chunks as small as a word, which I can then put into a software.

    So I have to calculate by myself (or maybe I can use the pattern used by someone else) how much time should pass between the study and the n-repetion of (let’s say) a page/chapter/paragraph/.

    In practical terms? How should I plan the work? I.e. how much time between studying and first/second/third repetition of the fragment I studied?

    Do you have any link which could help me in planning the study of a university module applying spaced repetition?

    Thanks a lot

    Luca

  16. Matt says:

    I am also a law student, and while I do use Anki, I have also developed some Applescript routines to integrate spaced repetition into VoodooPad, EagleFiler, Taskpaper, Popclip and OmniOutliner. I’ve posted most of these to my site. Check lawschoolmatt.com to see any are useful. The OmniOutliner script comes pretty close, I think, to matching the algorithm used in Anki. The advantage is you can keep using your subject outline, see things in context, and quiz yourself using the row text as the question and the note text as the answer.

  17. Pingback: Innovative Trends in der Hochschulbildung: Horizon-Report 2014 | innovation.virtuelle-ph.at

  18. Pingback: LPT: Better, faster and easier memorization through Spaced Repetition. | Awesome Facts

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.