Roger Craig Wins Jeopardy Championship with Knowledge Tracking

Roger Craig had a dream of being on Jeopardy, and he took a QS approach to making it happen. He downloaded all historical Jeopardy questions and answers into a database, clustering the questions by topic and keyword, and built a web tool around it to quiz himself. He visualized his answers to see where his knowledge gaps were and help him optimize his learning. Roger actually got to test his system by playing some actual rounds of Jeopardy, with surprising results! UPDATE: This week, Roger won the Jeopardy! Tournament of Champions and became the show’s 4th highest winner in history. Congratulations, Roger! If you haven’t seen his video below, check it out. (Filmed at the NY Quantified Self Show&Tell #13 at NYU ITP.)

This entry was posted in Videos and tagged , , , , , . Bookmark the permalink.

29 Responses to Roger Craig Wins Jeopardy Championship with Knowledge Tracking

  1. gwern says:

    Did I hear him correctly that he is reviewing his flashcards *randomly*?

    I can’t believe no one has told him about spaced repetition! What a waste.

  2. Pingback: Roger Craig on Knowledge Tracking | Quantified Self | AbundantKnowledge.info™

  3. Roger Craig says:

    @gwern , I’m not sure which sentence you’re referring to.
    I did randomly sample the corpus for diagnostic purposes, but reviewing of material is almost always in a spaced repetition framework.

    I have used spaced repetition software for many many years. e.g. Here’s a mention of my use of Anki in Stephen Baker’s book on the Watson matches, Final Jeopardy:
    http://books.google.com/books?id=CwLSrDUoVB0C&lpg=PA214&dq=craig%20anki&pg=PA214#v=onepage&q=craig%20anki&f=false

  4. Jake says:

    Very impressive. It makes my attempts to do something similar look like child’s play.

    So Roger, do you have any plans to open your tracker/website up to others?

  5. Roger Craig says:

    @Jake , Yes. In some format/way, but not right now. The main reason for holding back for now is that the vast majority of the questions are the intellectual property of Jeopardy!, not myself. I do plan on taking this project further though, stay tuned!

    • Matt says:

      Congratulations Roger. I am a college student and currently use a website called quizlet.com to study. You would be doing myself and millions of other students a great service if you would let us upload our own questions to your site to study. I would love to beta test it for you

  6. How did he manage to download old Jeopardy questions?! I want that too.

  7. Aaron says:

    I can think of several graduate level test prep sites that would heavily benefit from your method – namely the USMLE Step 1 (Medical Licensing Exam Step 1). Have you considered going that route with your program?

    • Stephen says:

      I’d really hope that medical licensing can’t benefit from cherry picking questions. Jeopardy’s great because it can’t humiliate too many of its viewers by asking difficult questions and just goes for easyish ones from a broad range of categories. I think medical licensing really should ask difficult questions over a relatively narrow area.

      • Kyle says:

        To some extent it takes more than just knowing random facts to do well on the USMLE as you must be able to work through medical problems, and the vast majority of students taking these exams have this base of knowledge to comprehend the questions on the USMLE. This will pretty much guarantee a student will pass the USMLE tests were originally designed for. Over the years, however, the test has become a measuring stick for residencies (job training) to judge medical students on. While a basic understanding of medical problems is necessary to pass the test, these questions are really one or two step problem based on “facts”. Knowing high yield facts can and does greatly improve ones score on the USMLE. This is shown but essentially every student using a book called “First AID” which is made up entirely of facts. As someone who has taken these tests and done very well, to score high on these exams you just study these facts. Someone who cannot work through medical problems will fail the test, but for the rest knowing the highest yield facts will separate them from the pack. You should learn how to understand and work through medical problems through medical school but when you get to the boards it really does come down to simple pieces of knowledge. I am not saying I agree how they test medical students but it is what it is. Scoring well on these exams largely determines what specialties you can go into. Students have an idea of many high yield facts but a program like this would be incredibly helpful for the USMLE as topics consistently come up and are repeated every year.

        • Kyle says:

          “This will pretty much guarantee a student will pass the USMLE tests were originally designed for.”
          sorry made changes and didnt reread, since I cant edit here is my edit for clarification:
          A basic understanding of medical problem solving should guarantee a student will pass the USMLE with a minimum amount of studying and originally the USMLE was designed to do just that- make sure students had a basic understanding of medical problem solving. Over the years…

  8. Robert LeRoy Parker says:

    What an incredible victory in the ToC! The back to back daily doubles in day one of the final was the greatest moment I’ve seen on Jeopardy. Legendary stuff.

  9. Maxim says:

    Roger,

    Is there any way to adapt this to studying for university students? Also, is there anywhere I can find more information on Knowledge Tracking, or “learning how to learn”?

    Thanks!

  10. Carl says:

    I have 4 Boys from ages 17 to 4.. To have a program such as yours that they could
    add their own data base from the texts in school would be a huge help..
    Please think along those lines also..
    Thanks

  11. gwern says:

    To the admins: reposting this article broke the old link http://quantifiedself.com/2011/09/roger-craig-on-knowledge-tracking/ (404), which breaks all links to the original one, including to my personal knowledge the link on Wikipedia, on my site, and LessWrong. Could you put in a redirect or something?

  12. wood says:

    hi roger,
    good job on the show! i watched it.
    just want to ask if it is ok, how did you parse the jarchive and put it on a database? were there any tools that you use to do this easily?

    thanks!

  13. Pingback: Le self tracking ou comment devenir le 4ème plus grand vainqueur de Jeopardy | Self Tracking France

  14. Pingback: Physics, Philosophy, and Everything in Between

  15. Pingback: 6 Counterintuitive Tricks to Beating Popular Game Shows | Chib

  16. MedStudent says:

    Hey Roger,

    I’ve been studying for the USMLE all year and I feel that my knowledge has plateaued, but I still only get 85% on every test. The 15% is not always in the same subjects. I was wondering if there is something I can do to to increase my score. I know Step1 is not a game, but I thought that maybe it also has a weakness.
    Please let me know if you have any ideas on maximizing my knowledge on high held info.

    Thanks in advance
    MedStudent

  17. Pingback: 6 Counter-Intuitive Ways to Beat Popular Game Shows « NeuroBolt

  18. Pingback: I’m Back! « Dru's Projects

  19. Jacob Bailey says:

    Roger. Please. Send me your program. I need it. I have been thinking about doing this for like 4 years and I’m 17. I do not possess the computer smarts to make the database as you did. Has anyone else made a similar program?

  20. Pingback: Quora

  21. Pingback: Self-Tracking and the Quantified Self Movement | Greatist

  22. Pingback: New Quantified Life Approach | Jeb Bradwell

  23. Pingback: The Best Strategies for Winning Jeopardy | Mediaite

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.