Roger Craig had a dream of being on Jeopardy, and he took a QS approach to making it happen. He downloaded all historical Jeopardy questions and answers into a database, clustering the questions by topic and keyword, and built a web tool around it to quiz himself. He visualized his answers to see where his knowledge gaps were and help him optimize his learning. Roger actually got to test his system by playing some actual rounds of Jeopardy, with surprising results! UPDATE: This week, Roger won the Jeopardy! Tournament of Champions and became the show’s 4th highest winner in history. Congratulations, Roger! If you haven’t seen his video below, check it out. (Filmed at the NY Quantified Self Show&Tell #13 at NYU ITP.)
What We're Reading
QS Radio Podcast
QS Access App
Make a Sparktweet
▁▃▇█▆█▆▄▃▂▃█▅
Map Your Moves Data
Download Your Fitbit Data
QS Meetup Groups
Partners
Sponsors
Friends of QS
We recently started a program to invite QS Toolmakers to contribute directly to funding our events. We call this program Friends of QS. If you would like to participate we invite you email us to learn more.
Did I hear him correctly that he is reviewing his flashcards *randomly*?
I can’t believe no one has told him about spaced repetition! What a waste.
Pingback: Roger Craig on Knowledge Tracking | Quantified Self | AbundantKnowledge.info™
@gwern , I’m not sure which sentence you’re referring to.
I did randomly sample the corpus for diagnostic purposes, but reviewing of material is almost always in a spaced repetition framework.
I have used spaced repetition software for many many years. e.g. Here’s a mention of my use of Anki in Stephen Baker’s book on the Watson matches, Final Jeopardy:
http://books.google.com/books?id=CwLSrDUoVB0C&lpg=PA214&dq=craig%20anki&pg=PA214#v=onepage&q=craig%20anki&f=false
Very impressive. It makes my attempts to do something similar look like child’s play.
So Roger, do you have any plans to open your tracker/website up to others?
@Jake , Yes. In some format/way, but not right now. The main reason for holding back for now is that the vast majority of the questions are the intellectual property of Jeopardy!, not myself. I do plan on taking this project further though, stay tuned!
Congratulations Roger. I am a college student and currently use a website called quizlet.com to study. You would be doing myself and millions of other students a great service if you would let us upload our own questions to your site to study. I would love to beta test it for you
How did he manage to download old Jeopardy questions?! I want that too.
@Peter, they are all online at J! Archive. Roger just downloaded the entire site and then created his database.
Did you watch even the first two minutes of the video? He names the website: The Jeopardy Archive (first Google result: http://www.j-archive.com/)
I can think of several graduate level test prep sites that would heavily benefit from your method – namely the USMLE Step 1 (Medical Licensing Exam Step 1). Have you considered going that route with your program?
I’d really hope that medical licensing can’t benefit from cherry picking questions. Jeopardy’s great because it can’t humiliate too many of its viewers by asking difficult questions and just goes for easyish ones from a broad range of categories. I think medical licensing really should ask difficult questions over a relatively narrow area.
To some extent it takes more than just knowing random facts to do well on the USMLE as you must be able to work through medical problems, and the vast majority of students taking these exams have this base of knowledge to comprehend the questions on the USMLE. This will pretty much guarantee a student will pass the USMLE tests were originally designed for. Over the years, however, the test has become a measuring stick for residencies (job training) to judge medical students on. While a basic understanding of medical problems is necessary to pass the test, these questions are really one or two step problem based on “facts”. Knowing high yield facts can and does greatly improve ones score on the USMLE. This is shown but essentially every student using a book called “First AID” which is made up entirely of facts. As someone who has taken these tests and done very well, to score high on these exams you just study these facts. Someone who cannot work through medical problems will fail the test, but for the rest knowing the highest yield facts will separate them from the pack. You should learn how to understand and work through medical problems through medical school but when you get to the boards it really does come down to simple pieces of knowledge. I am not saying I agree how they test medical students but it is what it is. Scoring well on these exams largely determines what specialties you can go into. Students have an idea of many high yield facts but a program like this would be incredibly helpful for the USMLE as topics consistently come up and are repeated every year.
“This will pretty much guarantee a student will pass the USMLE tests were originally designed for.”
sorry made changes and didnt reread, since I cant edit here is my edit for clarification:
A basic understanding of medical problem solving should guarantee a student will pass the USMLE with a minimum amount of studying and originally the USMLE was designed to do just that- make sure students had a basic understanding of medical problem solving. Over the years…
What an incredible victory in the ToC! The back to back daily doubles in day one of the final was the greatest moment I’ve seen on Jeopardy. Legendary stuff.
Roger,
Is there any way to adapt this to studying for university students? Also, is there anywhere I can find more information on Knowledge Tracking, or “learning how to learn”?
Thanks!
I have 4 Boys from ages 17 to 4.. To have a program such as yours that they could
add their own data base from the texts in school would be a huge help..
Please think along those lines also..
Thanks
To the admins: reposting this article broke the old link http://quantifiedself.com/2011/09/roger-craig-on-knowledge-tracking/ (404), which breaks all links to the original one, including to my personal knowledge the link on Wikipedia, on my site, and LessWrong. Could you put in a redirect or something?
hi roger,
good job on the show! i watched it.
just want to ask if it is ok, how did you parse the jarchive and put it on a database? were there any tools that you use to do this easily?
thanks!
Pingback: Le self tracking ou comment devenir le 4ème plus grand vainqueur de Jeopardy | Self Tracking France
Pingback: Physics, Philosophy, and Everything in Between
Pingback: 6 Counterintuitive Tricks to Beating Popular Game Shows | Chib
Hey Roger,
I’ve been studying for the USMLE all year and I feel that my knowledge has plateaued, but I still only get 85% on every test. The 15% is not always in the same subjects. I was wondering if there is something I can do to to increase my score. I know Step1 is not a game, but I thought that maybe it also has a weakness.
Please let me know if you have any ideas on maximizing my knowledge on high held info.
Thanks in advance
MedStudent
Pingback: 6 Counter-Intuitive Ways to Beat Popular Game Shows « NeuroBolt
Pingback: I’m Back! « Dru's Projects
Roger. Please. Send me your program. I need it. I have been thinking about doing this for like 4 years and I’m 17. I do not possess the computer smarts to make the database as you did. Has anyone else made a similar program?
Pingback: Quora
Pingback: Self-Tracking and the Quantified Self Movement | Greatist
Pingback: New Quantified Life Approach | Jeb Bradwell
Pingback: The Best Strategies for Winning Jeopardy | Mediaite