Tag Archives: seth roberts
My friend Seth Roberts, pioneering self-experimenter and personal scientist, died last Saturday. Seth’s sister Amy, made the announcement yesterday on his blog. The news was unexpected and very sad. A few things Seth taught me:
- Doing lots of experiments keeps you supplied with new ideas.
- With sensitive and reliable measurements, tiny experimental effects can yield surprising clues.
- The people who care about a problem most have the best chance of solving it, if they have access to tools.
Seth’s contributions as a colleague and teacher had many dimensions, but in thinking about him nonstop this morning what I find myself marveling at most is the unusual style he had in nearly every conversation. Seth became interested when he saw somebody thinking independently and, like the best teachers, he wanted to understand the process by which students and collaborators developed confidence their conjectures. Countless times, I heard Seth ask somebody “Why do you think that?” His challenge was direct and generous, for if you were willing to expose your reasons you could count on him to apply himself alongside you, thinking up ways to improve your investigation, make your measurements more practical, or give your analysis more logical or mathematical power. Seth was acutely aware that confidence in experimental results requires investment. More than any other experimentalist I’ve known, Seth respected the incremental nature of building confidence in our ideas and finding new ideas worthy of confidence. He encouraged us to do small experiments first, to find easier ways to collect data, and to avoid being tripped up at the outset by grandiose schemes of irrefutability.
We’ll collect and share some more impressions and memories of our friend and collaborator when the shock of this news lessens, but in the meantime I want to post a video of one of the talks he gave a few years ago about how to design personal experiments.
For further reading:
- Seth Roberts Wikipedia page
- Seth’s faculty page at Tsinghua University (bibliography is in English)
- A spot-on remembrance by Richard Sprague
In Richard Sprague’s post from today, linked directly above, there is this passage:
I found his blog, and discovered that he was living near me in Beijing. A famous professor like him – a New York Times bestselling author and all that – might be hard to get ahold of, but one day out of the blue I sent Seth an email, wondering if he’d like to get together for lunch. He replied in minutes and said sure, how about tomorrow?
That’s Seth exactly.
I have blogged many times about biohacker Tara Grant’s discovery that she slept much better if she took Vitamin D3 in the morning rather than later. Many people reported similar experiences, with a few exceptions. Lots of professional research has studied Vitamin D3 but the researchers appear to have no idea of this effect. They don’t control the time of day that subjects take D3 and don’t measure sleep. If the time of day of Vitamin D3 makes a big difference, measuring Vitamin D3 status via blood levels makes no sense. Quite likely other benefits of Vitamin D3 require taking it at the right time of day. Taking Vitamin D3 at a bad time of day could easily produce the same blood level as taking it at a good time of day.
I too had no idea of the effect that Grant discovered. I had taken Vitamin D3 several times — never in the morning — but after noticing no change stopped. I tested Grant’s discovery by taking Vitamin D3 at 8 or 9 am. First, taking it at 8 am, I gradually increased the dose from 2000 IU to 8000 IU. Then I shifted the time to 9 am. The experiment ended earlier than I would have liked because I had to fly to San Francisco.
When I woke up in the morning I rated how rested I felt on a 0-100 scale, where 0 = not rested at all and 100 = completely rested. I’d been using this scale for years. Here are the results (means and standard errors):
Vitamin D3 had a clear effect, but the necessary dose was more than 2000 IU. If Vitamin D3 acts like sunlight, you might think that taking it in the morning would make me wake up earlier. Here are the results for the time I woke up:
There was no clear effect of dosage on when I got up. Shifting the time from 8 am to 9 am may have had an effect (I wish I had 3 more days at 9 am).
Many people have reported that taking Vitamin D3 in the morning gave them more energy during the day. I usually take a nap in the early afternoon so I measured its effect on the length of those naps:
Maybe my naps were shorter with 6000 and 8000 IU at 8 am. It’s interesting that 4000 IU seemed to be enough to improve how rested how I felt but not enough to shorten my naps.
What do these results add to what we already know? First, the large-enough dose was more than 2000 IU. (A $22 million study of Vitamin D3 is using a dose of 2000 IU.) The dose needed to get more afternoon energy may be more than 4000 IU. Second, careful experimentation and records helped, even though many people found the effect so large it was easy to notice without doing anything special. For example, these results suggest the minimum dose you need to get the effect. Three, these support the value of supplements. Many people say it is better to get necessary nutrients from food rather than supplements. However, supplements allow much better control of dosage and timing and these results suggest that small changes in both can matter. I cannot imagine this effect being discovered with Vitamin D3 in food.
Note from Alex: Gwern Branwen also sent in this detailed post on Vitamin D and sleep.
I measure my arithmetic speed (how fast I do simple arithmetic problems, such as 3+ 4) daily. I assume it reflects overall brain function. I assume something that improves brain function will make me faster at arithmetic.
Two years ago I discovered that butter — more precisely, substitution of butter for pork fat — made me faster. This raised the question: how much is best? For a long time I ate 60 g of butter (= 4 tablespoons = half a stick) per day. Was that optimal? I couldn’t easily eat more but I could easily eat less.
To find out, I did an experiment. At first I continued my usual intake (60 g /day). Then I ate 30 g/day for several days. Finally I returned to 60 g/day. Here are the main results:
The graph shows that when I switched to 30 g/day, I became slower. When I resumed 60 g/day, I became faster. Comparing the 30 g/day results with the combination of earlier and later 60 g/day results, t = 6, p = 0.000001.
The change in error rates raised the possibility that the speed changes were due to movement along a speed-accuracy tradeoff function (rather than to genuine improvement, which would correspond to a shift in the function). To assess this idea, I plotted speed versus accuracy (each point a different day).
If differences between conditions were due to differences in speed-accuracy tradeoff, then the points for different days should lie along a single downward-sloping line. They don’t. They don’t lie along a single line. Within conditions, there was no sign of a speed-accuracy tradeoff (the fitted lines do not slope downward). If this is confusing, look at the points with accuracy values in the middle. Even when equated for accuracy, there are differences between the 30 g/day phase and the 60 g/day phases.
What did I learn?
1. How much butter is best. Before these results, I had no reason to think 60 g/day was better than 30 g/day. Now I do.
2. Speed of change. Environmental changes may take months or years to have their full effect. Something that makes your bones stronger may take months or years to be fully effective. Here, however, changes in butter intake seemed to have their full effect within a day. I noticed the same speed of change with pork fat and sleep: How much pork fat I ate during a single day affected my sleep that night (and only that night). With omega-3, the changes were somewhat slower. A day without it made little difference. You can go weeks without Vitamin C before you get scurvy. Because of the speed of the butter change, in the future I can do better balanced experiments that change conditions more often.
3. Better experimental design. An experiment that compares 60 g/day and 0 g/day probably varies many things besides butter consumption (e.g., preparing the butter to eat it). An experiment that compares 60 g/day and 30 g/day is less confounded. When I ate less butter, I ate more of other food. Compared to a 60 g/0 g experiment, this experiment (60 g/30 g) has less variation in other food. Another sort of experiment, neither better nor worse, would vary type of fat rather than amount. For example, replace 30 g of butter with 30 g of olive oil. Because the effect of eliminating 30 g/day of butter was clear, replacement experiments become more interesting — 30 g/day olive oil is more plausible as a sustainable and healthy amount than 60 g/day.
4. Generality. This experiment used cheaper butter and took place in a different context than the original discovery. I discovered the effect of butter using Straus Family Creamery butter. “One of the top premium butters in America, ” says its website, quoting Food & Wine magazine This experiment used a cheaper less-lauded butter (Land O’Lakes). Likewise, I discovered the effect in Berkeley. I did this experiment in Beijing. My Beijing life differs in a thousand ways from my Berkeley life.
The results suggest the value of self-experimentation, of course. Self-experimentation made this study much easier. But other things also mattered.
First, reaction-time methodology. In the 1960s my friend and co-author Saul Sternberg, a professor of psychology at the University of Pennsylvania, introduced better-designed reaction-time experiments to study cognition. They turned out to be far more sensitive than the usual methods, which involved measuring percent correct. (Saul’s methodological advice about these experiments.)
Second, personal science (science done to help yourself). I benefited from the results. Normal science is part of a job. The self-experimentation described in books was mostly (or entirely) done as part of a job. Before I collected this data, I put considerable work into these measurements. I discovered the effect of butter in an unusual way (measuring myself day after day), I tried a variety of tasks (I started by measuring balance), I refined the data analysis, and so on. Because I benefited personally, this was easy.
Third, technological advances. Twenty years ago this experiment would have been more difficult. I collected this data outside of a lab using cheap equipment (a Thinkpad laptop running Windows XP). I collected and analyzed the data with R (free). A smart high school student could do what I did.
There is more to learn. The outlier in the speed data (one day was unusually fast) means there can be considerable improvement for a reason I don’t understand.
In 1996, I accidentally discovered that if I stood a lot I slept better. If I stood 9 hours or more, I woke up feeling incredibly rested. Yet to get any improvement I had to stand at least 8 hours. That wasn’t easy, and after about 9 hours of standing my feet would start to hurt. I stopped standing that much. It was fascinating but not practical.
In 2008, I accidentally discovered that one-legged standing could produce the same effect. If I stood on one leg “to exhaustion” — until it hurt too much to continue — a few times, I woke up feeling more rested, just as had happened when I stood eight hours or more. At first I stood with my leg straight but after a while my legs got so strong it took too long. When I started standing on one bent leg, I could get exhausted in a reasonable length of time (say, 8 minutes), even after many days of doing it.
This was practical. I’ve been doing it ever since I discovered it. A few months ago I decided to try to learn more about the details. I was doing it every day — why not vary what I did and learn more?
One thing I wanted to learn was: how much was best? I would usually do two (one left leg, one right leg) or four (two left leg, two right leg). Was four better than two? What about three?
I decided to do something relatively sophisticated (for me): a randomized experiment. Every morning I would do two stands (one left, one right). In the evening I would randomly choose between zero, one, and two additional one-legged stands. Sometimes I forgot to choose. Here are the results for three sets of days: (a) “baseline” days (baseline(2), baseline(3), baseline(4)) before the randomized experiment and during the experiment when I forgot and (b) the “random” days (random 2, random 3, random 4) when I randomly choose and (c) a later set of days (”baseline 4″) when I did four one-legged stands every day.
Each morning, when I woke up I rated how rested I felt on a scale where 0 = not rested at all (as tired as when I went to sleep), and 100 = completely rested, not tired at all.
The main results are that three was better than two and four was better than three. The three/four difference was large enough compared to the two/four difference to suggest that five might be better than four. The similarity between random 4 and baseline 4 means that the amount of one-legged standing on previous days doesn’t matter much. For example, on Monday night it doesn’t matter how much I stood on Sunday.
A month ago I posted this graph, which shows how long I needed to type the answer to simple arithmetic problems (7-5, 4*1, 9+0). I tested myself with about 40 problems once or twice per day. Because I’d been doing this for a long time, I no longer improved due to practice. Then, at the end of July 2010, I started improving again.
In September I moved from Berkeley to Beijing. I was worried that in Beijing my scores would get worse. Perhaps I couldn’t get good flaxseed oil or butter. Maybe I would suffer from the air pollution. Maybe I would eat contaminated food. But my scores got better in Beijing.
In November I learned about benefits of cold showers. So I tried them. I took cold showers that lasted about 5 minutes. I liked the most obvious effect (less sensitivity to cold).
Maybe a bigger “dose” would produce a bigger effect. Maybe the mood improvement cold showers were said to cause would be clearer. So I increased the “dose” in two ways: (a) more water flow (I stopped holes in the shower head) and (b) lower water temperature. After a week or so with the stronger dose, I saw I was gaining weight. It could be the cold showers, I thought. Fat acts as insulation and I couldn’t think of another plausible explanation. So I went from cold showers back to warm showers (48 degrees C.) — this time with greater water flow. My warm showers were 5-10 minutes long.
I began to lose weight, suggesting that the cold water did cause weight gain. More surprising was that my arithmetic speed (time to do simple arithmetic, such as 7-3, 8*4) began to decrease. Here is a graph of the results.
Update: 10/19/10 – Study is now open to users at http://genomera.com/studies/butter-mind
Will eating one of these fats improve your math performance? Based on Seth Roberts’ butter and math
study, recently presented at a Bay Area Quantified
Self Show & Tell, during which Seth ate half a stick of butter each day and performed better in math, we expect the answer to be yes.
Seth was able to reduce his time by 30 milliseconds. Will others who try a similar experiment experience the same change?
In the Butter Mind study, to be run from October 23 – November 12, I will test the hypothesis that butter improves math performance. (note: there has been a slight shift in the dates.)
This study is meant to
mimic Seth Robert’s study, with the addition of a coconut oil group.
Many thanks to Seth for his advice and help getting this started!
Why the addition coconut oil? I have a pet theory that the cognitive enhancement Seth received may be from the high concentration of Medium Chain Triglycerides in butter, also present in coconut oil, which has been linked to positive effects on those with Alzheimer’s Syndrome. Seth has not tried coconut oil, so cannot report on its effects on his math scores.
Obviously, no study is perfect – and this one is no exception! It’s a test I was interested in trying myself after seeing Seth’s presentation — but I realized it would be far more fun and interesting to include others! This will be fun for me, and I hope for you, too. At the very least, will get data from a group over a 21-day period, but we may even get a few curious surprises.
I am currently looking for Butter Mind participants, who will perform a math test daily for 21 days and be in one of the following groups: butter eaters, coconut oil eaters, and controls, who will eat no additional fat but will perform the same math test as the fat-eaters.
To qualify for the study, you must be willing to eat 4TBS of butter or coconut oil (sticking to the same one) – or nothing extra – for 7 days and do a 32 problem simple math test for 21 days. You must have access to the internet to submit your scores.
will be randomly selected to be in the Butter, Coconut Oil, or Control group
- Participants will be responsible for purchasing butter or coconut oil, if in either of those groups
- The study
will take place for 21 days: from Oct 23 – Nov 12
study will be divided into 3 sets of 7 days
o Part I. Oct 23 – 29: Perform simple
math quiz daily + No additional
o Part II. Oct 30 – Nov 5: “Fat.” Perform simple
math quiz daily + Butter OR
Coconut Oil. For Controls, just the quiz.
o Part III. Nov 6 – 12: Perform simple
math quiz daily + No additional
- Non-control participants
will ingest 4 Tablespoons of either
Butter or Coconut Oil during the “Fat” portion of the study
will be asked to share lifestyle information before the study and asked to join an online group to track their data. Extra sharing (thoughts, epiphanies) is encouraged but optional.
will statistically analyzed, hacked and visualized (and new studies brainstormed) during Science
Hack Day, November 13-14, Institute
for the Future. www.sciencehackday.com. You can join for the Science Hack Day portion only by registering here.
results will be posted to the QS blog throughout the study
- Interested participants will receive a form requesting data on lifestyle factors several days before the study begins. I will update this post with a link to the form when it is ready.
For more information
or to join, send an email with “Butter Mind” in the subject line to:
Eri Gentry: firstname.lastname@example.org
Eri is co-founder of BioCurious, Citizen Science guest author at the Make mag blog, and is happiest when she gets to be a guinea pig. Eri hasn’t eaten butter in 8 years but will try it (or anything) for a better mind.
At the last Bay Area Quantified Self Show&Tell, Seth Roberts presented new findings on his “Arithmetic and Butter” experiment. Seth does arithmetic problems every morning as a measure of his brain function. He found that eating half a stick of butter every day shaved 30 milliseconds off his time to solve the problems. Does butter improve brain function, or is Seth endangering his life, as a cardiologist in the audience worried? Catch the excitement in the video below.
It all started when Seth Roberts wondered why his self-experiments were so effective.
Over 12 years, this psychology professor and regular at Quantified Self meetups tested new ideas, observed himself, and learned a number of significant things – new theories about mood, weight loss, sleep-wake cycles, and daily habits necessary for optimal living.
Seth’s exploration of why his self-experiments work so well is now in press for the journal Medical Hypotheses. You can read the full text here, and here’s an abstract:
Over 12 years, my self-experimentation found new and useful ways to improve sleep, mood, health, and weight. Why did it work so well? First, my position was unusual. I had the subject-matter knowledge of an insider, the freedom of an outsider, and the motivation of a person with the problem. I didn’t need to publish regularly. I didn’t want to display status via my research. Second, I used a powerful tool. Self-experimentation about the brain can test ideas much more easily (by a factor of about 500,000) than conventional research about other parts of the body. When you gather data, you sample from a power-law-like distribution of progress. Most data helps a little; a tiny fraction of data helps a lot. My subject-matter knowledge and methodological skills (e.g., in data analysis) improved the distribution from which I sampled (i.e., increased the average amount of progress per sample). Self-experimentation allowed me to sample from it much more often than conventional research. Another reason my self-experimentation was unusually effective is that, unlike professional science, it resembled the exploration of our ancestors, including foragers, hobbyists, and artisans.
Here’s to the power of self-experiments, and self-experimenters!