Tag Archives: advisory board
Thomas, from Germany, wrote in recently with a question for our QS advisory board:
do you have an idea how to track wisdom? i would like to start a project involving “mental models” as promoted by charles munger. but how do i know that this is working? how can someone track thequality of his decisions and understanding of the world?.
I would start by writing down each day a few of my decisions. Perhaps just one decision per day. So I would have a slowly growing list of decisions. Then I would later come back and rate each one. I’d need to develop a rating scheme; it might have more than one dimension. For example, one dimension might be importance, another might be time frame, a third might be how expected the outcome was. In other words, I would start with the simplest easiest project that might shed some light on how to do it. It’s really important to make the first steps as easy as possible. But it’s also important to make it something you do every day, so that you acquire a habit. So you start by acquiring a tiny amount of data each day..
Do you want to be more creative? Justin Wehr does, and he sent in this question for the QS advisory board.
Name: Justin Wehr
Purpose: My objective is to measure creative thoughts so I can figure out how to
Variables tracked: I have some variables related to this, but not enough. For example,
I have a notebook I carry around with me so I can write something down
every time I think of/hear something interesting that I want to
remember. So I think I could crudely measure creative thoughts with
something like “number of lines written in notebook per unit time”.
However, I do not track time spent reading or which lines written in
the notebook occurred during reading. But my hope is that I could set
up an experiment that would not be too complicated — that is the
advice I would like from the QS advisors.
How do I set up an experiment to determine when I have have the most
creative thoughts … when I am reading, when I am thinking, or when I do
intermittent periods of reading and thinking? What should I measure,
how should I measure it, is it practical, and how do I analyze the
See what Gary Wolf of Quantified Self and Gary King of Harvard had to say…
If you make a change to your daily routine or try a new medication, how do you know if it is working?
And if you have a question about your self-tracking for our advisors, let me know.
Bard’s Symptom Tracking Experiment:
My purpose is to correlate whether taking a particular medication helps to alleviate specific symptoms.
Medications and symptoms are tracked in a Google doc, http://spreadsheets.google.
My question is, imagine 2 columns, A representing whether medication was taken (0 or 1) and B measuring some relevant symptom (value x). The median being 99 in column B next to any 1′s in column A, and 100 in column B next to any 0′s in column A. Standard deviation 1 (or any SD really, I just wanted to find a formula that could still find a high correlation with such a small deviation, which shouldn’t be hard considering the huge amount of data that I have). This simulates a pill that works, on average, to decrease the symptoms by 1 point. Not a huge change, but extremely consistent, so worth identifying.
I would like a formula that returns the “strength” of the correlation, which in this example is approx. 100%, given a large enough data set. Any help would be greatly appreciated.
Neil Rubens’ Answer:
If I understood your questions correctly you may consider using the following two approaches to analyze your data.
1. There are many different ways of measuring dependence between variables (besides correlation); this wiki link on “dependence measurement” should provide a good place to start.
2. You can use a “statistical hypothesis test” to establish whether the difference in treatments are statistically significant (even if this difference is very small) — unlikely to have occurred by chance.
I hope this at least partially answers your questions.
David Goldberg and Teresa Lunt’s answer:
I’m not sure correlation is the best way to think about this, since one of the variables (the A column in your notation) takes on only two values, either 0 or 1.
It might make more sense to consider the two sets of symptom values, S1 for subjects who didn’t take the medication, and S2 for for those who did. Then you can use the tests developed for comparing two sets of numbers. Here are three common tests.
1. The t-test. Using the free ‘R’ statistical package on this data (where x=S1, y=S2) gives:
Welch Two Sample t-test
data: x and y
t = 4.105, df = 797.703, p-value = 4.459e-05
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
mean of x mean of y
This not only says there is a statistically significant difference (p
= .00004), but tells you that with 95% confidence, the difference in
means between the two sets is between 0.5 and 1.4. In other words,
the symptom value in the control (no medication) group is likely to be
at least 0.5 more than in the experimental group.
2. Another possibility is the Wilcoxon rank-sum test. If you think the
symptom values are nowhere near having a Gaussian distribution, then
this would be more apppropriate. For your data (again using ‘R’)
Wilcoxon rank sum test with continuity correction
data: x and y
W = 93025.5, p-value = 6.297e-05
alternative hypothesis: true location shift is not equal to 0
Again the test shows the two sets are unequal, since p is so small (p
= 0.00006). However, you don’t get the confidence interval for the
difference of means.
3. If the data aren’t Gaussian and you want the confidence interval for
the difference in means, consider using the bootstrap.
> fn = function()
+ mean(sample(x, length(x), replace=TRUE)) – mean(sample(y, length(y), replace=TRUE))
This gives a similar 95% confidence interval as the t-test: (.47, 1.4) vs (.49, 1.4)
Thanks to Bard for the question and to Neil, David and Teresa for their answers! Brilliant and experimenting QS readers, please send in your questions and we’ll do our best to find answers for you.
Jeremy Johnson sent in this question for the illustrious QS Scientific Advisory Board, so we set about finding an answer for him. Gordon Bell and Seth Roberts responded with lightning speed! Jeremy’s question and their answers are below. If you have a question about your self-tracking that you’d like some help with, let me know.
My purpose is to track multiple variables related to sleep, exercise, diet and supplements to make evidence-based decisions
to increase my energy level.
Variables are tracked in a Google doc, http://docs.google.com/Doc?
My question is, what existing tool (iPhone or web) will minimize the burden of data collection so this is sustainable?
Gordon Bell’s Answer:
A couple of devices will focus on energy expended:
- BodyBugg captures energy expended using several measuring transducers;
- a plain old pedometer like the one from Oregon Scientific gives steps taken that is a stab at energy expended and it goes into your computer with no fuss or muss – all it requires is logging on to a site to be fed monthly.
Energy input (aka diet) is probably the most important and hardest to deal with. There are several packages like FitWatch that allow you to count calories. I did this for a few weeks to get the hang of calorie costs. A good kitchen scale is important.
Weight is important and change is just the difference of energy in – energy out. This says you really know your body. If you are overweight, reducing weight is clearly the easiest place to get energy!
Drugs: caffeine, vitamins, alcohol, etc. are all inputs I don’t understand or want to comment on. Less is probably more though.
Zeo looks interesting for sleep. A friend uses it. I haven’t bothered to try it because I am generally in a state of: What, me worry?
Stress. The BodyBugg actually tracks this through their skin resistivity sensor, but you can’t get hold of it. The company BodyMedia that sells them the device might make it available. This would tell how much stress one is under each day. A diary will have to suffice for now.
Seth Roberts’ Answer:
Jeremy, I think you are starting too big. You are trying to record too much. If I were you I would start by trying to record one thing day after day. It would be something I wanted to improve — maybe sleep or energy level.
After I’d managed to record one thing daily for several weeks then I would start doing little experiments. I would take one thing I can vary — say, how much coffee I drink. I start tracking it — measuring it each day. After several weeks, I would intentionally change it — say, drink less coffee — and see what happens for several weeks.
That’s three steps.
Step 1: Measure one thing you want to improve. Do that for several weeks.
Step 2: In addition, measure one thing you can easily control (e.g., coffee consumption, exercise) that might affect what you want to improve. Do that for several weeks.
Step 3: Change that thing you can easily control. See what effect that change has.
In other words, try to do the smallest easiest thing that will put you closer to your goal. That tiny little goal will be turn out to be much harder to reach than you imagine. In the beginning, the smallest easiest thing is to measure one variable day after day.
Thanks to Jeremy for the question and to Gordon and Seth for their answers! Wonderful QS readers, please send in your questions and we’ll do our best to find answers for you.
Do you need help with your self-tracking data analysis? Is there a specific problem or burning question about your experiment design that you’d love some guidance on? Gary and I are proposing an idea to help – read on for details!
We’ve gathered an amazing Quantified Self Scientific Advisory Board to be part of our community. It’s a star group of international scientists involved in data analysis, data visualization, and self-experimentation. In alphabetical order, they are:
- Alex Bangs, Human Predictive Biosimulation, Entelos
- Gordon Bell, MyLifeBits, author of Total Recall, Microsoft Research
- Jeff Heer, Collaborative Data Visualization and Flare/Prefuse, Stanford
- Gary King, Quantitative Social Science and n=1 experiments, Harvard
- Teresa Lunt, Director of Computer Science Lab, PARC
- Seth Roberts, Self-Experimentation guru, author of Shangri-La Diet, Berkeley and Beijing
- Neil Rubens, Data Mining, University of Electro-Communications, Tokyo
The experiment we’d like to do is to encourage Quantified Self members to formulate questions about the personal data that they are trying to work with. Post them as comments or send them to me. We will make sure the questions are interesting and at least partially answerable, pass them along to the appropriate Advisor, and publish the questions and responses here on the Quantified Self blog, as a way to get discussion going and add value to everyone involved.
So let us know what you think, and start asking questions!