Tag Archives: advisory board

How to Track Wisdom

Thomas, from Germany, wrote in recently with a question for our QS advisory board:

do you have an idea how to track wisdom? i would like to start a project involving “mental models” as promoted by charles munger. but how do i know that this is working? how can someone track the
quality of his decisions and understanding of the world?
QS advisor Seth Roberts rose to the challenge of answering Thomas’ question:
I would start by writing down each day a few of my decisions. Perhaps just one decision per day. So I would have a slowly growing list of decisions. Then I would later come back and rate each one. I’d need to develop a rating scheme; it might have more than one dimension. For example, one dimension might be importance, another might be time frame, a third might be how expected the outcome was. In other words, I would start with the simplest easiest project that might shed some light on how to do it. It’s really important to make the first steps as easy as possible. But it’s also important to make it something you do every day, so that you acquire a habit. So you start by acquiring a tiny amount of data each day.
If you have a burning tracking-related question, write to us and we’ll try to get it answered for you! Also, if you have an alternative answer for Thomas on how to track wisdom, please leave a comment below.
Posted in Discussions | Tagged , , | 17 Comments

How To Measure and Maximize Creative Thoughts

Do you want to be more creative? Justin Wehr does, and he sent in this question for the QS advisory board.


Name: Justin Wehrjustinwehr.jpg

Purpose: My objective is to measure creative thoughts so I can figure out how to
maximize them.

Variables tracked: I have some variables related to this, but not enough. For example,
I have a notebook I carry around with me so I can write something down
every time I think of/hear something interesting that I want to
remember. So I think I could crudely measure creative thoughts with
something like “number of lines written in notebook per unit time”.
However, I do not track time spent reading or which lines written in
the notebook occurred during reading. But my hope is that I could set
up an experiment that would not be too complicated — that is the
advice I would like from the QS advisors.

How do I set up an experiment to determine when I have have the most
creative thoughts … when I am reading, when I am thinking, or when I do
intermittent periods of reading and thinking? What should I measure,
how should I measure it, is it practical, and how do I analyze the

See what Gary Wolf of Quantified Self and Gary King of Harvard had to say…

Continue reading

Posted in Discussions | Tagged , , , , , | 6 Comments

How To Measure Small Effects in Your Data

If you make a change to your daily routine or try a new medication, how do you know if it is working?

This was the question Bard sent in for the QS Scientific Advisory Board. His challenge was met by Neil Rubens, Teresa Lunt and David Goldberg. Read Bard’s question and their answers below.

And if you have a question about your self-tracking for our advisors, let me know.


Bard’s Symptom Tracking Experiment:

My purpose is to correlate whether taking a particular medication helps to alleviate specific symptoms.

Medications and symptoms are tracked in a Google doc, http://spreadsheets.google.com/pub?key=tTFplEr7avEhkaxOrnxqEEA&single=true&gid=0&output=html

My question is, imagine 2 columns, A representing whether medication was taken (0 or 1) and B measuring some relevant symptom (value x). The median being 99 in column B next to any 1′s in column A, and 100 in column B next to any 0′s in column A. Standard deviation 1 (or any SD really, I just wanted to find a formula that could still find a high correlation with such a small deviation, which shouldn’t be hard considering the huge amount of data that I have). This simulates a pill that works, on average, to decrease the symptoms by 1 point. Not a huge change, but extremely consistent, so worth identifying.

I would like a formula that returns the “strength” of the correlation, which in this example is approx. 100%, given a large enough data set. Any help would be greatly appreciated.


Neil Rubens’ Answer:

neilrubens.jpgHi Bard,

If I understood your questions correctly you may consider using the following two approaches to analyze your data.

1. There are many different ways of measuring dependence between variables (besides correlation); this wiki link on “dependence measurement” should provide a good place to start.

2. You can use a “statistical hypothesis test” to establish whether the difference in treatments are statistically significant (even if this difference is very small) — unlikely to have occurred by chance.

I hope this at least partially answers your questions.



David Goldberg and Teresa Lunt’s answer:

davidgoldberg.jpgHi Bard,

I’m not sure correlation is the best way to think about this, since one of the variables (the A column in your notation) takes on only two values, either 0 or 1.

It might make more sense to consider the two sets of symptom values, S1 for subjects who didn’t take the medication, and S2 for for those who did.  Then you can use the tests developed for comparing two sets of numbers.  Here are three common tests.

1. The t-test.  Using the free ‘R’ statistical package on this data (where x=S1, y=S2) gives:

> t.test(x,y)

Welch Two Sample t-test

data:  x and y

t = 4.105, df = 797.703, p-value = 4.459e-05

alternative hypothesis: true difference in means is not equal to 0

95 percent confidence interval:

0.4905078 1.3894922

sample estimates:

mean of x mean of y

99.795    98.855

This not only says there is a statistically significant difference (p

= .00004), but tells you that with 95% confidence, the difference in

means between the two sets is between 0.5 and 1.4.  In other words,

the symptom value in the control (no medication) group is likely to be

at least 0.5 more than in the experimental group.

teresalunt.jpg2. Another possibility is the Wilcoxon rank-sum test.  If you think the

symptom values are nowhere near having a Gaussian distribution, then

this would be more apppropriate.  For your data (again using ‘R’)

> wilcox.test(x,y)

Wilcoxon rank sum test with continuity correction

data:  x and y

W = 93025.5, p-value = 6.297e-05

alternative hypothesis: true location shift is not equal to 0

Again the test shows the two sets are unequal, since p is so small (p

= 0.00006).  However, you don’t get the confidence interval for the

difference of means.

3. If the data aren’t Gaussian and you want the confidence interval for

the difference in means, consider using the bootstrap.

> fn = function()

+   mean(sample(x, length(x), replace=TRUE)) – mean(sample(y, length(y), replace=TRUE))

> replicates=replicate(1000,fn()

)> quantile(replicates, c(0.025, 0.975))

2.5%     97.5%

0.4674375 1.3500000

This gives a similar 95% confidence interval as the t-test:  (.47, 1.4) vs (.49, 1.4)

Palo Alto Research Center


Thanks to Bard for the question and to Neil, David and Teresa for their answers! Brilliant and experimenting QS readers, please send in your questions and we’ll do our best to find answers for you.

Posted in Discussions | Tagged , , , , | 3 Comments

What Tools Should I Use To Make Tracking Easier?

Jeremy Johnson sent in this question for the illustrious QS Scientific Advisory Board, so we set about finding an answer for him. Gordon Bell and Seth Roberts responded with lightning speed! Jeremy’s question and their answers are below. If you have a question about your self-tracking that you’d like some help with, let me know.


Jeremy’s Energy Experiment:jeremy.jpg

My purpose is to track multiple variables related to sleep, exercise, diet and supplements to make evidence-based decisions
to increase my energy level.

Variables are tracked in a Google doc, http://docs.google.com/Doc?docid=0AcCCHmQcRfM7ZGRmbjZkd3dfOTdmanZ3N3Jqbg&hl=en&invite=CL2n6L0B

My question is, what existing tool (iPhone or web) will minimize the burden of data collection so this is sustainable?


Gordon Bell’s Answer:

GordonBell.jpgJeremy might start by tracking (aka recording) exercise, diet, sleep, work, his overall mood/stress level, and compare with how he feels about his energy level.
A couple of devices will focus on energy expended:
- BodyBugg captures energy expended using several measuring transducers;
- a plain old pedometer like the one from Oregon Scientific gives steps taken that is a stab at energy expended and it goes into your computer with no fuss or muss – all it requires is logging on to a site to be fed monthly.
Energy input (aka diet) is probably the most important and hardest to deal with. There are several packages like FitWatch that allow you to count calories. I did this for a few weeks to get the hang of calorie costs.  A good kitchen scale is important.
Weight is important and change is just the difference of energy in – energy out. This says you really know your body. If you are overweight, reducing weight is clearly the easiest place to get energy!
Drugs: caffeine, vitamins, alcohol, etc. are all inputs I don’t understand or want to comment on. Less is probably more though.
Zeo looks interesting for sleep. A friend uses it. I haven’t bothered to try it because I am generally in a state of: What, me worry?
Stress.  The BodyBugg actually tracks this through their skin resistivity sensor, but you can’t get hold of it. The company BodyMedia that sells them the device might make it available.  This would tell how much stress one is under each day.  A diary will have to suffice for now.


Seth Roberts’ Answer:

sethroberts.jpgI use R to collect data. R is free open-source software. I can use it without internet access. On the other hand I cannot carry it around with me. If R is too difficult, I might use Google Docs.

Jeremy, I think you are starting too big. You are trying to record too much. If I were you I would start by trying to record one thing day after day. It would be something I wanted to improve — maybe sleep or energy level.

After I’d managed to record one thing daily for several weeks then I would start doing little experiments. I would take one thing I can vary — say, how much coffee I drink. I start tracking it — measuring it each day. After several weeks, I would intentionally change it — say, drink less coffee — and see what happens for several weeks.

That’s three steps.
Step 1: Measure one thing you want to improve. Do that for several weeks.

Step 2: In addition, measure one thing you can easily control (e.g., coffee consumption, exercise) that might affect what you want to improve. Do that for several weeks.

Step 3: Change that thing you can easily control. See what effect that change has.

In other words, try to do the smallest easiest thing that will put you closer to your goal. That tiny little goal will be turn out to be much harder to reach than you imagine. In the beginning, the smallest easiest thing is to measure one variable day after day.


Thanks to Jeremy for the question and to Gordon and Seth for their answers! Wonderful QS readers, please send in your questions and we’ll do our best to find answers for you.

Posted in Discussions | Tagged , , , , , | 7 Comments

Introducing The Quantified Self Advisory Board!

Do you need help with your self-tracking data analysis? Is there a specific problem or burning question about your experiment design that you’d love some guidance on? Gary and I are proposing an idea to help – read on for details!

We’ve gathered an amazing Quantified Self Scientific Advisory Board to be part of our community. It’s a star group of international scientists involved in data analysis, data visualization, and self-experimentation. In alphabetical order, they are:

- Alex Bangs, Human Predictive Biosimulation, Entelos
- Gordon Bell, MyLifeBits, author of Total Recall, Microsoft Research
- Jeff Heer, Collaborative Data Visualization and Flare/Prefuse, Stanford
- Gary King, Quantitative Social Science and n=1 experiments, Harvard
- Teresa Lunt, Director of Computer Science Lab, PARC
- Seth Roberts, Self-Experimentation guru, author of Shangri-La Diet, Berkeley and Beijing
- Neil Rubens, Data Mining, University of Electro-Communications, Tokyo

The experiment we’d like to do is to encourage Quantified Self members to formulate questions about the personal data that they are trying to work with. Post them as comments or send them to me. We will make sure the questions are interesting and at least partially answerable, pass them along to the appropriate Advisor, and publish the questions and responses here on the Quantified Self blog, as a way to get discussion going and add value to everyone involved.

So let us know what you think, and start asking questions!

Posted in News and Pointers | Tagged , | 5 Comments