Tag Archives: science

Daniel Gartenberg: The Role of QS in Scientific Discovery

Posted on August 10, 2012 by Alexandra Carmichael

Today’s breakout session preview for the upcoming QS conference comes from Daniel Gartenberg, organizer of the Washington DC QS meetup group. Here is Daniel describing his session “Is QS Science? The Role of QS in Scientific Discovery:”

——

Do you believe in the power of using Quantified Self to solve some of science’s toughest questions, but have concerns about the validity of QS data?  There is actually a long scientific tradition of N=1 studies (i.e. studies conducted with a single participant).  Additionally, there are various advantages of N=1 studies, such as repeated, longitudinal, and naturalistic data.  These advantages of N=1 studies enable the personalization of treatments because they can take into account individual differences.

Yet N=1 studies are atypical in current scientific research.  We will be discussing why the scientific community frowns on N=1 studies, and how we can alleviate some of the scientific community’s concerns regarding QS.  This involves understanding what makes something ‘Science.’ Additionally, this will involve identifying threats to validity when conducting studies and QS research.  Threats to validity include, but are not limited to: Mortality, History, Maturation, Treatment Fidelity, Treatment Interaction, Compensatory Rivalry, Regression Towards the Mean, and Reactivity.

If done correctly, QS can be a new standard in scientific rigor, but this will require a concerted and collaborative effort by the QS community that will involve developing a system where QSers can post their data to the cloud and have the data aggregated and analyzed across individuals (e.g. curetogether).  The potentials and challenges for creating a QS database will be discussed.

Come to this breakout session if you are trying to make sense of your QS data, are interested in the scientific method, are interested in data analysis techniques, or want to create systems and tools that make QS data meaningful to the general population.

Posted in Conference | Tagged , , , | 4 Comments

Gustavo Glusman on Introducing QS into the Scientific Community

Posted on July 8, 2012 by Alexandra Carmichael

Gustavo Glusman is a member of Leroy Hood’s group at the Institute for Systems Biology. At a recent Hood group retreat, the main topic of conversation was Quantified Self! In the video below, Gustavo gives a fascinating recap of the retreat, including how the researchers talked about QS, what experiments they did on themselves, and the main challenges they see with QS from a scientific perspective. (Filmed by the Seattle QS Show&Tell meetup group.)

Gustavo Glusman: Introducing QS in the scientific community (Seattle) from David Reeves on Vimeo.

Posted in Videos | Tagged , , | 1 Comment

QS 101: The Science of Self Experimentation

Posted on May 3, 2012 by Ernesto Ramirez

This special guest post in our ongoing QS 101 series comes to us from Dan Gartenberg, our great QS-Washington DC meetup organizer, and his fellow graduate students in the Human Factors and Applied Cognition program at George Mason University.

Turning Scientific Concerns into Strengths for Quantified Self Experimentation

By Dan Gartenberg, Ewart de Visser, and Jonathan Strohl

Quantified Self and Science are not oil and water. They are intertwined with one another and have a long history together. Though some scientists may not hold QS in high regard, and have the following claims:

“these studies lack validity!”

“a study of a single individual will not generalize to the broader population”

“the possibility of experimenter bias makes your findings highly suspicious and inconclusive.”

“is your effect even real?”

While these are legitimate concerns, QS is science.  And if we keep in mind the scientific method when conducting QS research, this strengthens the validity of our QS projects.

History speaks for itself.  QS studies are actually in line with an age-old scientific tradition: The n=1 study. Back in the day, scientists did not have large labs and took to experimenting on themselves. For example, Hermann Ebbinghaus, one of the first cognitive psychologists, conducted experiments on himself to reveal the process of learning and forgetting.  As a scientist he used a level of rigor that was expected of scientists at the time, and more importantly, Eddinghaus contemplated reasonable mechanisms that explained his results.  Science gives us the tools to make precise measurements, and QS, with its emphasis on improvement of the self, provides a social framework for people to discuss novel phenomena.  In this article we demonstrate how science lends itself to QS and how the scientific method provides us with useful tools for self-discovery.  We first recommend a framework for conducting QS experiments and then discuss the scientific methods to keep in mind.

1)  Achieve your goal:  Unlike most experimental research, in QS our main objective is more often than not self improvement.  A frequently used approach to improve yourself is by throwing the kitchen sink at the problem until you get the sought after effect.  When we make this process social, we can then discuss with others what they are doing and how they think they are being affected by what they are doing.  Based on this information, we get a better understanding of how to most effectively modify our behaviors for the desired outcome.

2)  Use a simple design.  When presenting your data the scientific establishment might criticize your conclusions because it was not the gold standard “double blind randomized control trial.”  But running the right type of design isn’t the be-all-end-all of good science.  If you see a difference and have a reasonable mechanism that explains the difference, with no viable alternative explanations – you are solid.

3) Stats don’t matter as much.  Just graph it! One of the biggest sources of confusion is how to analyze QS data and knowing the right stats to run.  But statistics are only really useful when predicting small effects or for more complex prediction models.  In QS, any change is usually meaningful.  For example, if you are tracking your mood and you see a small improvement it is likely meaningful to you.  So don’t worry too much about statistics.

To discuss the roles of QS and science, we’ll use a dataset that we generated as a case study.  After reading the 4 Hour Body by Tim Ferris, three friends all had the same goal of losing weight.  None of us were extremely overweight at the time, but we could all stand to lose about 10-pounds.  This inspired us to create the 10-Pound Challenge, where we competed to lose weight and either did a slow carb diet or a low carb diet.  We then weighed ourselves every morning.

Quantified Friends: The 10-Pound Challenge:

Here are some issues and concepts that you should consider when making sense of your findings and how Science and QS can benefit one another:

Threat to validity Definition 10-pound application
Mortality  When your manipulation affects the likelihood of whether or not you respond to the measure of interest (i.e. you don’t respond to a survey out of embarrassment). There are almost no skipped days over the course of our study.  This demonstrates how QS can be used as a way to address the problem of mortality by making data collection more social.  This makes people more accountable and motivated to input their data.
History When external events from the environment impact the variables of interest. The diet was made social when we shared our progress with one another. This socialization, where we competed to lose weight, may explain our progress.  We knew that this resulted in alternative explanations, but it didn’t matter because we simply wanted to lose weight and were pulling out all the stops to help us reach our goal.
Maturation When over the course of a study you have changed in other ways that have confounded the impact of the variables in your experiment. In QS our goal is to actually change and mature.  In our QS project the intention was to lose weight and the mechanism was only as important as it was necessary to understand and use in order to promote our increased weight loss.
Treatment Fidelity  In science we usually compare a treatment group to a control group, but what if the treatment is not much different from the control? (i.e. there is not a good counterfactual). We administered relative treatments based on each of our unique situations (this is a common issue with QS).  For example, one of us already had a relatively strict diet.  He was able to make precise modifications to his diet in order to promote weight loss, whereas; the other two QSers made broader changes. This prevented us from making precise claims about how the diet affected weight loss.  Though we still got the general idea that the diet worked.
Treatment Interaction  When the variable that you manipulate interacts with other variables that explain the outcome. When undertaking the 10-pound challenge we frequently told people about the challenge.  This in turn made us more accountable for what we consumed due to social pressure.  In this example, the response to the treatment interacted with the social environment in a way that made us consume healthier meals and lose weight.  Since in QS we are not as fixated on control, we can see how these interactions unfold in the environment and discuss them with others in order to confirm or deny our intuitions.  This provides us with ways to explore new ideas and mechanisms.
Compensatory Rivalry  When the control group is aware that they are not getting the treatment and in turn seeks out other alternatives. In the case of the 10-pound challenge, there was no control group.  Control groups do not play a large role in QS because of the focus on self-improvement.  This is an issue that can be addressed by the scientific method of an A-B-A design where the QSer acts as their own control group.
Regression Towards the Mean  This intuitive premise from science is the basic idea that at the extremes of a behavior you are increasingly likely to gravitate towards the mean. QSers should be particularly sensitive to this because people frequently try to improve on a behavior when they are at an all time low or an all time high.
Reactivity  When your response is affected by external factors, for example, social desirability. In QS, reactivity can actually be used to improve upon outcomes. In our example, there was a social desirability to discuss what was and was not working.  At one point the team members independently agreed that eating too many slow-carbs (legumes) was hindering their progress.  We then made the appropriate changes to our behaviors and found increased improvements.

And these are just some of the threats to validity that QSers can consider to improve their projects. So look out for more to come!

We would like to thank Dr. Patrick McKnight and the MRES group (http://mres.gmu.edu) for providing helpful insights on our QS project.

 

 

Posted in QS 101, Uncategorized | Tagged , , , , | 1 Comment