Like many people paying attention to the press around Quantified Self, self-tracking, and wearable technology I was intrigued by the many articles that focused on a newly published research letter in the Journal of the American Medical Association. The letter, Accuracy of Smartphone Applications and Wearable Devices for Tracking Physical Activity Data, authored by Meredith A. Case et al., described a laboratory study that examined a few different smartphone applications and self-tracking devices. Specifically, they tested the accuracy of steps reported by the three different apps: Moves (Galaxy S4 and iPhone 5s), Withings Health Mate (iPhone 5s), and the Fitbit app (iPhone 5s), three wrist-worn devices: Nike Fuelband, Fitbit Flex, and the Jawbone UP24, and three waist-worn devices: Fitbit One, Fitbit Zip, and the Digi-Walker SW-200. Participants walked on a treadmill at 3.0 MPH for trials of 500 steps and 1500 steps while a research assistant manually counted the actual steps taken. Here’s what they found:
As the data from this research isn’t available we’re left to rely on the authors description of the data. They state that differences in observed vs device recorded steps counts “ranged from−0.3% to 1.0% for the pedometer and accelerometers [waist], −22.7%to −1.5% for the wearable devices [wrist], and −6.7% to 6.2% for smartphone applications [phone apps].” Overall the authors concluded that devices and smartphone apps were generally accurate for measuring steps. However, much of the press around this study dipped into the realm of sensationalism or attention grabbing headlines, for instance: Science Says FitBit Is a Joke.
Part of our work here at Quantified Self Labs is to encourage and help individuals make sense of their own data. After reading this research letter, or one of the many articles which covered it, you might be asking yourself, “I wonder if my device is accurate?” or “Should I be using a step tracking device or just my phone?” In the interest of helping people make sense of their data so that they can come to their own conclusions I decided to do a quick analysis of my own personal data.
For this analysis I examined the step data derived from my Fibit One and the Moves app I have installed on my iPhone 5. (Important note: the iPhone 5 does not have the M7 or M8 chip present on the 5s and 6/6+, respectively, which natively tracks steps.) I had a sneaking suspicion that my data experience differed from the findings of Case and her colleagues. Specifically, I had a hypothesis that the data from every day tracking via the Moves app would be significantly different than data from my Fitbit One.
First, I downloaded and exported my daily aggregate Fitbit data for 2014 using our Google Spreadsheets Fitbit script. I then exported my complete Moves app data via their online web portal. To create a daily aggregate step value from my Moves data I collapsed all activities in the summary_2014.csv file for each day. (Side note: We’ll be publishing a series of how-to’s for doing simple data transformations like this soon). This allowed me to create a file with daily aggregate step data from both Moves and my Fitbit for each day of 2014. Unfortunately I did not have my Fitbit for the first few weeks of 2014 so the data represents steps counts for 342 days (1/24/14 to 12/31/14).
I found that my Fitbit One consistently reports a higher number of total steps per day than my Moves app. Overall, for the 342 days I had 689,192 more steps reported by Fitbit than by the Moves app. The descriptive information is included in the table below:
Another way to look at this is by visualizing both data sets across the full time-frame:
There a few interesting things to point out in this dataset. On two days I have 0 steps reported from my Moves app. One day, Moves was unable to connect with their online service due to me being in an area with little to no cell signal. On the other day my phone was off, probably due to an iOS 8 release and having to reboot my phone a few times.
It is also clear to me that differences in data are related to how I wear my Fitbit and use my phone. For my Fitbit, it is basically on my hip from the time I wake up until the time I go to bed each night. However, my phone isn’t always “on my body” throughout the day. I think this is probably the case for more people.
Since I wear my Fitbit at all times some of the data it captures erroneously is included in the total step count. For instance, for the last few months in this data set I was commuting about 10 miles per day during the week by bike. This data is accurately captured as cycling by Moves, but captured as steps by my Fitbit. Therefore some over-reporting by Fitbit is present in the data.
For my own data I found that the Fitbit reports higher steps on most, if not all days, than the Moves app on my iPhone 5. There are a few caveats with this data and analysis that are worth mentioning. First, this exploration was intended to begin a conversation around the real-world use of activity monitoring apps and devices, and the data they collect. It was not intended as a statement on truth or validity (however I would welcome the help of a volunteer to follow me around with a manual clicker counting all my steps). Second, this analysis was undertaken in part to help you understand that scientists of all types, be it citizen or academic, have the ability to work with their own data in order to come to their own conclusions about what works or doesn’t work for them. Lastly, this analysis was completed very quickly and I am sure that other individuals may have different ideas about how to explore and analyze the data. For this reason I’m posting the daily aggregate values in a open Google Spreadsheet here.
We are happy to welcome this guest post by Madeleine Ball. Madeleine is the Senior Research Scientist at PersonalGenomes.org, co-founder of the upcoming Open Humans project, and the Director of Research at the Harvard Personal Genome Project. She can be found online @madprime.
Unpacking “data ownership”
It’s worth unpacking this phrase. What do we mean by “data ownership”? If we want to see changes, we need to start with a little more clarity.
Legally, data is not property. There is no copyright ownership of facts, as they are not “creative work”: the United States Supreme Court famously established this in the landmark case Feist vs. Rural. They are not patents, there is no invention. They are not trademarks. There is no “intellectual property” framework for data.
Yes, data is controlled: through security measures, access control, and data use agreements that legally restrict its usage. But it’s not owned. So let’s set aside the word “ownership” and talk about what we really want.
Control over what others do
One thing we might want is: “to control what others do with our data”. Whom they share it with, what they use it for. Practically this can be difficult to enforce, but the legal instruments exist.
Think about what you really want. Are you opposed to commercial use of your data? Look for words like “sell”, “lease”, and “commercial”. Are you concerned about privacy? Look for words like “share”, “third-parties”, and “aggregate” – and if individual data is shared, find out what that data is.
Companies won’t change if nobody is paying attention and nobody knows what they want. We can encourage change by getting specific, and by paying more attention to current policies. Raise awareness, criticize the bad actors, and praise the good ones.
Personal data access and freedom
The flip side of “data control” is our own rights: what can we do with our own data? We want access to our personal data, and the right to use it.
This idea is newer, and it has a lot of potential. This was what Tim Berners-Lee called for, when he called for data ownership last fall.
“That data that [firms] have about you isn’t valuable to them as it is to you.”
I think it’s worth listening, when the inventor of the world wide web thinks we should have a right to our data.
So let’s spell it out. Let’s turn this into a list of freedoms we demand. We should be inspired by the free software and free culture movements, which advocate for other acts of sharing with users and consumers. In particular, inspired by Richard Stallman’s “Four Freedoms” for free software, I have a suggested list.
Three Freedoms of Personal Data Rights
Raw data access – Access to digital files in standard, non-proprietary file formats.
Without raw data, we are captive to the “interface” to data that a data holder provides. Raw data is the “source code” underlying this experience. Access to raw data is fundamental to giving us the freedom to use our data in other ways.
Freedom to share - No restriction on how we share our data with others.
Typically, when data holders provide access to data, their data use agreements limit how this data may be shared. These agreements are vital to protecting user privacy rights when third parties have access, but we have the right make our own sharing decisions about our own data.
Unrestricted use – Freedom to modify and use our data for any purpose.
Data use agreements can also impose other limitations on what individuals can do with data. Any restriction imposed on our use of our data impinges on our personal data rights. Freedom for personal data means having the right to do anything we wish with data that came from us.
In the short term, access to raw data can seem obscure and irrelevant: most users cannot explore this data. But like the source code to software, access to this data has great potential: a few will be able to use it, and they can share their methods and software to create new tools.
Raw data access is also an opportunity for us to share for the greater good, on our own terms. We could share this data with research studies, to advance knowledge and technology. We could share data with developers, to develop software around it. We could share it with educators, with artists, with citizen scientists. We could even cut the red tape: dedicate our data public domain and make it a public good.
This work is licensed under a Creative Commons Attribution 4.0 International License.
— Richard Sprague (@sprague) January 27, 2015
Richard Sprague is interested in understanding peak performance. Over the last few years he’s been tracking various aspects of this life to try and understand what helps and what hinders. Inspired by our friend and renowned self-experimenter, Seth Roberts, Richard decided to test if consuming fish oil affected his response time. Using a simple reaction time test developed by Seth to test if butter made him smarter, Richard tested himself when he was and was not taking fish oil pills. In this talk, Richard explores his data and discusses what he found out when he ran his analysis.
Today’s post comes to us from PF Anderson, Emerging Technologies Informationist for the Health Sciences, University of Michigan and a member of the QS Ann Arbor meetup group. It first appeared on her excellent Emerging Technologies Librarian blog and we’re happy to republish it here.
Last week, I felt really lucky that I was able to make it to the first Quantified Self Meetup of the New Year (thanks to Nancy Gilby for the ride!). This session was held at the UMSI Entrepreneurship Center. Roughly ten people came, and I’m not sharing names even though they said I could because I’m not sure I got the names down right. The group included a wide range of types of people: corporate folk, students, entrepreneurs, faculty, alumni, and independents. The conversation was fast, dynamic, and overlapping, so I couldn’t catch everything. I will talk about what I did catch of the IDEAS and the GADGETS. That’s what’s really fun, eh?
What the Meetup group page SAYS they are interested in (as a sampling) is pretty extensive.
“Aging in Place Technology • Behavior change and monitoring • Caregiving of digital patients • Chemical Body Load Counts • Citizen science• Digitizing Body Info • Medical Self-Diagnostics • Lifelogging• Location tracking • Non-invasive Probes• Mindfulness and wisdom tracking • Parenting through monitoring/ tracking • Personal Genome Sequencing • Psychological Self-Assessments • Risks/Legal Rights/Duties • Self Experimentation • Sharing Health Records • Wearable Sensemaking”
What’s even more interesting is what people said they were interested in as they went around the table.
- aging population
- big data
- data visualization
- legal advice
- patient communities
- personal genomics
- sleep tracking
In 2007, after collapsing while rushing to board a train, Hugo Campos was diagnosed with hypertrophic cardiomyopathy, and an ICD (implantable cardioverter defibrillator) was implanted in his chest to track and regulate his heart rhythm. To his great surprise, he discovered that it was very difficult to gain access to the data being generated inside his own body. Today we’re inaugurating what we hope will be a regular series of “QS Conversations” about data access with an interview with Hugo about his long battle for the right to see what’s happening inside himself.
Ernesto: Why does access to your ICD data seem so important to you?
Hugo: I have a computer with firmware, processor and memory regulating my every heartbeat, wired into my heart, and buried inside my body. I can’t even see it. A corporation in the cloud, located out of state, has a wireless, transparent access to a device that’s implanted in my body, but the only control I have is to unplug the remote monitoring unit in my house to prevent them from getting the data. This creates a very unsettling feeling of not having autonomy. I’m paying thirty thousand dollars for a device, having it implanted inside my body, and then being locked out of it.
Ernesto: Was there something that happened that set you on this path?
Hugo: Yes. For a long time I’d been on my spouse’s health care plan, but when he decided to freelance and quit his job, I couldn’t get health insurance. This was before the Patient Protection and Affordable Care Act. Kaiser denied me because I had a heart condition. Anthem Blue Cross denied me as well. Now put yourself in my shoes. Here I am, being denied access to the device because the system “knows better” and I could harm myself, but now they can’t give me service at all.
Ernesto: You can only get access to your ICD through the medical system, but the medical system won’t take you because you have an ICD?
Hugo: Right, so I had to figure out a way to protect myself. I looked at it as kind of an extension of my Second Amendment rights. I’m not particularly pro-gun, but I look at it as the ability to defend myself. If the system was really unavailable, I have to at least be able to interrogate my ICD. So, I went on eBay and bought a pacemaker programmer that gave me full, unrestricted access to my implanted device. I can change its programming, shut it off, deliver therapy, and do as I wish. In fact, it’s the same machine that clinics use. I also went to Greenville, South Carolina, and took a class on how to program ICDs and pacemakers. I thought, “Okay, I may not become a cardiac electrophysiologist by any stretch of imagination, but”–to use the firearm metaphor again–”at least I have a basic understanding of gun safety so I don’t shoot myself.”
As you may know, we’re very interested in how HealthKit is shaping and extending the reach of personal self-tracking data. Last week, during Apple’s quarterly earnings call, Tim Cook mentioned that “There’s also been incredible interest in HealthKit, with over 600 developers now integrating it into their apps.” (emphasis mine).
@fat32io just a heads up. All the data in HealthKit is NOT backed up into iCloud. Unless encrypted local backup all data lost on reset
— Daniel Yates (@astralpilgrim) February 3, 2015
@astralpilgrim yikes, didn’t know that.
— fat32io (@fat32io) February 3, 2015
@fat32io yup. Lost all my data history last week doing a reset. Spoke to apple Genius Bar who told me about it.
— Daniel Yates (@astralpilgrim) February 3, 2015
@fat32io it’s the encryption that is key. They said its due to future sharing of health data with docs etc, requires encryption
— Daniel Yates (@astralpilgrim) February 3, 2015
For those of you that are unfamiliar with backup options for your iOS device. Here’s a quick gif to walk you through the process of encrypting your iOS backup so that you can restore your HealthKit data if anything happens to your device:
In 2013 Beth Martin was dealing with a failing startup, starting a new venture and working so much she moved her office into her bedroom to limit the time between waking and starting work. After a series of additional changes led her to near breakdown she decided to take six months off to rewrite her life. In this talk, presented at the Berlin QS meetup group, Beth describes the series of self-imposed challenges she created for herself and what she learned while tracking them and their impact on her life.
Philosophy, bicycles and brains, opinions on tracking sleep, learning from actually tracking sleep, and visualizing work through vigilant self-report – all these and more in our reading list below. Enjoy!
Sleep apps and the quantified self: blessing or curse? by Jan Van den Bulck. Here at QS Labs, we’re very interested in how the academic and research world is colliding with those of us using tools of measurement previously restricted to science. In this Letter to the Editor, published in the Journal of Sleep Research, the author lays out an interesting set of opinions about the increasing availability and use of commercial sleep tracking devices. (You can access the full pdf here.)
Measuring Brainwaves to Make a New Kind of Bike Map for NYC by Alex Davies. Readers of the QS website may remember a great show&tell talk we featured back in May of 2014. In that talk, Arlene Ducao discussed her MindRider Project, an EEG tracking bicycle helmet. In this short piece, we learn that Arlene has continued this awesome work and has produced MindRider Maps Manhattan, exposing the brain data of 10 cyclists as they transversed New York City.
Big Data and Human Rights, a New and Sometimes Awkward Relationship by Kathy Wren. Earlier this year the AAAS Science and Human Rights Coalition held a meeting to discuss the intersection of personal data collection and human rights. This short article describing some of the key discussion points is a great place to start if you’re exploring what “big” and personal data means to you and your use of the tools and services that collect it. (Videos of the meeting are also available.)
How Theory Matters: Benjamin, Foucault, and Quantified Self—Oh My! by Jamie Sherman. A very interesting and thought-provoking essay here on the nature of self-tracking and data collection framed against the works of Michel Foucault and Walter Benjamin. We count ourselves lucky to have Jamie as an active member and observer of our QS community.
But taken together, Foucault and Benjamin suggest that the penetration of data into daily life is part of a larger shift underway, and that changes we can already see in social life, politics, and labor are not unrelated, but rather intimately linked.
Compulsory Quantified Self by Gwyneth Olwyn. I think it’s good practice to try and expose ourselves to all sides of the conversation around self-tracking, the positive and the negative. In this blog post Gwyneth describes a few ideas about the purpose and outcomes of self-tracking, especially when the self is superseded by the demands of others (such as in a workplace wellness program).
Sleep Data Analysis with R by Ryan Quan. Ryan has been tracking his sleep with the Sleep Cycle app for the last two years. In this excellent post he explores and plots his data (yay export!) to see when he goes to sleep, how long he sleeps, and what really makes up “quality sleep.” Love the fact that he included his R code and sample data. Go Ryan!
Quantifying Goals Using Key Performance Indicators (KPIs) by Bob Troia. No data in this post, but I found it particularly inspiring to see how Bob was planning on keeping track of his goals for this year. If you’re looking for ideas for tracking your 2015 goals and Key Performance Indicators this is a great place to start.
The Resume Of The Future by Eric Boam. The above is one of the two beautiful visualizations created by Eric to explore his daily work activity and interactions. This visualization shows what he was actually spending his time on. How did he collect the data? Well, he used the Reporter App to ask himself three questions: “where are you, what are you doing, and who are you with?” Make sure to read his post, he developed very interesting insights through collecting this data.
Weight Loss: What Really Works? by Emi Nomura and Laura Borel. Another fascinating data analysis project here by the Jawbone data science team. They examined the behaviors of a group of users who lost at least 10% of their starting weight vs users with no weight loss and found that the biggest difference in behavior was tracking meals.
Mapping my Last Two Years of Runs and Rides
While browsing the r/dataisbeautiful subreddit I stumbled upon this interesting tool/company that visualizes the maps of your runs and bike rides by connecting to your Runkeeper or Strava account. Above I’ve included my 2013 and 2014 maps. Clearly I need to find some new running routes in my neighborhood. (click through to enlarge)
QS Access Links
As part of our new work highlighting stories, issues, and innovations related to personal data access we’re going to start publishing a short collections links in this space. As this works grows be on the lookout for a new Access Newsletter from QS Labs.
Who Should Have Access to Your DNA?
What FDA developments in Diabetes mean for FDA approval in Digital Health
Open consent, biobanking and data protection law: can open consent be ‘informed’ under the forthcoming data protection regulation?
WTF! It Should Not Be Illegal to Hack Your Own Car’s Computer
Unique in the shopping mall: On the reidentifiability of credit card metadata
Majority of Consumers Want to Own the Personal Data Collected from their Smart Devices
Who Owns Patient Data
Los Angeles County Supervisors OK Creation of Open-Data Website
This morning President Barack Obama announced a new Precision Medicine Initiative, a key $215 million piece of the proposed 2016 budget. Much has been written since last week’s State of the Union, when this initiative was first mentioned by President Obama. In brief, the initiative is an investment in new programs and funding initiatives at major government bodies that influence the current and future health of all Americans, including the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and the Office of the National Coordinator for Health Information Technology (ONC). These programs will focus on developing “a new model of patient-powered research that promises to accelerate biomedical discoveries and provide clinicians with new tools, knowledge, and therapies to select which treatments will work best for which patients.”
There is a lot of information being circulated about this new initiative, and we’ve collected some links below, but we’d like to highlight something directly related to our interests in self-tracking data, personal data access, and new models of participatory research. In this morning’s announcement President Obama mentioned a long-term goal of creating a participatory research cohort comprised of 1 million volunteers who will be called upon to share personal medical record data, genetic samples, biological samples, and diet and lifestyle information. This is truly an ambitious goal and we are happy to see the President take care to mention the importance of including patients and the individuals who collect this data in the decision making and research process. For example, here is the description of this specific program from the NIH Precision Medicine Infographic
Here at QS Labs, we’re dedicated to helping create and grow a culture that enables everyone to generate personal meaning from their personal data. Sharing, participation, and exploring new models of discovery are a core themes we’re exploring as part of our QS Access work. We’ll be following this initiative as it moves from today’s announcement to tomorrow’s reality. Be sure to stay tuned to our QS Access Channel for more updates as we learn more.
Learn more about the Precision Medicine Initiative
NIH mini site describing the initiative
White House Blog: The Precision Medicine Initiative: Data-Driven Treatments as Unique as Your Own Body
FACT SHEET: President Obama’s Precision Medicine Initiative
A New Initiative on Precision Medicine by Francis Collins and Harold Varmus (New England Journal of Medicine).