Tag Archives: activity tracking
When the Apple Watch was announced I started waiting with bated breath to see how it could be useful for Quantified Self and self-tracking purposes. Of course this means staying up late and making sure I had one on order as soon as possible. I put in my order shortly after midnight on launch day for a 42mm Space Gray with the black sport band.
On May 19th my Apple Watch arrived, coincidentally just after we wrapped on our first Bay Area Apple Watch Users Group meeting (which was fantastic and I highly recommend joining). I set it up and started figuring out how it worked as an activity tracker. I have a keen interest in activity tracking, not just as a self-tracker, but also as a graduate student studying how people use activity tracker data to understand and impact their lives. In that vein, I’ve been a consistent Fitbit user for over four years, transitioning from the original Fitbit to the Ultra, and then to my current Fitbit One. I’m a big fan of the Fitbit and use it as my personal “gold standard” for activity tracking. It’s accurate, consistent, and easy to use. Does that hold true for the Apple Watch? Let’s find out.
What did I do?
I wore my Apple Watch every day, from the moment I woke up to when I went to sleep at night. I set up my charging station on my nightstand, which is also where my Fitbit One spends its nights. I wasn’t thinking about this data analysis when I first started wearing the watch, but looking back over the past month I am confident saying that if I was wearing my Fitbit I was also wearing the watch.
This data analysis includes data from May 20th to June 23rd, or 35 days of data collection. My activities varied as a normal function of my work and life, meaning I didn’t purposefully mix things up or engage in activities just for testing purposes. Many days were sedentary, some days had longer walking periods, and in the 35 days I ran seven times at distances between four and nine miles.
How did I do it?
Exporting the data from both the Fitbit and the Apple Watch is not a trivial task, but thanks to a few pieces of software I was able to access and analyze both data sets.
The Apple Watch stores the data it collects in Apple’s Health app using Healthkit. A quick glance into the Health app indicates that it is storing minute-level step data from the Apple Watch. Apple built in a data export function for the Health app, but it’s in a proprietary XML format that I’m not super familiar with. Thankfully there is QS Access. Our team at QS Labs created simple app that connects to Apple Health and allows you to export your data in a easy to use .csv file.
To export my data I first made sure that the Apple Watch had the highest priority for the data sources that feed the “steps” data for Apple Health. This is important because all newer iPhones (5s, 6, 6+) also natively create step data and store it in the Health app. I then used QS Access to create a data export for steps. I chose the hourly function as it’s the highest level of granularity the QS Access app currently offers for data export.
Fitbit recently introduced a data export feature. While this is a great step forward for them, and for their millions of users, the export feature is a bit limited. You can only export daily aggregate data and only one month of data is exportable at a time. Since I had access to hourly data from the Apple Watch I wanted to match that granularity.
I turned to my good friend, and past colleague, Aaron Coleman. Aaron runs a unique startup called Fitabase, which was built to help researchers, organizations, and individuals get easy access to activity tracker data. I spun up my account at Fitabase, which has been collecting and storing my Fitbit data for the last few years, chose the date range and downloaded my hourly step data.
I wanted to get right to my core question, “How accurate is the Apple Watch compared to the Fitbit One?” so I imported both data files into Google Spreadsheets, did a bit of data formatting, created a pivot table, then made some simple graphs. The full data set is available here if you want explore more complex statistics or visualizations.
What Did I Learn?
When compared to the Fitbit One, the Apple Watch is fairly accurate for step tracking. What do I mean by fairly accurate? Let’s dive into the data.
When I explored my daily step totals it appeared that the Apple Watch counts more steps than my Fitbit One, but not that many more. Here’s the data you need to know:
- Fitbit Total Steps: 308,955
- Apple Watch Total Steps: 317,971
- >Difference: 9,016 or 2.91% of the total steps (counted by Fitbit)
I created a difference category by subtracting Fitbit steps from Apple Watch steps for each day. This allowed me to see how different the data was day over day. The mean difference indicated that Apple Watch counted 258 steps more per day on average. Important to note that the daily difference was highly variable with a standard deviation of 516 steps. Looking at the scatterplot and histogram below you can see a few clear outliers, but what appears to be an otherwise normal(ish) distribution for the difference in step counts.
What about when we look at a higher level of granularity? I also explored the hourly steps data and compared the Fitbit and Apple Watch. On average the Apple Watch counted 11 more steps per hour than the Fitbit One during this period. Again, this was highly variable with a standard deviation of 85 steps, and a range from overcounting by 462 steps to undercounting by 696 steps. I haven’t yet filtered out sleep time (0 steps) so the mean difference per hour in this data set is likely skewed low.
I also looked into one more question that I though was interesting. Is there a significant difference in daily or hourly step data as a function of the total steps? Or, more simply, when I’m more active does the Apple Watch still stay consistent?
It appears that being more active doesn’t have a significant impact on how accurate the Apple Watch is tracking and counting steps. I created scatterplots for this relationship and added a simple linear trendline. In both cases, the trendline indicated that only a small amount of variability in the difference between the devices was accounted for by the total steps taken.
I’m not ready to give up my Fitbit just yet, but I was happy to see that the Apple Watch is an accurate step tracking device. Of course there are caveats to this data set. It’s somewhat small, a little over a month of data, and I didn’t do any “ground truth” testing where I counted my actual steps. However, I feel more confident now that whether I’m walking around my apartment, my nieghborhood, or going on runs, the Apple Watch will accurately reflect those activities.
Like most other runners who are using the Apple Watch I’m interested to dive into the heart rate data to test it’s accuracy. I’ve already collected a few runs, but will doing a bit more testing to compare to other common heart rate trackers.
When we decide to track one thing, we sometimes find that we are indirectly tracking something else. That is the theme of today’s talk.
When Mark Leavitt was 57, he found out that he had heart disease, a condition that runs in his family. Mark set about making some life changes. He tracked his weight while adopting a low-fat diet. His tracking showed him that he was making progress and that progress encouraged him to keep tracking. But once Mark’s weight loss stalled and then started to backslide (though he had maintained his diet) his desire to track dwindled and was then snuffed out by a major life event.
Though he was ostensibly tracking weight, this experience gave him some insight into his motivation. He began to build a mental model of his willpower. When was it strong? When was it weak? Using his background as a doctor to make assumptions on the nature of his willpower, he used the tracking of other lifestyle changes, such as movement and strength-training, to test those assumptions and better understand how to follow through on his intentions.
Watch below to see what Mark found worked for him and if you would like to see how Mark’s keeping up with his habits, you can check out his live dashboard here.
In 2013 Eric Boyd started using a Nike FuelBand to track his activity. Not satisfied with the built in reporting the mobile and web applications were delivering he decided to dive into the data by accessing the Nike developer API. By being able to access the minute-level daily data Eric was able to make sense of his daily patterns, explore abnormalities in his data, and learn a bit more about how the FuelBand calculated it’s core metrics. Watch Eric’s talk from our 2013 Quantified Self Europe Conference to hear more about Eric’s experience.
Here at QS Labs we’ve been curious about the differences between two of the most popular devices among self-trackers: The Nike+ FuelBand and the FitBit. I’m the latest experimenter on this topic, and since January I’ve been wearing a FuelBand on my left wrist and a FitBit (original model) in the right hand coin pocket of my jeans. The FitBit almost always counts significantly more steps than the FuelBand.
The details are interesting. When Bastian compared his FuelBand vs his Fitbit, he found a slight correlation between his activity level and the difference in the number of steps they counted. In other words, them more active he was, the more the two devices disagreed. When Ernesto did his FuelBand vs Fitbit test, his numbers closely matched. My data is more like Bastian’s, but with the effect of high activity even clearer. Look at the graph below. On the vertical axis is the difference in step count, by day. On the horizontal access is the number of daily steps Fitbit counted. The higher the number of “FitBit steps,” the more likely it is that “Fuelband steps” are much lower.