Owning My Quantified Self Data
Topics
other
Aaron Parecki
After years of collecting Quantified Self data, Aaron Parecki began moving more of his data onto his personal website rather than letting it sit in someone else's cloud. This insures that his data will stick around even after apps and devices go away.
Tools
Micropub
Links
Slides
Transcript
Show
Your data ends being fragmented because it lives on a silo that’s specific to the device. So, my Jawbone sleep data is in the Jawbone. And then in January I got a Fitbit and now my sleep data is in the Fitbit API. And you know that’s kind of silly, but even with services still around they often limit what you can do, and they have limits on getting your data out of them sometimes. Sometimes they make breaking API changes that limit what you can get out of them later.
And often these services aren’t as reliable as you want them to be. They get acquired. They shutdown. They discontinue product lines. And I’m not saying I’m against using third party tools, I just always want to have the data out of it, even if these services shut down.
Wouldn’t it be amazing if there was a way to funnel all your Quantified Self data into a single place under your own control and allowing you to chose which devices you wanted to you and try out new devices, all of that data was in a place that you control.
Well, I’m not here to tell you about a shiny new QS aggregator setup, but I am here to tell you about how I actually do this today, and how I collect all the data into a place that I control, which is my website.
For the pat couple of years I’ve been working with a W3C group to develop the Micropub API, and my website supports this API and I’ve created various tools and scripts that normalize the data from all the various tools into Micropub requests that get sent to my website.
So for example, I use Foursquare to check in everywhere I go, and I made a tool, that kind of like if that sends all the check-in data from Foursquare to my website as a Micropub request, and then I can get my check-ins on my site.
For my weight I use the Withings scale and I couldn’t figure out their API, so I used if and that converts it to a Micropub request which sends it to my site.
I’ve been tracking sleep since I got a Jawbone in 2011, and if you remember a Jawbone in 2011 you won’t be surprised at all when I tell you that I’ve gone through about dozen of their devices. And my last one actually broke in January this year, so I finally switched to a Fitbit which is what I have now. But, thanks to that I’ve already been importing all that sleep data to my website, when I got the Fitbit I was just a matter of data munching to change the format to send it to my website. And not I still have the continuous dataset.
I also used Strava and Runkeeper for bike rides and runs and imported my data from those, but now I use my own GPS tracking app and it just sends the data directly to my website, so that’s an even shorter path there.
My phone doesn’t usually keep a GPS lock when I’m on an airplane, only if I’m in a window seat. So I hooked up some code to grab the flight data from the flight API and that sends it to my site and I get a nice little full path of the flight to Europe.
For a long time, I wanted to track what I eat and drink, and never found a sustainable solution, so I finally prototyped an app to figure out how much effort I was willing to put in to actually do that. I found that I could really only manage a short text description and occasionally a photo. So, I turned that into a really small web app called Teacup, which I’ve been using for a few years now and it gives me a quick way to enter the common things that I eat and drink and attach photos.
So, in addition to the Quantified Self data that I track, I also post my tweets to my website. And this is a small app called Quill and I also use the Microsoft dot blog IOS app. And I post those directly to my website and my website posts a copy to Twitter. So, anything you’re reading on Twitter is actually just a copy of what I put on my website.
With photos it doesn’t quite work that way because Instagram doesn’t have a very good API. So I first have to post to Instagram and then this other app turns into a Micropub request and posts it to my website.
So I’ve got all this stuff on my website and it’s pretty fun because this is what it looks like when you go to the URL for us which is a big day and it’s got all of my posts from every data source all just in a long list and you can see everything from each day all in one place.
And if you zoom out to the month view, then it turns into a calendar view and you have a little icon for each post, so you can see which day’s I post more photos or bookmark more things on my site. And it also shows the city name of the city I was in and able to do when I’m travelling.
And you can zoom out one more step to show the whole year at a glance and it’s a heatmap view of how many posts of different places are all on each day.
So this is all fun and this is cool but it’s a lot of work to do this right now, and I wrote a lot of code to make this happen, and dealt with a lot of APIs and a lot of frustrating API changes, and it’s not how it should be and there are ways to do this without code as well. People have got some pretty amazing Google spreadsheets rigged up where they select all the data sources and they automatically enter them into their spreadsheets or manually enter.
And it’s great that it doesn’t require writing code, but it’s still a lot of work to set these things up. Now there are some services that help bridge the gap, things like Gyroscope and (Exist? 06:12) and a couple of others. And you log in and connect all your accounts from all the different services and then they aggregate everything.
Ultimately, these are just band-aids for the problem though because even these services come and go. And sometimes they will shut down and sometimes they will take the aggregated data with them. And you know, frankly I don’t trust that these aggregator services will be around longer than I can keep my data myself in a place that I control.
And that’s why open standards like Micropub are important because we ultimately need to read the devices themselves to support open standards so that you can pick any device up and point it at any server and keep the data in a place that you control.
In the meantime, with Micropub you can at least start getting some of your data into your website. And this is actually this is live. A bunch of people are using it. You can go to (ownyourform.P3Kio? 07:05) and if you login it’ll walk you through setting up your website and Foursquare accounts and post your check-ins.
Here’s a handful of people who are using it and these are check-ins on their website. And you can see that everybody has a different idea of what check-ins should look like. And it’s their own choice and their own website and they can do it however they want and it’s pretty fun.
So, I would like to live in a world where I have control over my data and I don’t have to wrangle APIs to get that data. I want to use a mix of my own tools and I want to use tools other people build. I want to use my Jawbone toady and my Fitbit tomorrow and have that continuous dataset in a place that I control and with open standards like Micropub I believe we can get there.
Thank you very much.