Low Friction Personal Data Collection

Low Friction Personal Data Collection

And these are the things I’ve tried that I can’t really sustain, they last maybe only a month at a time or a couple of days at a time for various reasons. Things I have been able to sustain over extended periods of time, more than a year things like location at 5 seconds or one second intervals and I know exactly where I’ve been since 2008, and I’ve been tracking my sleep and weight consistently since almost exactly two years ago now. So if you go to my website aaronparecki dot com you can actually see where I am right now, but I’ve reduced that information down to one decimal point of Lat long, so you so you only know where I am within half a mile or so. But you also see my time zone which is actually more important.

If you send me an SMS with a specific keyword if you know my phone number so you will get my exact location, so you can find out where I am if I’m trying to come meet you somewhere. And you also get my time zone if I’m travelling to another city.

I tried to do this a long time ago with a Hardware GPS logger. The problem with this one is there is an SD card that records to and an extra battery pack, so you’re carrying a phone and this thing, and it’s a lot to charge that device and to download that data every night so it was real time I had to download it.

Finally in 2008, there was a couple of smartphones on the market that had GPS in them and this is when I was able to start the project and actually sustain it. So the key here was that it was my phone, also I was already charging it all the time, and it had a live connection to the server, so the data is in real time and I can always do things with it in real time.

I’ve been working on this since 2008 and the technology has gone through various iterations and I’m using it on the iPhone right now. This is the app I wrote a couple of years ago.

So this all goes up to my server which store every single row of data, data points from anywhere from one to five seconds apart and I can query by timestamp or by location to find out where I’ve been.

The SMS that you send to me take this ridiculous path where – sorry, the time on API is an important point. In order to get this data with my time zone there is this terraform project that we’ve been working on just getting the lat long you can find other geo spatial information.

The SMS takes this ridiculous path, where it actually goes to my Google voice number which sends me an email which then I catch through my Gmail filer which goes to a server, sending an email back to that return address which sends you back n SMS. It actually happens in like under half a second which is pretty amazing. But then this is an m=image which is also out in the gallery of my location of over four years in Portland. Unfortunately the script is a little too long to screen but you can go take a look at the print version, and it shows my patterns as they have changed over time and my path condenses in recent years and was much more spread out previously.

You can also go find my weight on my website, which is interesting. I’ve been wanting to track my weight for a long time, and you know I tried the paper pad method and obviously that’s not really sustainable except for a short period of time project. So finally with this wireless scale that’s what was able to let me sustain this project. And the key here is it actually requires no more effort than a regular scale. And that’s the important point and that’s what lets me keep doing this continuously.

And to get this is a similar ridiculous dataflow, where it goes through email change and stuff. But it works and it’s set up now and it will just keep working and I don’t have to touch it.

So this is about two years of my weight data now, and day to day my weight doesn’t change that much, maybe point two or point five pounds at a time, so I don’t really notice much even if it goes up and down plus or minus five pounds over a few months. So I looked at this and thought that’s interesting, what’s that peak. It turns out that’s when my startup was required, and so I guess a lot of stress relief makes my weight go down, so that was nice. But I didn’t even notice that was happening as it was happening until I looked at this full two years of data.

The same thing with my sleep, I’ve tried to try a number of ways to track my sleep, from the paper pad method next to my bed and a couple of different devices. So you can go to my site and see this and you’ll see the city I slept in and the time zone, and that’s coming from my location which I have previously.

I tried it with the sleep cycle app, which it’s a cool app, it gives you a nice graph but I can’t really keep it up because you have to plug it in next to your bed and there’s not always a plug next to your bed if you’re travelling. And then the phone gets hot and you can’t put it under the pillow and it might blow up when you’re sleeping.

I tried using the original Fitbit and it was pretty good except you wear in around your waist and o track your sleep you have to put it on your wrist. So you have to every night get in this ritual of take it off your waist, wrap it around a little armband thing, and that was just a little too much work for me to do and I couldn’t keep it up. So this is what I’ve been using since 2011, the Jawbone band finally.

It’s always on my wrist and I never have to worry about whether it’s on. The battery lasts for like 10 days, so I don’t have to worry about charging it if I’m travelling. So I’ve been able to keep that up and this goes up to my database as well and I take the timestamp and then I enrich that with my location from my GPS data as well as the time zone which I mentioned previously because you have to correct time zones when travelling.

Then you can slice and dice this data and get some neat things out of it, like top cities I’ve slept in. this is sorted by average amount of sleep per night, also showing the variation in nights for each city. And the two peaks here, one of them was Berlin where I was recovering from jet lag, that’s almost 17 hours of sleep, which is a ton and apparently it happened at no other time. And South by Southwest a couple of years ago in Austin that was like you know basically a nap, two hours that night.

This is another way to slice and dice it which is looking at week nights versus weekends, and this is about two years again and you can see that there’s actually a very large variation in weekend verse sleep nights. Over the las three months I’ve been consistently sleeping more on weekends which is good. I’m still trying to figure out why these occurred like I need to go back and look at my other data like check in on Foursquare or what I was eating and that kind of thing.

So basically I’m just setting up enough systems that are in place that can collect data from all over these different sources, pull it all together and let me query them and let me combined them together in ways, and in the meantime publish some of this stuff on my site which I feel would be fun for other people to see as well. Because a lot of these patterns are only visible if you have a very long extended dataset, not just a month at a time or a week at a time, so basically I’m just trying to set these systems up so that i can passively collect it. once I get the system in place I just want it to keep working without me thinking about it at all and that way have have lots of data to work with later. I might think of something two years from now that I wasn’t to analyze, but it’s only going to work if I have two years of data of which I now will have two years of data if I can start collecting it now.

So low friction, create these devices that let you collect data passively so that you can spend time on analyzing it later.

That’s all I’ve got, thanks.