Learning about Biases and Gaps in my Self-Collected Data

So, I’ve been collecting data on myself for a long time. I started in high school. I’ve got notebooks dating back to 1998 with diet and exercise records in them. And I’ve been using Jump since grad school and I actually work for Jump now, but it was really seeing what other people did with their own Quantified Self data that inspired me to put the two together.

So what kind of data have I been collecting on myself over the years? It’s mostly been information about my exercise habits, what I’ve been eating and all of a sudden I’ve been collecting in an effort to try to manage weight issues that I’ve had throughout my adult life. But I’ve also collected other information on other bio markers as well.

So creating this historical graph of my lifetime weight and history took a lot of digging, so my baby book – that is actually a picture of my baby book, through medical records, my notebooks, and information more recently from digital and wireless scales.

But as I start to add some notes and some context to this graph, you can see that a story starts to emerge, about a story about stress-related weight gain that I’ve experienced, and also pregnancy related weight gain that I had in my adult life.

Now, as I added some color coded notes to my graph, you can see in blue when I’m tracking I actually do fairly well in these things. It’s when I don’t track in the notes in red when things tend to go in the wrong direction for me.

So I’m in 2009 I hit a high and non-pregnant weight. I decided I was going to recommit to tracking. So when I’m actively trying to manage my weight, I actually adopt these constellation and behaviors that altogether both influenced weight and various other biomarkers. Now interestingly, I never made this connection until recently, but I I’m a Libra, so using a star consolation and that’s being the stars for Libra actually is not a bad way to represent this sort of many to many relationships, so that I have this balance thing going on.

It’s great to know that 23andme told me that I couldn’t blame my genetics, but I could blame the stars for this lifelong struggle in this area. But actually weight turns out to not be actually such a bad segregate variable for assessing how I’m doing. So my cholesterol composition tends to be best when I’m in my ideal weight range and go out of whack when I’m not. Although, obviously you need to pay more attention to Mark Cuban because I need more frequent measurements on that plot.

My blood pressure also seems to correlate fairly well with my weight, although in the middle I’ve got this variation due to age and fitness differences. I mean, whether I was sick or ill or time the measurements were taken, but there is a relationship there.

Obviously, what I eat and how much I eat also impacts my weight and my biomarkers, and I’ve collected a lot of data on this over the years. Honestly, people used to think that I was crazy because back in the day of notebooks and nutritional reference books, here I was noting down all the information about macro nutrients and totaling up my calories. That was so much work compared to what I have to do today. Now I’ve got apps and frequent food lists, and I can get everything out in digital form is so much easier, and it is so much worth the time investment to get the kind of data that I have now.

First, we also have to make a little bit of an investment in figuring out the happy path for getting your data out of the system to using and into a tool together and I’ve done that. Jump is my tool of choice and it sits on my desk every day so that’s what I’ve been using. And that lets me look across my whole dataset and say okay, for example, calories I’ve eaten, typically, I’m 4 to 6 meals per day 1500 to 2500 cal, and this is over nearly a four-year period.

And I can also find these weird outliers in my data. So for example the days that I ball parked my calories, that’s without actually logging any foods, and then there’s days that I skipped logging, was that a fast day, did I stop logging, was it a deficit day. If I was looking at someone else’s data it was may be harder to tell, but I know my own patterns.

Now I also found that I had this very high level of redundancy in my food log data. So you would expect this picking from a database, you pick different items for representing the same thing. I did a lot of cleanup on that, and I also classified foods into food groupings so that I could make meaningful visualization of those, so I could do things like access, which food, and which food groupings that contributed the most calories for my food logs over about a four-year period. So yes, I have an addiction to chocolate candies, embarrassing, but the slide will be off in a couple of seconds.

But really I found you know, that graph was over a four-year period. But my patterns really changed a lot over time. So being able to interactively filter and kind of focus down on specific areas and look more closely was really important when looking at this data.

So for example, I collected data. When I was pregnant and I had a two-week period where I ate a burrito every single day. And I avoided dairy and I was feeling nauseous and later in my pregnancy on the bottom these patterns changed. Dairy came back and I have not touched a burrito since and I have the data to back that up.

But anyway, after my son was born, which is this reference line here, clearly the sleep efficiency tanks, you can see that in the smoother on the top. But what I didn’t expect to see in my sleep data was a seasonal variation on the bottom, the sort of undulating patterns and the smoother you can see that. That was a surprise to me on my miss fed on my daily data.

And I felt really good about my sleep data and I knew that I had religiously wore the armband every night, but the daytime activity data turned out to be a totally different story, and it was kind of upsetting to meet actually because I thought I was doing such a good job. But actually, when I started to look at my activity data I thought oh I’m so much less active during these hot Carolina summers that makes a lot of sense, but in reality I knew I was active so what was going on here.

As I started digging more deeply into the data, I started to realise that in an effort to avoid having tan lines on my upper arm when the armband is worn, I have been wearing it less in the summer. So my summer data was really poor as a result, so my wear time completely correlated with my activity, which that was when I knew that I needed a new device. And before the next summer time, so the tan lines weren’t an issue.

And actually, I needed it to kind of recharge after a very interactive spring, so my joy at getting my hourly database was then just crushed by my realization that in 2015, the red line was my least active year yet since I started using an activity monitor.

I’ve also adopted some new tracking projects to kind of get me re-motivated like creating selections filters and custom body maps using Jump so that I can use my work out data and seeing how my patterns have changed over time, which has been really fun. And you can stop off at my office hour for more information about that.

So I’ve really only scratch the surface since starting to combined across my data types. I think there’s a lot more that I can do with this in the future, but I’m really just starting out there.

But I wanted to share one of my very favorite visualizations that I’ve created during this project. So it’s a graph that has muscle elements and it tracks my pregnancy, weight changes over the top and maintenance in the middle we have days that I was eating in either a surplus or a deficit, and then maintenance over time and on the bottom is weekly macro nutrients.

It would have been so helpful to have had this graph earlier on because it so clearly shows me the relationship between you know eating too much and then gaining weight and eating less and losing weight over time and how I eat as much as I burn and maintain just really helps to convince me. it also helped convince my family, so my mom and my sisters actually adopted the tracking behavior after seeing the success that I had with it. And to collectively between the four of us had lost over 200 pounds, so that’s been a really awesome amount for us.

I wanted to say a really special thanks to Zane Greg who’s a colleague of mine. He developed graph builder and helped me to improve a lot of the visualizations that I had. Hope you can stop by and see me at office hours. You can check me out on the Jump blog or my own blog or chat on Twitter.

Thank you very much.