Tag Archives: Personal Analytics
Stefean Heeke wondered why only large companies were using “big data” and analytics, and then he decided to do something similar. In this talk Stefan describes a few different self-tracking and personal analytics projects that gave him unique insights into his own life. Not satisfied with only understanding and improving himself, he also applied his analytical skills to his soccer team. In the video below Stefan explains his experiments, his takeaways, and what happens when you apply data to help improve a soccer team. (Filmed at the NY QS Meetup group.)
On March 8th, 2012 Stephen Wolfram opened the curtains and gave the world a glimpse of his own self-tracking and personal analytics practice. It was jaw-dropping. It was dense. It was beautiful. And, it might have shown us the future of what the Quantified Self could become. We were lucky enough to have Dr. Wolfram answer a few questions about his data, his personal insights, and the future of personal analytics.
First, some background. Stephen Wolfram is a no ordinary self-tracker. Reading through his biography has honestly given me a case of the envies. First published scientific paper by age 15. Five years later a PhD from CalTech in theoretical physics. He then went on to study and develop the field of scientific computing. This led him on a path towards trying to understand the underlying principles that drive the complex systems we often observe in nature. Of course he also had to invent his own computational engine to help discover those principles and in 1988 Wolfram Research released Mathematica. Not one to rest on his laurels he then set out to develop pioneering projects and in 2009 oversaw the release of Wolfram|Alpha, a new kind of computational knowledge engine.
During this time he also set up some really neat systems to store, process, and analyze different streams of data that make up his day-to-day life. I’ll just go over a few here that I found particularly interesting.
Phone Calls. Throughout all the development of his groundbreaking work and running a successful company, he has managed to operate as a remote CEO. So this means a lot of time on the phone. A lot.
We can clearly see that when you examine the available data Stephen spends more than 4 hours per day on the phone. But suppose you need to speak to him. When is the time he’s already most likely on the phone?
Probably best to call him between 11 AM and 6 PM. Although, maybe the weekend evenings would also be good. Personally I found this really interesting that his weekend and weekday evening phone use probabilities were so closely matched.
Steps. If you know me you know how much time I spend thinking about steps. I’m an avid Fitbit user and really believe in the power of subtle passive data tracking for physical activity. Stephen has been wearing a digital pedometer for a few years and has kept some amazing records of his daily activity:
And, I’ll be completely honest here. I was a little beside myself to learn that Stephen and I are kindred spirits when it comes to our use of treadmill desks:
There’s no mystery to this: years ago I decided I should take some exercise each day, so I set up a computer and phone to use while walking on a treadmill. (Yes, with the correct ergonomic arrangement one can type and use a mouse just fine while walking on a treadmill, at least up to—for me—a speed of about 2.5 mph.)
Those are just a few of the numbers and analyses he described in his fascinating blog post. Be sure to give it a read, but before you do read below for our interview with Stephen and a summary of thoughts on what a computational quantified self might become.
Q: There has been a lot of reaction around the internet regarding your post with many people astounded at how much data you’ve collected. In your estimation how much of that data is passively collected/recorded versus actively collected/recorded?
Stephen Wolfram: It’s essentially all passive. I’ve had systems set up at different times, then I’ve just let the systems run. And after a decade or two they’ve accumulated a lot of data.
I should say that quite a few of the systems are set up to send me mail each day with a report on the previous day (how much I typed; how many steps I took; etc.). I find this a useful form of self-awareness and self-management. But it has the side effect that it checks that the systems are still running. Systems that aren’t checked “on“ have a nasty habit of decaying and failing.
About active data collection: I’m frankly too busy (and perhaps too lazy) for that. Last year I decided to record everything I ate. I kept it up for the whole year, but it was a pain. I decided I’d wait for that data until it could be more automated (which it will be soon).
Q: The main point of contention that many people are making after learning about your data collection and analysis is along the the line of “Who has the time for that?” How would you answer that question and where do you see the field of personal informatics and analysis going to meet the needs of people who just don’t have the time?
SW: It has taken only a tiny amount of my time over the last 20 years or so! It did take time to set up the systems. But not to run them.
The analysis now has taken time too. And there’s lots more that should be done as well.
Both the systems and the analysis were made vastly easier by the fact that we used Mathematica for all of them. I’m hoping that we can set up systems that let other people use what we’ve built to do their own personal analytics–with essentially no expenditure of time.
Q: You’ve obviously spent a lot of time with your personal data, and mention that you have even more in reserve that you didn’t expand upon. What was the most surprising thing you’ve learned in so far in your exploration?
SW: I wish I’d had more time to spend with my data. The blog post and the data behind it are really only scratching the surface. Almost every plot we made, I said, “Hmmm … that’s interesting.” Often it was just confirming some impression about my life that I already had. But sometimes I learned things. An example that surprised me a bit was that so much of my email I end up processing late at night; I thought I was keeping up better during the day. I’d been thinking of going to sleep earlier … but I’d clearly have to find another scheme for my email then….
Q: Have there been or did you experience any negative effects from either tracking or analyzing your data? Where there any surprises about yourself that you weren’t expecting that weren’t positive?
SW: Not really. My children sometimes tease me about my obsessive data keeping, but that’s about it. I had thought perhaps I might be able to see some kind of degradation in my performance over the past 20 years, which would have been sobering … but there’s nothing that I’ve found so far. Quite to the contrary, in fact: it seems, for example, that I’m getting more done than I did 20 years ago.
Now, there are pieces of tracking that would be nice to do, but that I don’t feel comfortable doing. An example is keeping an image stream. I have a camera to wear around my neck, but it would be too weird for me in social situations to have it be there, so I have essentially never used it.
Q: Now that you’ve looked at such a large portion of your longitudinal personal data I wonder if you’ve changed your behavior in any way. Was there any point during the analysis, or possibly during previous analyses of this or other data that helped you decide to change something in your life, and if yes, what was it?
SW: About 10 years ago I did some analysis of my email flow, and concluded that it was better to wait awhile before responding to internal emails, because they tended to be on threads that “resolved themselves”, so I didn’t need to spend time.
There are lots of details that I’ve changed. An example is making sure spelling mistakes, especially in people’s names, get immediately resolved in e.g. calendar entries … because if they’re not, it becomes hopeless to search for them later.
A bigger thing some time ago from personal analytics is that I discovered that whenever I took a trip, my work was perturbed for quite awhile afterwards. That made me take fewer trips. Though recently my children have got me to take more trips … and I’m happy to find that technology has advanced to the point where I can avoid getting so behind as a result of them ….
Q: A majority of your post really gets into the correlational structure of multiple data sets. One would say that this is the epitome of post-hoc analysis. How do you think you could use the information you’re collecting now to help you make real-time decisions about your life and your behavior? What kind of system or data would have to be in place to make that worthwhile?
SW: As I’ve mentioned, I do have daily email sent to me, with data on quite a few things. I find that somewhat useful in regulating my behavior (“I didn’t get enough done yesterday; I’d better be more focused today”, etc.). Also, I think it always puts me in a good mood when I see that the previous day was very productive, and it helps me be more productive then.
I’ve thought about having some real-time displays. An example I recently set up is a count-up timer (really it’s just an iPad looking at a web page) that tells me how long I’ve been asleep. I also quite often look at my digital pedometer to see if I’ve walked enough steps yet in a day.
Q: Something that keeps popping up in my exploration of Quantified Self and people who use personal data tracking devices and services is the issue of “getting credit”. You yourself express regret over not having more information from earlier in your life. Some might argue that this dependence borders on the obsessive or unhealthy. That we are offloading our human intuition to these data capture systems. How do you balance your intuition and instinct with what you learn and access through your personal data collection and analysis?
SW: I am spoiled by personally having a rather good memory, especially for facts, names, etc. So I’ve always had the issue of remembering a lot about what happened in the past. Of course, with actual data I continually learn how sparse my memory is.
I’ve been fortunate enough to have all sorts of experiences in my life, and I’ve learned a lot from them. And the more I can remember the experiences, the better I can make use of what I learned from them. And the more I am able to repeat my successes, and avoid repeating my mistakes.
I’m definitely a person who likes to do things, not just contemplate my past life. But I like to build on my past life to continually be able to do more things, and have more personal satisfaction.
Q: Lastly, this post has been a big inspiration to a lot of people and has been shared and passed around by many. Are there any last insights or thoughts you’ld like to share with our community?
SW: I think it’s great that your community is working in these directions. There’s a lot of “best practices” to figure out, that will probably affect our lives a lot in the future. I’m looking forward to seeing what your community figures out!
So there you have it. Some very insightful words from the man behind the numbers. Now that I’ve read through his wonderful answers to my overly wordy questions, spent some time watching his wonderful TED talk, and doing some late night thinking, I’m starting to wonder about how his process and systems might be a window into our quantified futures.
The reason this post is entitled, The Computational Quantified Self, is because I think that Stephen is onto to something brilliant here with his process of having Wolfram|Alpha Pro handle all of the processing for him. One of the things that I hear time and time again from individuals who are either interested in or just starting their own self-tracking practice is, “What next?” That is to say, there are a lot of people out there who either do not have the skills, knowledge, or time to handle the analysis needed to make sense of the large data sets they’re creating. So what happens when there is a system to do that work for you? Maybe that system is something from Wolfram Research, or Google, or some new startup we don’t even know about yet. But surely there will be people who embrace those systems to help them make decisions. Better decisions. Informed decisions. Personalized data driven decisions that enhance and improve their lives.
That sounds like a pretty amazing world to live in.
Special thanks to Stephen Wolfram for taking time out of his busy schedule to answer our questions and to his staff for making this possible. This was a special edition of our regular Numbers from Around the Web series. If you have data you’ld like to have featured in the series you can contact the author here.