Tag Archives: data visualization
Media artist Danielle Roberts (check out her Reverse Calendar), found her curiosity piqued by a scientific paper claiming that the well-known calming benefits of being in nature can be achieved by merely looking at pictures of natural landscapes.
At QS Europe 15, Danielle will present her “Virtual View” project, which combines image and sound for immersion in a constructed natural environment – with a twist. Virtual View feeds your physiological signal back into the system, subtly altering based on your response.
Join us in Amsterdam for QS Europe on September 18th & 19th, 2015. It will be an incredible two full days of talks, breakout discussions, and working sessions. We look forward to seeing you there!
We’re excited to share another round of personal data visualizations from our QS community. Below you’ll find another five visualizations of different types of personal data. Make sure to check out Part 1, Part 2, and Part 3 as well!
Name: Damien Catani
Description: This is an overview of how I have been doing today against my daily habit targets. Yes, I had a good sleep!
Tools: I used a website I’ve been building for the purpose of setting and tracking all goals in life: goalmap.com
Name: Bethany Soule
Description: This is my pomodoro graph. I average four 45 minute pomodoros per day on my work, and I track them here. This is where most of my productivity occurs! There’s some give and take.
Tools: The graph is generated by Beeminder. I use a script I wrote to time my pomodoros and submit them to Beeminder when I complete them. The script also announces them in our developer chat room, so there’s also some public accountability there as well.
Name: Steven Zhang
Description: This plot shows the time I first go to sleep, against quality of day (a subjective metric I plot at the end of every day). What this tells me is that if I get a full night’s sleep of 8 hours, for every hour I got to bed, I can expect a .16 decrease in my QoD rating, which, given my range of QoD around 2 to 4, is about a 5% decrease in quality of day.
Tools: Sleep as Android to track sleep and some python scripts for ETL.
- Normal sleep
- 3. Trying to achieve normal sleep, but failing to
Tools: Tableau for visualization. Sleep as Android for logging sleep.
Name: Eric Jain
Description: Benford’s Law states that the most significant digits of numbers tend to follow a specific distribution, with “1″ being the most common digit, followed by “2″ etc. But my daily step counts show a slightly different distribution: The fall-off from “1″ to “2″ is larger than expected, and the frequency of digits larger than “5″ increases rather than decreases. Is this pattern typical for step counts? Could suspicious distributions be used to detect cheaters?
Tools: Fitbit, Zenobase, Tableau
Stay tuned here for more QS Gallery visualizations in the coming weeks. If you’ve learned something that you are willing to share from seeing your own data in a chart or a graph, please send it along. We’d love to see more!
As the calendar turns over to a new year, it’s useful to look back and see what the last 365 days have been all about. Looking back is always easier when you have something to look back on, and, no surprise here, self-tracking is a great help for trying to figure out how things went. That’s what makes this time of year so interesting for someone like myself. I spend a good deal of my time trying to track down real-world examples of people using personal data to explore their lives. Sometimes it’s easy, and sometimes it’s hard finding people willing to expose themselves and their data. However, when late December rolls around, I perk up because this is the time for those yearly reviews.
I’ve spent the last few weeks gathering up some great examples from individuals from all over the world. I hope the following examples inspire you to track something new in 2015 and maybe share it with the QS community in person at a local meetup, at our QS15 Global Conference, or in our social channels. Okay, let’s dive in!
My Year 2014 in Numbers #QuantifiedSelf by Ragnar Heil. A brief, but fun post detailing a year of music, travel, and location checkins.
2014: A Year in Review with iPhone Pedometer Data by Geoffrey Litt. I really enjoyed this very thorough exploration of a year’s worth of pedometer data gathered from the Argus app (iOS). Not satisfied with just looking at his total step count for the year, Geoffrey ran a series of data explorations. Among my favorite, his visualization of his daily rhythms:
2014 in Numbers – My Life Behind the Command Line by Quincy Larson. Work, wellness (sleep and running) and reading – it’s all here. I like the idea of tracking what you’ve read by writing one tweet per book.
2014, Quantified by Sarah Gregory. Sarah does an amazing job of capturing and showcasing her 2014 activities in this beautifully simple post. With a balance of pure quantitative information and qualitative insights I found this review especially compelling. (It was also nice to see that she used our “How to Download Your Fitbit Data” tutorial.)
2014 in Numbers by Donald Noble. Speaking of our Fitbit data download tutorial, here’s a short post about a year’s worth of steps – 4.15 million steps to be precise.
Three Years of Running Data: 1,153km with Nike+ and Mind by Todd Green. As you can see from the title, this post details three years of running, but as a runner myself I always like peeking into other runner’s data. (Todd also has a fantastic post from early 2014 about tracking every penny he spent in 2013.)
Food, Glorious Food by Peter Chambers. A fun post detailing what Peter and his family ate for dinner nearly every day of 2014. One juicy bit – the most common meal? Chili – Peter’s favorite!
2014 in Numbers by Jill Homer. With the help of her Strava app, Jill details her cycling and running from 2014. Click for the numbers, stay for the gorgeous photos.
I wrote every day in 2014: Here’s an #infographic by Jamie Todd Rubin. It’s great fun following Jaime’s blog. He’s relentless on his journey of daily writing (and is quite the active Fitbit user as well). What was 2014 like for his writing? Over 500,000 words – almost enough to take on Tolstoy’s War and Peace. Plus, the visualization is great (click through for the full version):
2014 Stats by Dan Goldin. Amazing data gathered from a self-designed Google spreadsheet that includes mood, sleep, food, and drink.
Tracking My Life in 2014 by Mike Shea. Mike tracks his life using his own custom designed “Lifetracker app.” This includes his rating on six aspects of his life, daily activities, media, and location. In this post he turns his 8,400 rows of data into elegant visualizations and interesting analysis:
A Year in Review of Personal Data, Should be, well, Personal. By Chris Dancy. As always, Chris has an interesting and entertaining post about his 2014 data and how it compares to 2013.
Why #DIYPS N=1 data is significant (and #DIYPS is a year old!) by Dana Lewis. Along with her co-investigator, Scott Leibrand, Dana has been on a journey to better control, understand, and generate knowledge about her type 1 diabetes through augmenting CGM data, devices, and alerts. What started as project to make alarms more clear and useful has morphed into a full on DIY closed loop pancreas. In this post, Dana explores what they’ve learned over the last year of data collection. Truly inspiring work:
My Quantified Self Lessons Learned in 2014 by Paul LaFontaine. In this post Paul recounts what he’s learned from his various QS experiments during 2014, with a focus on stress and hear rate variability. Make sure to also take a peak at his 2014 Review and Gear Review.
2014 Year in Webcam and Screenshots by Stan James. We’ve featured Stan and his great LifeSlice project here on QuantifiedSelf.com before. It’s an ingenious little lifelogging application that tracks your computer use through webcam shots, self-assessments, and screenshots. Check out this post to see a fun representation of his data.
2014 by Kyle McDonald. A very interesting diary of a year.
What 2439 Reports Taught Me by Sam Bew. We highlighted this great post in our What We’re Reading a few weeks ago, but it deserves another mention here. Sam analyzes the data collected from using the Reporter iOS app and writes about what he learned.
2014 Personal Annual Report by Jehiah Czebotar. Coffee, travel, Citi bike trips, software development, laptop battery life, and webcam shots – all included in this amazing page. Presented without narrative or explanation, but meaningful nonetheless. The coffee consumption visualization is not to be missed (click through for the interactive version):
2014: My Year in Review by Sachin Monga. A mix of quantitative and qualitative data from Sachin.
My Q4 2014 Data Review by Brandon Corbin. While not a full “year in review” here, I still found this post compelling. Brandon created his own life tracking application, Nomie, and then crunched the numbers from the 60 different things he is tracking. Some great examples of learning from personal data in here.
20140101 – 20141231 (2014). Noah Kalina started taking a photo of himself on January 11, 2000. On the 15th anniversary of his “everyday” project he published his 2014 photos.
When I was spending late nights searching for variations on “2014”+”data”+”my year in review” I stumbled upon quite a few posts detailing reading stats. Here’s a good selection of what I can only assume is a big genre:
2014 Reading Stats and Data Sheets by Kelly Jensen. A great place to start if you want to track your own reading in 2015. Kelly provides links to three excellent spreadsheet examples.
My Year in Reading by Jon Page. Short and to the point, but a great exploration of format, genre, and authors.
My Year in Reading: 2014 by Annabel Smith.
My Year in Books, Unnecessarily Charted by Jane Bryony Rawson.
Well, that it for now. Special thanks to Beau Gunderson, Steven Jonas, Nicholas Felton (and many others) for sending in links and tips on where to find many of the above mentioned work. If you have a data-driven year in review please reach our via email or twitter and we’ll add it to the list!
If you’re interested in learning about how people generate meaning from their own personal data we invite you to join us for our QS15 Global Conference. It’s a great place to share your experience, learn from others, and get inspired by leading experts in the growing Quantified Self Community. Early bird tickets are on sale. We hope to see you there.
If you’ve made it this far here’s a fun treat: Warby Parker made neat little tool you can use to generate a silly personal annual report.
In 2009 Tim Ngwena switched on Last.fm and he’s been running in across all his devices ever since. Earlier this year he decided to take a deep dive into his listening data to see what he could learn.
I realized that I was listening to the same old thing and I began to think about changing what I was listening to. But how can I change? Where can I start? I also wanted to learn something about my music, what I was listening to and who was behind the sounds. I decided to focus on music because it was doable.
In this talk, presented at the London QS meetup group, Tim explains how he was able to make sense of almost five years of data and learn more about himself and his listening habits.
What Did Tim Do?
Tim explored his music data along side additional information such as location data from Moves to learn about his musical tastes, listening habits, and explore new visualization and data analysis techniques.
How Did He Do It?
Tim exported his data, used the Last.fm API and some data cleaning and organizational tools to create a simplified and extensive database of his music listening history and associated data. He then visualized that data using Tableau.
What Did He Learn?
Tim learned a lot about himself and what the music he listens to says about him. He describes a few of the most interesting below,
Basically 80% of my listening comes form 10% of the artists that I have in my library.
I’ve listened to Erykah Badu for over a week (7.2 days). It led me to ask what is she saying to me?
Monday is my jam time. I’m listening from the morning into the evening.
I listen to music mostly when I’m walking.
Tim also learned a lot through the process of designing and creating his data visualization. The visualization, which you can explore here, made him think about being able to see the big picture when he has so much linked data.
I think context is important and you need to see all that information in one place and the tools I’m using allows me to do this.
Two weeks ago we announced the release of the QS Access App so you could access your HealthKit data in tabular format for personal exploration, visualization, and analysis. In that short period of time, we’ve seen a good number of downloads and positive feedback.
We know from our experiences hosting in-person and online communication about personal data that seeing real-world examples of what is possible is what inspires people to engage and ask questions of their own data. With that in mind we’re excited to announce our QS Access Visualization Showcase.
We are looking to you, our amazing community of trackers, designers, and visualizers, to show use what you can do with data gathered from using the QS Access App. Make heatmaps in D3, complete analyses and visualizations in Wizard, or just make meaningful charts in Excel. If you’re visualizing your QS Access data we want to see it.
We also know that data visualization design and creation is not trivial work. To support the community and help expose the visualization work we’ll be awarding free tickets to our QS15 Global Conference & Exposition to individuals who use QS Access to create unique and interesting visualizations. We’ve earmarked two tickets (a $700 value) for outstanding work. If you’re selected, we’ll also work with you to showcase your work at the QS15 Conference and Exposition so other community members and attendees can explore and learn from their own data.
If you’re in the Bay Area come to our QS Meetup on November 11th at the Berkeley Skydeck. You can showcase your visualization and tell our community what you’ve learned from accessing and visualizing your data.
HealthKit is still new and the number of apps that integrate with it is growing by the day. At QS Labs we’ve done a bit of work making simple visualizations that are meaningful to us.
Steps and Sedentary Activity
Gary has an iPhone 5s which has native step tracking. We used the QS Access app to export his hourly step totals and made these simple line graphs in Excel. You can read more about what he learned from these simple data visualizations here.
How Much Do I Run?
Ernesto is an avid runner and enjoys running along the quiet trails in Los Angeles. He was interested to see how often he actually runs and if there’s any pattern to his running. Using a well-designed D3 template he was able to make a calendar heatmatp of his running distance.
If you don’t have any HealthKit data to work with, or just want to play with some example data we’ve created a few files that you can use as examples. Download the files below from our GitHub account and make sure to read the documentation to understand where the data is coming from. Descriptions of the data files and sources are available in our QS Access Data Examples repo on Github.
At our 2013 Quantified Self Global Conference we were excited to share a variety of beautiful and insightful data visualizations from our community. In the months leading up to the conference we asked attendees to send in their own personal data visualizations along with a short description. In our 6 years of hosting Quantified Self meetups and events, as well as running this website, our forum, and social channels, we’ve seen the power of data visualization as a story telling medium. We exist in part to help people tell their stories – about the data they collect, the changes they create, and the insights and new knowledge they’re excited to share.
Today we’re sharing a few of our favorite visualizations from past conferences. The images and descriptions below represent a wide a variety of tracking experiences and techniques, and we hope to showcase eve more unique personal data projects at our upcoming QS15 Conference & Exposition.
Tracking Sleep by Anita Lillie
This is concatenation of screenshots from my sleep app. Most sleep apps don’t let you zoom out like this and still see daily/nightly detail, so I just made it myself. I like that it shows how almost-consistent I am with my sleep, and made me ask new questions about the “shape” of a night of sleep for me.
2.5 Years of My Weight by Mette Dyhrberg
I gained a lot of insights from this heat map. The most obvious weight gain was no surprise — that’s when I periodically don’t track. In any case, the big picture patterns are easily identified with a heat map. Realized looking at this heat map that the point of no return was mid-April 2012 — my data shows that was when I switched protein shakes with an egg based breakfast. I have since experimented and seen that protein shake in the morning seems to keep my blood sugar more stable and as a result my weight under control!
One Month of Blood Sugar by Doug Kanter
This is a visualization of one month of my blood sugar readings from October 2012. I see that my control was generally good, with high blood sugars happening most often around midnight (at the top of the circle).
Tracking Productivity by Nick Winter
My percentile feedback graph of my development productivity helps my motivation.
Six Months of My Life by David El Achkar
This is my life during the past six months. Each square = 15 minutes. Each column = 1 day. This picture represents 138 days or 3,000+ activities.
My Thesis Self Portrait by Sara M. Watson
Here’s a period of a few days of webcam images taken using Stan James’ LifeSlice during the final days of editing my thesis on Quantified Self uses of personal data. Serious business!
Sleep and Meaningful Work by Robby Macdonell
In an average work day, I don’t consider communication (email, instant message, etc) to be terribly meaningful work. I’d much rather be working on building software. Getting more sleep the night before increases the amount of meaningful work I’m likely to do in a day.
70 Days of Pulse by Laurie Frick
Pulse rate over 24 hours for 70 days from my Basis watch. Grey=null, blues=85
Rain Ashford is a PhD student in the Art and Computational Technology Program at Goldsmiths, University of London. Her work is based on the concept of “Emotive Wearables” that help communicate data about ourselves in social settings. This research and design exploration has led her to create unique pieces of wearable technology that both measure and reflect physiological signals. In this show&tell talk, filmed at the 2013 Quantified Self Europe Conference, Rain discusses what got her interested in this area and one of her current projects – the Baroesque Barometric Skirt.
We’re back with another great set of articles, show&tells, and visualizations for you.
How to Make Government Data Sites Better by Nathan Yau. Government entities are some of the largest holders of interesting data. Nathan focuses this article on the difficulties of accessing and making sense of data from the United States Centers for Disease Control and offers some good ideas on how to make it better.
Project Eavesdrop: An Experiment At Monitoring My Home Office by Steve Henn. What happens when you start monitoring yourself in the same manner the NSA might be doing? The author employs some technical help to learn what data leaks are possible and what you can figure out from your digital trails.
Sitting is Bad for You. So I Stopped. For a Whole Month. by Dan Kols. As a past frequent user of a treadmill desk and a sedentary behavior researcher I found this article intriguing. Yes, a bit silly in nature, but an interesting look at what happens when you go to the extreme. I especially enjoyed the integration of personal tracking in the piece.
Analyzing Squash Performance Using Fitbit by Ben Sidders. Ben sought out to see if he could learn anything from his step data to improve his squash playing. In this post he explains how he used R to access his data and plot it against his squash records, which he also records.
My Life As Seen Through Fitbit. Reddit user, VisionsofStigma, plots a year and a half of Fitbit data to find out what is related to the rise and fall of his activity.
Freeing My Fitbit Data by Bonnie Barrilleaux. Bonnie used our instructions for accessing Fitbit data in Google Spreadsheets then used Python to visualize her data. I especially like the histogram pictured at left. If you want to visualize your Fitbit data, she’s included her code in the post.
Diurnal Plot of Fitbit Data by Matthew Gaudet. Matthew was inspired by the diurnal plots from Stephen Wolfram’s in-depth personal data review. He implemented the same methods to better understand his activity.
Iconic History by Shan Huang. As part of a the Interactive and Computational Design class at Carnegie Mellon University, Shan created a Chrome browser extension that visualizes your browser history. More about the project here.
Visualizing Last.fm History by Andy Cotgreave. Andy has been using Last.fm since 2006 to track his music listening activity. As a data scientist he was interested in what he could learned from all that data. In this four-part series, he explores his data along side data from eight of his friends. (If you want explore your Last.fm data you can export it using this awesome CSV export tool.)
From the Forum
23andMe Data Analysis Tools
Want to receive the weekly What We Are Reading posts in your inbox? We’ve set up a simple newsletter just for you. Click here to subscribe.
In the Quantified Self community we focus on projects and ideas that help people access and get meaning out their personal data, including the information you can collect with your smartphone. If you have an iPhone, Android, or Windows phone you’re already have carrying of the world’s most sophisticated self-tracking tools. The GPS, accelerometer, the microphone, all of these tiny sensors make up a great set of tools you can use to understand how you move around the world.
I’m going to focus this short “how to” on geolocation data and mapping your movement, specifically using data gathered by the Moves application. Moves is a passive activity and location tracking tool available for the iPhone and Android. We’ve written a bit about it in the past and had a chance to interview their CEO, Sampo Karjalainen. I’ve been using it since May, 2013 and I wanted to share some neat tools and methods for getting a bit more out of the data Moves collects.
I find that visualizing my data on a map to be incredibly powerful. It might by my inner cartographer, but seeing my patterns of movement (or lack there of) in reference to known places and landmarks is a great mechanism for inducing recall and reflection on where I’ve been and what I’ve done. Hopefully you’ll use one of the tools or methods below to map you data and learn something new!
Moves Connected Apps
Like many self-tracking applications and devices, Moves has a API that many different developers have built services on top of. Here are just a few of the services that allow you to see your data on a map. Be advised that each of these services has access to your data. Make sure to read their Terms of Service before agreeing to the data transfer.
WebTrack. This is by far the most utilitarian data mapping tool. However, you shouldn’t get discouraged by the lack of fancy design because it gives you an very unique data view. When you use Moves on your phone you typically only see the “storyline” and the detected places you’ve spent time at. However, Moves is constantly pinging and recording your location when it detects movement. WebTrack allows you to see all those movement points by hovering over the associated timestamp.
Fluxtream. You might know Fluxtream as Friend of QS and a great open-source data aggregation tool. They’ve set up a “Moves Connector” that allows you to import and visualize your Moves data. Because Fluxtream is set up as an aggregation and visualization tool you can also map other interesting data sets. Want to know where you were tweeting last week? Fluxtream will map it for you. (You can see me tweeting on a CalTrain ride between San Francisco and Palo Alto below.)
Zenobase. Another interesting data aggregation service here. Zenobase treats your Moves data bit differently. Rather than importing all the movement geolocation data it focuses on your place data and visualizes those locations. I like the high-level view it start with, but make sure to keep zooming in to see more specific place data.
Resvan Maps. This mapping application adds a unique twist to the typical mapping visualizations. It will plot your places, paths, and categorize paths depending on the activity (transport, walking, running, and cycling). Additionally, you can create “analysis cirlces” and have the application compute the time you spent in a certain location you bound (it aggregates to hours:minutes per day).
MMapper. This method for mapping your data, developed by Nicholas Felton, is by far the most technical, but it produces some really neat visualizations. You’ll have to download Processing and follow the instructions Nicholas provides on the Github repository page here. The great thing here is that the mapping and data access is all happening locally.
Move-O-Scope. Another great mapping application here from the folks at Halftone.co. They’ve probably completed the most thorough mapping and exploration tools for your Moves data. After linking your Moves account you can explore maps by activity type, day of the week, and custom data ranges. Additionally, they’ve implemented a neat feature for exploring place data. You can see how many times you’ve visited a specific place, where you’ve come from and where you go next, what days you typically visit, and your typical time of day at that place. (See this post for background on why they created this nifty tool.)
Map It Yourself!
If you don’t want to trust your data to a third party, but you still want to explore your movement maps there is really great option for you. Our friend and co-organizer of the QS LA Meetup, Eric Blue, recently published a method for easily exporting your data: the Moves CSV Exporter. You’ll have to login and use the Moves pin system in order to download your data, but Traqs isn’t storing your data, just providing a way for you to access it. The tool allows you to download and explore your activity, summary, tracks and place data. We’ll focus on the place data for creating maps. You can also use your full tracks history for mapping all the geolocation points Moves collects.
Because this data is based on latitude/longitude coordinates there are many different methods available that you can use to map your data. I’m going to focus on two here: Google Fusion Tables and CartoDB (if you know of others share them in the comments or our forum).
Google Fusion Tables
Fusion Tables are a new Google Drive tool that you can use to store, analyze, and visualize many different types of data. Once you download your Moves places.csv file you can upload it to a new Google Fusion Table. Once you upload your data, which takes about 2 minutes, you’ll see a menu bar and three tabs: Rows, Cards, Map of longitude. Just click on the “Map” tab and you’ll see your data already placed on a map. If you want to see a heatmap rather than a point map just navigate to Tools -> Change Map and you’ll see an option for a heatmap on the lefthand side. This is just the tip of iceberg for mapping fusion table data. You can learn more about different mapping methods and tricks here.
CartoDB is a visualization and analysis engine for geospatial data. I’ve been using it to play around with a few of the different geolocation datasets that I have (I actively keep three). Although it is paid service, they do offer a free plan for smaller datasets, which is perfect for your Moves data. Again, you’ll have to upload your places.csv file to a new table once you set up your account. Once the data is uploaded there are quite a few different map visualization wizards you can use to view your data in different ways. Pesonaly I like playing with the “Torque” visualization that gives you a real feeling of space-time to your data.
TileMill is an interactive map design tool from the folks over at Mapbox. If you’re looking to create custom maps with your data that you can format, style, and share then this is a wonderful tool to use. At first glance it’s a little daunting because it looks like a mashup of a CSS editor and map tool. That actually gives it the unique power to drive customization. Don’t be afraid, it’s not too hard to get started with. Mapbox has provided a great “crashcourse” to get you started with importing data, saving it as a new layer on your map, and then manipulating how it looks on your screen. If you want to go just a bit farther you can also add legends and informative popups to describe your data points. Mapbox also offers a free hosting plan if you want to share your interactive maps on a webpage. For example check out my MovesMap here, where I added a quick styling to manipulate the point size in relation to the time spent at a location.
Hopefully you’ve learned something new from this. If you map your Moves data (or any other geolocation data) we want to see it! Leave a link in the comments, post it in the location mapping thread on the QS Forum or get in touch on twitter!