We hope you enjoy this weeks list. Feel free to submit articles, show&tell self-tracking stories, and QS data visualizations. Just email me!
Why can’t you track periods in Apple’s Health app? by Nat Buckley. With the recent re-release of Apple’s HealthKit enabled self-tracking and personal data system it no wonder that people are taking a long hard look at what data is being excluded. With the popularity of menstruation tracking apps (this app has nearly 30,000 ratings) it’s surprising this was overlooked. This excellent post is a must read on the topic.
Now That Cars Have Black Boxes, Am I Being Tracked? by Popular Science Editors. Questions and concerns about surveillance are becoming more commonplace. As someone who is looking to purchase a car in the next year or so I was happy to see this post come across my stream.
The Quantified Self community, lifelogging and the making of “smart” publics by Aristea Fotopoulou. I love it when people take a thoughtful look at the Quantified Self community and write about their experiences:
For me, the potential of QS for public participation lies in the show and tell meet-ups that constitute a central feature of this community. Meet-ups enable the exchange of stories about the success or failure of lifelogging practices; they allow people to connect and form synergies around common interests, and to explore wider questions such as personal data management and ownership. [...] members touch upon key political issues and create temporary spaces of dialogue: what happens to personal data, who has access to these data (is it private individuals, governments or corporations)? For what purposes (medical research)? And how can these data be interpreted (by algorithms, visualisations) and used to tell stories about people?
Stepping Down: Rethinking the Fitness Tracker by Sara M. Watson. Sara uses her personal journey of recovery from hip surgery to frame an interesting question: Should we trust our fitness trackers to prescribe movement goals?
Practical Statistical Modeling: The Dreaded After-School Carpool Pickup by Jamie Todd Rubin. Jamie wanted to understand if there was a way he could reduce how much time he spent waiting in line to pick up his son from school. Why not track it and model it!
Bulletproof Diet and Intermittent Fasting: 1.5 Year Results by Bob Troia. Bob takes a deep dive into his data to see if this particular diet is having beneficial health effects. Click for the great data, stay for the wonderful discussion and very, very thorough write-up.
Quotidian Record by Brian House. I’ve been a fan of Brian House since his early days visualizing Fitbit data. I was reminded of this work during a conversation about geolocation data and thought it would be a nice addition to our visualization list.
Visualizing My Daily Self-Management by Katie McCurdy.
What does my daily medication and self-management look like? How could I visualize this regimen? How can I communicate the ‘burden’ and work of caring for myself?
I decided to draw pictures of the things that I need to do on a daily basis; that way I could show the workshop attendees what my day was like instead of just telling them.
It’s Time to Eat by Karl Krehbiel. Karl, a data science intern at Jawbone used the data from their global community of users the determine the likelihood of food and drink consumption during the day. Really fun and interesting visualizations here.
Earlier this week we posted an update to our How To instructions for downloading your Fitbit data to Google Spreadsheets. This has been one of our most popular posts over the past few years. One of the most common requests we’ve received is to publish a guide to help people download and store their minute-by-minute level step and activity data. Today we’re happy to finally get that up.
The ability to access and download the minute-by-minute level (what Fitbit calls “intraday”) data requires one more step than what we’ve covered previously for downloading your daily aggregate data. Access to the intraday data is restricted to individuals and developers with access to the “Partner API.” In order to use the Partner API you must email the API team at Fitbit to request access and let them know what you intend to do with that data. Please note that they appear to encourage and welcome these type of requests. From their developer documentation:
Fitbit is very supportive of non-profit research and personal projects. Commercial applications require additional review and are subject to additional requirements. To request access, email api at fitbit.com.
In the video and instructions below I’ll walk you through setting up and using the Intraday Script to access and download your minute-by-minute Fitbit Data.
- Set up your FitBit Developer account and register an app.
- Go to dev.fitbit.com and sign in using your FitBit credentials.
- Click on the “Register an App” at the top right corner of the page.
- Fill in your application information. You can call it whatever you want.
- Make sure to click “Browser” for the Application Type and “Read Only” for the Default Access type fields.
- Read the terms of service and if you agree check the box and click “Register.”
- Request Access to the Partner API
- Email the API team at Fitbit
- They should email you back within a day or two with response
- Copy the API keys for the app you registered in Step 1
- Go to dev.fitbit.com and sign in using your FitBit credentials.
- Click on “Manage My Apps” at the top right corner of the page
- Click on the app you created in Step 1
- Copy the Consumer Key.
- Copy the Consumer Secret.
- You can save these to a text file, but they are also available anytime you return to dev.fitbit.com by clicking on the “Manage my Apps” tab.
- Set up your Google spreadsheet and script
- Open your Google Drive
- Create a new google spreadsheet.
- Go to Tools->Script editor
- Download this script, copy it’s contents, and paste into the script editor window. Make sure to delete all text in the editor before pasting. You can then follow along with the instructions below.
- Select “renderConfigurationDialog” in the Run drop down menu. Click run (the right facing triangle).
- Authorize the script to interact with your spreadsheet.
- Navigate to the spreadsheet. You will see an open a dialog box in your spreadsheet.
- In that dialog paste the Consumer Key and Consumer Secret that you copied from your application on dev.fitbit.com. Click “Save”
- Navigate back to the scrip editor window.
- Select “authorize” in the Run drop down menu. Click run (the right facing triangle).
- Select “authorize” in the Run drop down menu. This will open a dialog box in your spreadsheet. Click yes.
- A new browser window will open and ask you to authorize the application to look at your Fitbit data. Click allow to authorize the spreadsheet script.
- Download your Fitbit Data
- Go back to your script editor window.
- Edit the DateBegin and DateEnd variables with the date period you’d like to download. Remember, this script will only allow 3 to 4 days to be downloaded at a time.
- Select “refreshTimeSeries” in the Run drop down menu. Click run (the right facing triangle).
- Your data should be populating the spreadsheet!
If you’re a developer or have scripting skills we welcome your help improving this intraday data script. Feel free to check out the repo on Github!
How many times during the course of the day do you find your mental state drifting into negativity, feeling like you’re lost, or just plain stressed? How could you even keep track of this, and why would you want to?
What Did Paul Do?
Paul LaFontaine has been tracking what he calls “upsets” to better understand himself, the way he works, and to see if he can improve his mental and physiological response and recovery.
Upsets are something physiological that were happening beneath the surface, and they’re trackable. It didn’t have to be emotional, but there had to be a signal. This project is part of an longer ongoing study. Before this current iteration I manually logged over 3,000 upsets and what I found is that most of my upsets were self-induced. I’d be in a calm environment, but then become upset about something. I wanted to use technology because I was afraid of bias and I know I was missing some upsets.
How Did He Do It?
I used the HeartMath EMWave2 that measures heart rate variability and indicates when you’re in and out of coherence. When I was out of coherence I captured that as an upset. I would stop what I was doing and use an audio recorder to keep track of the time, how long I was upset, the reason, and what method I used to recover. I tracked 71 sessions (each session was 25-45 minutes) totaling 42 hours of tracking time. I logged 1292 upsets during this period.
What Did He Learn?
Paul analyzed his data and found some very interesting insights about his upsets, his reasons for being upset, and the effectiveness of his recovery techniques.
I found that I was triggering an upset every 2 minutes. My wife said something must be wrong with me, but this stayed relatively constant through the tracking period. I started to think of it like skiing a mogul course. The moguls didn’t move, it was about how effective I could move through them. And, dealing with upsets is like playing whack-a-mole. They come fast and furious and every second counts.
For recovery I was able to find that my most effective technique was breathing. By returning to six breaths per minute routine I was able to improve recovery time from 33 seconds to 17.8 seconds. It was the primary way I could remove myself from being upset and make myself calmer.
We want to thank Paul for presenting this great QS project at the Bay Area QS Meetup group. Make sure to watch the full talk below to learn more about Paul’s methods and findings, then hop over to his website where you can read about how he tracked his stress during this talk.
We’re posting a quick note today to let you know that we’ve updated our “How To Download Your Fitbit Data” post. It now included separate instructions for both the old and new versions of Google Spreadsheets. This is just the first in a series of planned updates. We hope to post additional updates to allow you to have deeper access to your Fitbit data including, heart rate, blood pressure, and daily goal data.
If you’re using this how-to we’d love to hear from you! Are you learning something new? Making interesting data visualizations? Discussing the data with your health care team? Let us know. You can email us or post here in the comments.
We hope you enjoy this week’s list!
Big Data in the 1800s in surgical science: A social history of early large data set development in urologic surgery in Paris and Glasgow by Dennis J Mazur. An amazing and profoundly interesting research paper tracing the use of “large numbers” in medical science. Who knew that is all began with bladder stones!
Civil Rights, Big Data, and our Algorithmic Future by Aaron Rieke, David Robinson and Harlan Yu. A very thorough and thoughtful report on the role of data in civil and social rights issues. The report focuses on four areas: Financial Inclusion, Jobs, Criminal Justice, and Government Data Collection and Use.
Caution in the Age of the Quantified Self by J. Travis Smith. If you’ve been following the story of self-tracking, data privacy, and data sharing this article won’t be all that surprising. Still, I can’t help but read with fascination the reiteration of tracking fears, primarily a fear of higher insurance premiums.
Patient Access And Control: The Future Of Chronic Disease Management? by Dr. Kaveh Safavi. This article is focused on providing and improving access and control of medical records for patients, but it’s only a small mental leap to take the arguments here and apply them all our personal data. (Editors note: If you haven’t already, we invite you to take some time and read our report: Access Matters.)
Perspectives of Patients with Type 1 or Insulin-Treated Type 2 Diabetes on Self-Monitoring of Blood Glucose: A Qualitative Study by Johanna Hortensius, Marijke Kars, and Willem Wierenga, et al. Whether or not you have experience with diabetes you should spend some time reading about first hand experiences with self-monitoring. Enlightening and powerful insights within.
Building a Sleep Tracker for Your Dog Using Tessel and Twilio by Ricky Robinett. Okay, maybe not strictly a show&tell here, but this was too fun not to share. Please, if you try this report back to us!
Digging Into my Diet and Fitness Data with JMP by Shannon Conners, PhD. Shannon is a software development manager at JMP, a statical software company. In this post she describes her struggle with her weight and her experience with using a BodyMedia Fit to track her activity and diet for four years. Make sure to take some time to check out her amazing poster linked below!
The following two visualizations are part of Shannon Conners’ excellent poster detailing her analysis of data derived from almost four years of tracking (December 2010 through July 2014). The poster is just excellent and these two visualizations do not do it justice. Take some time to explore it in detail!
Tracking Energy use at home by reddit user mackstann.
“The colors on the calendar represent the weather, and the circles represent how much power was used that day. The three upper charts are real-time power usage charts, over three different time spans. I use a Raspberry Pi and an infrared sensor that is taped onto my electric meter. The code is on github but it’s not quite up to date (I work on it in bits and pieces as time permits I have kids).”
Today we are excited and honored to announce that Steven Jonas has joined QS Labs as our Senior Editor/Information Architect. As has been the case with previous additions to QS Labs, we welcome Steven as a friend and fellow community member. Steven serves as a co-organizer of the Portland QS meetup group, and has participated as our speaker coordinator for our past two conferences.
In addition to his work supporting our global QS community, Steven is an active self-tracker, having engaged in many different projects. We’ve been delighted to highlight a few of those here on the QS website. We invite you to welcome Steven and get to know him a bit by exploring the posts linked below.
Photo by Mark Krynsky
Like many of us, James Norris remembers his first kiss. Unlike many of us, he also knows who it was with, where it was, and his age. How does he know this information? When he was 13, he realized that he forgot some detail about his life that he thought was important. To prevent that from happening again, he decided to carry around sticky notes to record important life events and has been doing it ever since. Fast forward 15 years and James has recorded 1,500 “firsts.” Watch this talk, presented at the Washington DC QS meetup group, to hear James talk about the data he collects, and the lessons he’s learned along the way.
At our 2013 Quantified Self Global Conference we were excited to share a variety of beautiful and insightful data visualizations from our community. In the months leading up to the conference we asked attendees to send in their own personal data visualizations along with a short description. In our 6 years of hosting Quantified Self meetups and events, as well as running this website, our forum, and social channels, we’ve seen the power of data visualization as a story telling medium. We exist in part to help people tell their stories – about the data they collect, the changes they create, and the insights and new knowledge they’re excited to share.
Today we’re sharing a few of our favorite visualizations from past conferences. The images and descriptions below represent a wide a variety of tracking experiences and techniques, and we hope to showcase eve more unique personal data projects at our upcoming QS15 Conference & Exposition.
Tracking Sleep by Anita Lillie
This is concatenation of screenshots from my sleep app. Most sleep apps don’t let you zoom out like this and still see daily/nightly detail, so I just made it myself. I like that it shows how almost-consistent I am with my sleep, and made me ask new questions about the “shape” of a night of sleep for me.
2.5 Years of My Weight by Mette Dyhrberg
I gained a lot of insights from this heat map. The most obvious weight gain was no surprise — that’s when I periodically don’t track. In any case, the big picture patterns are easily identified with a heat map. Realized looking at this heat map that the point of no return was mid-April 2012 — my data shows that was when I switched protein shakes with an egg based breakfast. I have since experimented and seen that protein shake in the morning seems to keep my blood sugar more stable and as a result my weight under control!
One Month of Blood Sugar by Doug Kanter
This is a visualization of one month of my blood sugar readings from October 2012. I see that my control was generally good, with high blood sugars happening most often around midnight (at the top of the circle).
Tracking Productivity by Nick Winter
My percentile feedback graph of my development productivity helps my motivation.
Six Months of My Life by David El Achkar
This is my life during the past six months. Each square = 15 minutes. Each column = 1 day. This picture represents 138 days or 3,000+ activities.
My Thesis Self Portrait by Sara M. Watson
Here’s a period of a few days of webcam images taken using Stan James’ LifeSlice during the final days of editing my thesis on Quantified Self uses of personal data. Serious business!
Sleep and Meaningful Work by Robby Macdonell
In an average work day, I don’t consider communication (email, instant message, etc) to be terribly meaningful work. I’d much rather be working on building software. Getting more sleep the night before increases the amount of meaningful work I’m likely to do in a day.
70 Days of Pulse by Laurie Frick
Pulse rate over 24 hours for 70 days from my Basis watch. Grey=null, blues=85
Like anyone who has ever been bombarded with magazine headlines in a grocery store checkout line, Kouris Kalligas had a few assumptions about how to reduce his weight and improve his sleep. Instead of taking someone’s word for it, he looked to his own data to see if these assumptions were true. After building up months of data from his wireless scale, diet tracking application, activity tracking devices, and sleep app he spent time inputing that data into Excel to find out if there were any significant correlations. What he found out was surprising and eye-opening.
This video is a great example of our user-driven program at our Quantified Self Conferences. If you’re interest in tell your own self-tracking story, or want to hear real examples of how people use data in their lives we invite you to register for the QS15 Conference & Exposition.
Before we get to this week’s list we want to make sure you know about our recent conference announcement. This week we announced our QS15 Conference & Exposition. This will be our seventh conference and is sure to be an amazing event. We invite you to register today!
Now on with the good stuff!
Why Big Data Won’t Cure Us by Gina Neff. A great research paper in the aptly name journal, Big Data. Dr. Neff specifically focuses on the perils of assuming “all the data” will solve the numerous health healthcare problems and then lays out five elements to consider as data, big and small, becomes part of our healthcare experience.
More Than Meets the Eye: NASA Scientists Listen to Data by Kasha Patel. Apparently the scientists studying the sun have so much data to sift through that listening to signals is a valuable alternative to visualizing it. (via our friend Joost Plattel)
Quantified Dating, Relationships, and Sex by Kitty Ireland. A great series of three posts by Kitty that explores a variety of examples of using self-tracking in the most intimate of situations – dating, long-term relationships, and sex.
A Look Back At the Evolution of Wearable Tech. In the wake of the recent Apple Watch announcement I love being able to look back at the history of different how technology has made inroads into our lives.
The Baby Measureur by Erich Morisse. Erich is a proud father of a new child and like any new dad with data skills he started tracking some important metrics such as feeding time, feeding duration, and of course diaper changing!
A Day at Burning Man, Visualized Through Health Tracker Data by Gregory Ferenstein. Gregory takes his Basis Band to Burning Man and shows us what he learned.
My Most Intimate Self Portrait by Scott Ogle. Scott has a wonderful post here about a visualization of his almost 30,000 text messages.
If I look closely, I can see a new job, vacations and a death in the data. I can even see where I moved past it all and stopped feeling the need to communicate so much. It may just be text messages, but it all correlates to things that are really real.
And all of it is captured in this graph.
9 Days in Amsterdam – Tracking my Mobility in Bicycle Wonderland by Patrick Stotz. Patrick traveled to Amsterdam and tracked his stay using OpenPaths. I especially enjoyed how he was able to segment his means of transportation. If you’re interested in maps I suggest take a look at his great checklist for making geodata visualizations and this list of geodata tools.
What Time of Day Do People Run by Data @ Runkeeper. As a runner I can’t get enough of these visualizations and data analyses.