How to Download Minute-by-Minute Fitbit Data

IntradayDataChart

Earlier this week we posted an update to our How To instructions for downloading your Fitbit data to Google Spreadsheets. This has been one of our most popular posts over the past few years. One of the most common requests we’ve received is to publish a guide to help people download and store their minute-by-minute level step and activity data. Today we’re happy to finally get that up.

The ability to access and download the minute-by-minute level (what Fitbit calls “intraday”) data requires one more step than what we’ve covered previously for downloading your daily aggregate data. Access to the intraday data is restricted to individuals and developers with access to the “Partner API.” In order to use the Partner API you must email the API team at Fitbit to request access and let them know what you intend to do with that data. Please note that they appear to encourage and welcome these type of requests. From their developer documentation:

Fitbit is very supportive of non-profit research and personal projects. Commercial applications require additional review and are subject to additional requirements. To request access, email api at fitbit.com.

In the video and instructions below I’ll walk you through setting up and using the Intraday Script to access and download your minute-by-minute Fitbit Data.

  1. Set up your FitBit Developer account and register an app.
    • Go to dev.fitbit.com and sign in using your FitBit credentials.
    • Click on the “Register an App” at the top right corner of the page.
    • Fill in your application information. You can call it whatever you want.
    • Make sure to click “Browser” for the Application Type and “Read Only” for the Default Access type fields.
    • Read the terms of service and if you agree check the box and click “Register.”
  2. Request Access to the Partner API
    • Email the API team at Fitbit
    • They should email you back within a day or two with  response
  3. Copy the API keys for the app you registered in Step 1
    • Go to dev.fitbit.com and sign in using your FitBit credentials.
    • Click on “Manage My Apps” at the top right corner of the page
    • Click on the app you created in Step 1
    • Copy the Consumer Key.
    • Copy the Consumer Secret.
    • You can save these to a text file, but they are also available anytime you return to dev.fitbit.com by clicking on the “Manage my Apps” tab.
  4. Set up your Google spreadsheet and script
    • Open your Google Drive
    • Create a new google spreadsheet.
    • Go to Tools->Script editor
    • Download this script, copy it’s contents, and paste into the script editor window. Make sure to delete all text in the editor before pasting. You can then follow along with the instructions below.
    • Select “renderConfigurationDialog” in the Run drop down menu. Click run (the right facing triangle).
    • Authorize the script to interact with your spreadsheet.
    • Navigate to the spreadsheet. You will see an open a dialog box in your spreadsheet.
    • In that dialog paste the Consumer Key and Consumer Secret that you copied from your application on dev.fitbit.com. Click “Save”
    • Navigate back to the scrip editor window.
    • Select “authorize” in the Run drop down menu. Click run (the right facing triangle).
    • Select “authorize” in the Run drop down menu. This will open a dialog box in your spreadsheet. Click yes.
    • A new browser window will open and ask you to authorize the application to look at your Fitbit data. Click allow to authorize the spreadsheet script.
  5. Download your Fitbit Data
    • Go back to your script editor window.
    • Edit the DateBegin and DateEnd variables with the date period you’d like to download. Remember, this script will only allow 3 to 4 days to be downloaded at a time. 
    • Select “refreshTimeSeries” in the Run drop down menu. Click run (the right facing triangle).
    • Your data should be populating the spreadsheet!

If you’re a developer or have scripting skills we welcome your help improving this intraday data script. Feel free to check out the repo on Github!

Posted in Lab Notes, Personal Projects | Tagged , , , , , , , | 1 Comment

Paul LaFontaine: Upset Every Other Minute

How many times during the course of the day do you find your mental state drifting into negativity, feeling like you’re lost, or just plain stressed? How could you even keep track of this, and why would you want to?

PaulLF_upsets

What Did Paul Do?
Paul LaFontaine has been tracking what he calls “upsets” to better understand himself, the way he works, and to see if he can improve his mental and physiological response and recovery.

Upsets are something physiological that were happening beneath the surface, and they’re trackable. It didn’t have to be emotional, but there had to be a signal. This project is part of an longer ongoing study. Before this current iteration I manually logged over 3,000 upsets and what I found is that most of my upsets were self-induced. I’d be in a calm environment, but then become upset about something. I wanted to use technology because I was afraid of bias and I know I was missing some upsets.

How Did He Do It?

I used the HeartMath EMWave2 that measures heart rate variability and indicates when you’re in and out of coherence. When I was out of coherence I captured that as an upset. I would stop what I was doing and use an audio recorder to keep track of the time, how long I was upset, the reason, and what method I used to recover. I tracked 71 sessions (each session was 25-45 minutes) totaling 42 hours of tracking time. I logged 1292 upsets during this period.

What Did He Learn?
Paul analyzed his data and found some very interesting insights about his upsets, his reasons for being upset, and the effectiveness of his recovery techniques.

I found that I was triggering an upset every 2 minutes. My wife said something must be wrong with me, but this stayed relatively constant through the tracking period. I started to think of it like skiing a mogul course. The moguls didn’t move, it was about how effective I could move through them. And, dealing with upsets is like playing whack-a-mole. They come fast and furious and every second counts.

For recovery I was able to find that my most effective technique was breathing. By returning to six breaths per minute routine I was able to improve recovery time from 33 seconds to 17.8 seconds. It was the primary way I could remove myself from being upset and make myself calmer.

We want to thank Paul for presenting this great QS project at the Bay Area QS Meetup group. Make sure to watch the full talk below to learn more about Paul’s methods and findings, then hop over to his website where you can read about how he tracked his stress during this talk.

Posted in Videos | Tagged , , , , , | Leave a comment

Downloading Fitbit Data: Update

We’re posting a quick note today to let you know that we’ve updated our “How To Download Your Fitbit Data” post. It now included separate instructions for both the old and new versions of Google Spreadsheets. This is just the first in a series of planned updates. We hope to post additional updates to allow you to have deeper access to your Fitbit data including, heart rate, blood pressure, and daily goal data.

If you’re using this how-to we’d love to hear from you! Are you learning something new? Making interesting data visualizations? Discussing the data with your health care team? Let us know. You can email us or post here in the comments.

ERFitbit_092214

Click to view the interactive version.

Posted in Lab Notes | Tagged , , , | 2 Comments

Meetups This Week

This week will see eight meetups in three countries on three continents and will show the different forms that Quantified Self meetups take. There will be longstanding groups like New York, who are putting on their 26th meetup, while Miami will be hosting their very first. London will be convening over 100 people with formal presentations, while Ashland will have an engaging discussion within a small group.

Show&Tell presentations will include Vienna’s Leonard Tulipan on how he applied his startup’s classification of “vanity metrics” vs. “actionable metrics” to his own medical data. Oxford member John Courouble will talk about the data around his weight loss plateau.

In addition to the standard show&tell format, Sydney will have breakout groups, while the Minnesota group will discuss the tools that their community is using. The Vienna and Portland groups will have guest speakers, including Vestigen, a company that tracks blood biomarkers for disease and Zenobase, which helps people aggregate and visualize their data, respectively.

If you are near any of these special events, you will definitely want to attend. To see when the next meetup in your area is, check the full list of the over 100 QS meetup groups in the right sidebar. Don’t see one near you? Why not start your own!

Monday (September 22)
London, England
Sydney, Australia

Tuesday
(September 23)
Ashland, Oregon
Miami, Florida
Minneapolis, Minnesota
Vienna, Austria

Wednesday (September 24)
New York City, New York
Oxford, England

Thursday (September 25)
Portland, Oregon

Posted in Meetups | Tagged , | Leave a comment

What We Are Reading

We hope you enjoy this week’s list!

Articles
Big Data in the 1800s in surgical science: A social history of early large data set development in urologic surgery in Paris and Glasgow by Dennis J Mazur. An amazing and profoundly interesting research paper tracing the use of “large numbers” in medical science. Who knew that is all began with bladder stones!

Civil Rights, Big Data, and our Algorithmic Future by Aaron Rieke, David Robinson and Harlan Yu. A very thorough and thoughtful report on the role of data in civil and social rights issues. The report focuses on four areas: Financial Inclusion, Jobs, Criminal Justice, and Government Data Collection and Use.

Caution in the Age of the Quantified Self by J. Travis Smith. If you’ve been following the story of self-tracking, data privacy, and data sharing this article won’t be all that surprising. Still, I can’t help but read with fascination the reiteration of tracking fears, primarily a fear of higher insurance premiums.

Patient Access And Control: The Future Of Chronic Disease Management? by Dr. Kaveh Safavi. This article is focused on providing and improving access and control of medical records for patients, but it’s only a small mental leap to take the arguments here and apply them all our personal data. (Editors note: If you haven’t already, we invite you to take some time and read our report: Access Matters.)

Perspectives of Patients with Type 1 or Insulin-Treated Type 2 Diabetes on Self-Monitoring of Blood Glucose: A Qualitative Study by Johanna Hortensius, Marijke Kars, and Willem Wierenga, et al. Whether or not you have experience with diabetes you should spend some time reading about first hand experiences with self-monitoring. Enlightening and powerful insights within.

Show&Tell
Building a Sleep Tracker for Your Dog Using Tessel and Twilio by Ricky Robinett. Okay, maybe not strictly a show&tell here, but this was too fun not to share. Please, if you try this report back to us!

Digging Into my Diet and Fitness Data with JMP by Shannon Conners, PhD. Shannon is a software development manager at JMP, a statical software company. In this post she describes her struggle with her weight and her experience with using a BodyMedia Fit to track her activity and diet for four years. Make sure to take some time to check out her amazing poster linked below!

Visualizations
The following two visualizations are part of Shannon Conners’ excellent poster detailing her analysis of data derived from almost four years of tracking (December 2010 through July 2014). The poster is just excellent and these two visualizations do not do it justice. Take some time to explore it in detail!

SC_calorieweight

SC_sleep

Tracking Energy use at home by reddit user mackstann.

EnergyApp

“The colors on the calendar represent the weather, and the circles represent how much power was used that day. The three upper charts are real-time power usage charts, over three different time spans. I use a Raspberry Pi and an infrared sensor that is taped onto my electric meter. The code is on github but it’s not quite up to date (I work on it in bits and pieces as time permits I have kids).”

From the Forum
Help With Livestrong Data Export
Need Help Deciding Which Device
New to Fitness Tracking

Posted in What We're Reading | Tagged , , , , , , , , | Leave a comment

Welcome Steven Jonas

skjonas2

Today we are excited and honored to announce that Steven Jonas has joined QS Labs as our Senior Editor/Information Architect. As has been the case with previous additions to QS Labs, we welcome Steven as a friend and fellow community member. Steven serves as a co-organizer of the Portland QS meetup group, and has participated as our speaker coordinator for our past two conferences.

In addition to his work supporting our global QS community, Steven is an active self-tracker, having engaged in many different projects. We’ve been delighted to highlight a few of those here on the QS website. We invite you to welcome Steven and get to know him a bit by exploring the posts linked below.

Memorizing my Daybook
Tracking Stress
Stress Out Loud

Photo by Mark Krynsky

Posted in Lab Notes | Tagged , , , | 1 Comment

James Norris: A Life of Firsts

Like many of us, James Norris remembers his first kiss. Unlike many of us, he also knows who it was with, where it was, and his age. How does he know this information? When he was 13, he realized that he forgot some detail about his life that he thought was important. To prevent that from happening again, he decided to carry around sticky notes to record important life events and has been doing it ever since. Fast forward 15 years and James has recorded 1,500 “firsts.” Watch this talk, presented at the Washington DC QS meetup group, to hear James talk about the data he collects, and the lessons he’s learned along the way.

Posted in Videos | Tagged , , , | 1 Comment

Visualizing Our Quantified Self

At our 2013 Quantified Self Global Conference we were excited to share a variety of beautiful and insightful data visualizations from our community. In the months leading up to the conference we asked attendees to send in their own personal data visualizations along with a short description. In our 6 years of hosting Quantified Self meetups and events, as well as running this website, our forum, and social channels, we’ve seen the power of data visualization as a story telling medium. We exist in part to help people tell their stories – about the data they collect, the changes they create, and the insights and new knowledge they’re excited to share.

Today we’re sharing a few of our favorite visualizations from past conferences. The images and descriptions below represent a wide a variety of tracking experiences and techniques, and we hope to showcase eve more unique personal data projects at our upcoming QS15 Conference & Exposition.

Tracking Sleep by Anita Lillie

This is concatenation of screenshots from my sleep app. Most sleep apps don’t let you zoom out like this and still see daily/nightly detail, so I just made it myself. I like that it shows how almost-consistent I am with my sleep, and made me ask new questions about the “shape” of a night of sleep for me.



2.5 Years of My Weight by Mette Dyhrberg

I gained a lot of insights from this heat map. The most obvious weight gain was no surprise — that’s when I periodically don’t track. In any case, the big picture patterns are easily identified with a heat map. Realized looking at this heat map that the point of no return was mid-April 2012 — my data shows that was when I switched protein shakes with an egg based breakfast. I have since experimented and seen that protein shake in the morning seems to keep my blood sugar more stable and as a result my weight under control!



One Month of Blood Sugar by Doug Kanter

This is a visualization of one month of my blood sugar readings from October 2012. I see that my control was generally good, with high blood sugars happening most often around midnight (at the top of the circle).



Tracking Productivity by Nick Winter

My percentile feedback graph of my development productivity helps my motivation.



Six Months of My Life by David El Achkar

This is my life during the past six months. Each square = 15 minutes. Each column = 1 day. This picture represents 138 days or 3,000+ activities.



My Thesis Self Portrait by Sara M. Watson

Here’s a period of a few days of webcam images taken using Stan James’ LifeSlice during the final days of editing my thesis on Quantified Self uses of personal data. Serious business!



Sleep and Meaningful Work by Robby Macdonell

In an average work day, I don’t consider communication (email, instant message, etc) to be terribly meaningful work. I’d much rather be working on building software. Getting more sleep the night before increases the amount of meaningful work I’m likely to do in a day.



70 Days of Pulse by Laurie Frick

Pulse rate over 24 hours for 70 days from my Basis watch. Grey=null, blues=85

Posted in Conference, QS Gallery | Tagged , , , , , | 2 Comments

Kouris Kalligas: Analyzing My Weight and Sleep

Like anyone who has ever been bombarded with magazine headlines in a grocery store checkout line, Kouris Kalligas had a few assumptions about how to reduce his weight and improve his sleep. Instead of taking someone’s word for it, he looked to his own data to see if these assumptions were true. After building up months of data from his wireless scale, diet tracking application, activity tracking devices, and sleep app he spent time inputing that data into Excel to find out if there were any significant correlations. What he found out was surprising and eye-opening.

This video is a great example of our user-driven program at our Quantified Self Conferences. If you’re interest in tell your own self-tracking story, or want to hear real examples of how people use data in their lives we invite you to register for the QS15 Conference & Exposition.

Posted in Conference, Videos | Tagged , , , , , , , , , | 3 Comments

Meetups This Week

As the summer begins to close, QS groups are getting together all over to talk and share their experiences. During this next week, there are 8 groups getting together in four different countries. To see when the next meetup in your area is, check the full list of the over 100 QS meetup groups in the right sidebar. Don’t see one near you? Why not start your own!

Monday (September 15)
Austin, Texas
This meetup will feature fascinating talks on baby measurements and turning tracking data into art.

Tuesday (September 16)
Dublin, Ireland
All of the talks at this meetup will have a sports theme.

Wednesday (September 17)
Cincinnati, Ohio
The Cincinnati group will host a couple of great guest speakers who will talk about heart rate variability and wearable application development.

Raleigh, North Carolina
A nice, low-key meetup for the group in Raleigh.

Dallas/Fort Worth, Texas
A group of trackers in the North Texas area will be getting together to talk about their self-tracking.

Thursday (September 18)
Groningen, Netherlands
This should be an intriguing meetup with talks on pH values, measuring physical load, and QS in health care.

Reno, Nevada
Reno is a brand new group and this meetup should be an auspicious start.

Manchester, England
The Manchester group will be getting together for a splendid conversation about QS.

Posted in Meetups | Tagged , | Leave a comment