Topic Archives: Lab Notes
MyFitnessPal is one of the leading dietary tracking tools, currently used by tens of millions of people all around the world to better track and understand the foods they consume every day. Their mobile apps and online tools allow individuals to enter foods and keep track of their micro- and macro-nutrient consumption, connect additional devices such as fitness trackers, and connect with their community – all in the name of weight management. However, there is no natively available method for easily accessing your dietary data for personal analysis, visualization, or storage.
With a bit of digging in the MyFitnessPal help section we can see that they have no official support for data export. However, they mention the ability to print reports and save PDF files that contain your historical data. While better than some services, a PDF document is far from easy to use when you’re trying to make your own charts or take a deeper look into your data.
We spent some time combing the web for examples of MyFitnessPal data export solutions over the last few days. We hope that some of these are useful to you in your ongoing self-tracking experiences.
MyFitnessPal Data Downloader: This extension allows you to directly download a CSV report from your Food Report page. (Chrome only)
MyFitnessPal Data Export: This extension is tied to another website, FoodFastFit.com. If you install the extension, it will redirect you back to that site where your data is displayed and you can download the CSV file. (Chrome only)
ExportMFP: A simple bookmark that will open a text area with comma-separated values for weight and calories, which you can copy/paste into your data editor of choice.
MyFitnessPal Reports: A bookmarklet that allows you to generates more detailed graphs and reports.
MyFitnessPal Analyser: Accesses your diet and weight data. It requires you to input your password so be careful.
Export MyFitnessPal Data to CSV: Simple web tool for exporting your data.
FreeMyDiary: A recently developed tool for exporting your food diary data.
MyFitnessPal Data Access via Python: If you’re comfortable working with the Python language, this might be for you. Developed by Adam Coddington, it allows access to your MyFitnessPal data programmatically
MFP Extractor and Trend Watcher: An Excel Macro, developed by a MyFitnessPal user, that exports your dietary and weight data into Excel. This will only work for Windows users.
Access MyFitnessPal Data in R: If you’re familiar with R, then this might work for you.
QS Access + Apple HealthKit
If you’re an iPhone user, you can connect MyFitnessPal to Apple’s HealthKit app to view your MyFitnessPal data alongside other data you’re collecting. You can also easily export the data from your Health app using our QS Access app. Data is available in hourly and daily breakdowns, and you should be able to export any data type MyFitnessPal is collecting to HealthKit.
Two weeks ago we announced the release of the QS Access App so you could access your HealthKit data in tabular format for personal exploration, visualization, and analysis. In that short period of time, we’ve seen a good number of downloads and positive feedback.
We know from our experiences hosting in-person and online communication about personal data that seeing real-world examples of what is possible is what inspires people to engage and ask questions of their own data. With that in mind we’re excited to announce our QS Access Visualization Showcase.
We are looking to you, our amazing community of trackers, designers, and visualizers, to show use what you can do with data gathered from using the QS Access App. Make heatmaps in D3, complete analyses and visualizations in Wizard, or just make meaningful charts in Excel. If you’re visualizing your QS Access data we want to see it.
We also know that data visualization design and creation is not trivial work. To support the community and help expose the visualization work we’ll be awarding free tickets to our QS15 Global Conference & Exposition to individuals who use QS Access to create unique and interesting visualizations. We’ve earmarked two tickets (a $700 value) for outstanding work. If you’re selected, we’ll also work with you to showcase your work at the QS15 Conference and Exposition so other community members and attendees can explore and learn from their own data.
If you’re in the Bay Area come to our QS Meetup on November 11th at the Berkeley Skydeck. You can showcase your visualization and tell our community what you’ve learned from accessing and visualizing your data.
HealthKit is still new and the number of apps that integrate with it is growing by the day. At QS Labs we’ve done a bit of work making simple visualizations that are meaningful to us.
Steps and Sedentary Activity
Gary has an iPhone 5s which has native step tracking. We used the QS Access app to export his hourly step totals and made these simple line graphs in Excel. You can read more about what he learned from these simple data visualizations here.
How Much Do I Run?
Ernesto is an avid runner and enjoys running along the quiet trails in Los Angeles. He was interested to see how often he actually runs and if there’s any pattern to his running. Using a well-designed D3 template he was able to make a calendar heatmatp of his running distance.
If you don’t have any HealthKit data to work with, or just want to play with some example data we’ve created a few files that you can use as examples. Download the files below from our GitHub account and make sure to read the documentation to understand where the data is coming from. Descriptions of the data files and sources are available in our QS Access Data Examples repo on Github.
On Wednesday this week we learned that the QS Access app we submitted to the Apple store was approved. This means you can download the QS Access app on iTunes. We hope you’ll find it useful. Our app is a very simple tool for accessing HealthKit data in a table so that you can explore it using Numbers, Excel, R, or any other CSV compatible tool.
It is still early days for HealthKit, but my conversations with toolmakers at Quantified Self events convinces me that there will be many device and software makers that integrate with Apple’s platform for collecting and analyzing personal data. I hope this will allow more people to learn from their own data by reflecting on changes over time and by combining multiple data streams – such as activity, sleep, and nutrition – into a single visualization for comparison.
To give you your HealthKit data in tabular format, we’ve had to simplify it. QS Access shows your data in either “hourly” or “daily” chunks. These won’t be appropriate for all uses, but many interesting questions can be asked of data that is presented as a time series using hourly and daily values. This is just a starting point, and we’re looking forward to making it do more based on your feedback.
We very much hope that if you learn something from your data using QS Access, you’ll share your project by participating in a Quantified Self Show&Tell meetup and by joining us at QS15 Conference and Exposition next year in San Francisco. Suggestions about the app itself and interesting examples of usage can be shared with us directly by emailing us: firstname.lastname@example.org,
The QS Access App was authored by our long time QS Labs friend and collaborator Robin Barooah.
NOTE: The methods detailed in this post no longer work. The post is kept here for archival purposes, but unfortunately you will not be able to access your Fitbit data by following the steps.
Earlier this week we posted an update to our How To instructions for downloading your Fitbit data to Google Spreadsheets. This has been one of our most popular posts over the past few years. One of the most common requests we’ve received is to publish a guide to help people download and store their minute-by-minute level step and activity data. Today we’re happy to finally get that up.
The ability to access and download the minute-by-minute level (what Fitbit calls “intraday”) data requires one more step than what we’ve covered previously for downloading your daily aggregate data. Access to the intraday data is restricted to individuals and developers with access to the “Partner API.” In order to use the Partner API you must email the API team at Fitbit to request access and let them know what you intend to do with that data. Please note that they appear to encourage and welcome these type of requests. From their developer documentation:
Fitbit is very supportive of non-profit research and personal projects. Commercial applications require additional review and are subject to additional requirements. To request access, email api at fitbit.com.
In the video and instructions below I’ll walk you through setting up and using the Intraday Script to access and download your minute-by-minute Fitbit Data.
- Set up your FitBit Developer account and register an app.
- Go to dev.fitbit.com and sign in using your FitBit credentials.
- Click on the “Register an App” at the top right corner of the page.
- Fill in your application information. You can call it whatever you want.
- Make sure to click “Browser” for the Application Type and “Read Only” for the Default Access type fields.
- Read the terms of service and if you agree check the box and click “Register.”
- Request Access to the Partner API
- Email the API team at Fitbit
- They should email you back within a day or two with response
- Copy the API keys for the app you registered in Step 1
- Go to dev.fitbit.com and sign in using your FitBit credentials.
- Click on “Manage My Apps” at the top right corner of the page
- Click on the app you created in Step 1
- Copy the Consumer Key.
- Copy the Consumer Secret.
- You can save these to a text file, but they are also available anytime you return to dev.fitbit.com by clicking on the “Manage my Apps” tab.
- Set up your Google spreadsheet and script
- Open your Google Drive
- Create a new google spreadsheet.
- Go to Tools->Script editor
- Download this script, copy it’s contents, and paste into the script editor window. Make sure to delete all text in the editor before pasting. You can then follow along with the instructions below.
- Select “renderConfigurationDialog” in the Run drop down menu. Click run (the right facing triangle).
- Authorize the script to interact with your spreadsheet.
- Navigate to the spreadsheet. You will see an open a dialog box in your spreadsheet.
- In that dialog paste the Consumer Key and Consumer Secret that you copied from your application on dev.fitbit.com. Click “Save”
- Navigate back to the scrip editor window.
- Select “authorize” in the Run drop down menu. Click run (the right facing triangle).
- Select “authorize” in the Run drop down menu. This will open a dialog box in your spreadsheet. Click yes.
- A new browser window will open and ask you to authorize the application to look at your Fitbit data. Click allow to authorize the spreadsheet script.
- Download your Fitbit Data
- Go back to your script editor window.
- Edit the DateBegin and DateEnd variables with the date period you’d like to download. Remember, this script will only allow 3 to 4 days to be downloaded at a time.
- Select “refreshTimeSeries” in the Run drop down menu. Click run (the right facing triangle).
- Your data should be populating the spreadsheet!
If you’re a developer or have scripting skills we welcome your help improving this intraday data script. Feel free to check out the repo on Github!
We’re posting a quick note today to let you know that we’ve updated our “How To Download Your Fitbit Data” post. It now included separate instructions for both the old and new versions of Google Spreadsheets. This is just the first in a series of planned updates. We hope to post additional updates to allow you to have deeper access to your Fitbit data including, heart rate, blood pressure, and daily goal data.
If you’re using this how-to we’d love to hear from you! Are you learning something new? Making interesting data visualizations? Discussing the data with your health care team? Let us know. You can email us or post here in the comments.
Today we are excited and honored to announce that Steven Jonas has joined QS Labs as our Senior Editor/Information Architect. As has been the case with previous additions to QS Labs, we welcome Steven as a friend and fellow community member. Steven serves as a co-organizer of the Portland QS meetup group, and has participated as our speaker coordinator for our past two conferences.
In addition to his work supporting our global QS community, Steven is an active self-tracker, having engaged in many different projects. We’ve been delighted to highlight a few of those here on the QS website. We invite you to welcome Steven and get to know him a bit by exploring the posts linked below.
Photo by Mark Krynsky
Someday, you will have a question about yourself that impels you to take a look at some of your own data. It may be data about your activity, your spending at the grocery store, what medicines you’ve taken, where you’ve driven your car. And when you go to access your data, to analyze it or share it with somebody who can help you think about it, you’ll discover…
Now is the time to work hard to insure that the data we collect about ourselves using any kind of commercial, noncommercial, medical, or social service ought to be accessible to ourselves, as well as to our families, caregivers, and collaborators, in common formats using convenient protocols. In service to this aim, we’ve decided to work on a campaign for access, dedicated to helping people who are seeking access to their data by telling their stories and organizing in their support. Although QS Labs is a very small organization, we hope that our contribution, combined with the work of many others, will eventually make data access an acknowledged right.
The inspiration for this work comes from the pioneering self-trackers and access advocates who joined us last April in San Diego for a “QS Public Health Symposium.” Thanks to funding support from the Robert Wood Johnson Foundation, and program support from the US Department of Health And Human Services, Office of the CTO, and The Qualcomm Institute at Calit2, we convened 100 researchers, QS toolmakers, policy makers, and science leaders to discuss how to improve access to self-collected data for personal and public benefit. During our year-long investigation leading up to the meeting, we learned to see the connection between data access and public health research in a new light.
If yesterday’s research subjects were production factors in a scientist’s workshop; and if today’s participants are – ideally – fully informed volunteers with interests worthy of protection; then, the spread of self-tracking tools and practices opens the possibility of a new type of relationship in which research participants contribute valuable craft knowledge, vital personal questions, and intellectual leadership along with their data.
We have shared our lessons from this symposium in a full, in-depth report from the symposium, including links to videos of all the talks, and a list of attendees. We hope you find it useful. In particular, we hope you will share your own access story. Have you tried to use your personal data for personal reasons and faced access barriers? We want to hear about it.
You can tweet using the hashtag #qsaccess, send an email to email@example.com, or post to your own blog and send us a link. We want to hear from you.
The key finding in our report is that the solution to access to self-collected data for personal and public benefit hinges on individual access to our own data. The ability to download, copy, transfer, and store our own data allows us to initiate collaboration with peers, caregivers, and researchers on a voluntary and equitable basis. We recognize that access means more than merely “having a copy” of our data. Skills, resources, and access to knowledge are also important. But without individual access, we can’t even begin. Let’s get started now.
An extract from the QSPH symposium report:
[A]ccess means more than simply being able to acquire a copy of relevant data sets. The purpose of access to data is to learn. When researchers and self-trackers think about self-collected data, they interpret access to mean “Can the data be used in my own context?” Self-collected data will change public health research because it ties science to the personal context in which the data originates. Public health research will change self-tracking practices by connecting personal questions to civic concerns and by offering novel techniques of analysis and understanding. Researchers using self-collected data, and self-trackers collaborating with researchers, are engaged in a new kind of skillful practice that blurs the line between scientists and participants… and improving access to self-collected data for personal and public benefit means broadly advancing this practice.
Quantified Self Labs is wholeheartedly dedicated to creating conferences and events that are safe and comfortable for everyone involved. This means providing a harassment-free experience for every participant, regardless of gender identity and expression, sexual orientation, ability, ethnicity, socioeconomic status, physical appearance, and beliefs.
Providing this experience for our members requires a concrete, visible commitment to the community. We introduced our first anti-harassment policy for the Quantified Self Global Conference in 2013. Quantified Self Boston has followed suit and introduced a Code of Conduct for all their events in April, 2014. Our Bay Area QS meetup has recently implemented this Code of Conduct as well. Meetup members from the Quantified Self Community have also started three women’s* meetups (QSXX San Francisco, QSXX Boston, QSXX New York City) to promote inclusion and safe spaces for sharing.
If you are involved in Quantified Self as an organizer or meetup member we encourage you to read our anti-harassment policy and the QS Boston Code of Conduct and use them as a basis for your own work in creating events that are welcoming to all.
Editor’s Note: Thanks to our friends Amelia Greenhall and Maggie Delano for their leadership. For more analysis and practical resources in fighting sexism in technology culture, see: The Ada Initiative.
* All women and people who identify or have identified as significantly non-male are welcome at QSXX, including those with non-binary and/or trans* gender identities.
My friend Seth Roberts, pioneering self-experimenter and personal scientist, died last Saturday. Seth’s sister Amy, made the announcement yesterday on his blog. The news was unexpected and very sad. A few things Seth taught me:
- Doing lots of experiments keeps you supplied with new ideas.
- With sensitive and reliable measurements, tiny experimental effects can yield surprising clues.
- The people who care about a problem most have the best chance of solving it, if they have access to tools.
Seth’s contributions as a colleague and teacher had many dimensions, but in thinking about him nonstop this morning what I find myself marveling at most is the unusual style he had in nearly every conversation. Seth became interested when he saw somebody thinking independently and, like the best teachers, he wanted to understand the process by which students and collaborators developed confidence their conjectures. Countless times, I heard Seth ask somebody “Why do you think that?” His challenge was direct and generous, for if you were willing to expose your reasons you could count on him to apply himself alongside you, thinking up ways to improve your investigation, make your measurements more practical, or give your analysis more logical or mathematical power. Seth was acutely aware that confidence in experimental results requires investment. More than any other experimentalist I’ve known, Seth respected the incremental nature of building confidence in our ideas and finding new ideas worthy of confidence. He encouraged us to do small experiments first, to find easier ways to collect data, and to avoid being tripped up at the outset by grandiose schemes of irrefutability.
We’ll collect and share some more impressions and memories of our friend and collaborator when the shock of this news lessens, but in the meantime I want to post a video of one of the talks he gave a few years ago about how to design personal experiments.
For further reading:
- Seth Roberts Wikipedia page
- Seth’s faculty page at Tsinghua University (bibliography is in English)
- A spot-on remembrance by Richard Sprague
In Richard Sprague’s post from today, linked directly above, there is this passage:
I found his blog, and discovered that he was living near me in Beijing. A famous professor like him – a New York Times bestselling author and all that – might be hard to get ahold of, but one day out of the blue I sent Seth an email, wondering if he’d like to get together for lunch. He replied in minutes and said sure, how about tomorrow?
That’s Seth exactly.
In 2013, Kate, Ernesto, and I created a Friends of QS program to help fund our work at QS Labs, which includes putting on our QS conferences and events, maintaining the web site and video programming, and creating opportunities for our vibrant global community to connect. We reached out to friends in our network of self-trackers, toolmakers, inventors, and entrepreneurs, and their response was inspiring. Last year we welcomed the contributions of more than 30 organizations and individuals to our inaugural group of Friends.
As we gear up for our 2014 Quantified Self Conference in Amsterdam, we are proud to acknowledge new and returning Friends of QS. If you believe in our work and would like to support it, please consider joining. We welcome your participation. Just get in touch!
Our Friends of QS include:
|AMAX is the manufacturer of datacenter solutions for big data analytics (Hadoop) and OpenStack, including architecture based on Facebook’s Open Compute design. Passionate believers that data is the key breakthrough to a fuller understanding of this world, AMAX is looking to collaborate with startups in need of dynamic computing power.|
|Beddit produces an ultra thin film sensor that fits under the sheets on your bed and tracks your sleeping patterns, heart rate, breathing, snoring, movements and environment. No wearable sensors. In the morning, the app tells you how you slept and how to turn your data into actionable ideas for improving your sleep and wellness.|
|Beeminder is a goal-tracking tool with teeth. Connect a QS gadget or app (Fitbit, RescueTime, etc) and Beeminder plots your progress towards your goal on a Yellow Brick Road. Stay on track and Beeminder is free. Go off the road and you (literally) pay the price.|
|Calit2 is a multidisciplinary research institution jointly run by UCSD and UCI devoted to conducting cutting-edge research discovering new ways that emerging technologies can improve the state’s economy and citizens’ quality of life.|
|dacadoo is a Swiss startup with a mission to improve users’ health and fitness. The platform calculates a personal health score, a number from 1 (low) to 1’000 (high), to indicate current health status. By integrating gamification and social networking principles, dacadoo motivates participants to be active in a fun and easy way by tracking and comparing personal health, lifestyle and fitness.|
|Chris Dancy’s name and avatar are synonymous with the future of work, edutainment, technically enabled external evolution, and his quantified life (existence). Since 2007, Chris has spoken around the globe on emerging trends, and has been featured in TechCrunch, Wired magazine, Bloomberg TV, and NPR Radio as the most connected human on Earth.|
|Douglass Winthrop is an SEC registered investment advisor with offices in New York and San Francisco. The firm believes in ”self knowledge through numbers” on many fronts and manages $1 billion for individuals, families, trusts, and endowments. Douglass Winthrop is owned and managed by its principals.|
|DreamsCloud is a social platform and resource for people of all ages with an interest in exploring dreams and the unconscious mind. Through its website, mobile apps for all smart devices, and a strong presence in social media, DreamsCloud delivers entertaining and informative content, tools and resources, and a community where users can interact and explore dream meanings.|
|You were born to move. Ergo Depot’s hand selected line of height adjustable desks and ergonomic seating encourages natural positions and movement. They empower people to function more efficiently, feel better, and live longer. Ergo Depot is evolving the way humans work.|
|Fluxtream is an open-source non-profit personal data visualization framework to help you make sense of your life and compare hypotheses about what affects your well-being. Using Fluxtream, you can bring together and explore physiological, contextual, and observational data from many devices and apps on a common timeline.|
|Since 1998, Gordon Bell has been working on the MyLifeBits project with Jim Gemmell – a quest to understand how you store everything in your life in cyberspace. After QS2012 he became a “trackee” of health data using CMU’s Bodytrack holding BodyMedia, Heartrate and other data.|
|LUMO BodyTech in on a mission to give the body a voice. They aim to bring good posture and movement back into our daily lives using the latest sensor and mobile technologies, starting with their first product LUMOback.|
|Dr. Mark Drangsholt is Professor and Chair of Oral Medicine at the University of Washington. He is a passionate enthusiast and supporter of the QS movement. His focus is on strengthening the bridge between academics and QS, and science and QS.|
|Mindful Cyborgs the audio present shock on mindfulness, cyborgs, contemplative computing, bio/lifehacking and unhacking, frictionless existence, quantified self netocracy, robotics and digital duality. Hosts, Wired, TechCrunch writer, founder of Technoccult Klint Finley and Global tech fluffer and data exhaust cartographer Chris Dancy.|
|Narrative (formerly Memoto) is a Swedish startup with the goal of giving everyone a photographic memory. Their tiny camera and GPS device with no controls clips on and takes photos as you go. The App organizes them. This all works together to give you pictures of every moment of your life, complete with information on when and where you took them.|
|Naveen Selvadurai is a Quantified Self enthusiast and an internet entrepreneur and co-founder of location-based social networking site, FourSquare. He worked previously at Socialight, Sony Music and Sun Microsystems.|
|OMsignal is developing a line of bio-sensing apparel that continuously tracks your biometrics. Embedded sensors in the apparel monitor your heart rate, breathing and activity while the OMsignal app displays your data in real-time on your mobile phone. OMsignal fits seamlessly into your everyday life. Technology woven into life.|
|Open mHealth envisions a world where disparate mobile health applications can be harnessed to deliver data-driven patient stories. Collaborating with a community of developers, clinicians, researchers, and business leaders, Open mHealth is a non-profit building an open software architecture that emphasizes modularity and reusability of digital health data.|
|Project Addapp is a platform created by two guys passionate about self-tracking. It allows users to pull data from multiple different tracking tools to create experiments to see, for example, how training affects sleep or how calorie intake affects workouts.|
|The Quantified Self Institute is an experimental collaboration between the Hanze University of Applied Sciences (Groningen, the Netherlands) and QS Labs to bridge the gap between science and the QS community. It is a network of QS users/makers, researchers, students, companies and other institutions that support the mission to encourage a healthy lifestyle through technology, science and fun.|
|Aware of the importance of data tracking in improving quality and way of life, reconti is developing a platform for logging measurements and analysis related to physical activity and health improvement. The aim is to identify connections and correlations between multiple results from a variety of sources. They also collaborate with other organizations in the mHealth community.|
|RescueTime is a Seattle-based company whose tools are used by more than 600,000 people worldwide to get an accurate picture of how they spend their time each day to help them be more productive. RescueTime is launching new consumer, business, and mobile offerings in Q3 2013.|
|Rock Health s powering the future of the digital health ecosystem, bringing together the brightest minds across disciplines to build better solutions. Rock Health funds and supports startups building the next generation of technologies transforming healthcare.|
|Sen.se has created an open platform (currently in beta) called Open.Sen.se for those who want to imagine, prototype and test new devices, installations, scenarios, and applications for this globally interconnected world.|
|Sensoplex is a one-stop developer & manufacturer of Wireless Sensor Modules and customized wearable devices to OEM’s focusing on the sports, fitness & wellness markets. SensoPlex’ expertise in motion sensors, wireless and design for manufacturing minimizes OEM’s risk, cost & time to market.|
|The Stir Kinetic Desk is an automated desk that seamlessly transitions between sitting and standing positions with a simple double tap. It’s designed with the purpose to help promote an active lifestyle based on motion, health and inspiration. The Stir Kinetic Desk learns your habits and adapts itself to your routine.|
|Many health conditions are correlated with the microbiome — from asthma to diabetes, autism to depression, irritable bowel, Crohn’s, heart disease, and more. uBiome gives the public access to cutting edge DNA sequencing technology. You can find out what’s in your microbiome, and how you correlate with others in our data set and with existing studies of the microbiome.|
|Ubisoft, a leader in entertainment software, is introducing O.zen, a motivational wellness training program. O.zen uses a unique heart rate sensor to create a personalized coaching plan in order to help manage stress and increase your vitality through playful breathing exercises.|
|ViviTouch® actuators, from Bayer MaterialScience LLC, produce a cinematic sensation that bridges the senses of seeing, hearing and feeling. ViviTouch® HD Feel is used in mobile, gaming, and wearable devices to bring high definition feel to entertainment and lifestyle experiences. ViviTouch® 4D Sound makes headphones come alive by amplifying the hearing experience.|
|VSP Global® is a group of leading companies, working together to meet and exceed the needs of eyecare professionals, clients, and it’s 60 million members. VSP wants to fully support the quantified self movement and it’s members who may want to be involved to find self-empowerment.|