Topic Archives: QS Resource
In the Quantified Self community we focus on projects and ideas that help people access and get meaning out their personal data, including the information you can collect with your smartphone. If you have an iPhone, Android, or Windows phone you’re already have carrying of the world’s most sophisticated self-tracking tools. The GPS, accelerometer, the microphone, all of these tiny sensors make up a great set of tools you can use to understand how you move around the world.
I’m going to focus this short “how to” on geolocation data and mapping your movement, specifically using data gathered by the Moves application. Moves is a passive activity and location tracking tool available for the iPhone and Android. We’ve written a bit about it in the past and had a chance to interview their CEO, Sampo Karjalainen. I’ve been using it since May, 2013 and I wanted to share some neat tools and methods for getting a bit more out of the data Moves collects.
I find that visualizing my data on a map to be incredibly powerful. It might by my inner cartographer, but seeing my patterns of movement (or lack there of) in reference to known places and landmarks is a great mechanism for inducing recall and reflection on where I’ve been and what I’ve done. Hopefully you’ll use one of the tools or methods below to map you data and learn something new!
Moves Connected Apps
Like many self-tracking applications and devices, Moves has a API that many different developers have built services on top of. Here are just a few of the services that allow you to see your data on a map. Be advised that each of these services has access to your data. Make sure to read their Terms of Service before agreeing to the data transfer.
WebTrack. This is by far the most utilitarian data mapping tool. However, you shouldn’t get discouraged by the lack of fancy design because it gives you an very unique data view. When you use Moves on your phone you typically only see the “storyline” and the detected places you’ve spent time at. However, Moves is constantly pinging and recording your location when it detects movement. WebTrack allows you to see all those movement points by hovering over the associated timestamp.
Fluxtream. You might know Fluxtream as Friend of QS and a great open-source data aggregation tool. They’ve set up a “Moves Connector” that allows you to import and visualize your Moves data. Because Fluxtream is set up as an aggregation and visualization tool you can also map other interesting data sets. Want to know where you were tweeting last week? Fluxtream will map it for you. (You can see me tweeting on a CalTrain ride between San Francisco and Palo Alto below.)
Zenobase. Another interesting data aggregation service here. Zenobase treats your Moves data bit differently. Rather than importing all the movement geolocation data it focuses on your place data and visualizes those locations. I like the high-level view it start with, but make sure to keep zooming in to see more specific place data.
Resvan Maps. This mapping application adds a unique twist to the typical mapping visualizations. It will plot your places, paths, and categorize paths depending on the activity (transport, walking, running, and cycling). Additionally, you can create “analysis cirlces” and have the application compute the time you spent in a certain location you bound (it aggregates to hours:minutes per day).
MMapper. This method for mapping your data, developed by Nicholas Felton, is by far the most technical, but it produces some really neat visualizations. You’ll have to download Processing and follow the instructions Nicholas provides on the Github repository page here. The great thing here is that the mapping and data access is all happening locally.
Move-O-Scope. Another great mapping application here from the folks at Halftone.co. They’ve probably completed the most thorough mapping and exploration tools for your Moves data. After linking your Moves account you can explore maps by activity type, day of the week, and custom data ranges. Additionally, they’ve implemented a neat feature for exploring place data. You can see how many times you’ve visited a specific place, where you’ve come from and where you go next, what days you typically visit, and your typical time of day at that place. (See this post for background on why they created this nifty tool.)
Map It Yourself!
If you don’t want to trust your data to a third party, but you still want to explore your movement maps there is really great option for you. Our friend and co-organizer of the QS LA Meetup, Eric Blue, recently published a method for easily exporting your data: the Moves CSV Exporter. You’ll have to login and use the Moves pin system in order to download your data, but Traqs isn’t storing your data, just providing a way for you to access it. The tool allows you to download and explore your activity, summary, tracks and place data. We’ll focus on the place data for creating maps. You can also use your full tracks history for mapping all the geolocation points Moves collects.
Because this data is based on latitude/longitude coordinates there are many different methods available that you can use to map your data. I’m going to focus on two here: Google Fusion Tables and CartoDB (if you know of others share them in the comments or our forum).
Google Fusion Tables
Fusion Tables are a new Google Drive tool that you can use to store, analyze, and visualize many different types of data. Once you download your Moves places.csv file you can upload it to a new Google Fusion Table. Once you upload your data, which takes about 2 minutes, you’ll see a menu bar and three tabs: Rows, Cards, Map of longitude. Just click on the “Map” tab and you’ll see your data already placed on a map. If you want to see a heatmap rather than a point map just navigate to Tools -> Change Map and you’ll see an option for a heatmap on the lefthand side. This is just the tip of iceberg for mapping fusion table data. You can learn more about different mapping methods and tricks here.
CartoDB is a visualization and analysis engine for geospatial data. I’ve been using it to play around with a few of the different geolocation datasets that I have (I actively keep three). Although it is paid service, they do offer a free plan for smaller datasets, which is perfect for your Moves data. Again, you’ll have to upload your places.csv file to a new table once you set up your account. Once the data is uploaded there are quite a few different map visualization wizards you can use to view your data in different ways. Pesonaly I like playing with the “Torque” visualization that gives you a real feeling of space-time to your data.
TileMill is an interactive map design tool from the folks over at Mapbox. If you’re looking to create custom maps with your data that you can format, style, and share then this is a wonderful tool to use. At first glance it’s a little daunting because it looks like a mashup of a CSS editor and map tool. That actually gives it the unique power to drive customization. Don’t be afraid, it’s not too hard to get started with. Mapbox has provided a great “crashcourse” to get you started with importing data, saving it as a new layer on your map, and then manipulating how it looks on your screen. If you want to go just a bit farther you can also add legends and informative popups to describe your data points. Mapbox also offers a free hosting plan if you want to share your interactive maps on a webpage. For example check out my MovesMap here, where I added a quick styling to manipulate the point size in relation to the time spent at a location.
Hopefully you’ve learned something new from this. If you map your Moves data (or any other geolocation data) we want to see it! Leave a link in the comments, post it in the location mapping thread on the QS Forum or get in touch on twitter!
Update: Want to make your own Sparktweet? We made a simple tool that you can use. Check it out here!
I was stumbling around Twitter the other day when I was confronted with something new and different:
— Steve Cavendish (@scavendish) April 5, 2013
Apparently that little data representation is not all that new and different. Way back in 2010 Alex Kerin figured out that Twitter was accepting unicode and decide to play around and see if it could represent data. Lo and behold it could and a SparkTweet was born:
▁▁▂▂▃▄▄█▁▁▂ ▃▄▄▅▆▁▁▂▂▃▄▄▅▆▁▁▂▂▃▄▄▅▆ Can you guess what I’m coding in Excel? Eh? Eh?
— Alex Kerin (@AlexKerin) June 9, 2010
Before we get into how you too can start populating your Twitter feed and Facebook (I checked and it worked there as well) with representations of your own Quantified Self data let’s dive into some history.
a small intense, simple, word-sized graphic with typographic resolution. Sparklines mean that graphics are no longer cartoonish special occasions with captions and boxes, but rather sparkline graphic can be everywhere a word or number can be: embedded in a sentence, table, headline, map, spreadsheet, graphic.
In another wonderful book, The Visual Display of Quantitative Information, Tufte describes sparklines as “datawords: data-intense, design-simple, word-sized graphics.“ Of course, those of us in the QS community are deeply interested not only in data, but also in how data operates in society, what is means as a cultural artifact that is discussed and exchanged in language both written and verbal. This interest iswhat initially piqued my curiosity. The movement of data and a dataword distributed among text and publicly expressed in a tweet. I can’t help but wonder, what does this mean for how we think about and express data about our world?*
If you want display quantitative data in your Twitter stream it shouldn’t take you all that long to get started. Lucky for us Alex Kerin has provided a nifty little Excel workbook that will generate the unicode that can be pasted into your tweet. Just download this workbook and follow the simple instructions! Soon you’ll be able to send out tweets just like this:
My 30-day step history: ▄ ▄ ▄ ▅ ▅ ▅ ▄ ▆ ▄ █ █ ▅ ▁ ▃ ▆ ▅ ▁ ▄ ▇ ▃ ▅ ▆ ▂ ▂ ▅ ▃ ▄ ▄ ▅ ▄ #QuantifiedSelf
— Ernesto Ramirez (@eramirez) April 11, 2013
Now you’re ready and able to go forth and tweet your data! If you use a sparktweet to express your Quantified Self data be sure to let us know in the comments or tweet at us with #sparktweet and/or #quantifiedself.
*Of course the use of sparktweets is not without controversy in the world of data visualization. For more discussion on sparktweets and their utility I suggest you start here.
Editors Note: We’ve updated this post to reflect Google’s move to a new version of their spreadsheet application. The newest version no longer support the Script Gallery mentioned here. We have included a link in the instruction steps below that allows you to use the old version of Google Spreadsheets. We’ll keep an eye out and update this post again if the old version is taken down. We’ve also included a new set of instructions if you’d like to use the new Google Spreadsheets. This involves slight editing to a simple script (updated 09/22/14).
Interested in downloading your minute-by-minute Fitbit data? Check out our new how-to here! (updated 09/26/14)
We’ve updated the script to reflect Fitbit’s move to only accept HTTPS requests to access their API. Make sure to update your own scripts if you’ve modified the one linked below in Step 4 (updated 10/15/14).
If you’re like me, then you’re always looking for new ways to learn about yourself through the data you collect. As a long time Fitbit user I’m always drawn back to my data in order to understand my own physical activity patterns. Last year we showed you how to access your Fitbit data in a Google spreadsheet. This was by far the easiest method for people who want to use the Fitbit API, but don’t have the programming skills to write their own code. As luck would have it one of our very own QS Meetup Organizers, Mark Leavitt from QS Portland, decided to make some modifications to that script to make it even easier to get your data. In this video below I walk you through the steps necessary to setup your very own Fitbit data Google spreadsheet.
Step-by-step instructions after the jump. Continue reading
We’ve already published this QS Show&Tell talk by Mark Drangsholt about using self-tracking to identify the triggers of his heart problems, lessen their frequency, and make good decisions about treatment. I’m re-posting it here to focus on attention on the interesting and powerful method Mark used, the case-crossover design, and invite you to think about whether this has promise for your own self-tracking projects.
Mark is a professor and chair of oral medicine at the University of Washington School of Dentistry. He’s a triathlete and long time self-tracker. He is in good physical condition, but suffers from heart ailments that are frightening and dangerous. For instance, he has tachycardia (sudden acceleration of heart rate). At times his heart goes from 60 to 220 beats per minute. It feels like his heart is going to jump out of his chest. He also has atrial fibrillation, with palpitations, a feeling of immanent doom, and a sense that he is choking.
“The first time it happened in 2003 I really thought I was dying,” Mark says in his talk. He had always assumed that if he ever had a heart attack he, of all people, would know to pick up the phone and call 911, but the opposite happened. He just thought to himself “this is it,” and slumped down in his chair. Fortunately, he survived, and when he recovered he asked himself whether he could identify the triggers of these unpleasant events and avoid them. He created a simple Excel table of all episodes for one year, on which he recorded information about his attacks.
Mark is an expert on evidence based medicine, so he was naturally curious about what kind of evidence his self-tracking data contained. In standard reference material on medical evidence, students learn about a hierarchy that goes something like this:
- 1 or more randomized controlled trials
- 1 or more cohort studies
- 1 or more case-control studies
- 1 or more case-series
- expert opinion without above evidence
Mark’s self-tracking data didn’t naturally fit with any of these approaches. To understand whether these triggers actually had an effect on his arrhythmias, he used a special technique originally proposed by the epidemiologists Murray Mittleman and K. Malcolm Maclure. A case-crossover design is a scientific way to answer the question: “Was the patient doing anything unusual just before the onset of the disease?” It is a design that compares the exposure to a certain agent during the interval when the event does not occur to the exposure during the interval when the event occurs.
Using this method, Mark discovered that events linked to his attacks included high intensity exercise, afternoon caffeine, public speaking to large groups, and inadequate sleep on the previous night. While these were not surprising discoveries, it was interesting to him to be able to rigorously analyze them, and see his intuition supported by evidence.
“A citizen scientist isn’t even on the conventional evidence pyramid,” Mark notes. “But you can structure a single subject design to raise the level of evidence and it will be more convincing.”
Please let us know if you use this method in your own projects. We’ll post more reports when we have them.
REFERENCES AND GUIDES
There are some tricks to doing a good case-crossover study on yourself. Mark’s video provides a basic introduction. For technical details, this detailed introduction to case-crossover design by Yue-Fang Chang especially useful.
The seminal paper on case-crossover design is “The Case-Crossover Design: A Method for Studying Transient Effects on the Risk of Acute Events” by Malcom Maclure. (1991) [PDF] A search on Google Scholar for case-crossover design will get you deep into this literature. Unfortunately very little of it involves the kind of n-of-1 studies we’re usually interested in, but there are many technical details that may contain clues for dedicated experimenters.
One paper that will be of special interest is this one: “Should We Use a Case-Crossover Design?” by K. Malcolm Maclure and his collaborator Murray Mittleman. (2000) [PDF] In the midst of discussing technical details important for scientists proposing to use this method in studies funding by research grants whose reviewers may not be familiar with it, Maclure and Mittlemen describe using case-crossover analysis to retrospectively understand more about the death of Maclure’s father. I quote the relevant section below:
We did an n-of-1 case-crossover study of hypothesized triggers of repeated syncope experienced by Kenneth Maclure (MM’s father), who was diagnosed with sick sinus syndrome and died of fatal MI at age 73 during a morning swim, after several other potential triggers. The target person times wereKenneth’s 62nd–74th years (and subsequent years if he had lived longer). The study base comprised the years 1980–1981 and 1986, during which there were 33 instances of syncope. We restricted the study base to those years because his wife, Margaret, was willing to review only 3 years of her diaries because the memories rekindled her grief. We had no intention to generalize the findings to other individuals, only to other years. Our goal was to identify triggers to which Kenneth may have been susceptible and to test Margaret’s general hypothesis, “Perhaps I should have done more to help him avoid stress.” Hypothesized triggers included visitors to the home, trips out of town, eating out, unusual exertion, and so on. The 24-h period before an episode of syncope was classified as a case day. Each case day was matched with a control day, the same 24-h period 2 weeks before. Margaret was surprised by our null findings and relieved some lingering feelings of guilt.
Many people think the Quantified Self mostly involves physical metrics: heart rate, sleep, diet, etc. but what about what goes on in our brains? Can we quantify that? There have been several inspiring Quantified Self talks about tracking learning and memory. This post will collect all them into one place, along with good resources for further exploration.
Memorization is only a small part of learning, but it in many circumstances it is unavoidable. There is an ideal moment to practice what you want to memorize. Practice too soon and you waste your time. Practice too late and you’ve forgotten the material and have to relearn it. The right time to practice is just at the moment you’re about to forget. If you are using a computer to practice, a spaced repetition program can predict when you are likely to forget an item, and schedule it on the right day.
In this graph, you can see how successive reminders change the shape of the forgetting curve, a pattern in our mental life that was first discovered by one of the great modern self-trackers, Hermann Ebbinghaus. With each well-timed practice, you extend the time before your next practice. Spaced repetition software tracks your practice history, and schedules each review at the right time.
Convenient tools to take advantage of fast memorization techniques have been around since Piotr Wozniak began developing his Supermemo software in the early 1980s. (I wrote a profile of Wozniak for Wired in 2008, which is cited in some of these talks.) Many of us in the Quantified Self use spaced repetition. We’ve put together this page to list resources, share experiences, and invite comments and questions. We hope you find it useful. If you do, please contribute some knowledge or questions to the comments.