Topic Archives: Personal Projects

Psychogeography By Sabastian Meier and Katrin Glinka

In honor of today being the last day of the existence of Moves, the app from which so many Quantified Self projects drew their location data, I thought I’d post this artwork by Sabastian Meier and Katrin Glinka, who constructed city models based on connecting Moves data with their memories.

For discussion of the demise of Moves, exporting, and alternatives, see this topic in the QS Forum: Moves shutting down? Oh, no! But note, today is the last day.

 

 

Posted in Personal Informatics, Personal Projects | Tagged , , | Leave a comment

Use TwArχiv to analyze your Twitter archives

Screen Shot 2018-01-08 at 15.22.38

We are happy to welcome this guest post on a community tool by Bastian Greshake Tzovaras. Bastian is the director of research at the Open Humans project. He can be found online at @gedankenstuecke. -Steven

I’ve built a Twitter analysis web application that’s open to everyone to use and learn from. Often the best data for learning something about yourself are data you’ve already collected; sometimes without even being explicitly aware of collecting it. Social media activity, for example. We often send off Facebook posts or tweets with very little thought about the metadata that we generate in doing so. Where was I when I made that post? What time was it? What type of content did it contain? Did I retweet or reply to another person’s post? And, of course, what did my post contain?

This data can be extremely powerful – for others. The language you use in your Tweets can be used to predict your age as well as your income. Twitter uses the data to gather information about your likes, dislikes, and possessions – among other topics. But what if you want to learn about yourself with your own Twitter data?

The tool I created allows anybody to explore their own Twitter archive in detail. First, you’ll want to request your archive from Twitter. It will contain all the tweets you have ever sent, with not only the text but all the metadata as well. To look at these metadata, go to my small web application called TwArχiv (pronounced tw-archive), which allows you to upload your data and explore it using interactive graphs.

For instance, you can see how the nature of the tweets you send change over time. Are you replying more to people than you used to or is it all just retweets by now? For my own data it seems that finishing up my PhD work had quite an impact, starting in late 2016. With less procrastination I wrote fewer unprompted tweets. Instead, replying to people became more central to my Twitter experience.

Screen Shot 2018-01-08 at 16.00.59

There is also plenty of research on gender bias in social media usage and whose voices are being amplified, with men being overwhelmingly favored.  TwArχiv allows one to do some soul searching on this. It tries to predict the gender of the people you interact with based on their first names and shows you whether your reply and retweet behaviour is gender-balanced.

My own graphs show that I had (and have) a good way to go here. Especially 2010 is wildly off when it comes to the gender representation in my Twitter interactions. What happened during that time? I was politically active in the German Pirate Party, which was infamous for being a “boys club”.

Screen Shot 2018-01-08 at 15.23.28

If you have geolocation enabled on your tweets, you can get an idea of where you tweet. With a fully zoomable map, TwArχiv allows you to explore the globe on all scales to see the broader picture as well as street-level tweet distributions. As a first attempt of seeing movement patterns, you can also get a time-stamped version of the map that highlights locations one tweet at a time.

If you want to give a try with your own archive, you can head to TwArχiv.org. The data storage is handled by Open Humans and by default your archive and the resulting visualizations will be private. (You can choose to make them public, though, to share them with your friends and followers – mine are here!).

A note: The Twitter archive does not contain any direct messages but only your tweets, so if you have a public Twitter account the archive is basically all your “public Twitter interactions”.

If you have ideas on how to extend the functionality of TwArχiv or you want to code your own Twitter archive analysis, you could even get funding to do so: The Open Humans’ mini-grants of USD 5,000 for projects that will enrich the Open Humans ecosystem are a perfect fit for this kind of data visualization and analysis.

Posted in Personal Projects | Tagged , , , , | Leave a comment

Hot Stuff: Body Temperature Tracking and Ovulatory Cycles

For the past eight months I’ve been tracking my temperature every minute using small, wireless sensors.

I work in a lab that recently showed minute-by-minute body temperature can tell you fascinating things about female physiology, at least in mice. Using temperature, we can tell what day a mouse will ovulate, whether or not it will become pregnant within hours of pairing with a male, and in the same time, whether or not its pregnancy will be successful. Just as interesting, the temperature reveals that some mice have stable ovulatory cycles and some don’t. We wanted to see if any of this holds up in humans (read: lab mates, a sporting family member and myself). I’ll show you what we did, what we found, and how to get started if you’d like to start tracking your temperature too.

Why Temperature?
Think of metabolism as a continuous symphony and body temperature as the din that carries through the concert hall walls.

Many of the metabolic reactions taking place throughout our bodies generate small amounts of heat and are actually coordinated in a similar way to musical chords. For example, during the luteal phase of the menstrual cycle progesterone levels will pulse in concert with estradiol, often following a luteinizing hormone pulse occurring 15 minutes prior. These fluctuations, as well as other things that affect metabolism (ovulating, eating a meal, etc.), translate into small temperature ripples which register on the surface of the body.

Temperature has long been used as a predictor of ovulation. But most temperature based techniques rely on a single measurement per day. Limiting data collection to one time point per day is the equivalent of listening to the symphony only at what we hope is the crescendo of each piece: with training, we might identify the chord, but we’ll still miss most of the show.

What are we doing and what have we seen already?
To see if we could use high temporal resolution temperature to recapitulate any of our previous findings, we began monitoring distal (wrist), axial (arm-pit) and core (ahem, core) temperatures every minute, using small devices called iButtons. We’ve seen some interesting things so far. I’ve shown the temperature data as a heat map, because it allows you to see many measurements while giving a clear picture of the overall pattern of rising and falling average temperatures over the course of 28 days.

HeatMap_LabMate

Temperature Can Predict the Start of Menstruation.
In the graph above, which uses my lab mate’s data, you’ll see that the range of temperatures she passes through in a day shifts a little higher every day leading up to the start of spotting/ menstruation. This timing is clear in her data, but it isn’t identical for everyone, though. My cycles are irregular and the chart below shows that menstruation starts when my average temperature reaches its highest level of the month. Note that this can be more than once per 28 days, as in the month graphed below.

heatmap_Azure

In my mom’s case, the heat map below clearly shows the shift between follicular (cooler) and luteal (warmer) phases. I’ve outlined the profiles of the Progesterone (P) and Estrogen (E) that my mom takes each day as part of hormone replacement therapy. In the valley where both hormones are low, she transitions from follicular to luteal phase. This corresponds to a temperature increase, and a few days later she gets her period.

HeatMap_Mom

These findings keep us coming back for more: more subjects and more longitudinal data for each of us. Perhaps the differences we have observed between us support that there are different ‘types’ of cyclers in the population, just as there are different body types. And maybe the temperature features we have in common will apply to other women.

So how do we gather the data (and how might you)?

ibutton_pile_2

iButtons are about the size of the button on your jeans, and one side has a sensor which is worn pressed to the skin. A sweat band is enough to secure one button to your wrist, and the axial button can be tucked into a bra strap or secured with a non-irritating skin tape (here is my favorite so far). Body temperature shouldn’t ever fluctuate more than a couple of degrees C, so devices with high precision are key. This model is accurate up to .0625 C. Both the resolution and the sampling rate can be user-specified, meaning you can take very precise measurements very frequently. I find that anywhere between one and three-minute resolution works well to capture changes throughout the day.

iButtons don’t ever need to charge, but the data needs to be read once the memory fills up. Depending on the sampling rate, that’s every 3-7 days. At the end of a recording period, the ibutton is touched to its reader, and a simple interface allows the user to view the data and export it as a csv. iButton will plot the data, but it won’t do any further analysis. We’ve taken these csv outputs into Matlab and Python for our analyses, and because they are widely used formats, anyone could make graphs and start to play with their data. I’m not associated with the company, but I’m excited to share what we’re finding and want others to know how to jump in. An ibutton and a reader together cost about $100.

Interested?
Temperature tracking is a scavenger hunt: we don’t know precisely what we’re looking for, but clues keep turning up that lead us in interesting and verifiable directions. Multiple hormonal systems in our bodies (the stress axis, the digestive system, the thyroid axis) affect body temperature, and the reproductive system is just one of those. This raises the question: could we see predictable changes in temperature associated with a long run, a large meal, or a bad night of sleep? Probably. Mapping the personal, research, and clinical applications of high temporal resolution body temperature tracking will take time and user participation. Luckily, it gives interesting and useful personal feedback along the way.

Posted in Personal Projects | Tagged , , , , , | Leave a comment

Robby Macdonell: The Data Is in, I am a Distracted Driver

“When I see someone driving towards me with their face buried in their phone, I get gloriously indignant about it.”

Robby Macdonell has given great talks on transportation logging and time-tracking. Here, he combined those two data streams, using Automatic and RescueTime, to prove that he does not use his phone while driving nearly as often as other drivers.

Only the data didn’t agree.

Watch how Robby confronts the realization that he is more distracted than he thought and the changes he made because of it.

Posted in Personal Projects, QSEU15, Videos | Tagged , , , , , , | Leave a comment

Tuning My Brain With Music For Reading, Programming, and Archery by Rocio Chongtay

In this fascinating talk Rocio Chongtay shares her novel and thoughtfully designed experiments in using music to adjust her concentration and relaxation depending on what she’s doing. Using a consumer EEG device from Neurosky, Rocio tried different types of music while tracking the relaxation and concentration dimensions identified by the Neurosky algorithm. She had experience experimenting with Neurosky in her lab, and then turned these techniques on understanding something about her own mind.

 

Posted in Conference, Personal Projects, QSEU15 | Tagged , , , , , | Leave a comment

Re-Living My Life with Mood Tracking by Kouris Kalligas

Kouris Kalligas, a long time participant and contributor at Quantified Self meetings, is the creator of the very easy to use data aggregation service AddApp. AddApp is an iPhone app that makes it simple to gain insights from data gathered on dozens of different devices. While running his startup, Kouris has also been doing ongoing self-tracking experiments. At QS Europe 2014, he gave an excellent show&tell talk about his sleep, diet, and exercise data. In the talk below, he discusses using mood data in combination with calendar data to reflect on the relationship between emotion, experience, and self-image.

Posted in Conference, Personal Projects, QS15 | Tagged , , , , | Leave a comment

Randy Sargent on Using Spectrograms to Visualize Heart Rate Variability

Let’s start 2016 with a very interesting talk by Randy Sargent about how to visualize the very large data sets produced by some kinds of self-tracking. Randy’s idea about using spectrograms, normally used for audio signals, to create a portrait of your own time series data, is completely novel as far as I know. If you have tried something similar, please get in touch.

 

Posted in Conference, Personal Informatics, Personal Projects, QS15, Videos | Tagged , , , , | Leave a comment

Effects of A Year in Ketosis by Jim McCarter

In this fascinating short talk by geneticist Jim McCarter, we see detailed data about the  effects of a ketogenic diet: lower blood pressure, better cholesterol numbers,and vastly improved daily well being.  Jim also describes the mid-course adjustments he made to reduce side effects such as including muscle cramps and increased sensitivity to cold.

Jim begins: “When I tell my friends I’ve given up sugar and starch and get 80% of my calories from fat, the first question I get is: Why?”

The rest of the talk is his very clear answer.

Posted in Conference, Personal Projects, QS15, Videos | Tagged , , , , | Leave a comment

Gary Wolf: Even when I'm active, I'm sedentary


Gary21634

We recently released our QS Access app, which allows you to see HealthKit data in tabular format. Not very many tools feed data into HealthKit yet, but Apple’s platform does pick up step data gathered by the iPhone itself. I have step data on HealthKit going back about two weeks. When Ernesto Ramirez and I were playing around with QS Access, loading the data into Excel and looking at some simple charts, I learned something: Even when I’m active, I’m sedentary.

Gary3334

My daily step totals ranged from a depressing 3334 steps on Thursday, September 18 to an inspiring 21,634 steps on Friday, September 25, but – as these charts clearly show – even  on the extreme days my activity was concentrated into relatively short periods when I got up from my desk and went out to do something. Most hours, every day, were spent with hardly any movement at all. I’m sitting at my desk, and sitting at my desk some more, and sitting at my desk still more. That’s probably not good. No, not good at all.

Pulling my data out of HealthKit and seeing a few simple charts gave me a bit of insight that I hope will lead to a change in how much I sit. It was a great to be able to easily make some simple analysis of my data. I hope you’ll find QS Access useful also (you can learn more about it here). Please share what you learn in the QS Access thread in the QS Forum or by emailing us about your projects: qsaccess@quantifiedself.com.

Posted in Personal Informatics, Personal Projects, QS Access | Tagged , , , , , | 2 Comments

How to Download Minute-by-Minute Fitbit Data

NOTE: The methods detailed in this post no longer work. The post is kept here for archival purposes, but unfortunately you will not be able to access your Fitbit data by following the steps.

IntradayDataChart

Earlier this week we posted an update to our How To instructions for downloading your Fitbit data to Google Spreadsheets. This has been one of our most popular posts over the past few years. One of the most common requests we’ve received is to publish a guide to help people download and store their minute-by-minute level step and activity data. Today we’re happy to finally get that up.

The ability to access and download the minute-by-minute level (what Fitbit calls “intraday”) data requires one more step than what we’ve covered previously for downloading your daily aggregate data. Access to the intraday data is restricted to individuals and developers with access to the “Partner API.” In order to use the Partner API you must email the API team at Fitbit to request access and let them know what you intend to do with that data. Please note that they appear to encourage and welcome these type of requests. From their developer documentation:

Fitbit is very supportive of non-profit research and personal projects. Commercial applications require additional review and are subject to additional requirements. To request access, email api at fitbit.com.

In the video and instructions below I’ll walk you through setting up and using the Intraday Script to access and download your minute-by-minute Fitbit Data.

  1. Set up your FitBit Developer account and register an app.
    • Go to dev.fitbit.com and sign in using your FitBit credentials.
    • Click on the “Register an App” at the top right corner of the page.
    • Fill in your application information. You can call it whatever you want.
    • Make sure to click “Browser” for the Application Type and “Read Only” for the Default Access type fields.
    • Read the terms of service and if you agree check the box and click “Register.”
  2. Request Access to the Partner API
    • Email the API team at Fitbit
    • They should email you back within a day or two with  response
  3. Copy the API keys for the app you registered in Step 1
    • Go to dev.fitbit.com and sign in using your FitBit credentials.
    • Click on “Manage My Apps” at the top right corner of the page
    • Click on the app you created in Step 1
    • Copy the Consumer Key.
    • Copy the Consumer Secret.
    • You can save these to a text file, but they are also available anytime you return to dev.fitbit.com by clicking on the “Manage my Apps” tab.
  4. Set up your Google spreadsheet and script
    • Open your Google Drive
    • Create a new google spreadsheet.
    • Go to Tools->Script editor
    • Download this script, copy it’s contents, and paste into the script editor window. Make sure to delete all text in the editor before pasting. You can then follow along with the instructions below.
    • Select “renderConfigurationDialog” in the Run drop down menu. Click run (the right facing triangle).
    • Authorize the script to interact with your spreadsheet.
    • Navigate to the spreadsheet. You will see an open a dialog box in your spreadsheet.
    • In that dialog paste the Consumer Key and Consumer Secret that you copied from your application on dev.fitbit.com. Click “Save”
    • Navigate back to the scrip editor window.
    • Select “authorize” in the Run drop down menu. Click run (the right facing triangle).
    • Select “authorize” in the Run drop down menu. This will open a dialog box in your spreadsheet. Click yes.
    • A new browser window will open and ask you to authorize the application to look at your Fitbit data. Click allow to authorize the spreadsheet script.
  5. Download your Fitbit Data
    • Go back to your script editor window.
    • Edit the DateBegin and DateEnd variables with the date period you’d like to download. Remember, this script will only allow 3 to 4 days to be downloaded at a time. 
    • Select “refreshTimeSeries” in the Run drop down menu. Click run (the right facing triangle).
    • Your data should be populating the spreadsheet!

If you’re a developer or have scripting skills we welcome your help improving this intraday data script. Feel free to check out the repo on Github!

Posted in Lab Notes, Personal Projects | Tagged , , , , , , , | 42 Comments