Tag Archives: openSNP
Enjoy this week’s list!
Would You Share Private Data for the Good of City Planning? by Henry Grabar. The use of personal, and typically private, data for municipal planning and research is becoming more common. Strava, Uber, and other companies are passing along their user data to government bodies interested in understanding their constituents. In this article, past projects are described and new ideas are put forth about this growing trend.
The social network for people who want to upload their DNA to the Internet by Daniela Hernandez. A wonderful piece of journalism on the growing OpenSNP platform for open user-donated genetic data. Take some time and read the whole thing. (Full disclosure: My 23&Me data is available on OpenSNP.org.)
What Cognition-as-a-Service should mean? by Debidatta Dwibedi. The promise of fitness trackers for many is that the use of them will improve one’s fitness. Dwibedi expresses the desire for a tool to make one wiser by helping the user avoid logical fallacies. There are tools that can help, like spaced repetition.
Connected Car: Quantified Self becomes Quantified Car by Melanie Swan.
Sensors sensors everywhere
Near and far
On your wrist
In your home
And in your car.
What On Kawara’s Analog Wisdom at the Guggenheim Has to Offer a Digital World by Ben Davis. A fantastic peek into “On Kawara: Silence” a recently opened retrospective hosted at the Guggenheim.
He was making art about the “quantified self”—the contemporary self-improvement craze for tracking and charting one’s personal data—not just before the fitbit, but before the handheld calculator.
What My Hearing Aid Taught Me About the Future of Wearables by Ryan Budish. A great article here about how to think about possible ways our technology with change and shape the world around us. Special consideration is given to our ever evolving relationship with the tools of wearable computing.
I tried to quantify my sex life—and I am appalled (NSFW language) by Miles Klee. I went back and forth whether to include this here, but in the end I think it’s important to expose tracking of all types.
How I audited my daily media habits and improved the way I read by Lydia Laurenson. Lydia was concerned with the amount of bad content she was reading on the web.For a month, she rated the articles she read according to a 5-point scale with categories like “I’m actually angry I clicked this link” and “Wow, this is really cool or useful. I’m glad I saw this.” With these ratings, she was able to see which publications produced good contents, and which outlets gave her recommendations worth her time. You can check out her (empty) tracking spreadsheet here.
The Quantified Chef by Dan Brown. Dan doesn’t fancy himself a self-tracker, but was interested in understanding his cooking habits as the main dinner cook for his family. Some interesting finds and thoughts about what it means to collect data on yourself.
Using a Log Book and Excel To Assess Time Use by Morris Villarroel. Morris spoke about how he uses journals to track his life at our 2014 QS Europe Conference. In this post, he explains how he transfers hand-written data into Excel for more in-depth analysis.
Sid Lee Dashboard. Sid Lee, a creative agency, outfitted it’s Paris office with multiple sensors and data gathering systems powered by Arduinos to feed a beautiful real-time data dashboard. Make sure to click through for the interactive site and watch their short video.
Two Thousand And Fourteen by Tyler Baird. A sentence or two cannot do this amazing work justice. Click, read, and take in the 8,760 hours of Tyler’s tracked life.
The BMJ Today: Patient Centered Care
Health Data Exploration Project Announces Agile Research Project Awards
FDA makes official its hands-off approach to regulating health apps and medical software
Small thoughts on large cohorts
Selling your right of privacy at $5 a pop
New sensors are peeking into previously invisible or hard to understand human behaviors and information. This has led to many researchers and organizations developing an interest in exploring and learning from the increasing amount of personal self-tracking data being produced by self-trackers. Even though individuals are producing more and more personal data that could possibly provide insights into health and wellness, access to that data remains a hurdle. Over the last few years a few different projects, companies, and research studies have launched to tackle this data access issue. As an introduction to this area, we’ve put together a short list of three interesting projects that involve donating personal data for broader use.
Developed and administed by the WikiLife foundation, the DataDonors platform allows individuals to upload and donate various forms of self-report and Quantified Self data. Data is currently available to the public at no cost in an aggregated format (JSON/CSV). Data types includes physical activity, diet, sleep, mood, and many others.
OpenSNP is an online community of over 1600 individuals who’ve chosen to upload and publicly share their direct-to-consumer genetic testing results ( 23andMe, deCODEme or FamilyTreeDNA) . Genotype and phenotype data is freely available to the public.
Open Paths is an Android and iOS geolocation data collection tool developed by the New York Times R&D Lab. It periodically collects, transmits, and stores your geolocation in a secure database. The data is available to users via an API and data export functions. Additionally, users can grant access to their data to researchers who have submitted projects.
We’ll be expanding this list in the coming weeks with additional companies, projects, and research studies that involve personal self-tracking data donation. If you have one to share comment here or get in touch.
We talk about very frequently here on the QS website about tools, methods, and systems that help us understand ourselves. When it comes to the self there may be nothing more fundamental to understanding our objective ourness than our basic genetic makeup. Many of you have probably undergone or have thought of using Direct-To-Consumer genetic testing to better understand your phenotypes, disease risk, or even your ancestry. That’s all great, and I’ve spent a lot of valuable time combing through my own genetic data, but like most data true power lies in large datasets that provide observations across many individuals. So how do you participate in that type of sharing and learning? Enter the team at the openSNP.org. Today we talk with Bastian Greshake, one the developers behind the openSNP project.
How do you describe openSNP? What is it?
The too long, didn’t read version: A open platform which allows people to share their genetic information and traits, which are suspected to be at least partially genetically predisposed, which also tries to annotate those genetic variants with primary scientific literature. The data can be exported from openSNP through the website or through APIs, making it easy to re-use the data.
A longer version: openSNP has basically two target groups and users may as well fit in both categories.
First there are customers of Direct-To-Consumer (DTC) genetic testing like 23andMe who want to share their genetic information with the public for various reasons. Those can use openSNP to release their genetic data into the public domain using the Creative Commons license which is applied to the data uploaded and entered in openSNP.
As genetic information is interesting but not very useful to analyze the effect of genetic variants on bodily traits those users can also enter information about traits which might genetically influenced and create new possible categories which all other users then can enter. Those traits range from the more obvious ones, like eye and hair color, to more exotic ones like political ideology. A few weeks ago we also created a method for users to also connect their Fitbit accounts to openSNP to make the collection of data easier and more standardized. The genetic effects on activity, sleep habits and weight loss/gain can more easily be analyzed in this fashion.
We also mine the databases of Mendeley, the Public Library of Science and the SNPedia to annotate the genetic variants users carry. This allows customers of DTC testing to find out what the recent scientific literature is able to tell them about their genetic variants. While the SNPedia is a crowd-curated Wiki, Mendeley and the Public Library of Science link back to primary literature, in the latter case even to Open Access literature which is full text available for everyone.
The second group of users who are interested in openSNP are scientists and citizen scientists who are interested in using the data for their own studies, be it to figure out what genetics can tell us about our ancestry or which effects single variants have on disease risks or other traits. The data can be downloaded from openSNP in bulk or more granularly accessed through a JSON-API and the Distributed Annotation System, a standard in Bioinformatics, which for example is used to visualize the data.
Both groups can profit from the commenting features which allows users to communicate about traits and individual genetic variants. The internal message system of openSNP also facilitates further communication, for example to share details about shared traits and diseases or to allow people who want to use the data to get back in touch with the people who uploaded the data. The latter one enables the direct exchange between those two user-groups in a bidirectional way: Researchers can ask questions about traits and people who have shared their data have a back channel as well and can get notified about the results researchers have made.
What’s the backstory? What led to it?
It more or less began with me getting my genetic information analyzed by 23andMe myself. After I received the results I published the data in a git repository on GitHub to make it available for others who might benefit of having more data. As I started to dig deeper into my own results and the raw data I wanted to have more data sets myself, to be able to compare the results. But unfortunately there wasn’t a single resource for such data. Some people also had published their data on GitHub, others on their own websites, collected publicly available data sets in a Google Spreadsheet or participated in projects like the Personal Genome Project.
This was quite frustrating: Finding the data was hard and it most often there was no additional data about traits attached. And more often than one would expect there was also no way to contact to people who made the data public. So the idea to create a platform to solve this problem grew and I contacted some friends to see if they were interested in doing such a platform, just for fun. We started out with the basic idea of creating a platform where people could upload their genetic data along with some traits they have. A couple of weeks after we started to work on the project we stumbled upon the APIs of Mendeley & the Public Library of Science and thought it might be cool to include additional data about the genetic variants as well. During the development we came up with more and more features, like the openSNP APIs. All in all the project is still growing and we’re working on adding and refining features.
What impact has it had? What have you heard from users?
We submitted the first release of openSNP to the 2011 PLOS/Mendeley Binary Battle, a competition interested in creative ways to use their APIs and won the first prize. We also secured a small grant from the German Wikimedia Foundation, which allowed us to genotype over 20 people, mainly from underrepresented groups, to diversify the available data. Those persons have now released their genetic data on openSNP as well. Right now we have over 250 genetic data sets on openSNP and just short of 600 registered users. Those numbers don’t sound to impressive in the age of one billion people on Facebook. But to put it into perspective: Genetic testing is still a niche thing and before openSNP was released there were about 40-50 of those data sets publicly available.
The feedback of our users has been very positive. Many users come up with new ideas for features they like to see added and we are really open to those suggestions and critiques. Many of the API methods, which are now implemented (and the whole Distributed Annotation System), are only in place because user let us know they wanted them. I know of users who are actively using openSNP to learn more about their test results and are in an active exchange with other users with similar traits. And while the amount of data we have so far doesn’t really allow scientifically sound studies there are already people using the data, for example there are users who run their self-written analysis-tools over the openSNP-data sets and report the results back to the users, which is amazing.
What makes it different, sets it apart?
Of course we’re not really the first to think of such an idea but are more or less a remix. For example 23andMe themselves do use the data of consenting customers for studies. They also provide questionaries about traits which users can take. But this data isn’t available to the public, due to (perfectly reasonable) concerns in terms of privacy, bio-ethics and liability. On the other hand there are projects like the Personal Genome Project, which publishes traits and genetic data of participants into the public domain. But due to similar reasons like with 23andMe the participation in the project isn’t open to everyone.
We feel that informed individuals should be in the position to share their data with the world, like they are already doing on their own websites, in an easy fashion. And of course we’re targeting a slightly different group: Probably over 150,000 people are customers of some DTC genetic testing, this is a huge potential data source which could be used to help us understand new and exciting things.
What are you doing next? How do you see openSNP evolving?
We’re still developing and refining openSNP. One of the biggest problems right now is the quality of the data for the additional traits. We have kept the process of adding data really open on purpose, to make it easy for people to provide additional information about themselves. Unfortunately this has the side-effect that the quality of the descriptions varies wildly. Those problems start of with regional idiosyncrasies: Is it “Eye Color” or “Eye Colour” and are you using the metric or the imperial system of units? And is your eye color blue or “Indeterminate brown-green with a subtle grey caste”? This granular data can be very useful, but for many applications it can be too specific. With the implementation of the Fitbit API we’ve taken a first step to keep the entering of data simple but unified at the same time. And we’re currently looking into other ways of how one could counter problems like this.
We’re also looking in more data sources to annotate the genetic variants listed in openSNP, to provide even more information for customers of DTC testing. And we’re also working on making our APIs more powerful. With the rOpenSci package there is already a great library which makes use of the APIs in the current state, but of course we would like to see more of those libraries.
And it’s hard to say in which direction openSNP will evolve as we are a bit dependent on the DTC genetic testing industry. More and more data, like Whole-Genome or Exome Sequencing, is generated and we are working on reflecting those changes on openSNP as well. And we’re open for any suggestions. So if you find that a feature is missing you should let us know, we will try to work out a way of how this might be usefully implemented.
Anything else you’d like to say?
First of all: We know, genetic information is sensitive and depending on where you are living there might not even be laws to protect you from discrimination based of your genes. Other countries, like the US with the Genetic Information Discrimination Act (GINA), have some mechanisms against this, but even those might not offer total protection in the end. And you should also keep in mind that your genetic information does not only give away details about yourself, but by design also about the next of kin. I think this is really important. If you are thinking about publishing your genetic data please keep those issues in mind. And if you come to the conclusion that this isn’t for you as you have to fear negative repercussions or just have a gut feeling of not really wanting to publish the data: Please don’t do it.
And what I also can’t stress enough is that openSNP is developed and run by a team of about four people and we are all doing this in our spare time as a fun project and as community service, without compensation. Some of us have day jobs, others are still studying and some even do both. So while we are doing our best to keep everything running it might sometimes take a while. But if you feel like contributing to the project please get in touch with us. We’d love to have more people in on this.
Authors note: Data sharing, especially genetic data, is a very sensitive topic in our community. I want to fully disclose my bias towards openness and sharing. I believe that our kindergarden teachers had it right when they taught us that sharing is one of the fundamental human traits we should all cultivate. To this end, I have participated in openSNP and you can view my genetic data here and my Fitbit data here.
This is the 18th post in the “Toolmaker Talks” series. The QS blog features intrepid self-quantifiers and their stories: what did they do? how did they do it? and what have they learned? In Toolmaker Talks we hear from QS enablers, those observing this QS activity and developing self-quantifying tools: what needs have they observed? what tools have they developed in response? and what have they learned from users’ experiences? If you are a “toolmaker” and want to participate in this series, contact Rajiv Mehta or Ernesto Ramirez.
I’m typing this post while flying back to Southern California after spending a few days at a “Big Data” conference in San Francisco. One of the best things about the conference was meeting the subject of today’s round of Numbers From Around the Web. I first stumbled upon Bastian because he’s the main instigator and developer behind a great project called openSNP. Simply put, openSNP is a place you can host your direct-to-consumer genomic data for the world to see, understand, play with, download, well you get the point. This is a really interesting phenomenon that deserves it’s own post, but we’re going to explore some really neat QS experimentation and learning Bastian engaged in to better understand his sleep.
So, I found out about Bastian because openSNP announced that they also built a method to link and host Fitbit data (you can do that here if you’re so inclined). Turns out Bastian is an avid Fitbit user and has been using it to explore his sleeping patterns. His first major insight from his data indicated that in about 5.5 years he should be sleeping 24hrs per day:
So I downloaded my data from openSNPand started playing around with it: I did a simple linear regression over the time series and could indeed find a trend towards more sleep. The regression came out as y = 0.5x + 417, which ± says that for each two days that pass I will sleep a minute longer, which also means that it will be about 2000 days (or 5.5 years) until I will sleep 24 hours a day.
So yes, obviously regression may not be the best tool in the statistical toolbox to understand sleep so he decided to examine another question: “Do I sleep better or worse when there is someone in bed with me?” Using his sleep and calendar data he was able to identify nights he spend alone and nights he slept next to a warm body and found some pretty interesting stuff.
You can clearly see here in his table of 80 days of sleep (60 alone vs 20 with a companion) that he actually tends to sleep worse when he is sharing his bed. While he spends more time in bed, he takes longer to fall asleep, spends more time awake, and is awakened more often. For those of you who are not statically inclined those p-values indicate the probability that the difference in the two categories is due to chance (you can learn more about p-values here).
Like many good scientists he dug deeper to make sure what he was observing wasn’t related to other confounding variables such as the day of week:
Bastian didn’t find any significant differences in sleep quality between weekend and week days for his sleeping situation, but as one might expect he’s less active and sleeps more on weekends.
So, while this analysis might seem simplistic, one of the great things about Bastian and what he’s developing at openSNP is his willingness to be open with his data. Do you have some ideas about what you might find about Bastian from his Fitbit data? Have another hypothesis about sleep? Well you can test it out by downloading his data! You can start by reading his excellent post about this sleep analysis here.
Every few weeks be on the lookout for new posts profiling interesting individuals and their data. If you have an interesting story or link to share leave a comment or contact the author here.