Illustration: Azure Grant

Azure Grant and Gary Wolf

September 3, 2019

Download the full paper here: “Design and Implementation of Participant-Led Research in the Quantified Self Community.“

THE QUANTIFIED SELF is about making personally relevant discoveries using our own self-collected data. We call this practice everyday science, a name that emphasizes its nonprofessional character. Lately we’ve begun organizing small group projects that show how collaboration can make individual projects easier. Sometimes, joining forces with others who share our question can make it possible to create both personal and generalizable health knowledge. Following the scholar Effy Vayena, we use the term “participant-led research” (PLR) to describe this approach.

In the Bloodtesters PLR, a group of about 24 of us measured our blood cholesterol as often as once per hour. Normally, blood cholesterol is measured once a year, the results are used to evaluate risk of heart attack or stroke. However, in our own data, we noticed that our blood cholesterol varied widely from day to day. In fact, every single person in our group crossed through a risk category during a series of measurements taken in the morning before eating anything.

There were many other interesting results. We found the PLR approach to be unexpectedly powerful. It yielded insights that were valuable to the individuals who conducted it, while also generating important results that bear on standard cardiovascular care.

We’ve written a detailed white paper documenting the design and implementation of Bloodtesters, hoping it will be useful to others who may want to follow in our footsteps. If you prefer a short version, the most important conclusions are excerpted below the line. Please get in touch if you’d like to think along with us about future participant-led research studies.

For two related papers based on this work, see:

“Free-Living Humans Cross Cardiovascular Disease Risk Categories Due to Daily Rhythms in Cholesterol and Triglycerides” in the Journal of Circadian Rhythms.

“Approaches to governance of participant-led research: a qualitative case study” in BMJ Open.

________

What is PLR?

Participant-led research (PLR) is a collaborative form of investigation in which researchers and participants are the same individuals. PLR attempts to render investigation equitable and transparent. Any professional researchers who are members of the group collect their own personal data and conduct their own personal research projects, in addition to sharing research skills and performing conventional research duties; meanwhile, all participants share in organizational, ethical and scientific responsibilities. (See: “Research led by participants: a new social contract for a new kind of research” by Effy Vayena.)

What Blood Testers Taught Us About the Process of Participant-Led Research and Self-Collected Data

Contrary to traditional study designs, this work did not exclusively rely on a team made up of academic researchers managing the participation of untrained subjects. Instead, Blood Testers utilized the skills of a variety of individuals in research, academic and non-academic communication, engineering, and data science as well as those with personal experience with self-tracking and cardiovascular illness. The project left us with many benefits, challenges, and questions.

Below, we offer a framework in which individuals of diverse professions can work together to gain individual knowledge and contribute to the scientific literature.

1. 1. Combining the role of researcher and participant encourages commitment to the project. By beginning a project as participants first, researchers/administrators second, individuals in the project took a common stake in its outcomes for both personal and altruistic reasons.
2. 2. Substantial, longitudinal learning was required for participants to conduct a rigorous self-experiment. This was true even of experienced self-trackers. For example, although individuals were interested in cholesterol when they signed on, many realized it required substantial study time to learn background material on the physiological functions of cholesterol.
3. 3. Choosing to participate is personal. Some individuals had negative past experiences with large lancets (e.g., having to use them to check blood sugar), and this was a deterrent from joining the project.
4. 4. Blood Testers required greater time commitment than would typical study. Three individuals wanted to participate but, despite their best intentions, were not able to make time to participate.
5. 5. Organizing a participant-led project requires close attention and flexibility in communication. We found that individuals who communicated more during the project were more likely to draw both personally and generally valuable conclusions from their projects. Individuals are (a) distributed and (b) operating with a high degree of autonomy, making consistent and adaptive communication essential.
6. 6. Physiological measures within an individual vary significantly and rhythmically over time, and these trends require careful analysis to interpret. Collection of high frequency physiological measures brings up fundamental questions about individual variability and biological rhythmicity. This brings up further questions about the proper interpretation of point measurements in many outputs (e.g., glucose, blood pressure, weight/body mass index).
7. 7. Approaching causality is very difficult in participant-led research; nonetheless, much personal learning occurs. Often, the variability observed among the first few trials of a personal experiment is sufficient to inspire the participant to shift their line of questioning, ultimately leading to an interesting—if not causal—personal takeaway.
8. 8. Personal data can be more difficult to interpret than impersonal data, but collaboration can provide balance. Reasoning “objectively” about oneself is hard. This an ongoing challenge, as even trained scientists may struggle in choosing and applying appropriate methods to the analysis of self-collected data. For instance, interpreting observations of variability in blood biomarkers is not intuitive; a tendency to interpret every rise and fall is easy to slip into, even for experienced self-trackers. Although this bias has led to the removal of self-experimentation from much scientific work, we learned that collaboration among participants & organizers can help reduce such bias (see 11).
9. 9. Publishing this work was a non-trivial process. Despite our early intention to not interact with the IRB/REB process, publication required obtaining an IRB/REB exemption. Even preprint servers require proof of ethical review. How can future PLR projects taking place outside academic institutions share generalizable findings when there is no free access to ethical review?
10. 10. The best way to disseminate this work is not known. This work was shared via scientific publication, conference talk, meetup, and blog. What is the most widely useful artifact of such a project? Is there a discrete, shareable unit of knowledge appropriate to PLR? We expect that something akin to a community database and “charticle” or Wikipedia page can fill this role eventually, allowing both professional researchers and academic individuals to share the same knowledge base—but we are not there yet.
11. 11. Self-collected, high temporal resolution data has both personal and generalizable value. Self-collected data refers to numerical or categorical information obtained by an individual about themselves, either by use of their own senses or through use of technology, as in the lipid samples collected during Blood Testers’ self experiments. Self-collected data has several advantages from a research perspective over traditional assays conducted by laboratory staff including high temporal resolution, ease of acquisition/scalability, and detailed individual annotation. Self-collected data also has associated challenges, largely due to skepticism (both real and unwarranted) about data quality and management as well as the complexity of measures that can be captured. Blood Testers showed us that these challenges can be overcome by bringing together diverse expertise.

Benefits of Participant-Led Research

Participant-led research, by definition, exists in service of both the participant and general scientific knowledge. The “human right to science” guaranteed in Article 27 of the Universal Declaration of Human Rights is brought to life in PLR. The fruits of discovery are broadly shared, and the learning associated with the research process itself accrues to the benefit of all involved. We noted a number of specific benefits of our particular project.

1. 1. Participants gained conceptual and practical skills relating to data interpretation. By collaborating with fellow participants who were researchers by profession, other participants gained exposure to concepts in statistics, programming, and interpretation of data. This included skills relating to reading new types of graphs, normal and non-normal distributions, field-specific knowledge in cardiovascular physiology, and the concept of static measures versus time series. These skills are not usually acquired without a specialized degree yet are valuable skills for decision making in a data-rich world.
2. 2. Participation encouraged active reasoning. Conducting a personal experiment is a way to question one’s assumptions in a non-judgmental and curious manner. This sort of reflection has been reported to promote self-awareness, patience, and curiosity—as attested to in the Blood Testers project and in meta analyses of the practice of self-tracking. For example, validation of self-collected measurements involved careful reflection on the context of the measurements and the capabilities of the instrumentation. These issues are common in scientific and medical practice but are normally engaged directly only by specialists. Additionally, participant presentations were occasions for developing new knowledge and skills.
3. 3. Learning was motivated by personal interest. Domain knowledge about cardiovascular health, risk, lipid metabolism and circadian rhythms was pursued in the context of personal health questions. An abstract idea of questionable personal relevance, such as a daily rhythm in cholesterol, became worthy of reflection when needed to explain variation in one’s own data. For some participants, this approach to learning about cholesterol and circadian rhythms was a key benefit.
4. 4. Time lag between research and dissemination was minimized. In contrast to traditional approaches to research participation, in which fruits of discovery are shared with participants after a long delay, the learning offered by PLR is ongoing and begins immediately.

Academic and Clinical Significance of Participant-Led Research

The QS community shares an interest in using empirical observation to explore personal questions. In this approach, self-collected time series are essential as they represent the self-investigator’s dynamic physiology. As nearly all biological systems are dynamic (e.g., ultradian, circadian, ovulatory, and seasonal rhythms), the time series data of interest to QS participants also hold interest to academic researchers. These data augment existing knowledge based on static measures with observations that are otherwise difficult or impossible to collect.

Access to within-individual time series is a significant barrier to exploring longitudinal, within-individual physiology. For traditional researchers, conducting large-scale human studies is costly, time consuming, and often creates siloed (rather than shared) data. PLR may be useful in addressing these challenges both by offering (1) the possibility of collaboration with self-trackers willing to take part in well-defined projects that collect time series data and (2) the possibility of collaboration with participants who have already collected extensive time series. Collaboration between the QS community; researchers in physiology, medicine, and chronobiology; and data scientists can generate novel insights into the dynamics of human physiology. However, for this potential to be realized, significant challenges in funding, instrumentation, data access, ethical review, lay education, and research dissemination must be addressed.

Road Map for Participant-Led Research

The QS Blood Testers project was a relatively simple experiment in PLR that reveals future prospects and challenges. We propose the following areas of work for stakeholders in the advancement of PLR.

1. 1. Collaborations across technical sectors to build an open technology stack for PLR, including open instrumentation that allows secure data access and control to participants.
2. 2. Development of training and educational materials that enable more people to do rewarding and productive self-experiments.
3. 3. Collaboration with data science and physiology specialists to develop shareable methods and frameworks, including data pipelines, suited to statistical and time series analysis of personal data.
4. 4. Partnership with open access movement to ensure participants’ access to personal data and relevant scientific literature.
5. 5 Creation of suitable frameworks for ethical review, allowing participants to work with academic and clinical partners who require traditional IRB/REB review to publish.
6. 6. Contribute to the development of a dissemination system for PLR that creates accessible alternatives to traditional disciplinary publishing, including informal publication and shared data analytical resources.