What makes a successful personal experiment?

As I continue trying to stretch the concept of experiment so that a wide audience understands applying a scientific method to life, I struggle with defining success. While the trite “You can always learn something” is true, I think we need more detail. At heart is the tension between the nature of experimentation’s trial-and-error process (I prefer the term Edisonian approach) – which means outcomes are unpredictable – and our need to feel satisfaction with our work. Here are a few thoughts.

Skillful discovery. Rather than being attached to a particular outcome, which we have limited control over, I’ve found it’s better to focus on becoming an expert discoverer and mastering the process of experimentation. Because you have complete control over what you observe and what you make of it, you are guaranteed success. Fortunately, there’s always room to develop your investigatory skills.

Fixing the game. At first it might seem contrived, but carefully choosing what you measure can help implement a scientific perspective on success. For example, instead of framing a diet experiment as “Did I lose weight?,” it is more productive to ask “How did my weight change?” The former is a binary measure (losing weight = success, not losing = failure) and one that you don’t necessarily have control over. After all, you are trying an experiment for the very reason that you don’t know how it will work out. The latter phrasing is better because it activates your curiosity and gives you some objectivity, what I call a “healthy sense of detachment.”

Improving models. As essentially irrational creatures, we run the risk of not questioning what we know. Updating our mental models of people, situations, and the world helps us to be more open to improvements. And the leading edge of that is the conflict between expectation (predicted outcome) and reality (actual results, AKA data). The quantified way to work that is by explicitly capturing our assumptions, testing them, taking in the results, and adjusting our thinking as necessary. This also leads to better predictions; from The Differences Between Innovation and Cooking Chili:

Of course, all of the experimental rigor imaginable cannot guarantee success. But it does guarantee that innovators learn as quickly as possible. Here, “learn” means something specific. It means making better predictions. As predictions get better, decisions get better, and you either fail early and cheap (a good outcome!) or you zero in quickly on something that works.

Getting answers. Another way to guarantee success is by going into an experiment with clearly formulated questions that your results will answer. Structured correctly, you know you will get answers to them. I think of it as regardless of what happens, you have found something out. (Hmm – maybe thinking of the process as active discovery is a richer concept than the generic “you learned something.”)

Designing for surprise. If the product of your experiment was not very surprising, then maybe you should question your choice of what you tried. Exciting experiments probe the unknown, which ideally means novelty is in store. Fill in the blank: “If you’re not surprised at the end of your experiment, then __.”

Zeroing in. Because we usually dream up experiments with a goal in mind, chances are we come out the other end having moved some amount in the direction of attaining that goal. Progress is a success, so give yourself a pat on the back.

Taking action. Finally, each experiment is a manifestation of personal empowerment, which is a major success factor in life. While health comes to mind (do difficult patients have better results?), I think generally the more we take charge of our lives, the closer we get to happiness.

What do you think?

 

[Image from lincolnblues]

(Matt is a terminally-curious ex-NASA engineer and avid self-experimenter. His projects include developing the Think, Try, Learn philosophy, creating the Edison experimenter’s journal, and writing at his blog, The Experiment-Driven Life. Give him a holler at matt@matthewcornell.org)

About Matthew Cornell

Matt is a terminally-curious ex-NASA engineer and avid self-experimenter. His projects include developing the Think, Try, Learn philosophy, creating the Edison experimenter's journal, and writing at his blog, The Experiment-Driven Life. Give him a holler at matt@matthewcornell.org
This entry was posted in Discussions and tagged , , . Bookmark the permalink.

4 Responses to What makes a successful personal experiment?

  1. Bridget says:

    I LOVE paragraph 4 – “not questioning what we know”. We are all guilty of this but as self-trackers, we are ahead of most people. As a person who doesn’t hold onto aspects of my life because they are “comfortable”: leaving a job, moving, leaving a relationship, I am accused of being fickle or irrational when in fact, I’m anything but that! I love how you explained an examined life in simple terms here. People who take the time to question are those who promote progress. Thanks for this post!

    • “People who take the time to question are those who promote progress.” – Very nice, Bridget. As someone who is *really* uncomfortable being outside my comfort zone, I really appreciate that you’ve put yourself in those situations. The struggle builds character, but sometimes it ain’t easy :-) Thanks for your thinking,

  2. Bo says:

    Paragraph 3 really hit home for me, since I’ve been backwards-thinking about my weight loss all this year; what a great example!

    Para 4 reminded me of the text I’m writing in my dissertation, because a “reputation system” is the formalization of these models and how to update them. But I’m also reminded of how, in my caffeine tracking experiment, that having “enough” data is so important — I got completely different results after 30 days versus 90 days… which leads to other topics on verifying unexpected results and external influences on your experiment. (I have always wondered why CompSci papers don’t include a “sources of error” section.)

    This post seems like a great list of ways to frame experimentation so that “success” is positive and encouraging, rather than demotivating. I would love to see it be the intro to a whole chapter on self-experimentation with examples on things that people might easily try in everyday life (like the weight example).

    • Great points, Bo. In Edison I’ve been thinking about how reputation relates to activity in a self-experimentation community. Very, very different from the rating “popular”. I also like your “sources of error” thought. What if humans came with such a section?

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.