Matthew Cornell

June 10, 2011

As I continue trying to stretch the concept of experiment so that a wide audience understands applying a scientific method to life, I struggle with defining success. While the trite “You can always learn something” is true, I think we need more detail. At heart is the tension between the nature of experimentation’s trial-and-error process (I prefer the term Edisonian approach) – which means outcomes are unpredictable – and our need to feel satisfaction with our work. Here are a few thoughts.

Skillful discovery. Rather than being attached to a particular outcome, which we have limited control over, I’ve found it’s better to focus on becoming an expert discoverer and mastering the process of experimentation. Because you have complete control over what you observe and what you make of it, you are guaranteed success. Fortunately, there’s always room to develop your investigatory skills.

Fixing the game. At first it might seem contrived, but carefully choosing what you measure can help implement a scientific perspective on success. For example, instead of framing a diet experiment as “Did I lose weight?,” it is more productive to ask “How did my weight change?” The former is a binary measure (losing weight = success, not losing = failure) and one that you don’t necessarily have control over. After all, you are trying an experiment for the very reason that you don’t know how it will work out. The latter phrasing is better because it activates your curiosity and gives you some objectivity, what I call a “healthy sense of detachment.”

Improving models. As essentially irrational creatures, we run the risk of not questioning what we know. Updating our mental models of people, situations, and the world helps us to be more open to improvements. And the leading edge of that is the conflict between expectation (predicted outcome) and reality (actual results, AKA data). The quantified way to work that is by explicitly capturing our assumptions, testing them, taking in the results, and adjusting our thinking as necessary. This also leads to better predictions; from The Differences Between Innovation and Cooking Chili:

Of course, all of the experimental rigor imaginable cannot guarantee success. But it does guarantee that innovators learn as quickly as possible. Here, “learn” means something specific. It means making better predictions. As predictions get better, decisions get better, and you either fail early and cheap (a good outcome!) or you zero in quickly on something that works.

Getting answers. Another way to guarantee success is by going into an experiment with clearly formulated questions that your results will answer. Structured correctly, you know you will get answers to them. I think of it as regardless of what happens, you have found something out. (Hmm – maybe thinking of the process as active discovery is a richer concept than the generic “you learned something.”)

Designing for surprise. If the product of your experiment was not very surprising, then maybe you should question your choice of what you tried. Exciting experiments probe the unknown, which ideally means novelty is in store. Fill in the blank: “If you’re not surprised at the end of your experiment, then __.”

Zeroing in. Because we usually dream up experiments with a goal in mind, chances are we come out the other end having moved some amount in the direction of attaining that goal. Progress is a success, so give yourself a pat on the back.

Taking action. Finally, each experiment is a manifestation of personal empowerment, which is a major success factor in life. While health comes to mind (do difficult patients have better results?), I think generally the more we take charge of our lives, the closer we get to happiness.

What do you think?

[Image from lincolnblues]

(Matt is a terminally-curious ex-NASA engineer and avid self-experimenter. His projects include developing the Think, Try, Learn philosophy, creating the Edison experimenter’s journal, and writing at his blog, The Experiment-Driven Life. Give him a holler at matt@matthewcornell.org)