How to Write a Good Experimentation Hypothesis

According to Optimizely, a hypothesis is a prediction you create before running an experiment. A hypothesis clearly states what you’ll change, what you believe the experiment’s outcome will be, and why you think so. Running the experiment will either prove or disprove your hypothesis.

Your hypothesis should express your assumption or prediction about how a certain variable or factor will affect your goal. When defining it, keep the following guidance in mind.

• Be specific about what you’re trying to achieve.
• Choose how you will measure the experiment and in what audience segment. Your metric(s) should be directly related to your goals, such as conversions, revenue, bounce rate, or time on the page.
• Keep your sample size in mind. For example, if you know your website traffic, you can use a time-bound scenario.

Your hypothesis can follow a general structure of, “If _____, then _____ because _____.”

We’ll dive in and share two examples of experiment hypotheses, including common questions you may ask yourself.

You may want to increase your conversion rate on a landing page. But how do you do that, and how much do you want to increase it? What about the duration for which you want that increase to happen? As you can see, it’s important to ask yourself the right questions to get to the specifics of your requirements and ensure a valid sample size.

Experimentation Hypothesis Examples

Example 1

If you deploy a pop-up with a call-to-action (CTA) to a valuable case study on the landing page, then it will increase the conversion rate by 5% within three months because it captures the user’s attention and information on a high-traffic page and allows them to download a valuable case study.

This is great, and you share this with your design on the team, but the designer hates pop-ups. They agree that pop-ups grab the user's attention, so your designer advocates for a variation of the pop-up that uses animation to show the pop-up and then moves it to the side. Hence, it’s less disruptive to the user viewing the content on your landing page. You agree that it would be interesting to test, so you determine the metrics you’ll need to track the success of this test, which audience segment(s) it can be applied to, and what sample size you’ll need to achieve a statistically significant result before you set up this experiment.

Example 2

If you deploy two design variations of a pop-up with a CTA to a valuable case study on the landing page, then it will increase the conversion rate by 5% within 3 months because it captures the user’s attention and information on a high-traffic page and allows them to download a valuable case study.

You can then vet this hypothesis with your team before you figure out which metrics you’ll need to track this experiment’s success, which audience segment(s) it can be applied to, and the sample size you’ll need to achieve a statistically significant result. Once you have that information, you can set up and run your experiment.

The Importance of a Good Experiment Hypothesis

Running an experiment without a hypothesis means its results will be unfocused and won’t yield definitive information you can use to improve your digital experiences. By writing a good experiment hypothesis, you’ll have a concrete idea of what you’re trying to achieve, how you will measure it, and how much data you need to form a solid conclusion. Then you can take what you learn to improve your digital experiences in ways that will make a major impact like increasing leads, conversions, and revenue.

Need help writing an experiment hypothesis or two for your organization? Contact us. Our digital marketing experts can assist you with developing and running experiments that will take the guesswork out of your efforts so you can make changes you know will resonate with your audiences.