Product development is confronted with the constant challenge of supplying the customer with a product that exactly meets his needs. In our new blog series, etventure’s product managers provide an insight into their work and approach. The focus is on hypothesis-driven product development. In the first part of the series, we show why and how to define a verifiable hypothesis as the starting point for an experiment.

For the development of new products, features and services as well as the development of start-ups, we at etventure rely on a hypothesis-driven method that is strongly oriented towards the “Lean Startup”¹ philosophy. Having already revealed our remedy for successful product development last week, we now want to take a closer look at the first step of an experiment – the formulation of the hypothesis.

“Done is better than perfect.” – Sheryl Sandberg

Where do hypotheses come from?

Scientists observe nature and ask many questions that lead to hypotheses. Product teams can also be inspired by observations, personal opinions, previous experiences or the discovery of patterns and outliers in data. These observations are often associated with a number of problems and open questions.

• Who is our target group?
• Why does X do this and not that?
• How can person X be motivated to take action Y?
• How can we encourage potential users to sign up for our service?

First of all, it is important that the team meets for brainstorming and becomes creative. Subsequently, those ideas are selected that are “true” from the team’s point of view and are therefore referred to as hypotheses.

What makes a good hypothesis?

Unlike science, we cannot afford to spend too much time on a hypothesis. Nevertheless, one of the key qualifications of every product developer is to recognize a well-formulated hypothesis. The following checklist serves as a basis for this:

A good hypothesis…

• is something we believe to be true, but we don’t know for sure yet
• is a prediction we expect to arrive
• can be easily tested
• may be true or false
• includes the target group
• is clear and measurable

Assumption ≠ Fact

An assumption may be true, but it may also be false. A fact is always true and can be proven by evidence. Therefore, an assumption always offers an opportunity to learn something. If we already have strong evidence of what we believe in, we don’t need to test it again – there is nothing new to learn. However, we never accept anything as a fact until it has been validated. Awareness of this difference is essential for our product decisions. That’s why we keep asking ourselves questions: Do we have proof of our assumptions, are they facts, or does it end with assumption? In other words: Is it objectively measurable?

Prediction

Human behaviour is often “predictably irrational”.² This is because our brain uses shortcuts when processing information to save time and energy.³ This is also true in product development: We often tend to ignore evidence that our assumption might be wrong. Instead, we feel confirmed in existing beliefs. The good news is that these distortions are consistent and well known, so we can design systems to correct them. In order to avoid misinterpretations of the test results, it helps, for example, to make the following prediction: What would happen if my assumption was confirmed?

Testing

In order for hypotheses to be validated, it must be possible to test them in at least one, but preferably in different scenarios. Since both temporal and monetary resources are usually very limited, hypotheses must always be testable as easy as possible and with justifiable effort.

Testability and falsification

Learning means finding answers to questions. In product development, we want to know whether our assumption is true or not. When testing our ideas, we have to assume that both could happen. What is important is that both results are correct, both mean progress. This concept, is derived from science⁴ and helps to avoid an always applicable hypothesis such as “Tomorrow it will either rain or not”.

Target group

Product development should mainly focus on the customer’s needs. Therefore, the target group must be included in the formulation of the hypothesis. This prevents distortion and makes the hypotheses more specific. During development, hypotheses can be refined or the target audience can be adapted.

Clarity and measurability

And last but not least, a hypothesis must always be clear and measurable. Complex hypotheses are not uncommon in science, but in practice it must be immediately clear what is at stake. Product developers should be able to explain their hypotheses within 30 seconds to someone who has never heard of the subject.

Why formulate hypotheses?

Product teams benefit in many ways if they take the time to formulate a hypothesis.

• Impartial decisions: Hypotheses reduce the influence of prejudices on our decision-making.
• Team orientation: Similar to a common vision, a hypothesis strengthens team thinking and prevents conflicts in the experimental phase.
• Focus: Testing without hypothesis is like sailing without a goal. A hypothesis helps to focus and control the experimental design.

How can good hypotheses be formulated?

Various blogs and articles provide a series of templates that help to formulate hypotheses quickly and easily. Most of them differ only slightly from each other. Product teams can freely decide which format they like – as long as the final hypothesis meets the above criteria. We have put together a selection of the most important templates:

• We believe that [this ability] will lead to [this result]. We will know that we have succeeded when [we see a measurable sign].
• I believe that [target group] will [execute this repeatable action/use this solution], which for [this reason] will lead to [an expected measurable result].
• If [cause], then [effect], because [reason].
• If [I do], then [thing] will happen.
• We believe that with [activity] for [these people] [this result / this effect] will happen.

The following hypotheses have actually been used by us in the past weeks and months. During the test phase some of them could be validated, others were rejected.

• After 1,000 visits to the registration page, the reduction of registration steps from 3 to 1 increases the registration rate for new visitors by 25%.
• This subject line increases the opening rates for newsletter subscribers by 15% after 3 days.
• If we offer online training to our customers, the number of training sessions will increase by 35% within the next 2 weeks.
• We believe that the sale of a machine-optimized packaging material to our customers will lead to a higher demand for our packaging material. We will know that we have been successful if we have sold 50% more packaging material within the next 4 weeks.

How to turn hypotheses into experiments?

Formulating good hypotheses is essential for successful product development. And yet it is only the first step in a multi-step development and testing process. In our next article you will learn how hypotheses become experiments.

Further links:

1 Eric Ries: The Lean Startup

2 Predictably Irrational: The Hidden Forces that Shape Our Decisions

3 Cognitive Bias Cheat Sheet

4 Karl Popper