Overview
The primary distinction between generative research and evaluative experiments lies in having a clear, testable hypothesis. This guide teaches how to construct robust hypotheses for product management testing within GLIDR.
Key Elements of a Strong Hypothesis
A well-crafted hypothesis includes four components:
- The change - The single modification you're implementing
- The impact - Expected results from the change
- The metric - Measurable success or failure threshold
- The timebox - Duration for running the test
Template Structure
"This new feature will cause a 10 percent increase of new users visiting the homepage in 3 months."
Detailed Breakdown
The Change: One isolated modification to test (e.g., button color change or marketing campaign launch)
The Impact: Expected outcome if you alter variable x, then y should occur
The Metric: A measurement that needs to be hit or surpassed to determine success or establish when to pivot
The Timebox: Sufficient duration to collect meaningful data without unnecessary delays
Common Pitfalls to Avoid
- Testing multiple variables simultaneously prevents identifying which caused results
- Lacking measurable metrics makes determining success/failure impossible
- Disconnecting outcomes from experimental changes creates false causation
- Setting timeframes that are unreasonably long or short relative to company growth
Hypothesis Checklist
Ensure your hypothesis is:
- Simple and unambiguous
- Measurable
- Describing a relationship between two elements
- Clear in cause-and-effect
- Achievable
- Falsifiable (evidence could prove it wrong)
Adapted from The Real Startup Book, edited by Tristan Kromer and published by Kromatic, under a Creative Commons Attribution-ShareAlike 4.0 International License.