An official website of the United States government
Here’s how you know
Official websites use .gov A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS A lock ( Lock Locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Guidance on building better digital services in government
Latest News
Using a Hypothesis-Driven Approach in Analyzing (and Making Sense) of Your Website Traffic Data
At the Digital Analytics Program (DAP), some of the most frequently asked questions we get are “how can I get access to the DAP data?” and “what do I do with all this data?” We all know that data is knowledge, and knowledge is power, but once we have access to it and realize that it is, indeed, oceans of data , how do we not “drown” in it, and, perhaps more importantly, how do we make sense of it?
What Questions Are You Trying to Answer?
In research and data analysis, a hypothesis-driven approach is one of the main methods for using data to test and, ultimately, prove (or disprove) assertions. To do that, researchers collect a sufficient amount of data on the subject and then approach it with a specific hypothesis in mind. On the flipside, there’s also exploratory data analysis, where data analysts “dive” into data in search of patterns (or lack there of). Often times, exploratory data analysis creates a foundation for hypotheses, or feeds our assertions that then get used as part of hypothesis-driven testing.
So what does this all mean? It means that when you are attempting to perform an analysis using multidimensional website traffic data, you, too, should approach it with a goal in mind, or, at minimum, specific questions you want to be answered. Hence, when people ask us, “what do I do with all this data?” our response is “what questions are you trying to answer?”
Example: Real-Time Gov-Wide Visitors on #TaxDay
Let’s use this in a concrete example. On April 14, 2015, in a conversation with my team regarding the gov-wide website traffic on #TaxDay, I casually made an assertion that the number of real-time visitors on April 15, 2015, may reach 300,000. The assertion was based on the fact that in almost three years of the DAP’s life, we’ve seen several big spikes (up to 220K real-time visitors), which had happened before IRS.gov became part of the DAP. So, I figured, now that DAP monitors IRS.gov traffic, in addition to other large governement websites, perhaps we’ll see a big spike in real-time visitor traffic on #TaxDay because lots of people will be online filing their tax returns and/or extensions on the last day. My hypothesis was not based on any previously done exploratory analysis.
To see if we’d reach that new record of real-time traffic in DAP on #TaxDay, our DigitalGov team performed live blogging and hourly monitoring throughout the day yesterday to report on the real-time users on the .gov websites as part of the public dashboard . And…. the hypothesis did not hold true (and was rejected by reality). The highest number of real-time visitors gov-wide we got on #TaxDay for 2015 was just shy of 200K.
What Happens After the Hypothesis is Tested
So now that the results of my hypothesis are known, the big question is “why?”
Well, the data tells us that for the last three months, the “ Where’s My Refund ?” page has been consistently in the top performing pages with the highest number of real-time users and a total 115M+ pageviews since February 1, 2015. IRS-related pages overall have been dominating the top 20 active pages consistently in the last three months, suggesting that people were filing their taxes online during the last few months and, then, naturally, spent most of their time wondering when they get their refund. We did see a spike in the extension applications [PDF] downloads yesterday, which makes sense, but the number of visitors filling out the application was not high enough to bring us anywhere even close to 300K real-time visitors.
Interestingly, the uptick in overall traffic to .gov websites on #TaxDay was modest compared to previous events driven by weather, space shuttle launches, and asteroid fly-bys. Hmmm…. that may be a good topic for a different blog post.
In Conclusion
The DAP yields a lot of data and may be overwhelming but is a goldmine for exploratory and hypothesis-driven analyses. With the right questions in mind, this data can help the government to better understand its visitors and what brings them to agency websites, and ultimately, continuously create better web content and digital services for online visitors.
Originally posted by Marina Fox on Apr 16, 2015 Marina Fox
GSA | Washington, DC
Join a Community
- Artificial Intelligence (AI)
- Web Analytics and Optimization
- Web Managers
Get Started Building
Related Topics
2015-04-16-using-a-hypothesis-driven-approach-in-analyzing-and-making-sense-of-your-website-traffic-data.md
news/2015/04/2015-04-16-using-a-hypothesis-driven-approach-in-analyzing-and-making-sense-of-your-website-traffic-data.md
Link Shortcode
" >}}
Join 60,000 others in government and subscribe to our newsletter — a round-up of the best digital news in government and across our field.
Digital.gov
An official website of the U.S. General Services Administration
Lindsay Angelo
Hypothesis driven problem solving explained: tactics and training.
What is hypothesis driven problem solving? How do I apply hypothesis driven problem solving to business? What are the steps to hypothesis driven problem solving? This blog post explores all of these questions - and then some.
What is hypothesis-driven problem solving?
Hypothesis driven problem solving also known as "top-down problem solving" or "hypothesis driven thinking" is a form of problem-solving that starts with the answer and works backward to prove or disprove that answer. Practiced by the biggest consulting firms around the globe for its effectiveness in getting to the heart of the matter, hypothesis-driven thinking is rooted in the scientific method.
Whereas bottoms-up problem solving (a non-hypothesis-driven approach) analyzes the data/information to arrive at your answer, top-down identifies an answer and looks to data/information to validate it. Comparatively speaking, bottoms-up problem solving can be a never-ending process, whereas top-down is laser-focused - and for that reason, effective.
In business leadership, the typical problem-solving approach practiced tends to be bottoms-up problem solving, however if you take the time to learn and apply it, top-down is often much more effective, particularly when you’re dealing with a problem that’s defined and a tight timeline.
What is a hypothesis driven approach or method?
A hypothesis-driven approach is one where you state your assumptions about what you think the answer is, and then fact-find to validate or refute. This helps focus your data gathering on exactly what you need vs. "boiling the ocean". It also helps to ensure you’ve thought through the entirety of the problem and that there is rigor and structure in your thinking.
How do I apply hypothesis driven thinking?
The four steps to hypothesis-driven problem solving are simple. In a nutshell:
1) Define the problem
The first step is to define the problem. This may seem like an obvious step, but it's important to be clear about what you're trying to solve. Sometimes people jump right into solving a problem without taking the time to fully understand it. Defining the question you're trying to answer and your problem helps ensure that you're focusing on the right issue and prevents you from wasting time and energy. This may seem easy at first glance but to quote Albert the great “if I had an hour to solve a problem, I’d spend 55 minutes on the problem and 5 on the solution.” Defining your problem takes rumination and time.
2) Develop your initial hypothesis
The second step in hypothesis-driven thinking is to come up with an initial hypothesis. An initial hypothesis is a proposed explanation for an event or phenomenon that can be tested. It's important to note that a good hypothesis doesn't have to be correct – it just has to be plausible.
For example, let's say you're trying to improve customer satisfaction at your company. Your hypothesis could be that providing more customer service training will improve satisfaction, or perhaps hiring more seasoned employees/agents. This answer-driven approach gets you thinking early about the solution early on.
At this stage of the work, it's not uncommon to brainstorm multiple key hypotheses before you narrow the field.
From there, you're going to flesh out your logic taking a decision tree approach. That's thinking through what needs to be true for your hypothesis to be true. Fast forward and you will end up with a decision-tree with the 1st level being your hypothesis, the 2nd level being your supporting assumptions or logic and the 3rd level downward being the fact points that you'll need to uncover. This tree informs your work plan. Want a real world hypothesis tree to work from? Sign up to the right to claim your free hypothesis problem solving template.
3) Gather and analyze information to validate or refute your hypothesis
The third step is to gather information to validate or refute your hypothesis. This can be done in a number of ways, including surveys, interviews, focus groups, and data analysis.
It's important to note that you should constantly be gathering new information throughout the problem-solving process. You never want to stop learning – that's how you find the best solutions. In this step, ideally, you're looking for measurable evidence to validate or refute your assumptions. The term "acceptable evidence thresholds" is often used to describe the certainty you're looking to arrive at. Apply the 80/20 rule here - that’s enough evidence to get to 80% certainty. Many analytical approaches and methods can be used in this step.
Once you've gathered all of your information, it's time to analyze it and see if your hypothesis was correct.
4) Pivot your hypothesis and arrive at your solution
If the information you've gathered points to your hypothesis being correct, great! You can move on to step four. If not, don't worry – you can adjust your hypothesis and try again. Pivot to alternative hypotheses as many times as needed. Hypothesis-driven thinking is an iterative process.
With subsequent validation, eventually, you will arrive at your solution, backed by evidence. You've just shifted from many potential solutions to THE solution.
Hypothesis driven problem solving is a great way to solve complex problems. By breaking the problem into smaller parts, it's easier to develop and test hypotheses. And by analyzing the results, you can determine whether your hypothesis was correct.
It's more and more common to see hypothesis thinking used in a variety of fields, including as an approach to software development.
Looking to improve your problem-solving skills? Give hypothesis driven problem solving a try. It's a great way to systematically solve complex problems using an analytical approach - and gets you to results that are both accurate and timely. That’s why it’s the bread and butter of the biggest consulting companies around the world from McKinsey & Company to Bain & Company to BCG, and many more.
What is hypothesis driven consulting?
Hypothesis-driven consulting is applying a hypothesis-driven approach to complex problems that arise in client engagements. It's commonly used and coveted by top consulting firms, including McKinsey, due to its effectiveness at solving business problems. It's core to the Consulting process and some expect their interviewees to approach case studies and case study interviews by applying a hypothesis approach. Amongst many consulting tools, hypothesis thinking is clutch. Trying to land a role on a top-tier Consulting Team? Prepping for a Consulting Interview? Learn this tool and you will be ahead of the game.
What problems can I apply hypothesis driven thinking to?
You can apply this thinking to a broad array of problems/opportunities. How do I diversify my revenue streams? What kind of business models should I optimize for? What's the market size of China? What's our market share in the US? Why are we seeing an increase in costs? How do we grow our customer base? What's our customer preference when it comes to returns? Why are business class ticket sales down? Why are we seeing a decline in revenue? Why are fuel costs rising? What color should we paint the fence?
A hypothesis-driven approach is a proven problem-solving process that will elevate your strategic thinking, help you craft power business strategies, accelerate your effectiveness, and is an agile practice that serves you well in today's rapidly changing climate.
Does hypothesis-driven problem solving training exist?
Due to an overwhelming need, we've created just that, a virtual training course on hypothesis-driven training that’s nested within our Strategic Thinking training. Student learning outcomes include improving your effectiveness, thinking and acting with impact, and levelling up your problem-solving skills. We call it problem-based learning at its finest. Enter your email below to download the free template and we will email you when the course becomes available!
What is strategic thinking training?
It’s training that teaches you to think more strategically about your work, business - and life. In the context of business, it teaches you how to make powerful and effective strategic decisions in the context of your business strategy.
Is top-down thinking an experimental approach?
Yes, it's an experimental approach that closely resembles the scientific method.
What is hypothesis-driven development?
Hypothesis-Driven development is just another way to describe the process of taking a hypothesis-driven approach. Hypothesis-driven development is essentially a synonymous term.
Is top-down thinking used as an approach to software development?
Yes, resembling components of "agile thinking", hypothesis thinking is used as a common approach to software development. Frequent real-time feedback loops are applied to develop a product.
What's the process of hypothesis-driven thinking?
It's an easy four-step process - read the article above for more background knowledge.
Hypothesis-driven thinking can be used across many modalities and functions - what are some examples from strategy consulting?
Stay tuned for my next blog post - I’ll cover how the method is used in strategy and in strategic decision-making
What are some other use cases for hypothesis thinking?
Use cases range from macro to micro: identifying market trends, product development, crafting digital products, UX, driving customer demand, statistical analysis, statistical tests, continuous improvement, design sprints, and business development. Leadership teams often use it for solving organizational wide problems/issues and it can be applied to solve almost any typical strategy engagement.
Can hypothesis-thinking be used if trying to determine a market size? Or in trying to determine why a decrease in revenue? Or a decline in sales? Or an increase in costs?
Yes, it can used to help work through all of these questions/problems.
Which consulting firms and consulting partners use this method?
Most big firms, including Bain & Company, McKinsey & Company, BCG, etc. apply this tool in real life engagements.
Is the problem solving course self-directed learning?
Yes and it's accompanied by (un)formal discussion, hands-on practice, mini-cases and strategy cases, and inquiry-based learning. We've worked hard to create an engaging learning process designed to be delivered to you anytime, anywhere.
What are the course learning outcomes?
Learning and development outcomes can be found here
Who is this course for? Could my team benefit from it?
Any and all working professionals are welcome - we serve business leaders, individual contributors, executives and any team looking to level up their strategic thinking - come one come all.
Can this course help with my communication skills?
Yes, it includes a module on narrative-driven story-telling, a natural derivative of hypothesis problem-solving.
What's an issue tree?
An issue tree is a specific type of logic tree. See this blog post for issue trees, explained!
What's MECE?
MECE stands for Mutually, Exclusive, Collectively Exhaustive and forms the core of hypothesis driven thinking and logic trees. See this post for an explanation and guide on MECE.
About the Author
Lindsay provides growth and advisory services to purpose-driven brands. Named a global innovation leader and Women to Watch, you will find her at the intersection of strategy, story-telling and innovation. When she’s not collaborating with clients, she’s hitting TEDx and other stages across North America to deliver keynotes on the future of consumerism, strategy and innovation. Prior to advising and providing brand and marketing consulting services, Lindsay spent six years at lululemon crafting their global growth strategy, exploring new marketplace opportunities and growing the company into the number one yoga wear player in the world. Her experiences culminate in what she refers to as her sweet spot - where strategy, innovation and insights intersect, where the rational meets the emotive, where facts meet insights and where logic meets creativity.
HOME | MY WORK | THINKING | SPEAKING | CONSULTING | BIO | CONTACT
The Hypothesis Driven Approach: The key to the Consulting Mindset
- Case Interview: A comprehensive guide
- Pyramid Principle
- Hypothesis driven structure
- Fit Interview
- Consulting math
- What is it?
- How do they differ?
Taking an hypothesis driven approach to a problem means attempting to solve that problem by focussing on your best hypothesis as to the answer . This helps arrive at a solution quickly and efficiently.
The idea of being "hypothesis driven" is one of those concepts which is thrown around constantly, both in consulting circles and in the research world . With a quick Google search, you will find a slew of articles, books and videos about the hypothesis driven approach. Looking through these resources as ex-consultants left us quite puzzled, as the various authors all seemed to be trying to needlessly complicate a relatively simple concept .
To the uninitiated, the term "hypothesis driven" might either seem rather technical, complex or, alternatively, like a meaningless piece of management speak. However, the idea of taking an hypothesis driven approach to problems is a simple but genuinely powerful idea which should characterise your approach to all aspects of your interview .
Prep the right way
Let's get a better idea of what exactly we mean by the term "hypothesis driven", before seeing how this way of working differs from more standard methods.
1. What is it?
When we work in an hypothesis driven way, if the cause of some problem is A, B, C or D, we start by assuming the most ostensibly likely option - say A - is the cause and then checking whether this is true or not. If it is not, then we iterate the same process with the other options - B, C and D - in order of their likelihood until we find the correct answer.
This seems pretty sensible, but it is actually very different to how most people will work in practice . Especially if you are coming from an academic or research background, you will likely try to analyse the entirety of a system before going on to focus on any of its parts. This is all very rigorous and appropriate in its own context, but consultants are employed to generate solutions . Analysis is only a means to that end.
The hypothesis driven approach is both dynamic and efficient and means that you will always be moving forward towards a solution to the main problem. You will maintain tight focus and avoiding getting side-tracked trying to do everything.
The best way to appreciate the difference between hypothesis driven and more typical approaches to problem solving is to have a look at an example of how they differ in tackling the same case . To this end, let's take a look at two ways in which a candidate might respond to the following case prompt:
Revenues for a leading supermarket chain have been declining over the last 10 years. How can we address this?
2.1. Non-hypothesis driven Approach
Given this prompt, a candidate taking a non-hypothesis driven approach would likely come up with a list of possible reasons for the observed declining revenue. This would then be delivered back to the interviewer as a list of questions seeking more information as to each possible cause . Thus, the candidate might respond:
We know revenues have declined. I want to find out why:
- Did customers change their preferences?
- Which segment has shown most decline in volume?
- Is there a price war in the industry?
- ...etc, etc...
2.2. Hypothesis Driven Approach
By contrast, a second candidate taking a more hypothesis driven approach might tackle the case as follows:
Candidate: We know revenues have declined. This could be due to price or volume. Do we know how these changed? Interviewer: Prices have remained constant. Volume has declined in line with revenue. Candidate: Since we know volume is the problem, this could be because the market size has contracted or that the client's market share has been reduced. Do we know how each of these have changed? Interviewer: The market size has remained the same, but our market share has reduced. ...
3. How do they differ?
To understand how typical non-hypothesis driven and hypothesis driven approaches differ, we will compare them as to their impact on both analysis and communication, breaking down different factors within each of these areas.
Looking for an all-inclusive, peace of mind program?
Ostensibly, the approach taken by the first candidate might have seemed sensible enough. However, as we contrast the two approaches, we will see that this naive, non-hypothesis driven approach is inherently and critically flawed .
3.1. Analysis
3.1.1. Relevance
The first candidate basically generates a laundry list of questions for the interviewer. Of course, some of these queries might be relevant, but the candidate has done nothing to place them within a wider structure. This makes it unclear how their questions are relevant to one another and to the client's main issue.
As a result, even if the interviewer actually gives answers to this whole slew of questions (they might well not...), it will be far from clear as to how those answers relate to one another or feed into any single conclusion. For instance, say that the interviewer responds to the questions above by telling us that a larger decline in sales had been seen in the over 50s and that there was a price war with rival companies over certain product lines. These isolated facts do not actually tell us very much in and of themselves and certainly don't begin to suggest how we might begin to address the client's problem.
By contrast, the second candidate, taking an hypothesis driven approach, has been very effective in structuring the problem . They work from identifying two initial factors (price and volume) to drill down and eliminate irrelevant elements as they go. The structure they impose on the problem means that it is clear how each of their questions is ultimately relevant to the client's problem .
3.1.2. MECE
The main underlying reason why the questions asked by the first candidate are so inefficient is that they are not addressing MECE subjects (if you are not familiar with the MECE concept yet, see our article here ).
Their questions ask about overlapping subjects (are not mutually exclusive) and are also unlikely to cover all factors which might be relevant to the case (not collectively exhaustive). The overall result is that, even if the candidate is given answers to all their queries, the information they end up with will likely be both ambiguous and incomplete and they might out on some possibilities altogether.
For example, the first two questions - asking about changing customer preferences and the behaviours of certain segments - overlap in terms of what they are referring to. If, say, over 50s are looking elsewhere for healthier options, this will be reflected in the answers to both. The third question asks about price wars, but price is only one aspect of competition between firms. Customers might well be favouring competitors due to geographical convenience or better customer service rather than just due to price.
Everything you need in one place
The more structured nature of the hypothesis driven approach taken by the second candidate means that their questions are indeed MECE . Thus, questions refer to separate subjects and, between them, cover all possibilities at a given level of analysis. This ultimately allows the candidate to gain more useful information from fewer questions whilst not missing any important possibilities.
3.1.3. Answering the question
The most crucial difference between the two approaches is in whether they actually answer the question . The first candidate does not really get to grips with the actual problem and we can expect that they will fail to provide a single, clear answer for the supermarket CEO. By contrast, the second candidate makes use of their hypothesis driven approach to quickly narrow down the problem area and we can expect that they will quickly be able to deliver a decisive client recommendation.
You might think it is unlikely that you would fail to actually answer the question in a real interview. Surely, you are smarter than that, right? In fact, not giving a single clear solution is a major stumbling block for a large number of perfectly clever candidates, who often tend to conduct a broad analysis of the scenario generally whilst failing to relate this back to the client's concern.
This is especially common for candidates using traditional "framework" based case systems , where the generic scheme applied might not fully capture the case or address the client issue. You can see an example of this in our article on Problem Driven Structures , where we see how Victor Cheng's Mergers and Acquisitions framework fails to actually generate a solution when applied to a case.
The hypothesis driven approach is characterised by a constant focus on the main question . As we have seen above, all of the candidate's questions and analysis are ultimately driven by, and feed back into, addressing the client's main concern. Thus, the way an hypothesis driven approach structures analysis means that it is almost impossible to avoid arriving at a solution .
All the best resources in one place!
3.2. communication.
3.2.1. Impression
Your interviewer will be looking for an hypothesis driven approach to problem solving as a sign that you have the correct mindset for consulting. By taking an hypothesis driven approach, the second candidate demonstrates their ability to structure problems logically, to ask precise and relevant questions and generally to work in an efficient, focussed manner .
By contrast, the first candidate might have demonstrated a little general business knowledge, but fails to display any of these other traits. Rather, they will come off as over-eager and lacking reflective thought . The interviewer will find the barrage of queries associated with the standard approach vexing and the candidate risks appearing as if they are simply trying fish for answers from the interviewer, rather than being willing to actually analyse the problem for themselves. The second candidate asks questions more sparingly and gives a strong rationale for each in turn. This is precisely what the interviewer wants to see.
3.2.2. Navigation
The first candidate's disorganised approach to asking questions and lack of focus or clear direction risks losing the interviewer - who will not be able to follow their train of thought.
A candidate working on too many lines of reasoning at once will risk confusing not only the interviewer, but also themselves . Candidates who do not take a sufficiently hypothesis driven approach often become muddled or get bogged down in some tangential aspect of analysis. In terms of communication, this will manifest in candidates repeating themselves and jumping around between topics when talking to the interviewer, with a general lack of clear direction.
The second candidate's analysis is much more sophisticated. However, the more structured approach - particularly the fact that only one issue is being dealt with at a time - makes it much easier for the interviewer to follow . This also helps to prevent the candidate themselves getting muddled or sidetracked.
3.2.3. Guidance
On a pragmatic note, an interviewer might give a promising candidate a nudge in the right direction if they see that their analysis has overlooked something important or risks getting stuck in a dead end.
Impress your interviewer
The fact that the hypothesis driven approach of the second candidate makes their rationale so clear means that it will be easy for the interviewer to intervene in this manner. For the first candidate, the interviewer will probably not be able to be obliging. A helpful nudge on its own could make the difference between getting a job and being rejected - and it won't be possible if your interviewer has no idea what you are up to!
4. Next Steps
All of this has shown just how crucial it is to take an hypothesis driven approach to your case interview. The best way to ensure that you approach cases in an hypothesis driven manner is to use our priority driven structure case method . This highly structured case solving technique builds in an hypothesis driven approach at its core.
You can read more in our article on problem driven structures , whilst the MCC Academy teaches this method along with all the background knowledge and skills you will need to implement it effectively.
To be able to properly make use of an hypothesis driven approach to cases, you will need a good grasp of how to segment and the MECE principle that underlies valid segmentations and ultimately allows for problems to be structured in a hypothesis driven fashion.
We have focussed on the case interview here, but many of the same issues we have discussed around taking a structured, focussed approach will be just as relevant to your fit interview . Remember - the different portions of a consulting interview are really different ways for the interviewer to look for the same general capabilities and mindset. You can find out more about how to take a structured approach to your fit interview in our article here .
Find out more in our case interview course
Ditch outdated guides and misleading frameworks and join the MCC Academy, the first comprehensive case interview course that teaches you how consultants approach case studies.
Discover our case interview coaching programmes
Discover our career advancement programme
Account not confirmed
The 6 Steps that We Use for Hypothesis-Driven Development
Uptech is top-rated app development company
Table of content.
One of the greatest fears of product managers is to create an app that flopped because it's based on untested assumptions. After successfully launching more than 20 products, we're convinced that we've found the right approach for hypothesis-driven development.
In this guide, I'll show you how we validated the hypotheses to ensure that the apps met the users' expectations and needs.
What is hypothesis-driven development?
Hypothesis-driven development is a prototype methodology that allows product designers to develop, test, and rebuild a product until it’s acceptable by the users. It is an iterative measure that explores assumptions defined during the project and attempts to validate it with users’ feedbacks.
What you have assumed during the initial stage of development may not be valid for the users. Even if they are backed by historical data, user behaviors can be affected by specific audiences and other factors. Hypothesis-driven development removes these uncertainties as the project progresses.
Why we use hypothesis-driven development
For us, the hypothesis-driven approach provides a structured way to consolidate ideas and build hypotheses based on objective criteria. It’s also less costly to test the prototype before production.
Using this approach has reliably allowed us to identify what, how, and in which order should the testing be done. It gives us a deep understanding of how we prioritise the features, how it’s connected to the business goals and desired user outcomes.
We’re also able to track and compare the desired and real outcomes of developing the features.
The process of Prototype Development that we use
Our success in building apps that are well-accepted by users is based on the Lean UX definition of hypothesis. We believe that the business outcome will be achieved if the user’s outcome is fulfilled for the particular feature.
Here’s the process flow:
How Might We technique → Dot voting (based on estimated/assumptive impact) → converting into a hypothesis → define testing methodology (research method + success/fail criteria) → impact effort scale for prioritizing → test, learn, repeat.
Once the hypothesis is proven right, the feature is escalated into the development track for UI design and development.
Step 1: List Down Questions And Assumptions
Whether it’s the initial stage of the project or after the launch, there are always uncertainties or ideas to further improve the existing product. In order to move forward, you’ll need to turn the ideas into structured hypotheses where they can be tested prior to production.
To start with, jot the ideas or assumptions down on paper or a sticky note.
Then, you’ll want to widen the scope of the questions and assumptions into possible solutions. The How Might We (HMW) technique is handy in rephrasing the statements into questions that facilitate brainstorming.
For example, if you have a social media app with a low number of users, asking, “How might we increase the number of users for the app?” makes brainstorming easier.
Step 2: Dot Vote to Prioritize Questions and Assumptions
Once you’ve got a list of questions, it’s time to decide which are potentially more impactful for the product. The Dot Vote method, where team members are given dots to place on the questions, helps prioritize the questions and assumptions.
Our team uses this method when we’re faced with many ideas and need to eliminate some of them. We started by grouping similar ideas and use 3-5 dots to vote. At the end of the process, we’ll have the preliminary data on the possible impact and our team’s interest in developing certain features.
This method allows us to prioritize the statements derived from the HMW technique and we’re only converting the top ones.
Step 3: Develop Hypotheses from Questions
The questions lead to a brainstorming session where the answers become hypotheses for the product. The hypothesis is meant to create a framework that allows the questions and solutions to be defined clearly for validation.
Our team followed a specific format in forming hypotheses. We structured the statement as follow:
We believe we will achieve [ business outcome],
If [ the persona],
Solve their need in [ user outcome] using [feature].
Here’s a hypothesis we’ve created:
We believe we will achieve DAU=100 if Mike (our proto persona) solve their need in recording and sharing videos instantaneously using our camera and cloud storage .
Step 4: Test the Hypothesis with an Experiment
It’s crucial to validate each of the assumptions made on the product features. Based on the hypotheses, experiments in the form of interviews, surveys, usability testing, and so forth are created to determine if the assumptions are aligned with reality.
Each of the methods provides some level of confidence. Therefore, you don’t want to be 100% reliant on a particular method as it’s based on a sample of users.
It’s important to choose a research method that allows validation to be done with minimal effort. Even though hypotheses validation provides a degree of confidence, not all assumptions can be tested and there could be a margin of error in data obtained as the test is conducted on a sample of people.
The experiments are designed in such a way that feedback can be compared with the predicted outcome. Only validated hypotheses are brought forward for development.
Testing all the hypotheses can be tedious. To be more efficient, you can use the impact effort scale. This method allows you to focus on hypotheses that are potentially high value and easy to validate.
You can also work on hypotheses that deliver high impact but require high effort. Ignore those that require high impact but low impact and keep hypotheses with low impact and effort into the backlog.
At Uptech, we assign each hypothesis with clear testing criteria. We rank the hypothesis with a binary ‘task success’ and subjective ‘effort on task’ where the latter is scored from 1 to 10.
While we’re conducting the test, we also collect qualitative data such as the users' feedback. We have a habit of segregation the feedback into pros, cons and neutral with color-coded stickers. (red - cons, green -pros, blue- neutral).
The best practice is to test each hypothesis at least on 5 users.
Step 5 Learn, Build (and Repeat)
The hypothesis-driven approach is not a single-ended process. Often, you’ll find that some of the hypotheses are proven to be false. Rather than be disheartened, you should use the data gathered to finetune the hypothesis and design a better experiment in the next phase.
Treat the entire cycle as a learning process where you’ll better understand the product and the customers.
We’ve found the process helpful when developing an MVP for Carbon Club, an environmental startup in the UK. The app allows users to donate to charity based on the carbon-footprint produced.
In order to calculate the carbon footprint, we’re weighing the options of
- Connecting the app to the users’ bank account to monitor the carbon footprint based on purchases made.
- Allowing users to take quizzes on their lifestyles.
Upon validation, we’ve found that all of the users opted for the second option as they are concerned about linking an unknown app to their banking account.
The result makes us shelves the first assumption we’ve made during pre-Sprint research. It also saves our client $50,000, and a few months of work as connecting the app to the bank account requires a huge effort.
.png "hypothesis driven meaning")
Step 6: Implement Product and Maintain
Once you’ve got the confidence that the remaining hypotheses are validated, it’s time to develop the product. However, testing must be continued even after the product is launched.
You should be on your toes as customers’ demands, market trends, local economics, and other conditions may require some features to evolve.
Our takeaways for hypothesis-driven development
If there’s anything that you could pick from our experience, it’s these 5 points.
1. Should every idea go straight into the backlog? No, unless they are validated with substantial evidence.
2. While it’s hard to define business outcomes with specific metrics and desired values, you should do it anyway. Try to be as specific as possible, and avoid general terms. Give your best effort and adjust as you receive new data.
3. Get all product teams involved as the best ideas are born from collaboration.
4. Start with a plan consists of 2 main parameters, i.e., criteria of success and research methods. Besides qualitative insights, you need to set objective criteria to determine if a test is successful. Use the Test Card to validate the assumptions strategically.
5. The methodology that we’ve recommended in this article works not only for products. We’ve applied it at the end of 2019 for setting the strategic goals of the company and end up with robust results, engaged and aligned team.
You'll have a better idea of which features would lead to a successful product with hypothesis-driven development. Rather than vague assumptions, the consolidated data from users will provide a clear direction for your development team.
As for the hypotheses that don't make the cut, improvise, re-test, and leverage for future upgrades.
Keep failing with product launches? I'll be happy to point you in the right direction. Drop me a message here.
have a project for US?
Let’s build your next product! Share your idea or request a free consultation from us.
Mobile App Design in A Nutshell
How To Build A Product With User Centered Design Approach
How Cross-functional Teams Help Us In Building Better Digital Products?
Product Design Process For Great Mobile Apps
Sign up for our newsletter.
Tell us about your idea. We will reach you out.
Thanks for reaching out.
Uptech is a top-rated product development company. Over 6 years of work we've helped over 150 companies to build successful mobile and web apps .
Let’s discuss your development needs.
- Skip to primary navigation
- Skip to content
- Skip to footer
What Is A Hypothesis
Case interviews are often challenging because they are open-ended, with limitless possibilities of where you can go. There is some truth to this predicament – there is always more than just one way to crack a case. However, there are also ways to follow a system for every case you encounter that leads to fruitful results. One key component systematic way of consistently cracking case interviews involves being hypothesis-driven. But how do you do hypothesis-driven case interviews and to cover the bases, what is a hypothesis?
Hypothesis Definition
In the context of a consulting interview, a hypothesis definition is “a testable statement that needs further data for verification”. In other words, the meaning of a hypothesis is that it’s an educated guess that you think could be the answer to your client’s problem.
A hypothesis is therefore not always true. Instead, it is a starting point that ultimately leads you to the end point. Imagine your client comes to you with a problem, and the root cause is A, B, C, D, or E. Forming a hypothesis allows you to start with A, gather data to see if it’s correct, and if not, move onto B. You then keep going until you get to the right “letter” or answer to the case.
To be clear, you don’t always know your options upfront at the start of a case interview. Usually, after you gather data, you may find that option A was completely wrong, somewhat wrong, or right on track. Depending on the data, you either move onto a new hypothesis, revise it, or dig for more data, respectively. But for purposes of using it in a case interview, we feel this is a good hypothesis definition.
Hypothesis Example
Let’s use an example to shed some more light on what a hypothesis is, and how to use them in case interviews. Imagine your client is a shoe manufacturing company that has experienced a decrease in profitability over the past 12 months.
Non-Hypothesis Driven Approach
An approach without a hypothesis might result in a laundry list of questions like in the following exchange:
I understand that our client is looking to solve its profitability issues. I have identified a few areas that I’d like to look into.
- Q: Has our client’s customer base declined?
- A: No, the number of customers has actually increased.
- Q: Oh, interesting. Has the client’s market share decreased?
- A: No, market share has actually increased.
- Q: Got it. Then has there been an increase in costs?
- A: Yes, the shoe manufacturer recently invested in a new facility and has less negotiating power with its suppliers, driving up costs. Here is the data…
In this exchange, even though the candidate is getting closer to the right answer, there is no structure in the approach. The candidate is merely guessing potential problems rather than systematically getting to a solution.
Hypothesis Driven Approach
Using a hypothesis driven approach requires the following steps:
- State a hypothesis based on the provided information.
- Gather data to test the hypothesis.
- Revise hypothesis as needed or offer a completely new one if the data proves your original hypothesis wrong.
- Repeat steps 2-3 for additional buckets in your framework.
For example, you may start your hypothesis with a focus on revenue for a profitability issue. If you find that the reason is due to a decrease in volume, you may next hypothesize that the issue is due to an increase in competition. You then ask for data regarding the competition, and adjust your hypothesis accordingly to the data or lack thereof.
Hypothesis Driven Approach Example
Let’s next see what a hypothesis driven approach looks like:
I understand that our client is looking to solve its profitability issues. My hypothesis is that the client is experiencing a decrease in revenue due to intense competition in the shoe market.
- Q: Do we have any data on the sales volume?
- A: We do, volume has actually been increasing.
- Q: Oh, interesting. In that case, do we have any information on how prices have changed recently?
- A: Prices have stayed the same.
- Q: Got it. In that case, it seems like revenue is not the problem here. I would like to revise my earlier hypothesis and assume that our client is experiencing cost issues. Perhaps the fixed costs have increased due to investments or variable costs have increased due to an increase in raw material costs. Do we have any information on the fixed and variable costs?
- A: We do, both variable and fixed costs have increased dramatically over the past 12 months. Here is the data…
As you can see, in this exchange, the candidate is drilling down into a hypothesis and sounds structured in his or her approach. The interviewer can be sure that even if the candidate is provided with another problem, he or she would be able to systematically get to the answer.
To be clear, you don’t need to always state “my hypothesis is X.” In fact, it may sound too robotic in an actual interview. This is just an example to show you how the hypothesis-driven approach looks.
Using this approach ensures that you are displaying some of the key skills that consulting firms care about in the case interview: structure and clarity . If you can be hypothesis-driven in your case interview, you are displaying to your interviewer that you will be hypothesis-driven on the job. This means that you will be a much more efficient data collector, and conduct more efficient data analysis, to arrive at a solution quickly.
Do yourself a favor – use our hypothesis-driven case interview approach as you practice, and watch your performance soar.
Additional Reading:
- The Ladder Of Inference
- The Pyramid Principle
- Case Interview: Complete Prep Guide
Save your cart?
- What is Strategy?
- Business Models
- Developing a Strategy
- Strategic Planning
- Competitive Advantage
- Growth Strategy
- Market Strategy
- Customer Strategy
- Geographic Strategy
- Product Strategy
- Service Strategy
- Pricing Strategy
- Distribution Strategy
- Sales Strategy
- Marketing Strategy
- Digital Marketing Strategy
- Organizational Strategy
- HR Strategy – Organizational Design
- HR Strategy – Employee Journey & Culture
- Process Strategy
- Procurement Strategy
- Types of Value
- Competitive Dynamics
- Problem Solving
- Strategic Options
- Decision Making
- Process Tools
- Planning and Projects
- People Leadership
- Personal Development
- OnDemand Coaching
“A fact is a simple statement that everyone believes. It is innocent, unless found guilty. A hypothesis is a novel suggestion that no one wants to believe. It is guilty until found effective.”
– Edward Teller, Nuclear Physicist
During my first brainstorming meeting on my first project at McKinsey, this very serious partner, who had a PhD in Physics, looked at me and said, “So, Joe, what are your main hypotheses.” I looked back at him, perplexed, and said, “Ummm, my what?” I was used to people simply asking, “what are your best ideas, opinions, thoughts, etc.” Over time, I began to understand the importance of hypotheses and how it plays an important role in McKinsey’s problem solving of separating ideas and opinions from facts.
What is a Hypothesis?
“Hypothesis” is probably one of the top 5 words used by McKinsey consultants. And, being hypothesis-driven was required to have any success at McKinsey. A hypothesis is an idea or theory, often based on limited data, which is typically the beginning of a thread of further investigation to prove, disprove or improve the hypothesis through facts and empirical data.
The first step in being hypothesis-driven is to focus on the highest potential ideas and theories of how to solve a problem or realize an opportunity.
Let’s go over an example of being hypothesis-driven.
Let’s say you own a website, and you brainstorm ten ideas to improve web traffic, but you don’t have the budget to execute all ten ideas. The first step in being hypothesis-driven is to prioritize the ten ideas based on how much impact you hypothesize they will create.
The second step in being hypothesis-driven is to apply the scientific method to your hypotheses by creating the fact base to prove or disprove your hypothesis, which then allows you to turn your hypothesis into fact and knowledge. Running with our example, you could prove or disprove your hypothesis on the ideas you think will drive the most impact by executing:
1. An analysis of previous research and the performance of the different ideas 2. A survey where customers rank order the ideas 3. An actual test of the ten ideas to create a fact base on click-through rates and cost
While there are many other ways to validate the hypothesis on your prioritization, I find most people do not take this critical step in validating a hypothesis. Instead, they apply bad logic to many important decisions. An idea pops into their head, and then somehow it just becomes a fact.
One of my favorite lousy logic moments was a CEO who stated,
“I’ve never heard our customers talk about price, so the price doesn’t matter with our products, and I’ve decided we’re going to raise prices.”
Luckily, his management team was able to do a survey to dig deeper into the hypothesis that customers weren’t price-sensitive. Well, of course, they were and through the survey, they built a fantastic fact base that proved and disproved many other important hypotheses.
Why is being hypothesis-driven so important?
Imagine if medicine never actually used the scientific method. We would probably still be living in a world of lobotomies and bleeding people. Many organizations are still stuck in the dark ages, having built a house of cards on opinions disguised as facts, because they don’t prove or disprove their hypotheses. Decisions made on top of decisions, made on top of opinions, steer organizations clear of reality and the facts necessary to objectively evolve their strategic understanding and knowledge. I’ve seen too many leadership teams led solely by gut and opinion. The problem with intuition and gut is if you don’t ever prove or disprove if your gut is right or wrong, you’re never going to improve your intuition. There is a reason why being hypothesis-driven is the cornerstone of problem solving at McKinsey and every other top strategy consulting firm.
How do you become hypothesis-driven?
Most people are idea-driven, and constantly have hypotheses on how the world works and what they or their organization should do to improve. Though, there is often a fatal flaw in that many people turn their hypotheses into false facts, without actually finding or creating the facts to prove or disprove their hypotheses. These people aren’t hypothesis-driven; they are gut-driven.
The conversation typically goes something like “doing this discount promotion will increase our profits” or “our customers need to have this feature” or “morale is in the toilet because we don’t pay well, so we need to increase pay.” These should all be hypotheses that need the appropriate fact base, but instead, they become false facts, often leading to unintended results and consequences. In each of these cases, to become hypothesis-driven necessitates a different framing.
• Instead of “doing this discount promotion will increase our profits,” a hypothesis-driven approach is to ask “what are the best marketing ideas to increase our profits?” and then conduct a marketing experiment to see which ideas increase profits the most.
• Instead of “our customers need to have this feature,” ask the question, “what features would our customers value most?” And, then conduct a simple survey having customers rank order the features based on value to them.
• Instead of “morale is in the toilet because we don’t pay well, so we need to increase pay,” conduct a survey asking, “what is the level of morale?” what are potential issues affecting morale?” and what are the best ideas to improve morale?”
Beyond, watching out for just following your gut, here are some of the other best practices in being hypothesis-driven:
Listen to Your Intuition
Your mind has taken the collision of your experiences and everything you’ve learned over the years to create your intuition, which are those ideas that pop into your head and those hunches that come from your gut. Your intuition is your wellspring of hypotheses. So listen to your intuition, build hypotheses from it, and then prove or disprove those hypotheses, which will, in turn, improve your intuition. Intuition without feedback will over time typically evolve into poor intuition, which leads to poor judgment, thinking, and decisions.
Constantly Be Curious
I’m always curious about cause and effect. At Sports Authority, I had a hypothesis that customers that received service and assistance as they shopped, were worth more than customers who didn’t receive assistance from an associate. We figured out how to prove or disprove this hypothesis by tying surveys to transactional data of customers, and we found the hypothesis was true, which led us to a broad initiative around improving service. The key is you have to be always curious about what you think does or will drive value, create hypotheses and then prove or disprove those hypotheses.
Validate Hypotheses
You need to validate and prove or disprove hypotheses. Don’t just chalk up an idea as fact. In most cases, you’re going to have to create a fact base utilizing logic, observation, testing (see the section on Experimentation), surveys, and analysis.
Be a Learning Organization
The foundation of learning organizations is the testing of and learning from hypotheses. I remember my first strategy internship at Mercer Management Consulting when I spent a good part of the summer combing through the results, findings, and insights of thousands of experiments that a banking client had conducted. It was fascinating to see the vastness and depth of their collective knowledge base. And, in today’s world of knowledge portals, it is so easy to disseminate, learn from, and build upon the knowledge created by companies.
NEXT SECTION: DISAGGREGATION
DOWNLOAD STRATEGY PRESENTATION TEMPLATES
168-PAGE COMPENDIUM OF STRATEGY FRAMEWORKS & TEMPLATES 100-PAGE SALES PLAN PRESENTATION 186-PAGE HR & ORG STRATEGY PRESENTATION
BOOK SOME ONE-ON-ONE TIME WITH JOE
I hope you've gotten some new ideas and perspectives from Stratechi.com. If you want some one-on-one support from me, Joe Newsum , set up some time here . I'm a McKinsey alum who has also been the COO of the 9th fastest growing U.S. company, managed $120 million marketing budgets, led the transformation of 20,000 employees, successfully started two companies from scratch, and amassed a load of experience over my 25-year career. I really enjoy coaching clients and they get a ton of value too. You can see some of their testimonials here . I have deep experience with this topic, strategic planning, career development, scaling up, workshops, leadership, presentation development & delivery, ramping up new roles, and much more. Read my take on developing a strategy . Click here to learn more about me or book some time.
EXPLORE OTHER TYPES OF STRATEGIES
BIG PICTURE WHAT IS STRATEGY? BUSINESS MODEL COMP. ADVANTAGE GROWTH
TARGETS MARKET CUSTOMER GEOGRAPHIC
VALUE PROPOSITION PRODUCT SERVICE PRICING
GO TO MARKET DISTRIBUTION SALES MARKETING
ORGANIZATIONAL ORG DESIGN HR & CULTURE PROCESS PARTNER
EXPLORE THE TOP 100 STRATEGIC LEADERSHIP COMPETENCIES
TYPES OF VALUE COMPETITIVE DYNAMICS PROBLEM SOLVING
OPTION CREATION ANALYTICS DECISION MAKING PROCESS TOOLS
PLANNING & PROJECTS PEOPLE LEADERSHIP PERSONAL DEVELOPMENT
CLICK HERE FOR TONS OF FREE STRATEGY & LEADERSHIP TEMPLATES
STAY IN TOUCH
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Publications
- Account settings
- Advanced Search
- Journal List
- J Oral Maxillofac Pathol
- v.23(2); May-Aug 2019
Hypothesis-driven Research
Umadevi krishnamohan rao.
1 Department of Oral and Maxillofacial Pathology, Ragas Dental College and Hospital, Chennai, Tamil Nadu, India E-mail: [email protected]
As Oral Pathologists, we have the responsibility to upgrade our quality of service with an open mind attitude and gratitude for the contributions made by our professional colleagues. Teaching the students is the priority of the faculty, and with equal priority, oral pathologists have the responsibility to contribute to the literature too as a researcher.
Research is a scientific method of answering a question. This can be achieved when the work done in a representative sample of the population, i.e., the outcome of the result, can be applied to the rest of the population, from which the sample is drawn. In this context, frequently done research is a hypothesis-driven research which is based on scientific theories. Specific aims are listed in this type of research, and the objectives are stated. The scope of a well-designed methodology in a hypothesis-driven research equips the researcher to establish an opportunity to state the outcome of the study.
A provisional statement in which the relationship between two variables is described is known as hypothesis. It is very specific and offers the freedom of evaluating a prediction between the variables stated. It facilitates the researcher to envision and gauge as to what changes can occur in the listed specific outcome variables (dependent) when changes are made in a specific predictor (independent) variable. Thus, any given hypothesis should include both these variables, and the primary aim of the study should be focused on demonstrating the association between the variables, by maintaining the highest ethical standards.
The other requisites for a hypothesis-based study are we should state the level of statistical significance and should specify the power, which is defined as the probability that a statistical test will indicate a significant difference when it truly exists.[ 1 ] In a hypothesis-driven research, specifications of methodology help the grant reviewers to differentiate good science from bad science, and thus, hypothesis-driven research is the most funded research.[ 2 ]
“Hypotheses aren’t simply useful tools in some potentially outmoded vision of science; they are the whole point.” This was stated by Sean Carroll, from the California Institute of Technology, in response to Editor-In-Chief of “ Wired ” Chris Anderson, who argued that “biology is too complex for hypotheses and models, and he favored working on enormous data by correlative analysis.”[ 3 ]
Research does not stop by stating the hypotheses but must ensure that it is clear, testable and falsifiable and should serve as the fundamental basis for constructing a methodology that will allow either its acceptance (study favoring a null hypothesis) or rejection (study rejecting the null hypothesis in favor of the alternative hypothesis).
It is very worrying to observe that many research projects, which require a hypothesis, are being done without stating one. This is the fundamental backbone of the question to be asked and tested, and later, the findings need to be extrapolated in an analytical study, addressing a research question.
A good dissertation or thesis which is submitted for fulfillment of a curriculum or a submitted manuscript is comprised of a thoughtful study, addressing an interesting concept, and has to be scientifically designed. Nowadays, evolving academicians are in a competition to prove their point and be academically visible, which is very vital in their career graph. In any circumstance, unscientific research or short-cut methodology should never be conducted or encouraged to produce a research finding or publish the same as a manuscript.
The other type of research is exploratory research, which is based on a journey for discovery, which is not backed by previously established theories and is driven by hope and chance of breakthrough. The advantage of using these data is that statistics can be applied to establish predictions without the consideration of the principles of designing a study, which is the fundamental requirement of a conventional hypothesis. There is a need to set standards of statistical evidence with a much higher cutoff value for acceptance when we consider doing a study without a hypothesis.
In the past few years, there is an emergence of nonhypothesis-driven research, which does receive encouragement from funding agencies such as innovative molecular analysis technologies. The point to be taken here is that funding of nonhypothesis-driven research does not implicate decrease in support to hypothesis-driven research, but the objective is to encourage multidisciplinary research which is dependent on coordinated and cooperative execution of many branches of science and institutions. Thus, translational research is challenging and does carry a risk associated with the lack of preliminary data to establish a hypothesis.[ 4 ]
The merit of hypothesis testing is that it takes the next stride in scientific theory, having already stood the rigors of examination. Hypothesis testing is in practice for more than five decades and is considered to be a standard requirement when proposals are being submitted for evaluation. Stating a hypothesis is mandatory when we intend to make the study results applicable. Young professionals must be appraised of the merits of hypothesis-based research and must be trained to understand the scope of exploratory research.
Guidelines for Hypothesis-Driven Research
Hypotheses are a crucial part of the scientific thinking process, and most professional scientific endeavors are hypothesis-driven. That is, they seek to address a specific, measurable, and answerable question. A well-constructed hypothesis has several characteristics: it is clear, testable, falsifiable, and serves as the basis for constructing a clear set of experiments that will allow the student to discuss why it can be accepted or rejected based on the experiments. We believe that it is important for students who publish with JEI to practice rigorous scientific thinking through generating and testing hypotheses.
This means that manuscripts that merely introduce an invention, no matter how impressive it is, are not appropriate for JEI. Here are some common examples of unacceptable “hypotheses” relating to engineering projects:
- I hypothesize that my invention will work
- I hypothesize that I can build this invention
If your manuscript is related to engineering, please read our Guidelines for Engineering-Based Projects
This video goes over the general hypothesis requirements as they relate to research eligible for publication at JEI. It was created by one of our previous authors and current student advisory board members, Akshya Mahadevan!
When you assess whether your manuscript has a clear, well-constructed hypothesis, please ask whether it meets the following five criteria:
1. It IS NOT discovery or descriptive research
Some research is not hypothesis-driven. Terms used to describe non-hypothesis-driven research are ‘descriptive research,’ in which information is collected without a particular question in mind, and ‘discovery science,’ where large volumes of experimental data are analyzed with the goal of finding new patterns or correlations. These new observations can lead to hypothesis formation and other scientific methodologies. Some examples of discovery or descriptive research include an invention, explaining an engineered design like a program or an algorithm, mining large datasets for potential targets, or even characterizing a new species.
Another way to assess whether your research is hypothesis-driven is by analyzing the experimental setup. What variables in the experiment are independent, and which are dependent? Do the results of the dependent variable answer the scientific question? Are there positive and negative control groups?
2. It IS original
While your hypothesis does not have to be completely novel within the larger field of your research topic, it cannot be obvious to you, given the background information or experimental setup. You must have developed the hypothesis and designed experiments to test it yourself. This means that the experiments cannot be prescribed – an assigned project from an AP biology course, for example.
3. It IS NOT too general/global
Example 1: “Disease X results from the expression of virulence genes.” Instead the hypothesis should focus on the expression of a particular gene or a set of genes.
Example 2: “Quantifying X will provide significant increases in income for industry.” This is essentially untestable in an experimental setup and is really a potential outcome, not a hypothesis.
4. It IS NOT too complex
Hypothesis statements that contain words like “and” and “or” are ‘compound hypotheses’. This makes testing difficult, because while one part may be true the other may not be so. When your hypothesis has multiple parts, make sure that your experiments directly test the entire hypothesis. Possible further implications that you cannot test should be discussed in Discussion.
5. It DOES NOT misdirect to the researcher
The hypothesis should not address your capabilities. “Discovering the mechanism behind X will enable us to better detect the pathogen.” This example tests the ability of the researchers to take information and use it; this is a result of successful hypothesis-driven research, not a testable hypothesis. Instead, the hypothesis should focus on the experimental system. If it is difficult to state the hypothesis without misdirecting to the researcher, the focus of the research may be discovery science or invention-based, and should be edited to incorporate a properly formulated hypothesis.
Please contact the JEI Editorial Staff at [email protected] if you have any questions regarding the hypothesis of your research.
- Work together
- Product development
- Ways of working
Have you read my two bestsellers, Unlearn and Lean Enterprise? If not, please do. If you have, please write a review!
- Read my story
- Get in touch
- Oval Copy 2 Blog
How to Implement Hypothesis-Driven Development
- Facebook__x28_alt_x29_ Copy
Remember back to the time when we were in high school science class. Our teachers had a framework for helping us learn – an experimental approach based on the best available evidence at hand. We were asked to make observations about the world around us, then attempt to form an explanation or hypothesis to explain what we had observed. We then tested this hypothesis by predicting an outcome based on our theory that would be achieved in a controlled experiment – if the outcome was achieved, we had proven our theory to be correct.
We could then apply this learning to inform and test other hypotheses by constructing more sophisticated experiments, and tuning, evolving, or abandoning any hypothesis as we made further observations from the results we achieved.
Experimentation is the foundation of the scientific method, which is a systematic means of exploring the world around us. Although some experiments take place in laboratories, it is possible to perform an experiment anywhere, at any time, even in software development.
Practicing Hypothesis-Driven Development [1] is thinking about the development of new ideas, products, and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved. The process is iterated upon until a desirable outcome is obtained or the idea is determined to be not viable.
We need to change our mindset to view our proposed solution to a problem statement as a hypothesis, especially in new product or service development – the market we are targeting, how a business model will work, how code will execute and even how the customer will use it.
We do not do projects anymore, only experiments. Customer discovery and Lean Startup strategies are designed to test assumptions about customers. Quality Assurance is testing system behavior against defined specifications. The experimental principle also applies in Test-Driven Development – we write the test first, then use the test to validate that our code is correct, and succeed if the code passes the test. Ultimately, product or service development is a process to test a hypothesis about system behavior in the environment or market it is developed for.
The key outcome of an experimental approach is measurable evidence and learning. Learning is the information we have gained from conducting the experiment. Did what we expect to occur actually happen? If not, what did and how does that inform what we should do next?
In order to learn we need to use the scientific method for investigating phenomena, acquiring new knowledge, and correcting and integrating previous knowledge back into our thinking.
As the software development industry continues to mature, we now have an opportunity to leverage improved capabilities such as Continuous Design and Delivery to maximize our potential to learn quickly what works and what does not. By taking an experimental approach to information discovery, we can more rapidly test our solutions against the problems we have identified in the products or services we are attempting to build. With the goal to optimize our effectiveness of solving the right problems, over simply becoming a feature factory by continually building solutions.
The steps of the scientific method are to:
- Make observations
- Formulate a hypothesis
- Design an experiment to test the hypothesis
- State the indicators to evaluate if the experiment has succeeded
- Conduct the experiment
- Evaluate the results of the experiment
- Accept or reject the hypothesis
- If necessary, make and test a new hypothesis
Using an experimentation approach to software development
We need to challenge the concept of having fixed requirements for a product or service. Requirements are valuable when teams execute a well known or understood phase of an initiative and can leverage well-understood practices to achieve the outcome. However, when you are in an exploratory, complex and uncertain phase you need hypotheses. Handing teams a set of business requirements reinforces an order-taking approach and mindset that is flawed. Business does the thinking and ‘knows’ what is right. The purpose of the development team is to implement what they are told. But when operating in an area of uncertainty and complexity, all the members of the development team should be encouraged to think and share insights on the problem and potential solutions. A team simply taking orders from a business owner is not utilizing the full potential, experience and competency that a cross-functional multi-disciplined team offers.
Framing Hypotheses
The traditional user story framework is focused on capturing requirements for what we want to build and for whom, to enable the user to receive a specific benefit from the system.
As A…. <role>
I Want… <goal/desire>
So That… <receive benefit>
Behaviour Driven Development (BDD) and Feature Injection aims to improve the original framework by supporting communication and collaboration between developers, tester and non-technical participants in a software project.
In Order To… <receive benefit>
As A… <role>
When viewing work as an experiment, the traditional story framework is insufficient. As in our high school science experiment, we need to define the steps we will take to achieve the desired outcome. We then need to state the specific indicators (or signals) we expect to observe that provide evidence that our hypothesis is valid. These need to be stated before conducting the test to reduce the bias of interpretation of results.
If we observe signals that indicate our hypothesis is correct, we can be more confident that we are on the right path and can alter the user story framework to reflect this.
Therefore, a user story structure to support Hypothesis-Driven Development would be;
We believe < this capability >
What functionality we will develop to test our hypothesis? By defining a ‘test’ capability of the product or service that we are attempting to build, we identify the functionality and hypothesis we want to test.
Will result in < this outcome >
What is the expected outcome of our experiment? What is the specific result we expect to achieve by building the ‘test’ capability?
We will have confidence to proceed when < we see a measurable signal >
What signals will indicate that the capability we have built is effective? What key metrics (qualitative or quantitative) we will measure to provide evidence that our experiment has succeeded and give us enough confidence to move to the next stage.
The threshold you use for statistical significance will depend on your understanding of the business and context you are operating within. Not every company has the user sample size of Amazon or Google to run statistically significant experiments in a short period of time. Limits and controls need to be defined by your organization to determine acceptable evidence thresholds that will allow the team to advance to the next step.
For example, if you are building a rocket ship you may want your experiments to have a high threshold for statistical significance. If you are deciding between two different flows intended to help increase user sign up you may be happy to tolerate a lower significance threshold.
The final step is to clearly and visibly state any assumptions made about our hypothesis, to create a feedback loop for the team to provide further input, debate, and understanding of the circumstance under which we are performing the test. Are they valid and make sense from a technical and business perspective?
Hypotheses, when aligned to your MVP, can provide a testing mechanism for your product or service vision. They can test the most uncertain areas of your product or service, in order to gain information and improve confidence.
Examples of Hypothesis-Driven Development user stories are;
Business story.
We Believe That increasing the size of hotel images on the booking page Will Result In improved customer engagement and conversion We Will Have Confidence To Proceed When we see a 5% increase in customers who review hotel images who then proceed to book in 48 hours.
It is imperative to have effective monitoring and evaluation tools in place when using an experimental approach to software development in order to measure the impact of our efforts and provide a feedback loop to the team. Otherwise, we are essentially blind to the outcomes of our efforts.
In agile software development, we define working software as the primary measure of progress. By combining Continuous Delivery and Hypothesis-Driven Development we can now define working software and validated learning as the primary measures of progress.
Ideally, we should not say we are done until we have measured the value of what is being delivered – in other words, gathered data to validate our hypothesis.
Examples of how to gather data is performing A/B Testing to test a hypothesis and measure to change in customer behavior. Alternative testings options can be customer surveys, paper prototypes, user and/or guerilla testing.
One example of a company we have worked with that uses Hypothesis-Driven Development is lastminute.com . The team formulated a hypothesis that customers are only willing to pay a max price for a hotel based on the time of day they book. Tom Klein, CEO and President of Sabre Holdings shared the story of how they improved conversion by 400% within a week.
Combining practices such as Hypothesis-Driven Development and Continuous Delivery accelerates experimentation and amplifies validated learning. This gives us the opportunity to accelerate the rate at which we innovate while relentlessly reducing costs, leaving our competitors in the dust. Ideally, we can achieve the ideal of one-piece flow: atomic changes that enable us to identify causal relationships between the changes we make to our products and services, and their impact on key metrics.
As Kent Beck said, “Test-Driven Development is a great excuse to think about the problem before you think about the solution”. Hypothesis-Driven Development is a great opportunity to test what you think the problem is before you work on the solution.
We also run a workshop to help teams implement Hypothesis-Driven Development . Get in touch to run it at your company.
[1] Hypothesis-Driven Development By Jeffrey L. Taylor
More strategy insights
Why community-led is the future of innovation, marketing’s job to be done with katelyn bourgoin, how high performance organizations innovate at scale, read my newsletter.
Insights in every edition. News you can use. No spam, ever. Read the latest edition
We've just sent you your first email. Go check it out!
- Explore Insights
- Nobody Studios
- LinkedIn Learning: High Performance Organizations
IMAGES
VIDEO
COMMENTS
Strong hypotheses are most often written in the, “If A occurs, then B will occur” format and are presented as statements, not questions. Good hypotheses also are clear and keep variables in mind, defining them in easy-to-measure terms.
According to the scientific method, one must first formulate a question and then do background research before it is possible to make a hypothesis. The scientific method, of which the hypothesis is a key component, has long been used by sci...
According to Purdue University’s Online Writing Lab, personal correspondence is defined as either personal letters or newsletters. Personal letters include many sorts of communication, from holiday salutations to letters of complaint. Newsl...
In research and data analysis, a hypothesis-driven approach is one of the main methods for using data to test and, ultimately
Hypothesis driven problem solving also known as "top-down problem solving" or "hypothesis driven thinking" is a form of problem-solving that
Taking an hypothesis driven approach to a problem means attempting to solve that problem by focussing on your best hypothesis as to the answer. This helps
Hypothesis-driven development is a prototype methodology that allows product designers to develop, test, and rebuild a product until it's acceptable by the
Senior Manager, Strategy · Form a hypothesis about the problem and determine the data needed to test the hypothesis · Gather and analyze the
Continuous delivery demands the use of hypotheses, not requirements, to deliver what customers want. Developers embrace
If you can be hypothesis-driven in your case interview, you are displaying to your interviewer that you will be hypothesis-driven on the job.
And, being hypothesis-driven was required to have any success at McKinsey. A hypothesis is an idea or theory, often based on limited data
A provisional statement in which the relationship between two variables is described is known as hypothesis. It is very specific and offers the freedom of
Hypotheses are a crucial part of the scientific thinking process, and most professional scientific endeavors are hypothesis-driven.
Practicing Hypothesis-Driven Development[1] is thinking about the development of new ideas, products, and services – even organizational change