Learn vocabulary for formulating and discussing experiment hypotheses: null hypothesis, statistical significance, p-value, Type I and Type II errors.
0 / 15 completed
1 / 15
What does 'we hypothesize that X will lead to Y' communicate in a growth experiment?
A hypothesis statement frames the experiment's purpose: a specific, testable prediction linking a change (X) to an expected measurable outcome (Y). Good hypotheses are falsifiable and specify both the direction and magnitude of the expected effect.
2 / 15
What is the 'null hypothesis' in the context of an A/B test?
The null hypothesis (H₀) assumes no effect: the change being tested has no impact on the metric. Statistical testing aims to determine whether observed data provides sufficient evidence to reject H₀ in favour of the alternative hypothesis (H₁).
3 / 15
What does 'the test reached significance at p < 0.05' mean?
A p-value below 0.05 means there is less than a 5% chance of seeing this result by random chance if the null hypothesis were true. This threshold (alpha) is a convention — lower p-values indicate stronger evidence against the null hypothesis.
4 / 15
What is a 'Type I error' in hypothesis testing?
A Type I error (false positive) occurs when you conclude the treatment had an effect, but the observed difference was actually due to chance. The significance level (alpha) is the maximum acceptable probability of making a Type I error.
5 / 15
What is a 'Type II error' in hypothesis testing?
A Type II error (false negative) occurs when a real effect exists but the experiment fails to detect it — often due to insufficient statistical power, which is related to sample size and effect size. Power analysis before running an experiment helps minimise Type II error risk.
6 / 15
During a code review of the new user onboarding flow, Sarah says, 'I'm hypothesizing that reducing the number of required form fields will increase conversion rates. We'll track this with A/B testing.' What does she primarily mean by 'hypothesizing' in this context? hypothesize
Sarah's statement reflects the core of hypothesis testing: proposing an explanation (the hypothesis) for something she observes. Importantly, this explanation isn't a proven truth; it's a suggestion to be rigorously tested through experimentation. The key is that this 'hypothesis' can be either supported or rejected by data.
7 / 15
In a Slack channel discussing performance issues with a new API endpoint, Ben writes: 'Let's run a null hypothesis test to see if the increased latency is actually significant or just random variation.' What does Ben mean by 'null hypothesis' here? null
The 'null hypothesis' is a critical concept in hypothesis testing – it represents the default assumption that there is no effect or relationship. In this case, Ben assumes that the latency increase isn't *actually* significant; he's setting up a test to see if the data suggests otherwise. It's the opposite of the alternative hypothesis (that there *is* a significant change).
8 / 15
You're drafting a PR description for a new feature that automatically recommends products to users based on their browsing history. The description includes the following sentence: 'We hypothesize that personalized recommendations will increase click-through rates by at least 10%.' What does this statement imply about the process of validating this hypothesis? recommend
The statement 'hypothesize that… will increase by at least 10%' is an *operationalization* of the hypothesis – a specific, measurable prediction. Validating it necessitates designing and running an experiment to collect data on click-through rates and then statistically analyzing the results to see if the 10% threshold is met.
9 / 15
During a standup meeting, David says, 'I'm running a test to see if adding logging to the database query improves performance. I'm worried about a Type II error – missing a real performance issue because the test doesn't show a significant difference.' What type of error is David concerned about? error
A Type II error (false negative) occurs when you fail to reject the null hypothesis *despite* it being false. David is worried about this because he's concerned that his test won't detect a genuine performance improvement due to limitations in the experiment design (e.g., not enough data, too much noise). It's important to understand that failing to find significance doesn't mean there isn't an effect – just that the current test wasn't sensitive enough.
10 / 15
A senior engineer, Maria, is reviewing a pull request and comments: 'The results of this A/B test reached significance at p < 0.05.' What does the 'p < 0.05' part of this statement signify? significance
The 'p-value' (represented by 'p < 0.05') indicates the probability of observing data as extreme as, or more extreme than, what was actually observed, *assuming* that the null hypothesis is true. A p-value less than 0.05 means there's only a 5% chance of seeing such results if there were truly no difference between the groups (the null hypothesis). This provides evidence to reject the null hypothesis.
11 / 15
During a code review of the new user onboarding flow, Sarah says, 'I'm hypothesizing that reducing the number of required form fields will increase conversion rates. We'll track this with A/B testing.' What does she primarily mean by 'hypothesizing' in this context? hypothesize
Sarah's statement reflects the core of hypothesis testing: proposing an explanation (the hypothesis) for something she observes. Importantly, this explanation isn't a proven truth; it's a suggestion to be rigorously tested through experimentation. The key is that this 'hypothesis' can be either supported or rejected by data.
12 / 15
In a Slack channel discussing performance issues with a new API endpoint, Ben writes: 'Let's run a null hypothesis test to see if the increased latency is actually significant or just random variation.' What does Ben mean by 'null hypothesis' here? null
The 'null hypothesis' is a critical concept in hypothesis testing – it represents the default assumption that there is no effect or relationship. In this case, Ben assumes that the latency increase isn't *actually* significant; he's setting up a test to see if the data suggests otherwise. It's the opposite of the alternative hypothesis (that there *is* a significant change).
13 / 15
You're drafting a PR description for a new feature that automatically recommends products to users based on their browsing history. The description includes the following sentence: 'We hypothesize that personalized recommendations will increase click-through rates by at least 10%.' What does this statement imply about the process of validating this hypothesis? recommend
The statement 'hypothesize that… will increase by at least 10%' is an *operationalization* of the hypothesis – a specific, measurable prediction. Validating it necessitates designing and running an experiment to collect data on click-through rates and then statistically analyzing the results to see if the 10% threshold is met.
14 / 15
During a standup meeting, David says, 'I'm running a test to see if adding logging to the database query improves performance. I'm worried about a Type II error – missing a real performance issue because the test doesn't show a significant difference.' What type of error is David concerned about? error
A Type II error (false negative) occurs when you fail to reject the null hypothesis *despite* it being false. David is worried about this because he's concerned that his test won't detect a genuine performance improvement due to limitations in the experiment design (e.g., not enough data, too much noise). It's important to understand that failing to find significance doesn't mean there isn't an effect – just that the current test wasn't sensitive enough.
15 / 15
A senior engineer, Maria, is reviewing a pull request and comments: 'The results of this A/B test reached significance at p < 0.05.' What does the 'p < 0.05' part of this statement signify? significance
The 'p-value' (represented by 'p < 0.05') indicates the probability of observing data as extreme as, or more extreme than, what was actually observed, *assuming* that the null hypothesis is true. A p-value less than 0.05 means there's only a 5% chance of seeing such results if there were truly no difference between the groups (the null hypothesis). This provides evidence to reject the null hypothesis.
What will I practise in "Hypothesis Testing — Vocabulary"?
Learn vocabulary for formulating and discussing experiment hypotheses: null hypothesis, statistical significance, p-value, Type I and Type II errors.
How many exercises are in this module?
This module has 15 multiple-choice exercises, each with instant feedback and a full explanation of the correct answer.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to use with no account, sign-up, or paywall.
Do I need to create an account to do these exercises?
No account is required. Just click an option to answer — your score for this session is tracked automatically in the progress bar above.
What happens if I choose the wrong answer?
You'll immediately see which answer was correct, plus a full explanation covering the vocabulary and reasoning behind it — mistakes are where most of the learning happens.
Can I retry the exercises if I want a higher score?
Yes — use the "Try again" button on the results screen to reset and go through all the questions again.
Is my progress saved if I close the page?
No. Progress is tracked only for your current visit; reloading or leaving the page resets the counter. This keeps the exercise simple and account-free.
Where can I find more Growth & Experimentation Language exercises?
Browse the full Growth & Experimentation Language hub for related drills, or check the "Next up" link below to continue with a connected topic.
How is this different from reading an article on the same topic?
Articles explain vocabulary and concepts in prose; this exercise tests and reinforces that vocabulary through active recall with immediate feedback — the two work best together.
Who writes these exercises?
Every exercise is written by the CoderSlingo team, drawing on real workplace English used in IT roles, then reviewed for accuracy and clarity.