Practice A/B testing vocabulary for recommendation systems: test duration, CTR uplift, holdout groups, collaborative filtering vs. content-based comparison, and deployment decisions.
0 / 5 completed
1 / 5
'We A/B tested _____ filtering vs. content-based.' What type of recommendation algorithm uses user behavior data?
Collaborative filtering uses patterns from many users' behavior (what similar users liked) to generate recommendations, contrasted with content-based filtering which uses item attributes.
2 / 5
'The test ran for _____ weeks.' Why is test duration important in A/B testing recommendations?
Two weeks is a typical minimum for recommendation A/B tests — long enough to capture weekday/weekend behavior cycles and avoid novelty effects from users reacting to change.
3 / 5
'CTR uplift of 8%.' What does CTR measure in recommendation testing?
CTR (Click-Through Rate) measures what percentage of shown recommendations are clicked. An 8% uplift means the new model drives 8% more clicks than the control.
4 / 5
What is a 'holdout group' in A/B testing?
A holdout group receives no experimental treatment and serves as a long-term baseline to measure the cumulative impact of recommendation improvements over time.
5 / 5
'The winning model is deployed to _____ of users.' What percentage completes a full rollout?
After an A/B test confirms a winner, the winning model is typically rolled out to 100% of users, replacing the control. Partial rollouts may persist for monitoring purposes only.