Practise vocabulary for safe model rollouts: shadow mode, canary traffic splitting, A/B model testing, and rollback vocabulary.
0 / 10 completed
1 / 10
In ___ deployment, the new model receives production requests and generates predictions, but its outputs are never shown to users.
Shadow deployment mirrors live traffic to the new model without exposing its predictions to users. It validates behaviour, latency, and output distributions safely before any user-facing rollout.
2 / 10
In a ___ rollout, a small percentage of production traffic (e.g., 5%) is routed to the new model, with the rest still served by the current model.
Canary deployment gradually exposes the new model to real users (starting with 1–5%), monitoring for errors, latency regressions, and business metric impacts before expanding to 100%.
3 / 10
An A/B model test assigns users to ___ groups, measuring the business metric impact of each model version on real users.
In an A/B model test, users are randomly split into control (current model) and treatment (new model) groups. Statistical analysis of business metrics (CTR, conversion, revenue) determines whether the new model should be promoted.
4 / 10
If the new model shows degraded performance during a canary rollout, the correct action is to ___.
Rolling back means routing all traffic back to the previous champion model. Modern serving infrastructure supports instant rollback by updating traffic weights without redeployment.
5 / 10
The ___ rate during a canary rollout measures how often the new model's predictions differ from the old model on the same inputs.
Prediction divergence (disagree rate) measures how often the new and old models produce different outputs on identical inputs. High divergence warrants investigation before expanding the canary.
6 / 10
Sarah from the DevOps team just posted this comment on your PR describing the Shadow & Canary deployment for the new sentiment analysis model: 'We're routing 10% of traffic to the canary model to monitor performance against the baseline. It's crucial we track the drift rate closely, especially during the initial rollout.' What does Sarah *primarily* mean by 'drift rate'?
The 'drift rate' in this context refers to discrepancies between the new model's outputs and the established baseline. While latency is important, Sarah specifically highlights a difference in *prediction accuracy*. Tracking drift helps identify issues early on, ensuring the canary model isn't significantly impacting performance or introducing bias before wider adoption. Option D is simply about volume handled.
7 / 10
Mark, a data scientist, sent this Slack message: 'Just noticed the canary model's F1 score dropped by 5% overnight. We need to investigate immediately – could be a new dataset shift or something wrong with the model itself.' Considering the Shadow & Canary deployment strategy, what is Mark most concerned about?
Mark's message indicates a significant drop in the canary model's performance (F1 score). In a Shadow & Canary setup, this suggests a potential data drift issue – the input data the canary is seeing differs substantially from what it was trained on or what's currently being used in production. This is a critical concern because data drift can severely degrade model accuracy without necessarily indicating an error in the model itself.
8 / 10
You're writing a PR description for deploying a new version of your fraud detection model using the Shadow & Canary approach. Which statement best summarizes the purpose of this deployment strategy?
The core purpose of Shadow & Canary deployment is *parallel execution* and controlled monitoring. It's not about immediate replacement or automated scaling; instead, the new model operates in the shadows while its performance is meticulously assessed against the existing baseline. This allows for a gradual, risk-mitigated transition.
9 / 10
During a canary deployment of a new recommendation engine, you observe that the prediction rate (the number of predictions generated per second) is significantly higher than anticipated. What should be your *initial* course of action?
A sudden spike in prediction rate during a canary deployment is often an indicator of a problem – potentially inefficient code or unexpected data volume. Increasing compute resources should be your first step to handle the increased load *before* assuming it's normal behavior. Investigating the data pipeline is crucial if the increase is due to changes in input data.
10 / 10
The team is using a Shadow & Canary deployment for their new customer churn prediction model. The drift rate is consistently high (around 15%). What does this primarily suggest?
A persistently high drift rate indicates that the inputs the canary model is receiving are substantially different from those it was trained on. This often reflects changing customer behavior or a shift in the underlying data distribution – meaning the model isn't generalizing well to new data patterns, and therefore its accuracy will likely decline over time.
What will I practise in "Shadow & Canary Model Deployment Language"?
This module focuses on ML Model Serving — real workplace phrasing you'll use on the job. It contains 10 scenario-based multiple-choice questions with instant feedback.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to use with no account or sign-up required.
How many questions does this exercise have?
This module includes 10 questions. Each one gives an immediate right/wrong result plus a full explanation of the correct phrasing.
What happens if I answer a question incorrectly?
You'll see the correct answer highlighted straight away, along with a plain-English explanation of why it's right and why the other options don't fit — mistakes are part of the learning here.
Can I retry the exercise if I want a better score?
Yes — use the 'Try again' button on the results screen to reset your score and go through the questions again. There's no limit on attempts.
Who is this ML Model Serving exercise for?
It's aimed at IT professionals with working English who want to sound more natural and precise around ml model serving — useful whether you're preparing for real conversations at work or just building confidence with the vocabulary.
Do I need an account to track my progress?
No account is needed. Your progress through the exercise is tracked locally in your browser for the current session, and you can replay the module at any time.
How is this different from reading a blog article?
This exercise is an interactive drill that tests and reinforces specific phrasing through multiple-choice questions with instant feedback, while blog articles explain concepts and vocabulary in prose. The two work well together.
Where can I find more ML Model Serving exercises?
See the ML Model Serving hub for more modules like this one, or browse the full Exercises page for other IT-English topics.
Can I complete this exercise on my phone?
Yes — every exercise on CoderSlingo is fully responsive and works on phones and tablets, so you can practise anywhere.