5 exercises — choose the best-structured answer to common ML infrastructure interview questions. Focus on feature pipelines, GPU efficiency, model serving, and drift monitoring.
Structure for ML infrastructure design answers
Separate training from serving: pipelines and SLAs differ significantly
Name components precisely: feature store, model registry, serving runtime, monitoring
Address data and model drift: they are distinct and need separate detection strategies
0 / 10 completed
1 / 10
The interviewer asks: "Design an online feature pipeline for a real-time personalisation system serving 50,000 requests per second." Choose the answer that covers the critical design dimensions.
Option B is strongest: it introduces the three-tier feature classification with concrete latency budgets per tier, names specific technologies (Flink, Spark Structured Streaming, Redis), addresses the train-serve skew problem (the critical interview topic — same job writes both stores), covers Redis hot-key scaling, and includes feature monitoring. Option D's pre-compute approach fails for online features requiring last 5 minutes of activity — that data doesn't exist until it happens. Option C names the right architecture without implementation depth. Feature pipeline design: three-tier classification → computation tech per tier → serving latency budget → train-serve consistency mechanism → hot-key handling → monitoring.
2 / 10
The interviewer asks: "Our GPU cluster is at 45% utilisation. How would you investigate this and what would you do about it?" Choose the most systematic diagnostic answer.
Option D is strongest: it structures the investigation systematically (classify idle vs. compute first), names specific profiling tools (nvidia-smi dmon, PyTorch Profiler, Nsight Compute), identifies six distinct root causes with specific fixes for each, and includes multi-GPU and scheduling dimensions beyond single-GPU optimisation. Option B correctly identifies data loading as the most common cause but assumes the diagnosis without instrumenting first — premature optimisation. Option A also skips to a conclusion. GPU investigation: classify idle vs. compute → data pipeline profiling → memory bandwidth → batch size → multi-GPU overhead → scheduling gaps.
3 / 10
The interviewer asks: "Compare batch inference, online inference, and streaming inference — when would you use each?" Which answer covers the key design considerations for each pattern?
Option A is strongest: it defines each pattern with latency characteristics and concrete use cases, names specific infrastructure for each, and identifies the unique challenge per pattern — batch (parallelisation), online (cold start + p99), streaming (model versioning across rolling updates — the hardest challenge). Option D names the right technologies but doesn't explain use cases or the unique challenges of each pattern. Inference pattern comparison: latency × use case × infrastructure × unique challenge for each of the three modes.
4 / 10
The interviewer asks: "How do you detect and respond to model drift in production?" Choose the most complete monitoring and response strategy.
Option C is strongest: it distinguishes three drift types with separate detection and response strategies, names specific tests with threshold values (PSI > 0.2, KS p < 0.05), names real tooling (Evidently AI, WhyLogs, Arize), addresses the delayed label problem (concept drift is hard to detect without timely ground truth), defines a tiered response playbook, and specifies canary deployment for the retrained model. Option B is technically correct but shallow — no detection methods, tooling, or tiered response. Drift monitoring: three types → per-type detection tests and thresholds → tooling → concept drift proxy signals → delayed label strategy → tiered response → canary deployment.
5 / 10
The interviewer asks: "How would you reduce the cost of a training job currently taking 8 hours on 32 A100 GPUs?" Which answer gives the most practical cost-reduction strategy?
Option B is strongest: it mandates profiling first (the universally correct first step), provides eight concrete techniques with estimated impact (mixed precision 1.5–2× throughput, spot instances 60–90% cheaper), lists the checkpoint-and-resume requirement for spot instances, and includes the short hyperparameter sweep strategy before a full run. Option D's distillation requires training a separate model and doesn't directly reduce the current 8-hour job; ZeRO may already be in use and isn't guaranteed to reduce runtime. Training cost reduction: profile first → mixed precision → checkpointing → gradient accumulation → data pipeline → spot instances + checkpointing → parallelism efficiency → short ablation sweeps.
6 / 10
Context: You're reviewing a pull request submitted by David for deploying a new model to the staging environment. David's added a comment to the PR describing the deployment process:
"Deployed using kubectl scale – seems straightforward enough."
This scenario tests your understanding of effective code review communication. While scaling is a valid operation, simply stating 'kubectl scale' isn't sufficient. A robust deployment strategy requires consideration for monitoring, rollback procedures, and potential failure scenarios – aspects David's comment misses, highlighting the need for more detailed documentation.
7 / 10
Context: Sarah, a Senior ML Engineer, sends you this Slack message:
'Hey team, we're seeing some weird latency spikes in the recommendation engine. The logs aren't showing any obvious errors, but it feels like something's not quite right.'
The key here is proactive investigation. While a spike warrants attention, Sarah's message lacks crucial data for diagnosis. A good response would include requests for metrics (e.g., request latency, CPU/memory usage) to help pinpoint the root cause – this demonstrates a systematic approach to problem-solving that's vital in ML infrastructure.
8 / 10
Context: You're writing the description for a pull request proposing an update to your Kubernetes deployment configuration for a model serving application. The PR aims to increase the replica count based on observed traffic patterns.
Which of the following best describes the information you should include in the PR description?
PR descriptions should be clear and informative, not just a technical record. While detailing the configuration change itself is necessary, providing associated metrics (request volume, latency, error rates) demonstrates that you've considered the impact of the change on performance – this strengthens the justification for the update.
9 / 10
Context: During a daily standup meeting, your team lead asks:
'John, can you give us an update on the performance of the new training job for the fraud detection model?'
Standup updates should focus on progress and key metrics. While confirming a job ran is important, providing additional details – such as the training loss, GPU utilization, and estimated remaining time— gives the team valuable insight into the model's learning process and potential bottlenecks.
10 / 10
Context: You've identified that a training job is taking significantly longer than expected. Initial investigation suggests the primary bottleneck is GPU utilization remaining consistently low (around 20%) despite a full cluster being allocated to the job.
What's the most effective initial step to take?
Low GPU utilization suggests a problem beyond simply increasing the number of GPUs. Profiling the training code is crucial to identify inefficiencies like slow data loading, suboptimal algorithm implementations, or memory constraints that could be limiting GPU usage – this targeted approach avoids blindly scaling up resources.
What does "ML Infrastructure Engineer — Interview Questions — Best-Answer Practice" cover?
Practice answering ML Infrastructure Engineer interview questions in professional English. 5 exercises on feature pipelines, GPU utilisation, model serving, ML monitoring, and training cost optimisation.
How many questions are in this interview set?
This set has 10 exercises, each with a full explanation.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to use with no account, sign-up, or paywall.
Do these exercises include model answers?
Yes. Each interview question gives you several possible responses and asks you to pick the one that communicates most clearly and completely — the explanation then breaks down exactly why that answer works, including the specific vocabulary a strong candidate would use.
What if I choose an answer that isn't the strongest one?
You'll see which option was correct and read a full explanation of why it's stronger than the alternatives, plus the key vocabulary and phrasing worth reusing in a real interview.
Can I retry the questions?
Yes — use the "Try again" button on the results screen to reset and go through the set again.
Is this the same as a real technical or behavioural interview?
No — it's focused practice for the language side of interviewing: recognising which phrasing sounds precise and confident versus vague, and knowing the vocabulary interviewers expect for this role. It won't replace mock interviews, but it builds the vocabulary you'll need in one.
Where can I find interview prep for other roles?
Browse the full Interview exercises hub for 170+ modules covering behavioural, technical, and system design rounds across dozens of IT roles, or check the "Next up" link below to continue.
Do I need an account, and is my progress saved?
No account is needed. Progress is tracked only for your current visit — reloading or leaving the page resets the counter.
Who writes these interview questions?
Every question is written by the CoderSlingo team based on real technical interview patterns for this role, then reviewed for accuracy and clarity.