Practise vocabulary for making models faster and more efficient: quantisation, ONNX, batching, caching, and latency vs. throughput trade-offs.
0 / 10 completed
1 / 10
___ reduces model size and inference latency by representing weights in lower-precision formats like INT8 instead of FP32.
Quantisation replaces 32-bit floating-point weights with lower-precision integers (INT8, FP16). This reduces memory footprint by 4x and speeds up inference, with minimal accuracy loss when done carefully.
2 / 10
ONNX (Open Neural Network Exchange) is a format that allows models trained in one framework (e.g., PyTorch) to be served using a different ___ runtime.
ONNX standardises model representation across frameworks. A PyTorch model exported to ONNX can be served by ONNX Runtime, TensorRT, or CoreML — enabling framework-agnostic deployment and optimisation.
3 / 10
Request ___ groups multiple inference requests together and processes them as a single batch, improving GPU utilisation.
Batching collects multiple requests within a time window (dynamic batching) or fills a fixed batch size (static batching), then processes them together. This amortises GPU overhead across many requests, improving throughput.
4 / 10
___ measures how many inference requests a serving system can handle per unit of time — prioritised over latency for batch workloads.
Throughput (requests per second, tokens per second) measures serving capacity. Optimising for throughput (larger batches, more parallelism) often increases individual request latency — a key trade-off in serving architecture.
5 / 10
Serving a model at ___ means splitting it across multiple GPUs when it is too large to fit on a single device.
Model parallelism (tensor parallelism, pipeline parallelism) distributes model layers or tensors across multiple GPUs. Required for very large models (70B+ parameters) that exceed single-GPU memory.
6 / 10
David from the ML team posted this in Slack: 'Just ran a quick benchmark on our new sentiment analysis model. Latency is currently 15ms for single requests, but we're seeing significant performance degradation under load. We need to investigate further.' Which optimization technique should David prioritize investigating first?
David's message highlights performance degradation under load. Batching is directly addressing this by combining requests and improving GPU efficiency. Quantization focuses on precision reduction, pruning removes connections which can be complex to manage, and caching is typically used for static data rather than dynamic inference.
7 / 10
Sarah left this comment in a code review of the API endpoint serving the fraud detection model: 'The response time for this endpoint is consistently exceeding our SLA. We should consider using an ONNX runtime to ensure compatibility across different inference engines and potentially improve throughput.' What does Sarah's comment primarily refer to?
Sarah mentions 'ONNX runtime' – this is about using a standard format and engine. Model compilation involves optimizing the model for specific hardware, which is directly relevant to reducing response times. Serverless architecture deals with deployment, data streaming focuses on continuous input, and runtime optimization is the core concept she's addressing.
8 / 10
You're designing a new microservice for image recognition. The model is too large to fit entirely on a single GPU. Which strategy would be most appropriate for distributing the model's workload?
Model sharding involves splitting the model's parameters across multiple devices. This is a direct solution when the model exceeds a single GPU's memory capacity. Gradient accumulation accumulates gradients over multiple batches; tensor parallelism focuses on distributed operations, and mixed precision training handles numerical precision differences.
9 / 10
Mark in a standup update said: 'We're seeing high GPU utilization during peak hours with our recommendation engine. However, the average latency is still above our target.' Which metric should Mark primarily focus on optimizing to improve the user experience?
Mark's statement highlights high GPU utilization alongside unacceptable latency. While throughput is important, it's secondary to latency when user experience is concerned. Memory footprint and model accuracy relate to model design, not immediate performance optimization for a live service.
10 / 10
During a system monitoring review, the team noticed that their real-time fraud detection model was consistently experiencing high latency. The serving infrastructure logs indicated that requests were being queued due to limited GPU resources. Which technique could be most effective in addressing this bottleneck?
Increasing batch size allows the GPU to process multiple requests concurrently, directly addressing the queuing issue caused by limited resources. Dynamic scaling adjusts resources automatically; model distillation is about model compression, and data augmentation increases training data – neither addresses a bottleneck in real-time inference.
What will I practise in "Inference Optimisation Vocabulary"?
This module focuses on ML Model Serving — real workplace phrasing you'll use on the job. It contains 10 scenario-based multiple-choice questions with instant feedback.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to use with no account or sign-up required.
How many questions does this exercise have?
This module includes 10 questions. Each one gives an immediate right/wrong result plus a full explanation of the correct phrasing.
What happens if I answer a question incorrectly?
You'll see the correct answer highlighted straight away, along with a plain-English explanation of why it's right and why the other options don't fit — mistakes are part of the learning here.
Can I retry the exercise if I want a better score?
Yes — use the 'Try again' button on the results screen to reset your score and go through the questions again. There's no limit on attempts.
Who is this ML Model Serving exercise for?
It's aimed at IT professionals with working English who want to sound more natural and precise around ml model serving — useful whether you're preparing for real conversations at work or just building confidence with the vocabulary.
Do I need an account to track my progress?
No account is needed. Your progress through the exercise is tracked locally in your browser for the current session, and you can replay the module at any time.
How is this different from reading a blog article?
This exercise is an interactive drill that tests and reinforces specific phrasing through multiple-choice questions with instant feedback, while blog articles explain concepts and vocabulary in prose. The two work well together.
Where can I find more ML Model Serving exercises?
See the ML Model Serving hub for more modules like this one, or browse the full Exercises page for other IT-English topics.
Can I complete this exercise on my phone?
Yes — every exercise on CoderSlingo is fully responsive and works on phones and tablets, so you can practise anywhere.