Fine-Tuning Vocabulary
5 exercises — fine-tuning vs. RAG, LoRA and QLoRA, PEFT, instruction tuning vs. RLHF, and why dataset curation matters.
0 / 25 completed
1 / 25
A team debates: "Should we fine-tune a model on our support tickets, or just use RAG with our knowledge base?" How would you explain the fundamental difference between what fine-tuning and RAG each accomplish?
This behaviour-versus-knowledge distinction is the single most important framing for choosing between fine-tuning and RAG (they're also often combined).
Good fine-tuning use cases: teaching a consistent response format (always structured JSON with specific fields), adapting tone/style to match a brand voice, teaching a specialised skill (like classifying support tickets into specific categories used only by your company) — patterns that should apply consistently across many different inputs.
Good RAG use cases: answering questions about frequently-updated documentation, referencing a large body of facts too large to encode via fine-tuning, providing traceable/citable sources for an answer (harder to achieve with fine-tuning, since it's unclear which specific training example produced a given output) — especially when the underlying knowledge changes often, since fine-tuning would require expensive retraining for every update, while RAG's knowledge source is just a database that can be updated directly.
Good fine-tuning use cases: teaching a consistent response format (always structured JSON with specific fields), adapting tone/style to match a brand voice, teaching a specialised skill (like classifying support tickets into specific categories used only by your company) — patterns that should apply consistently across many different inputs.
Good RAG use cases: answering questions about frequently-updated documentation, referencing a large body of facts too large to encode via fine-tuning, providing traceable/citable sources for an answer (harder to achieve with fine-tuning, since it's unclear which specific training example produced a given output) — especially when the underlying knowledge changes often, since fine-tuning would require expensive retraining for every update, while RAG's knowledge source is just a database that can be updated directly.
2 / 25
A research paper mentions "using LoRA instead of full fine-tuning to reduce the cost of adapting the model." What is LoRA, and how does it reduce cost compared to full fine-tuning?
LoRA is the most widely adopted technique in the broader category of PEFT (Parameter-Efficient Fine-Tuning) methods, which all share the goal of adapting a large pretrained model to a new task or domain without the full cost of updating every one of its (often billions of) parameters.
How it works, at a high level: instead of directly modifying the original weight matrices, LoRA inserts small trainable "low-rank" matrices alongside them. During training, only these small matrices are updated; the original model weights stay completely frozen. At inference time, the LoRA matrices' contribution is added to the frozen weights' output.
Why this matters practically: a full fine-tune might require storing a complete new copy of a multi-gigabyte model for every task-specific variant, and require substantial GPU memory during training to hold gradients for every parameter. A LoRA adapter, by contrast, might be just megabytes in size (only the small added matrices), can be trained on much more modest hardware, and multiple LoRA adapters can even be swapped in and out against the same frozen base model — making it practical to maintain many specialised variants without the storage and compute cost multiplying with each one.
How it works, at a high level: instead of directly modifying the original weight matrices, LoRA inserts small trainable "low-rank" matrices alongside them. During training, only these small matrices are updated; the original model weights stay completely frozen. At inference time, the LoRA matrices' contribution is added to the frozen weights' output.
Why this matters practically: a full fine-tune might require storing a complete new copy of a multi-gigabyte model for every task-specific variant, and require substantial GPU memory during training to hold gradients for every parameter. A LoRA adapter, by contrast, might be just megabytes in size (only the small added matrices), can be trained on much more modest hardware, and multiple LoRA adapters can even be swapped in and out against the same frozen base model — making it practical to maintain many specialised variants without the storage and compute cost multiplying with each one.
3 / 25
A follow-up discussion mentions "QLoRA," described as "LoRA combined with quantisation, letting us fine-tune a 70B parameter model on a single GPU." How does QLoRA achieve this, building on what LoRA already does?
QLoRA demonstrates a common and valuable pattern in ML engineering: combining two separately useful techniques (quantisation for memory reduction, LoRA for parameter-efficient training) to achieve something neither could accomplish alone at the same scale.
The insight: in standard LoRA, the frozen base model still needs to be loaded in memory at reasonably high precision, which for very large models (tens of billions of parameters) still requires substantial, often multi-GPU, hardware, even though the trainable portion is small. QLoRA additionally quantises the frozen base model to a much lower precision (commonly 4-bit) before loading it, drastically cutting the memory footprint of the largest component (the frozen weights), while keeping the small trainable LoRA matrices at higher precision to preserve training quality.
Practical significance: this combination was specifically notable for making it feasible to fine-tune very large open-weight models (tens of billions of parameters) on a single consumer or prosumer-grade GPU, rather than requiring an expensive multi-GPU cluster — significantly lowering the barrier to custom model fine-tuning for smaller teams.
The insight: in standard LoRA, the frozen base model still needs to be loaded in memory at reasonably high precision, which for very large models (tens of billions of parameters) still requires substantial, often multi-GPU, hardware, even though the trainable portion is small. QLoRA additionally quantises the frozen base model to a much lower precision (commonly 4-bit) before loading it, drastically cutting the memory footprint of the largest component (the frozen weights), while keeping the small trainable LoRA matrices at higher precision to preserve training quality.
Practical significance: this combination was specifically notable for making it feasible to fine-tune very large open-weight models (tens of billions of parameters) on a single consumer or prosumer-grade GPU, rather than requiring an expensive multi-GPU cluster — significantly lowering the barrier to custom model fine-tuning for smaller teams.
4 / 25
A fine-tuning project plan includes a step called "instruction tuning" before mentioning a separate later step called "RLHF/preference tuning." What's the difference between these two stages, and why are they typically sequential?
This distinguishes two commonly conflated stages of what's sometimes loosely called "fine-tuning," each solving a different problem in turning a raw pretrained model into a helpful, well-behaved assistant.
Instruction tuning (or supervised fine-tuning, SFT) — trains on a dataset of (instruction, good response) pairs, teaching the model the basic behaviour pattern of following a request rather than just predicting statistically likely next words (a raw pretrained model might continue a question with more questions, rather than actually answering it).
RLHF / preference tuning — builds on an already instruction-tuned model, using comparisons (human raters, or increasingly AI raters, judging "which of these two responses is better") to further shape the model's behaviour along dimensions that are hard to specify as simple training examples — like appropriate tone, avoiding harmful content, or being appropriately concise versus thorough.
Why sequential: preference tuning works by nudging an already-competent model's behaviour in a preferred direction; it's far less effective (and less stable) as a way to teach a model basic instruction-following from scratch, which is why instruction tuning typically establishes that foundation first.
Instruction tuning (or supervised fine-tuning, SFT) — trains on a dataset of (instruction, good response) pairs, teaching the model the basic behaviour pattern of following a request rather than just predicting statistically likely next words (a raw pretrained model might continue a question with more questions, rather than actually answering it).
RLHF / preference tuning — builds on an already instruction-tuned model, using comparisons (human raters, or increasingly AI raters, judging "which of these two responses is better") to further shape the model's behaviour along dimensions that are hard to specify as simple training examples — like appropriate tone, avoiding harmful content, or being appropriately concise versus thorough.
Why sequential: preference tuning works by nudging an already-competent model's behaviour in a preferred direction; it's far less effective (and less stable) as a way to teach a model basic instruction-following from scratch, which is why instruction tuning typically establishes that foundation first.
5 / 25
A team preparing a fine-tuning dataset discusses "curating high-quality examples and removing near-duplicates" before training. Why does dataset curation matter as much as (or more than) the fine-tuning technique used?
Dataset quality is frequently the single highest-leverage factor in a fine-tuning project's success, often mattering more than the specific technique (LoRA vs. QLoRA vs. full fine-tuning) or even the amount of compute used.
Why curation matters so directly: fine-tuning works by adjusting the model to better match the patterns in its training examples. If those examples contain inconsistent formatting, factual errors, or repetitive near-duplicate content, the model will learn — and reproduce — those same issues, since it has no independent way to distinguish "correct" patterns from "what's in this training set" during the fine-tuning process itself.
Near-duplicate removal specifically matters because heavily over-represented examples (many near-identical variations of the same scenario) can cause the model to overfit narrowly to that pattern, at the expense of broader generalisation to the diversity of real-world inputs it will actually encounter after deployment.
Common industry finding, often cited in fine-tuning discussions: a smaller (hundreds to low thousands), carefully curated, diverse, high-quality dataset frequently outperforms a much larger but noisier dataset — this is why "data curation" is now treated as a distinct, valuable engineering discipline within ML teams, not a trivial preprocessing afterthought.
Why curation matters so directly: fine-tuning works by adjusting the model to better match the patterns in its training examples. If those examples contain inconsistent formatting, factual errors, or repetitive near-duplicate content, the model will learn — and reproduce — those same issues, since it has no independent way to distinguish "correct" patterns from "what's in this training set" during the fine-tuning process itself.
Near-duplicate removal specifically matters because heavily over-represented examples (many near-identical variations of the same scenario) can cause the model to overfit narrowly to that pattern, at the expense of broader generalisation to the diversity of real-world inputs it will actually encounter after deployment.
Common industry finding, often cited in fine-tuning discussions: a smaller (hundreds to low thousands), carefully curated, diverse, high-quality dataset frequently outperforms a much larger but noisier dataset — this is why "data curation" is now treated as a distinct, valuable engineering discipline within ML teams, not a trivial preprocessing afterthought.
6 / 25
Reviewer: 'The PR description says we're 'fine-tuning' the language model. I'm not entirely clear on what that *really* means in this context. Does it just mean retraining? Or is there a deeper implication?' You need to respond clearly and accurately to this comment, explaining the core concept of fine-tuning within an AI model training workflow. Consider the team's understanding that they're using a large pre-trained model.
This question tests understanding beyond a simple definition. The correct answer highlights that fine-tuning isn't *just* retraining; it's about adapting the model's weights based on new data. A common misconception is to think of it solely as adjusting prompts, which is prompt engineering. Importantly, the response needs to acknowledge the pre-trained nature of the model – the core knowledge remains intact while being subtly shifted towards the specific task.
7 / 25
Senior Dev: "Okay team, we're deploying this new chatbot. The API response times are consistently slow when users ask follow-up questions about the initial topic. We need to improve it."
Lead ML Engineer: "I've been experimenting with fine-tuning a smaller version of our core language model on a dataset of chat logs specifically related to customer support. It seems like that's helping, but it's still not fast enough."The correct answer highlights that fine-tuning adapts the model's parameters based on specific data. This process makes the model better at understanding and responding to the particular domain – in this case, customer support conversations – leading to improved response times. The other options misrepresent the core purpose of fine-tuning: it's not about scaling infrastructure or using RAG/LoRA directly; it's about adjusting the model itself for a specific use case.
8 / 25
During a Slack discussion about optimizing a chatbot's responses, a junior developer writes: 'I'm fine-tuning the model to improve its accuracy.' A senior engineer replies with concern: 'Are you just retraining it from scratch? That seems incredibly resource intensive!' What is the *primary* distinction the senior engineer is highlighting regarding 'fine-tuning' in this context?
The senior engineer is correctly pointing out that 'fine-tuning' doesn't imply retraining from scratch. A key misconception is that it equates to simply applying the same training data again. Instead, fine-tuning utilizes a pre-trained model and then adjusts its parameters – essentially learning *new* relationships within the existing knowledge base—making it far more efficient than starting with a blank slate. The core concept is adapting rather than replacing.
9 / 25
You're reviewing a PR description for a project that's using Retrieval Augmented Generation (RAG) to power a chatbot. The description states: 'We're augmenting the language model with relevant context retrieved from our internal documentation database.' A junior developer asks, 'So, we're just feeding the model more data? What's the core difference between RAG and fine-tuning?'The key difference lies in the modification of the AI model itself. Fine-tuning directly adjusts the model's parameters through training – essentially teaching it new patterns and associations. RAG, on the other hand, provides external information *during inference*—the chatbot's response generation—without altering the core model. Therefore, RAG supplements existing knowledge while fine-tuning fundamentally changes what the model 'knows'.
10 / 25
A team is building a chatbot for a legal firm. During a discussion about improving its responses to complex legal queries, one developer says, 'I'm fine-tuning the model on a dataset of case summaries and relevant statutes.' Another developer asks, 'What exactly does 'fine-tuning' mean in this context? Are we fundamentally changing the model's architecture or just adjusting its parameters based on this new data?'
The correct answer highlights that 'fine-tuning' involves adjusting the pre-trained model's parameters using the new data. It's crucial to understand that this doesn't involve rebuilding the entire model from scratch (option A), nor does it mean simply feeding more data into a static model (option D). Instead, fine-tuning leverages the existing knowledge within the larger language model and adapts it specifically to the nuances of legal case summaries and statutes (option B).
11 / 25
Reviewer: 'The PR description says we're 'fine-tuning' the language model. I'm not entirely clear on what that *really* means in this context. Does it just mean retraining? Or is there a deeper implication?' You need to respond clearly and accurately to this comment, explaining the core concept of fine-tuning within an AI model training workflow. Consider the team's understanding that they're using a large pre-trained model.
This question tests understanding beyond a simple definition. The correct answer highlights that fine-tuning isn't *just* retraining; it's about adapting the model's weights based on new data. A common misconception is to think of it solely as adjusting prompts, which is prompt engineering. Importantly, the response needs to acknowledge the pre-trained nature of the model – the core knowledge remains intact while being subtly shifted towards the specific task.
12 / 25
Senior Dev: "Okay team, we're deploying this new chatbot. The API response times are consistently slow when users ask follow-up questions about the initial topic. We need to improve it."
Lead ML Engineer: "I've been experimenting with fine-tuning a smaller version of our core language model on a dataset of chat logs specifically related to customer support. It seems like that's helping, but it's still not fast enough."The correct answer highlights that fine-tuning adapts the model's parameters based on specific data. This process makes the model better at understanding and responding to the particular domain – in this case, customer support conversations – leading to improved response times. The other options misrepresent the core purpose of fine-tuning: it's not about scaling infrastructure or using RAG/LoRA directly; it's about adjusting the model itself for a specific use case.
13 / 25
During a Slack discussion about optimizing a chatbot's responses, a junior developer writes: 'I'm fine-tuning the model to improve its accuracy.' A senior engineer replies with concern: 'Are you just retraining it from scratch? That seems incredibly resource intensive!' What is the *primary* distinction the senior engineer is highlighting regarding 'fine-tuning' in this context?
The senior engineer is correctly pointing out that 'fine-tuning' doesn't imply retraining from scratch. A key misconception is that it equates to simply applying the same training data again. Instead, fine-tuning utilizes a pre-trained model and then adjusts its parameters – essentially learning *new* relationships within the existing knowledge base—making it far more efficient than starting with a blank slate. The core concept is adapting rather than replacing.
14 / 25
You're reviewing a PR description for a project that's using Retrieval Augmented Generation (RAG) to power a chatbot. The description states: 'We're augmenting the language model with relevant context retrieved from our internal documentation database.' A junior developer asks, 'So, we're just feeding the model more data? What's the core difference between RAG and fine-tuning?'The key difference lies in the modification of the AI model itself. Fine-tuning directly adjusts the model's parameters through training – essentially teaching it new patterns and associations. RAG, on the other hand, provides external information *during inference*—the chatbot's response generation—without altering the core model. Therefore, RAG supplements existing knowledge while fine-tuning fundamentally changes what the model 'knows'.
15 / 25
A team is building a chatbot for a legal firm. During a discussion about improving its responses to complex legal queries, one developer says, 'I'm fine-tuning the model on a dataset of case summaries and relevant statutes.' Another developer asks, 'What exactly does 'fine-tuning' mean in this context? Are we fundamentally changing the model's architecture or just adjusting its parameters based on this new data?'
The correct answer highlights that 'fine-tuning' involves adjusting the pre-trained model's parameters using the new data. It's crucial to understand that this doesn't involve rebuilding the entire model from scratch (option A), nor does it mean simply feeding more data into a static model (option D). Instead, fine-tuning leverages the existing knowledge within the larger language model and adapts it specifically to the nuances of legal case summaries and statutes (option B).
16 / 25
Reviewer: 'The PR description says we're 'fine-tuning' the language model. I'm not entirely clear on what that *really* means in this context. Does it just mean retraining? Or is there a deeper implication?' You need to respond clearly and accurately to this comment, explaining the core concept of fine-tuning within an AI model training workflow. Consider the team's understanding that they're using a large pre-trained model.
This question tests understanding beyond a simple definition. The correct answer highlights that fine-tuning isn't *just* retraining; it's about adapting the model's weights based on new data. A common misconception is to think of it solely as adjusting prompts, which is prompt engineering. Importantly, the response needs to acknowledge the pre-trained nature of the model – the core knowledge remains intact while being subtly shifted towards the specific task.
17 / 25
Senior Dev: "Okay team, we're deploying this new chatbot. The API response times are consistently slow when users ask follow-up questions about the initial topic. We need to improve it."
Lead ML Engineer: "I've been experimenting with fine-tuning a smaller version of our core language model on a dataset of chat logs specifically related to customer support. It seems like that's helping, but it's still not fast enough."The correct answer highlights that fine-tuning adapts the model's parameters based on specific data. This process makes the model better at understanding and responding to the particular domain – in this case, customer support conversations – leading to improved response times. The other options misrepresent the core purpose of fine-tuning: it's not about scaling infrastructure or using RAG/LoRA directly; it's about adjusting the model itself for a specific use case.
18 / 25
During a Slack discussion about optimizing a chatbot's responses, a junior developer writes: 'I'm fine-tuning the model to improve its accuracy.' A senior engineer replies with concern: 'Are you just retraining it from scratch? That seems incredibly resource intensive!' What is the *primary* distinction the senior engineer is highlighting regarding 'fine-tuning' in this context?
The senior engineer is correctly pointing out that 'fine-tuning' doesn't imply retraining from scratch. A key misconception is that it equates to simply applying the same training data again. Instead, fine-tuning utilizes a pre-trained model and then adjusts its parameters – essentially learning *new* relationships within the existing knowledge base—making it far more efficient than starting with a blank slate. The core concept is adapting rather than replacing.
19 / 25
You're reviewing a PR description for a project that's using Retrieval Augmented Generation (RAG) to power a chatbot. The description states: 'We're augmenting the language model with relevant context retrieved from our internal documentation database.' A junior developer asks, 'So, we're just feeding the model more data? What's the core difference between RAG and fine-tuning?'The key difference lies in the modification of the AI model itself. Fine-tuning directly adjusts the model's parameters through training – essentially teaching it new patterns and associations. RAG, on the other hand, provides external information *during inference*—the chatbot's response generation—without altering the core model. Therefore, RAG supplements existing knowledge while fine-tuning fundamentally changes what the model 'knows'.
20 / 25
A team is building a chatbot for a legal firm. During a discussion about improving its responses to complex legal queries, one developer says, 'I'm fine-tuning the model on a dataset of case summaries and relevant statutes.' Another developer asks, 'What exactly does 'fine-tuning' mean in this context? Are we fundamentally changing the model's architecture or just adjusting its parameters based on this new data?'
The correct answer highlights that 'fine-tuning' involves adjusting the pre-trained model's parameters using the new data. It's crucial to understand that this doesn't involve rebuilding the entire model from scratch (option A), nor does it mean simply feeding more data into a static model (option D). Instead, fine-tuning leverages the existing knowledge within the larger language model and adapts it specifically to the nuances of legal case summaries and statutes (option B).
21 / 25
Reviewer: 'The PR description says we're 'fine-tuning' the language model. I'm not entirely clear on what that *really* means in this context. Does it just mean retraining? Or is there a deeper implication?' You need to respond clearly and accurately to this comment, explaining the core concept of fine-tuning within an AI model training workflow. Consider the team's understanding that they're using a large pre-trained model.
This question tests understanding beyond a simple definition. The correct answer highlights that fine-tuning isn't *just* retraining; it's about adapting the model's weights based on new data. A common misconception is to think of it solely as adjusting prompts, which is prompt engineering. Importantly, the response needs to acknowledge the pre-trained nature of the model – the core knowledge remains intact while being subtly shifted towards the specific task.
22 / 25
Senior Dev: "Okay team, we're deploying this new chatbot. The API response times are consistently slow when users ask follow-up questions about the initial topic. We need to improve it."
Lead ML Engineer: "I've been experimenting with fine-tuning a smaller version of our core language model on a dataset of chat logs specifically related to customer support. It seems like that's helping, but it's still not fast enough."The correct answer highlights that fine-tuning adapts the model's parameters based on specific data. This process makes the model better at understanding and responding to the particular domain – in this case, customer support conversations – leading to improved response times. The other options misrepresent the core purpose of fine-tuning: it's not about scaling infrastructure or using RAG/LoRA directly; it's about adjusting the model itself for a specific use case.
23 / 25
During a Slack discussion about optimizing a chatbot's responses, a junior developer writes: 'I'm fine-tuning the model to improve its accuracy.' A senior engineer replies with concern: 'Are you just retraining it from scratch? That seems incredibly resource intensive!' What is the *primary* distinction the senior engineer is highlighting regarding 'fine-tuning' in this context?
The senior engineer is correctly pointing out that 'fine-tuning' doesn't imply retraining from scratch. A key misconception is that it equates to simply applying the same training data again. Instead, fine-tuning utilizes a pre-trained model and then adjusts its parameters – essentially learning *new* relationships within the existing knowledge base—making it far more efficient than starting with a blank slate. The core concept is adapting rather than replacing.
24 / 25
You're reviewing a PR description for a project that's using Retrieval Augmented Generation (RAG) to power a chatbot. The description states: 'We're augmenting the language model with relevant context retrieved from our internal documentation database.' A junior developer asks, 'So, we're just feeding the model more data? What's the core difference between RAG and fine-tuning?'The key difference lies in the modification of the AI model itself. Fine-tuning directly adjusts the model's parameters through training – essentially teaching it new patterns and associations. RAG, on the other hand, provides external information *during inference*—the chatbot's response generation—without altering the core model. Therefore, RAG supplements existing knowledge while fine-tuning fundamentally changes what the model 'knows'.
25 / 25
A team is building a chatbot for a legal firm. During a discussion about improving its responses to complex legal queries, one developer says, 'I'm fine-tuning the model on a dataset of case summaries and relevant statutes.' Another developer asks, 'What exactly does 'fine-tuning' mean in this context? Are we fundamentally changing the model's architecture or just adjusting its parameters based on this new data?'
The correct answer highlights that 'fine-tuning' involves adjusting the pre-trained model's parameters using the new data. It's crucial to understand that this doesn't involve rebuilding the entire model from scratch (option A), nor does it mean simply feeding more data into a static model (option D). Instead, fine-tuning leverages the existing knowledge within the larger language model and adapts it specifically to the nuances of legal case summaries and statutes (option B).
Frequently Asked Questions
What will I practice in "Fine-Tuning Vocabulary — AI Prompting English Exercise"?
This is an AI Prompting exercise set. It walks through 25 scenario-based multiple-choice questions built around real usage of AI Prompting terminology that IT professionals encounter on the job.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to complete with no account, sign-up, or paywall.
How many questions are in this exercise?
This set contains 25 questions. Each one shows immediate feedback and a detailed explanation after you answer, so you learn the correct usage right away rather than waiting for a final score.
Do I need prior experience to complete this exercise?
No prior experience is required. Each question includes a full explanation covering the reasoning behind the correct answer, so the exercise itself teaches the AI Prompting vocabulary as you go.
Can I retry the exercise if I get questions wrong?
Yes — use the "Try again" button on the results screen to reset your answers and go through all the questions again. There is no limit on attempts.
Is my progress saved?
Your answers and score for the current session are tracked in the browser as you go. No account or login is needed, and there is nothing to install.
What if I don't understand a term used in a question?
Read the explanation shown after you answer each question — it breaks down the correct term in plain English with a real-world example. You can also check the site Glossary for quick definitions.
How is this different from reading a blog article on the topic?
Exercises like this one are interactive drills that test and reinforce specific vocabulary through multiple-choice questions, while blog articles explain concepts in prose. Practising here after reading builds active recall, not just passive recognition.
Where can I find more AI Prompting exercises?
See the AI Prompting exercises hub for the full set of related pages, or browse all exercise categories from the main Exercises index.
Can I use this exercise to prepare for a technical interview?
Yes — AI Prompting vocabulary comes up often in technical discussions and interviews. Pair this exercise with our dedicated Interview Preparation section for role-specific practice.