Evaluation Metrics Vocabulary
BLEU, ROUGE, perplexity, win rate, pass@k, hallucination rate — the metrics used to measure LLM quality.
Key vocabulary
- BLEU score — measures overlap between generated and reference text using n-gram precision; common in translation.
- ROUGE — recall-oriented metric comparing generated summaries to reference summaries.
- Perplexity — measures how well a language model predicts a text sample; lower is better.
- Pass@k — fraction of problems where at least 1 of k generated solutions is correct; used for code generation.
- Hallucination rate — proportion of model outputs containing factually incorrect or fabricated information.
0 / 37 completed
1 / 37
Your team reports a BLEU score of 42 for a translation model. What does the BLEU score measure?
BLEU (Bilingual Evaluation Understudy) compares n-gram overlap between a generated translation and one or more reference translations. It measures precision — how much of the generated text appears in the reference. Despite being widely used, BLEU is criticized for not capturing meaning or fluency well.
2 / 37
A colleague says “our summarization model has a high ROUGE-L score.” ROUGE-L measures:
ROUGE-L uses the Longest Common Subsequence (LCS) between the generated and reference text. Unlike ROUGE-1/2 which count n-gram overlaps, ROUGE-L considers word order. It is a recall-oriented metric, rewarding summaries that cover important content from the reference.
3 / 37
An eval report shows “pass@10 = 0.78” for a code generation model. This means:
pass@k measures whether at least one of k generated samples solves a problem. pass@10 = 0.78 means that for 78% of problems, if you generate 10 solutions, at least one is functionally correct. Higher k gives more generous estimates, which is relevant when using models in best-of-N sampling pipelines.
4 / 37
In the context of RAG (Retrieval-Augmented Generation) evaluation, “faithfulness” refers to:
Faithfulness in RAG evaluation asks: does the generated answer only contain claims that are supported by the retrieved documents? It is distinct from answer relevance (does the answer address the question?) and context recall (were the right documents retrieved?). RAGAS is a popular framework that measures all three.
5 / 37
Why might a team choose human evaluation over automatic metrics for evaluating a creative writing model?
Human evaluation is preferred when automatic metrics cannot capture what matters. For creative writing, qualities like engagement, originality, and appropriate tone require human judgment. The tradeoff is cost and scalability — hence the rise of LLM-as-judge as a middle ground between cheap-but-flawed automatic metrics and expensive-but-accurate human evaluation.
6 / 37
PR Title: Fix: Incorrect error message displayed during API call
During a code review of a newly submitted PR, Sarah comments to David: "I'm seeing some issues with the response from the payment gateway. The precision metric is consistently low – around 85% – and the recall is even lower at 60%. It seems like we're missing a significant number of successful transactions."
What does David likely mean by referring to 'precision' and 'recall' in this context?David is using standard information retrieval terminology. Precision (85%) reflects how accurately the payment gateway identifies *successful* transactions among all the transactions it reported as successful. Recall (60%) indicates what proportion of *actual* successful payments were correctly identified by the system – a low recall means many valid transactions are being missed. It's crucial to understand that precision and recall often have opposing relationships, and optimizing for one can negatively impact the other.
7 / 37
A developer, Mark, sends a Slack message to the team: "I've run some automated tests on the new model and the F1 score is quite low – around 0.2. I'm thinking we need to investigate why it's not performing well.". What does Mark likely mean by mentioning the 'F1 score'?
The F1 score is a valuable metric for imbalanced datasets—a common scenario in many machine learning applications. It provides a single number that balances precision and recall, giving a more nuanced view than accuracy alone. A low F1 score (0.2) suggests the model is struggling to achieve both high precision *and* high recall simultaneously, often indicating a class imbalance problem.
8 / 37
During a standup meeting, Elena says, "We're using the perplexity score to evaluate our new language model. It's currently at 25.". What does Elena likely mean by referring to 'perplexity'?
Perplexity is a standard metric in language modeling that quantifies how well a probability distribution predicts a sample. A lower perplexity score indicates the model assigns higher probabilities to the actual words in the sequence, meaning it's better at predicting and understanding the text. Therefore, a perplexity of 25 suggests the model isn't very good at predicting the language.
9 / 37
PR Title: Fix: Incorrect error message displayed during API call
During a code review of a newly submitted PR, Sarah comments to David: "I'm seeing some issues with the response from the payment gateway. The precision metric is consistently low – around 85% – and the recall is even lower at 60%. It seems like we're missing a significant number of successful transactions."
What does David likely mean by referring to 'precision' and 'recall' in this context?David is using standard information retrieval terminology. Precision (85%) reflects how accurately the payment gateway identifies *successful* transactions among all the transactions it reported as successful. Recall (60%) indicates what proportion of *actual* successful payments were correctly identified by the system – a low recall means many valid transactions are being missed. It's crucial to understand that precision and recall often have opposing relationships, and optimizing for one can negatively impact the other.
10 / 37
A developer, Mark, sends a Slack message to the team: "I've run some automated tests on the new model and the F1 score is quite low – around 0.2. I'm thinking we need to investigate why it's not performing well.". What does Mark likely mean by mentioning the 'F1 score'?
The F1 score is a valuable metric for imbalanced datasets—a common scenario in many machine learning applications. It provides a single number that balances precision and recall, giving a more nuanced view than accuracy alone. A low F1 score (0.2) suggests the model is struggling to achieve both high precision *and* high recall simultaneously, often indicating a class imbalance problem.
11 / 37
During a standup meeting, Elena says, "We're using the perplexity score to evaluate our new language model. It's currently at 25.". What does Elena likely mean by referring to 'perplexity'?
Perplexity is a standard metric in language modeling that quantifies how well a probability distribution predicts a sample. A lower perplexity score indicates the model assigns higher probabilities to the actual words in the sequence, meaning it's better at predicting and understanding the text. Therefore, a perplexity of 25 suggests the model isn't very good at predicting the language.
12 / 37
PR Title: Fix: Incorrect error message displayed during API call
During a code review of a newly submitted PR, Sarah comments to David: "I'm seeing some issues with the response from the payment gateway. The precision metric is consistently low – around 85% – and the recall is even lower at 60%. It seems like we're missing a significant number of successful transactions."
What does David likely mean by referring to 'precision' and 'recall' in this context?David is using standard information retrieval terminology. Precision (85%) reflects how accurately the payment gateway identifies *successful* transactions among all the transactions it reported as successful. Recall (60%) indicates what proportion of *actual* successful payments were correctly identified by the system – a low recall means many valid transactions are being missed. It's crucial to understand that precision and recall often have opposing relationships, and optimizing for one can negatively impact the other.
13 / 37
A developer, Mark, sends a Slack message to the team: "I've run some automated tests on the new model and the F1 score is quite low – around 0.2. I'm thinking we need to investigate why it's not performing well.". What does Mark likely mean by mentioning the 'F1 score'?
The F1 score is a valuable metric for imbalanced datasets—a common scenario in many machine learning applications. It provides a single number that balances precision and recall, giving a more nuanced view than accuracy alone. A low F1 score (0.2) suggests the model is struggling to achieve both high precision *and* high recall simultaneously, often indicating a class imbalance problem.
14 / 37
During a standup meeting, Elena says, "We're using the perplexity score to evaluate our new language model. It's currently at 25.". What does Elena likely mean by referring to 'perplexity'?
Perplexity is a standard metric in language modeling that quantifies how well a probability distribution predicts a sample. A lower perplexity score indicates the model assigns higher probabilities to the actual words in the sequence, meaning it's better at predicting and understanding the text. Therefore, a perplexity of 25 suggests the model isn't very good at predicting the language.
15 / 37
PR Title: Fix: Incorrect error message displayed during API call
During a code review of a newly submitted PR, Sarah comments to David: "I'm seeing some issues with the response from the payment gateway. The precision metric is consistently low – around 85% – and the recall is even lower at 60%. It seems like we're missing a significant number of successful transactions."
What does David likely mean by referring to 'precision' and 'recall' in this context?David is using standard information retrieval terminology. Precision (85%) reflects how accurately the payment gateway identifies *successful* transactions among all the transactions it reported as successful. Recall (60%) indicates what proportion of *actual* successful payments were correctly identified by the system – a low recall means many valid transactions are being missed. It's crucial to understand that precision and recall often have opposing relationships, and optimizing for one can negatively impact the other.
16 / 37
A developer, Mark, sends a Slack message to the team: "I've run some automated tests on the new model and the F1 score is quite low – around 0.2. I'm thinking we need to investigate why it's not performing well.". What does Mark likely mean by mentioning the 'F1 score'?
The F1 score is a valuable metric for imbalanced datasets—a common scenario in many machine learning applications. It provides a single number that balances precision and recall, giving a more nuanced view than accuracy alone. A low F1 score (0.2) suggests the model is struggling to achieve both high precision *and* high recall simultaneously, often indicating a class imbalance problem.
17 / 37
During a standup meeting, Elena says, "We're using the perplexity score to evaluate our new language model. It's currently at 25.". What does Elena likely mean by referring to 'perplexity'?
Perplexity is a standard metric in language modeling that quantifies how well a probability distribution predicts a sample. A lower perplexity score indicates the model assigns higher probabilities to the actual words in the sequence, meaning it's better at predicting and understanding the text. Therefore, a perplexity of 25 suggests the model isn't very good at predicting the language.
18 / 37
API Response: {
"metric": "Precision",
"value": "0.85",
"dataset": "Test Set A"
} Liam is reviewing the results of a new model for image classification. This response indicates that the model achieves 85% precision on Test Set A. What does 'precision' primarily measure in this context?Precision in machine learning measures the accuracy of a positive prediction. Specifically, it answers the question: 'Of all the instances predicted as positive, what proportion were actually positive?' Option A is incorrect because precision focuses on *true* positives. Option C describes recall, not precision.
19 / 37
During a sprint planning meeting, Alex says, "We're aiming for an F1 score of at least 0.6 on our fraud detection model. Let's track that closely.". What does Alex likely mean by referencing the 'F1 score'?
The F1 score is a harmonic mean of precision and recall. It balances both metrics, providing a single value that represents the model's overall performance when dealing with imbalanced datasets – a common scenario in fraud detection where fraudulent transactions are far rarer than legitimate ones. Option B describes accuracy, while options C and D refer to different concepts.
20 / 37
PR Description: 'This change improves the model's recall for low-confidence predictions by implementing a new thresholding strategy. We reduced false negatives.' What is the primary impact of reducing 'false negatives' in this scenario?Recall (also known as sensitivity) measures the proportion of *actual* positive instances that are correctly identified by the model. Reducing false negatives directly improves recall – meaning more true positive cases are successfully detected. Options A and D contradict the definition of recall, while option B is about stability, not recall.
21 / 37
Sarah, a data scientist, sends a Slack message to her team: 'I'm seeing high perplexity scores (around 50) on the new translation model. This suggests it's struggling with the nuances of the target language.' What does Sarah likely mean by referring to 'perplexity' in this context?
Perplexity is fundamentally a measure of uncertainty. Lower perplexity indicates that the model has a better understanding of the target language's probability distribution – meaning it predicts the next word more accurately. A high perplexity score signifies the model is struggling to predict the language effectively. Option A describes fluency, and options C & D are irrelevant.
22 / 37
Code Review Comment: 'I'm concerned about this metric – the area under the ROC curve is quite low. It suggests the classifier isn't effectively discriminating between classes.' What does the 'area under the ROC curve' primarily indicate in this scenario?The ROC curve (Receiver Operating Characteristic) plots the trade-off between true positive rate and false positive rate at various classification thresholds. The area under this curve provides a comprehensive measure of the model's ability to discriminate effectively – a higher AUC indicates better performance. Option A is accuracy; option C describes precision/recall, and option D relates to computational cost.
23 / 37
API Response: {
"metric": "Precision",
"value": "0.85",
"dataset": "Test Set A"
} Liam is reviewing the results of a new model for image classification. This response indicates that the model achieves 85% precision on Test Set A. What does 'precision' primarily measure in this context?Precision in machine learning measures the accuracy of a positive prediction. Specifically, it answers the question: 'Of all the instances predicted as positive, what proportion were actually positive?' Option A is incorrect because precision focuses on *true* positives. Option C describes recall, not precision.
24 / 37
During a sprint planning meeting, Alex says, "We're aiming for an F1 score of at least 0.6 on our fraud detection model. Let's track that closely.". What does Alex likely mean by referencing the 'F1 score'?
The F1 score is a harmonic mean of precision and recall. It balances both metrics, providing a single value that represents the model's overall performance when dealing with imbalanced datasets – a common scenario in fraud detection where fraudulent transactions are far rarer than legitimate ones. Option B describes accuracy, while options C and D refer to different concepts.
25 / 37
PR Description: 'This change improves the model's recall for low-confidence predictions by implementing a new thresholding strategy. We reduced false negatives.' What is the primary impact of reducing 'false negatives' in this scenario?Recall (also known as sensitivity) measures the proportion of *actual* positive instances that are correctly identified by the model. Reducing false negatives directly improves recall – meaning more true positive cases are successfully detected. Options A and D contradict the definition of recall, while option B is about stability, not recall.
26 / 37
Sarah, a data scientist, sends a Slack message to her team: 'I'm seeing high perplexity scores (around 50) on the new translation model. This suggests it's struggling with the nuances of the target language.' What does Sarah likely mean by referring to 'perplexity' in this context?
Perplexity is fundamentally a measure of uncertainty. Lower perplexity indicates that the model has a better understanding of the target language's probability distribution – meaning it predicts the next word more accurately. A high perplexity score signifies the model is struggling to predict the language effectively. Option A describes fluency, and options C & D are irrelevant.
27 / 37
Code Review Comment: 'I'm concerned about this metric – the area under the ROC curve is quite low. It suggests the classifier isn't effectively discriminating between classes.' What does the 'area under the ROC curve' primarily indicate in this scenario?The ROC curve (Receiver Operating Characteristic) plots the trade-off between true positive rate and false positive rate at various classification thresholds. The area under this curve provides a comprehensive measure of the model's ability to discriminate effectively – a higher AUC indicates better performance. Option A is accuracy; option C describes precision/recall, and option D relates to computational cost.
28 / 37
API Response: {
"metric": "Precision",
"value": "0.85",
"dataset": "Test Set A"
} Liam is reviewing the results of a new model for image classification. This response indicates that the model achieves 85% precision on Test Set A. What does 'precision' primarily measure in this context?Precision in machine learning measures the accuracy of a positive prediction. Specifically, it answers the question: 'Of all the instances predicted as positive, what proportion were actually positive?' Option A is incorrect because precision focuses on *true* positives. Option C describes recall, not precision.
29 / 37
During a sprint planning meeting, Alex says, "We're aiming for an F1 score of at least 0.6 on our fraud detection model. Let's track that closely.". What does Alex likely mean by referencing the 'F1 score'?
The F1 score is a harmonic mean of precision and recall. It balances both metrics, providing a single value that represents the model's overall performance when dealing with imbalanced datasets – a common scenario in fraud detection where fraudulent transactions are far rarer than legitimate ones. Option B describes accuracy, while options C and D refer to different concepts.
30 / 37
PR Description: 'This change improves the model's recall for low-confidence predictions by implementing a new thresholding strategy. We reduced false negatives.' What is the primary impact of reducing 'false negatives' in this scenario?Recall (also known as sensitivity) measures the proportion of *actual* positive instances that are correctly identified by the model. Reducing false negatives directly improves recall – meaning more true positive cases are successfully detected. Options A and D contradict the definition of recall, while option B is about stability, not recall.
31 / 37
Sarah, a data scientist, sends a Slack message to her team: 'I'm seeing high perplexity scores (around 50) on the new translation model. This suggests it's struggling with the nuances of the target language.' What does Sarah likely mean by referring to 'perplexity' in this context?
Perplexity is fundamentally a measure of uncertainty. Lower perplexity indicates that the model has a better understanding of the target language's probability distribution – meaning it predicts the next word more accurately. A high perplexity score signifies the model is struggling to predict the language effectively. Option A describes fluency, and options C & D are irrelevant.
32 / 37
Code Review Comment: 'I'm concerned about this metric – the area under the ROC curve is quite low. It suggests the classifier isn't effectively discriminating between classes.' What does the 'area under the ROC curve' primarily indicate in this scenario?The ROC curve (Receiver Operating Characteristic) plots the trade-off between true positive rate and false positive rate at various classification thresholds. The area under this curve provides a comprehensive measure of the model's ability to discriminate effectively – a higher AUC indicates better performance. Option A is accuracy; option C describes precision/recall, and option D relates to computational cost.
33 / 37
API Response: {
"metric": "Precision",
"value": "0.85",
"dataset": "Test Set A"
} Liam is reviewing the results of a new model for image classification. This response indicates that the model achieves 85% precision on Test Set A. What does 'precision' primarily measure in this context?Precision in machine learning measures the accuracy of a positive prediction. Specifically, it answers the question: 'Of all the instances predicted as positive, what proportion were actually positive?' Option A is incorrect because precision focuses on *true* positives. Option C describes recall, not precision.
34 / 37
During a sprint planning meeting, Alex says, "We're aiming for an F1 score of at least 0.6 on our fraud detection model. Let's track that closely.". What does Alex likely mean by referencing the 'F1 score'?
The F1 score is a harmonic mean of precision and recall. It balances both metrics, providing a single value that represents the model's overall performance when dealing with imbalanced datasets – a common scenario in fraud detection where fraudulent transactions are far rarer than legitimate ones. Option B describes accuracy, while options C and D refer to different concepts.
35 / 37
PR Description: 'This change improves the model's recall for low-confidence predictions by implementing a new thresholding strategy. We reduced false negatives.' What is the primary impact of reducing 'false negatives' in this scenario?Recall (also known as sensitivity) measures the proportion of *actual* positive instances that are correctly identified by the model. Reducing false negatives directly improves recall – meaning more true positive cases are successfully detected. Options A and D contradict the definition of recall, while option B is about stability, not recall.
36 / 37
Sarah, a data scientist, sends a Slack message to her team: 'I'm seeing high perplexity scores (around 50) on the new translation model. This suggests it's struggling with the nuances of the target language.' What does Sarah likely mean by referring to 'perplexity' in this context?
Perplexity is fundamentally a measure of uncertainty. Lower perplexity indicates that the model has a better understanding of the target language's probability distribution – meaning it predicts the next word more accurately. A high perplexity score signifies the model is struggling to predict the language effectively. Option A describes fluency, and options C & D are irrelevant.
37 / 37
Code Review Comment: 'I'm concerned about this metric – the area under the ROC curve is quite low. It suggests the classifier isn't effectively discriminating between classes.' What does the 'area under the ROC curve' primarily indicate in this scenario?The ROC curve (Receiver Operating Characteristic) plots the trade-off between true positive rate and false positive rate at various classification thresholds. The area under this curve provides a comprehensive measure of the model's ability to discriminate effectively – a higher AUC indicates better performance. Option A is accuracy; option C describes precision/recall, and option D relates to computational cost.
Frequently Asked Questions
What will I practice in "Evaluation Metrics Vocabulary | Coders Lingo"?
This is an AI Model Evaluation Language exercise set. It walks through 37 scenario-based multiple-choice questions built around real usage of AI Model Evaluation Language terminology that IT professionals encounter on the job.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to complete with no account, sign-up, or paywall.
How many questions are in this exercise?
This set contains 37 questions. Each one shows immediate feedback and a detailed explanation after you answer, so you learn the correct usage right away rather than waiting for a final score.
Do I need prior experience to complete this exercise?
No prior experience is required. Each question includes a full explanation covering the reasoning behind the correct answer, so the exercise itself teaches the AI Model Evaluation Language vocabulary as you go.
Can I retry the exercise if I get questions wrong?
Yes — use the "Try again" button on the results screen to reset your answers and go through all the questions again. There is no limit on attempts.
Is my progress saved?
Your answers and score for the current session are tracked in the browser as you go. No account or login is needed, and there is nothing to install.
What if I don't understand a term used in a question?
Read the explanation shown after you answer each question — it breaks down the correct term in plain English with a real-world example. You can also check the site Glossary for quick definitions.
How is this different from reading a blog article on the topic?
Exercises like this one are interactive drills that test and reinforce specific vocabulary through multiple-choice questions, while blog articles explain concepts in prose. Practising here after reading builds active recall, not just passive recognition.
Where can I find more AI Model Evaluation Language exercises?
See the AI Model Evaluation Language exercises hub for the full set of related pages, or browse all exercise categories from the main Exercises index.
Can I use this exercise to prepare for a technical interview?
Yes — AI Model Evaluation Language vocabulary comes up often in technical discussions and interviews. Pair this exercise with our dedicated Interview Preparation section for role-specific practice.