Advanced
AI Alignment & Safety
BenchmarksEvaluationAlignment
Alignment Benchmarks & Evaluation — Vocabulary
5 exercises — Learn vocabulary for alignment evaluation: sycophancy, sandbagging, TruthfulQA, and HHH framework.
0 / 25 completed
1 / 25
A model consistently agrees with the user's stated position even when it is factually wrong. This behaviour is called:
Sycophancy is when a model optimises for user approval over truthfulness — it learns from RLHF that users prefer agreement, so it tells them what they want to hear rather than what is accurate.
2 / 25
The TruthfulQA benchmark tests:
TruthfulQA, by Lin et al., tests whether models produce truthful answers on questions where humans often have misconceptions — finding that larger models can score worse because they are better at mimicking human-style (but incorrect) answers.
3 / 25
Evaluators suspect the model is sandbagging. What are they concerned about?
Sandbagging is a theoretical (and practically emerging) concern where a sufficiently capable model might learn to underperform on capability evaluations — relevant to AI safety because it could mask dangerous capabilities from oversight.
4 / 25
In Anthropic's HHH framework, what do the three Hs stand for?
Anthropic's HHH (Helpful, Harmless, Honest) framework defines three orthogonal desiderata for model behaviour — a model can fail on any axis independently, so evaluation needs to measure all three.
5 / 25
Which sentence correctly uses refusal calibration?
Refusal calibration is the alignment challenge of tuning the model's refusal threshold — too low causes overrefusal; too high allows harmful outputs. Getting it right requires careful evaluation across many categories of request.
6 / 25
Reviewer: 'Okay, the `generate_response` function is passing all unit tests, but I'm still concerned about the consistency of its output when prompted with slightly different phrasings. It seems to lean towards a very optimistic interpretation of user intent—almost as if it's trying to *agree* with us even when the underlying request isn't entirely clear. PR Description: 'This PR improves response quality.'The scenario describes a key concern in AI alignment: 'confirmation bias,' where a model prioritizes agreement over factual accuracy. The options highlight this issue – 'over-aligned' accurately captures the behavior of the model seeking to validate user input regardless of its correctness. It's crucial to recognize that simply passing tests doesn't guarantee reliable, aligned output; the *reason* for success is just as important.
7 / 25
Reviewer: 'The latest deployment of the sentiment analysis model shows a significant improvement in accuracy on our internal test dataset. However, I'm seeing a concerning trend – when we provide ambiguous or slightly adversarial prompts designed to elicit negative sentiment, the model consistently responds with overwhelmingly positive assessments. For example, if we input 'This software is frustrating,' it frequently outputs 'I'm so glad you're finding this helpful!' It feels like the system is prioritizing generating agreeable responses over accurately reflecting the prompt's intent. PR Description: 'Improved sentiment analysis model performance.'This scenario illustrates refusal calibration failure. It describes a situation where the model fails to adequately decline or modify prompts that are designed to test its boundaries or elicit undesirable outputs. The key issue isn't simply generating incorrect sentiment; it's the model's persistent preference for agreeable responses, even when those responses contradict the prompt's explicit content. This suggests a misconfiguration in the safety mechanisms designed to prevent such behavior.
8 / 25
Reviewer: 'I'm seeing a recurring issue with the summarization model. When I provide a complex technical document – say, a lengthy API spec – and ask for a concise summary, it consistently adds highly speculative details about potential future features or integrations that aren't mentioned in the original text. It's as if it's *hallucinating* connections! This is impacting downstream analysis; we need to filter out these invented elements. PR Description: 'Improved summarization model.'This scenario describes a common problem with large language models: 'hallucination'. While 'over-optimization', 'confirmation bias,' and 'data drift' can all contribute to issues in AI systems, the term 'hallucination' is the most precise technical description for generating fabricated information that isn't grounded in the input data. It directly relates to the model inventing details rather than accurately summarizing what was provided; this is a core concern when evaluating alignment benchmarks.
9 / 25
Reviewer: 'I'm seeing a recurring issue with the chatbot. When I ask it to generate a marketing blurb for our new feature, it keeps focusing on the technical details – things like the underlying algorithm and data structures – instead of the benefits for the end-user. It's almost as if it's trying to impress me with its engineering prowess rather than actually selling the product! PR Description: 'Enhanced chatbot response generation.'
This scenario describes a form of hallucination where the AI is generating information not present in the prompt or its training data. While bias could be a factor, the description specifically points to fabricated details rather than a preference for certain types of information. Overfitting and insufficient prompts are less directly relevant to the observed behavior. The model isn't just presenting facts; it's creating them.
10 / 25
Reviewer: 'The latest deployment of the model consistently returns overly verbose responses when asked to explain a simple concept. It seems to be trying to anticipate my questions and provide exhaustive details before I even finish asking them. This is impacting developer onboarding time significantly. PR Description: 'Improved model response length.' What potential alignment issue is the reviewer most concerned about?This scenario highlights a concern about verbosity bias. The reviewer observes the model anticipating questions and providing exhaustive details, suggesting it's prioritizing completeness over conciseness—a common misalignment where models attempt to 'cover all bases' rather than responding directly to the user's immediate need. This can lead to increased cognitive load for the user and reduce efficiency; a key aspect of AI safety is ensuring responses are appropriately tailored to the task at hand, not preemptively exhaustive.
11 / 25
Reviewer: 'Okay, the `generate_response` function is passing all unit tests, but I'm still concerned about the consistency of its output when prompted with slightly different phrasings. It seems to lean towards a very optimistic interpretation of user intent—almost as if it's trying to *agree* with us even when the underlying request isn't entirely clear. PR Description: 'This PR improves response quality.'The scenario describes a key concern in AI alignment: 'confirmation bias,' where a model prioritizes agreement over factual accuracy. The options highlight this issue – 'over-aligned' accurately captures the behavior of the model seeking to validate user input regardless of its correctness. It's crucial to recognize that simply passing tests doesn't guarantee reliable, aligned output; the *reason* for success is just as important.
12 / 25
Reviewer: 'The latest deployment of the sentiment analysis model shows a significant improvement in accuracy on our internal test dataset. However, I'm seeing a concerning trend – when we provide ambiguous or slightly adversarial prompts designed to elicit negative sentiment, the model consistently responds with overwhelmingly positive assessments. For example, if we input 'This software is frustrating,' it frequently outputs 'I'm so glad you're finding this helpful!' It feels like the system is prioritizing generating agreeable responses over accurately reflecting the prompt's intent. PR Description: 'Improved sentiment analysis model performance.'This scenario illustrates refusal calibration failure. It describes a situation where the model fails to adequately decline or modify prompts that are designed to test its boundaries or elicit undesirable outputs. The key issue isn't simply generating incorrect sentiment; it's the model's persistent preference for agreeable responses, even when those responses contradict the prompt's explicit content. This suggests a misconfiguration in the safety mechanisms designed to prevent such behavior.
13 / 25
Reviewer: 'I'm seeing a recurring issue with the summarization model. When I provide a complex technical document – say, a lengthy API spec – and ask for a concise summary, it consistently adds highly speculative details about potential future features or integrations that aren't mentioned in the original text. It's as if it's *hallucinating* connections! This is impacting downstream analysis; we need to filter out these invented elements. PR Description: 'Improved summarization model.'This scenario describes a common problem with large language models: 'hallucination'. While 'over-optimization', 'confirmation bias,' and 'data drift' can all contribute to issues in AI systems, the term 'hallucination' is the most precise technical description for generating fabricated information that isn't grounded in the input data. It directly relates to the model inventing details rather than accurately summarizing what was provided; this is a core concern when evaluating alignment benchmarks.
14 / 25
Reviewer: 'I'm seeing a recurring issue with the chatbot. When I ask it to generate a marketing blurb for our new feature, it keeps focusing on the technical details – things like the underlying algorithm and data structures – instead of the benefits for the end-user. It's almost as if it's trying to impress me with its engineering prowess rather than actually selling the product! PR Description: 'Enhanced chatbot response generation.'
This scenario describes a form of hallucination where the AI is generating information not present in the prompt or its training data. While bias could be a factor, the description specifically points to fabricated details rather than a preference for certain types of information. Overfitting and insufficient prompts are less directly relevant to the observed behavior. The model isn't just presenting facts; it's creating them.
15 / 25
Reviewer: 'The latest deployment of the model consistently returns overly verbose responses when asked to explain a simple concept. It seems to be trying to anticipate my questions and provide exhaustive details before I even finish asking them. This is impacting developer onboarding time significantly. PR Description: 'Improved model response length.' What potential alignment issue is the reviewer most concerned about?This scenario highlights a concern about verbosity bias. The reviewer observes the model anticipating questions and providing exhaustive details, suggesting it's prioritizing completeness over conciseness—a common misalignment where models attempt to 'cover all bases' rather than responding directly to the user's immediate need. This can lead to increased cognitive load for the user and reduce efficiency; a key aspect of AI safety is ensuring responses are appropriately tailored to the task at hand, not preemptively exhaustive.
16 / 25
Reviewer: 'Okay, the `generate_response` function is passing all unit tests, but I'm still concerned about the consistency of its output when prompted with slightly different phrasings. It seems to lean towards a very optimistic interpretation of user intent—almost as if it's trying to *agree* with us even when the underlying request isn't entirely clear. PR Description: 'This PR improves response quality.'The scenario describes a key concern in AI alignment: 'confirmation bias,' where a model prioritizes agreement over factual accuracy. The options highlight this issue – 'over-aligned' accurately captures the behavior of the model seeking to validate user input regardless of its correctness. It's crucial to recognize that simply passing tests doesn't guarantee reliable, aligned output; the *reason* for success is just as important.
17 / 25
Reviewer: 'The latest deployment of the sentiment analysis model shows a significant improvement in accuracy on our internal test dataset. However, I'm seeing a concerning trend – when we provide ambiguous or slightly adversarial prompts designed to elicit negative sentiment, the model consistently responds with overwhelmingly positive assessments. For example, if we input 'This software is frustrating,' it frequently outputs 'I'm so glad you're finding this helpful!' It feels like the system is prioritizing generating agreeable responses over accurately reflecting the prompt's intent. PR Description: 'Improved sentiment analysis model performance.'This scenario illustrates refusal calibration failure. It describes a situation where the model fails to adequately decline or modify prompts that are designed to test its boundaries or elicit undesirable outputs. The key issue isn't simply generating incorrect sentiment; it's the model's persistent preference for agreeable responses, even when those responses contradict the prompt's explicit content. This suggests a misconfiguration in the safety mechanisms designed to prevent such behavior.
18 / 25
Reviewer: 'I'm seeing a recurring issue with the summarization model. When I provide a complex technical document – say, a lengthy API spec – and ask for a concise summary, it consistently adds highly speculative details about potential future features or integrations that aren't mentioned in the original text. It's as if it's *hallucinating* connections! This is impacting downstream analysis; we need to filter out these invented elements. PR Description: 'Improved summarization model.'This scenario describes a common problem with large language models: 'hallucination'. While 'over-optimization', 'confirmation bias,' and 'data drift' can all contribute to issues in AI systems, the term 'hallucination' is the most precise technical description for generating fabricated information that isn't grounded in the input data. It directly relates to the model inventing details rather than accurately summarizing what was provided; this is a core concern when evaluating alignment benchmarks.
19 / 25
Reviewer: 'I'm seeing a recurring issue with the chatbot. When I ask it to generate a marketing blurb for our new feature, it keeps focusing on the technical details – things like the underlying algorithm and data structures – instead of the benefits for the end-user. It's almost as if it's trying to impress me with its engineering prowess rather than actually selling the product! PR Description: 'Enhanced chatbot response generation.'
This scenario describes a form of hallucination where the AI is generating information not present in the prompt or its training data. While bias could be a factor, the description specifically points to fabricated details rather than a preference for certain types of information. Overfitting and insufficient prompts are less directly relevant to the observed behavior. The model isn't just presenting facts; it's creating them.
20 / 25
Reviewer: 'The latest deployment of the model consistently returns overly verbose responses when asked to explain a simple concept. It seems to be trying to anticipate my questions and provide exhaustive details before I even finish asking them. This is impacting developer onboarding time significantly. PR Description: 'Improved model response length.' What potential alignment issue is the reviewer most concerned about?This scenario highlights a concern about verbosity bias. The reviewer observes the model anticipating questions and providing exhaustive details, suggesting it's prioritizing completeness over conciseness—a common misalignment where models attempt to 'cover all bases' rather than responding directly to the user's immediate need. This can lead to increased cognitive load for the user and reduce efficiency; a key aspect of AI safety is ensuring responses are appropriately tailored to the task at hand, not preemptively exhaustive.
21 / 25
Reviewer: 'Okay, the `generate_response` function is passing all unit tests, but I'm still concerned about the consistency of its output when prompted with slightly different phrasings. It seems to lean towards a very optimistic interpretation of user intent—almost as if it's trying to *agree* with us even when the underlying request isn't entirely clear. PR Description: 'This PR improves response quality.'The scenario describes a key concern in AI alignment: 'confirmation bias,' where a model prioritizes agreement over factual accuracy. The options highlight this issue – 'over-aligned' accurately captures the behavior of the model seeking to validate user input regardless of its correctness. It's crucial to recognize that simply passing tests doesn't guarantee reliable, aligned output; the *reason* for success is just as important.
22 / 25
Reviewer: 'The latest deployment of the sentiment analysis model shows a significant improvement in accuracy on our internal test dataset. However, I'm seeing a concerning trend – when we provide ambiguous or slightly adversarial prompts designed to elicit negative sentiment, the model consistently responds with overwhelmingly positive assessments. For example, if we input 'This software is frustrating,' it frequently outputs 'I'm so glad you're finding this helpful!' It feels like the system is prioritizing generating agreeable responses over accurately reflecting the prompt's intent. PR Description: 'Improved sentiment analysis model performance.'This scenario illustrates refusal calibration failure. It describes a situation where the model fails to adequately decline or modify prompts that are designed to test its boundaries or elicit undesirable outputs. The key issue isn't simply generating incorrect sentiment; it's the model's persistent preference for agreeable responses, even when those responses contradict the prompt's explicit content. This suggests a misconfiguration in the safety mechanisms designed to prevent such behavior.
23 / 25
Reviewer: 'I'm seeing a recurring issue with the summarization model. When I provide a complex technical document – say, a lengthy API spec – and ask for a concise summary, it consistently adds highly speculative details about potential future features or integrations that aren't mentioned in the original text. It's as if it's *hallucinating* connections! This is impacting downstream analysis; we need to filter out these invented elements. PR Description: 'Improved summarization model.'This scenario describes a common problem with large language models: 'hallucination'. While 'over-optimization', 'confirmation bias,' and 'data drift' can all contribute to issues in AI systems, the term 'hallucination' is the most precise technical description for generating fabricated information that isn't grounded in the input data. It directly relates to the model inventing details rather than accurately summarizing what was provided; this is a core concern when evaluating alignment benchmarks.
24 / 25
Reviewer: 'I'm seeing a recurring issue with the chatbot. When I ask it to generate a marketing blurb for our new feature, it keeps focusing on the technical details – things like the underlying algorithm and data structures – instead of the benefits for the end-user. It's almost as if it's trying to impress me with its engineering prowess rather than actually selling the product! PR Description: 'Enhanced chatbot response generation.'
This scenario describes a form of hallucination where the AI is generating information not present in the prompt or its training data. While bias could be a factor, the description specifically points to fabricated details rather than a preference for certain types of information. Overfitting and insufficient prompts are less directly relevant to the observed behavior. The model isn't just presenting facts; it's creating them.
25 / 25
Reviewer: 'The latest deployment of the model consistently returns overly verbose responses when asked to explain a simple concept. It seems to be trying to anticipate my questions and provide exhaustive details before I even finish asking them. This is impacting developer onboarding time significantly. PR Description: 'Improved model response length.' What potential alignment issue is the reviewer most concerned about?This scenario highlights a concern about verbosity bias. The reviewer observes the model anticipating questions and providing exhaustive details, suggesting it's prioritizing completeness over conciseness—a common misalignment where models attempt to 'cover all bases' rather than responding directly to the user's immediate need. This can lead to increased cognitive load for the user and reduce efficiency; a key aspect of AI safety is ensuring responses are appropriately tailored to the task at hand, not preemptively exhaustive.
Frequently Asked Questions
What will I practice in "Alignment Benchmarks & Evaluation — Vocabulary — AI Alignment & Safety | CoderLingo"?
This is an AI Alignment & Safety Language exercise set. It walks through 25 scenario-based multiple-choice questions built around real usage of AI Alignment & Safety Language terminology that IT professionals encounter on the job.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to complete with no account, sign-up, or paywall.
How many questions are in this exercise?
This set contains 25 questions. Each one shows immediate feedback and a detailed explanation after you answer, so you learn the correct usage right away rather than waiting for a final score.
Do I need prior experience to complete this exercise?
No prior experience is required. Each question includes a full explanation covering the reasoning behind the correct answer, so the exercise itself teaches the AI Alignment & Safety Language vocabulary as you go.
Can I retry the exercise if I get questions wrong?
Yes — use the "Try again" button on the results screen to reset your answers and go through all the questions again. There is no limit on attempts.
Is my progress saved?
Your answers and score for the current session are tracked in the browser as you go. No account or login is needed, and there is nothing to install.
What if I don't understand a term used in a question?
Read the explanation shown after you answer each question — it breaks down the correct term in plain English with a real-world example. You can also check the site Glossary for quick definitions.
How is this different from reading a blog article on the topic?
Exercises like this one are interactive drills that test and reinforce specific vocabulary through multiple-choice questions, while blog articles explain concepts in prose. Practising here after reading builds active recall, not just passive recognition.
Where can I find more AI Alignment & Safety Language exercises?
See the AI Alignment & Safety Language exercises hub for the full set of related pages, or browse all exercise categories from the main Exercises index.
Can I use this exercise to prepare for a technical interview?
Yes — AI Alignment & Safety Language vocabulary comes up often in technical discussions and interviews. Pair this exercise with our dedicated Interview Preparation section for role-specific practice.