Advanced
AI Alignment & Safety
GovernancePolicyRegulation
AI Governance & Policy Vocabulary
5 exercises — Learn AI governance and policy vocabulary: AI Safety Institute, voluntary commitments, frontier AI, capability thresholds, red-teaming requirements, and safety evaluations.
0 / 29 completed
1 / 29
What is an AI Safety Institute (AISI) in the context of AI governance?
AI Safety Institutes (e.g. the UK AISI, US AISI) are government-affiliated bodies whose mandate is to evaluate advanced AI models for safety risks — including dangerous capabilities, misuse potential, and systemic risks. They conduct pre-deployment testing and publish safety evaluations to inform regulation.
2 / 29
In AI governance, a voluntary commitment by an AI company typically means:
Voluntary commitments (e.g. the White House July 2023 commitments, the Seoul AI Safety Summit pledges) are self-regulatory pledges by AI companies — covering practices like pre-deployment red-teaming, information sharing with governments, and safety evaluations. They carry no legal penalties for non-compliance but create reputational and political accountability.
3 / 29
What is a frontier AI model in the context of AI policy and regulation?
"Frontier AI" refers to the most advanced AI models currently being developed — those at or near the capability frontier. Regulatory frameworks (e.g. the EU AI Act, UK AI Safety Summit) use this term to designate the AI systems that pose the greatest potential risk and therefore require heightened scrutiny, evaluation, and safety obligations.
4 / 29
A policy document states: "The model must pass safety evals before deployment." What does this mean in practice?
"Safety evals" refers to structured evaluations designed to test whether a model has dangerous capabilities — such as providing meaningful uplift to those seeking to create biological, chemical, nuclear, or radiological weapons, or assisting cyberattacks. Companies like Anthropic publish "responsible scaling policies" specifying capability thresholds that trigger deployment restrictions.
5 / 29
What does red-teaming mean specifically in an AI governance / pre-deployment context?
In AI governance, red-teaming means deliberately trying to make the AI system fail in harmful ways — eliciting dangerous information, bypassing safety guardrails, generating CSAM, or providing uplift for weapons. Policy frameworks increasingly require documented red-teaming before frontier model deployment, with results shared with relevant government bodies.
6 / 29
PR Description:
"Implemented new sentiment analysis model for customer support ticket triage. Initial tests show a 95% accuracy rate on our internal dataset. Model is trained using the latest GPT-4 Turbo variant and deployed via API endpoint /api/v1/sentiment."This question tests understanding of proactive AI governance in deployment. While high accuracy is important, simply stating it isn't enough; a responsible developer needs to acknowledge potential limitations and biases inherent in any large language model like GPT-4 Turbo. The description should outline how the model will be monitored for drift, and what processes are in place to address unexpected behavior – crucial elements of AI alignment beyond just initial accuracy metrics. Option D is too optimistic without these considerations.
7 / 29
Reviewer Comment: 'This model's output is drifting significantly over time. We need to implement more robust monitoring and potentially a 'kill switch' if it starts generating nonsensical responses. Also, consider adding logging around the API calls—we're not tracking how frequently this is being used.'This comment demonstrates an understanding of key concepts in AI governance beyond just raw performance. The 'kill switch' and monitoring are directly related to proactive safety measures – preventing unintended consequences or harmful outputs. Drift detection is crucial for maintaining model reliability over time, a concept central to alignment strategies. It's not about simply improving accuracy; it's about operationalizing the risk.
8 / 29
Slack Message
User A: "Just deployed the new AI assistant for onboarding. It's using the latest LangChain framework and a fine-tuned Llama 2 model. Initial feedback is positive!"
User B: "Sounds good, but have you considered incorporating techniques like 'constitutional AI' to guide its responses and mitigate potential harmful outputs? We need to think about alignment at this stage."User B's comment highlights an important consideration in AI governance: proactive alignment. 'Constitutional AI' refers to a technique where an AI is trained with a set of ethical principles – a 'constitution' – to guide its responses and reduce the risk of generating harmful or biased outputs. This demonstrates an understanding that simply deploying an AI doesn't guarantee it will behave safely or ethically, requiring ongoing efforts to ensure alignment with desired values.
9 / 29
During a code review for an AI model designed to summarize legal documents, the lead engineer presents this PR description:
`Implemented summarization model using BERT and fine-tuned on a dataset of court rulings. The model achieves 82% F1 score on our validation set. Output is serialized as JSON with fields: 'summary', 'confidence_score', 'source_document'. Deployed via the `summarize` API endpoint at /api/v1/summarize. The system automatically retrains weekly based on new rulings added to the source dataset.`
The reviewer comments: 'I'm concerned about the model's reliance on a constantly retraining process. Frequent retraining could introduce bias from recent, potentially skewed, court decisions and create instability in performance. Also, the confidence score seems arbitrary – it needs proper calibration.' Which of the following best addresses the reviewer's concerns regarding AI alignment and safety?
The correct answer (human oversight) directly tackles the reviewer's concerns about bias introduction through frequent retraining. The other options are less effective: simply filtering data doesn't guarantee unbiased training; removing confidence scores doesn't address the core instability issue caused by constant updates; and reducing retraining frequency alone won't prevent the model from drifting due to biases present in the underlying data. This highlights the importance of incorporating human judgment – a key aspect of AI alignment – into the development lifecycle, particularly when dealing with sensitive domains like legal information.
10 / 29
PR Description:
"Implemented new sentiment analysis model for customer support ticket triage. Initial tests show a 95% accuracy rate on our internal dataset. Model is trained using the latest GPT-4 Turbo variant and deployed via API endpoint /api/v1/sentiment."This question tests understanding of proactive AI governance in deployment. While high accuracy is important, simply stating it isn't enough; a responsible developer needs to acknowledge potential limitations and biases inherent in any large language model like GPT-4 Turbo. The description should outline how the model will be monitored for drift, and what processes are in place to address unexpected behavior – crucial elements of AI alignment beyond just initial accuracy metrics. Option D is too optimistic without these considerations.
11 / 29
Reviewer Comment: 'This model's output is drifting significantly over time. We need to implement more robust monitoring and potentially a 'kill switch' if it starts generating nonsensical responses. Also, consider adding logging around the API calls—we're not tracking how frequently this is being used.'This comment demonstrates an understanding of key concepts in AI governance beyond just raw performance. The 'kill switch' and monitoring are directly related to proactive safety measures – preventing unintended consequences or harmful outputs. Drift detection is crucial for maintaining model reliability over time, a concept central to alignment strategies. It's not about simply improving accuracy; it's about operationalizing the risk.
12 / 29
Slack Message
User A: "Just deployed the new AI assistant for onboarding. It's using the latest LangChain framework and a fine-tuned Llama 2 model. Initial feedback is positive!"
User B: "Sounds good, but have you considered incorporating techniques like 'constitutional AI' to guide its responses and mitigate potential harmful outputs? We need to think about alignment at this stage."User B's comment highlights an important consideration in AI governance: proactive alignment. 'Constitutional AI' refers to a technique where an AI is trained with a set of ethical principles – a 'constitution' – to guide its responses and reduce the risk of generating harmful or biased outputs. This demonstrates an understanding that simply deploying an AI doesn't guarantee it will behave safely or ethically, requiring ongoing efforts to ensure alignment with desired values.
13 / 29
During a code review for an AI model designed to summarize legal documents, the lead engineer presents this PR description:
`Implemented summarization model using BERT and fine-tuned on a dataset of court rulings. The model achieves 82% F1 score on our validation set. Output is serialized as JSON with fields: 'summary', 'confidence_score', 'source_document'. Deployed via the `summarize` API endpoint at /api/v1/summarize. The system automatically retrains weekly based on new rulings added to the source dataset.`
The reviewer comments: 'I'm concerned about the model's reliance on a constantly retraining process. Frequent retraining could introduce bias from recent, potentially skewed, court decisions and create instability in performance. Also, the confidence score seems arbitrary – it needs proper calibration.' Which of the following best addresses the reviewer's concerns regarding AI alignment and safety?
The correct answer (human oversight) directly tackles the reviewer's concerns about bias introduction through frequent retraining. The other options are less effective: simply filtering data doesn't guarantee unbiased training; removing confidence scores doesn't address the core instability issue caused by constant updates; and reducing retraining frequency alone won't prevent the model from drifting due to biases present in the underlying data. This highlights the importance of incorporating human judgment – a key aspect of AI alignment – into the development lifecycle, particularly when dealing with sensitive domains like legal information.
14 / 29
PR Description:
"Implemented new sentiment analysis model for customer support ticket triage. Initial tests show a 95% accuracy rate on our internal dataset. Model is trained using the latest GPT-4 Turbo variant and deployed via API endpoint /api/v1/sentiment."This question tests understanding of proactive AI governance in deployment. While high accuracy is important, simply stating it isn't enough; a responsible developer needs to acknowledge potential limitations and biases inherent in any large language model like GPT-4 Turbo. The description should outline how the model will be monitored for drift, and what processes are in place to address unexpected behavior – crucial elements of AI alignment beyond just initial accuracy metrics. Option D is too optimistic without these considerations.
15 / 29
Reviewer Comment: 'This model's output is drifting significantly over time. We need to implement more robust monitoring and potentially a 'kill switch' if it starts generating nonsensical responses. Also, consider adding logging around the API calls—we're not tracking how frequently this is being used.'This comment demonstrates an understanding of key concepts in AI governance beyond just raw performance. The 'kill switch' and monitoring are directly related to proactive safety measures – preventing unintended consequences or harmful outputs. Drift detection is crucial for maintaining model reliability over time, a concept central to alignment strategies. It's not about simply improving accuracy; it's about operationalizing the risk.
16 / 29
Slack Message
User A: "Just deployed the new AI assistant for onboarding. It's using the latest LangChain framework and a fine-tuned Llama 2 model. Initial feedback is positive!"
User B: "Sounds good, but have you considered incorporating techniques like 'constitutional AI' to guide its responses and mitigate potential harmful outputs? We need to think about alignment at this stage."User B's comment highlights an important consideration in AI governance: proactive alignment. 'Constitutional AI' refers to a technique where an AI is trained with a set of ethical principles – a 'constitution' – to guide its responses and reduce the risk of generating harmful or biased outputs. This demonstrates an understanding that simply deploying an AI doesn't guarantee it will behave safely or ethically, requiring ongoing efforts to ensure alignment with desired values.
17 / 29
During a code review for an AI model designed to summarize legal documents, the lead engineer presents this PR description:
`Implemented summarization model using BERT and fine-tuned on a dataset of court rulings. The model achieves 82% F1 score on our validation set. Output is serialized as JSON with fields: 'summary', 'confidence_score', 'source_document'. Deployed via the `summarize` API endpoint at /api/v1/summarize. The system automatically retrains weekly based on new rulings added to the source dataset.`
The reviewer comments: 'I'm concerned about the model's reliance on a constantly retraining process. Frequent retraining could introduce bias from recent, potentially skewed, court decisions and create instability in performance. Also, the confidence score seems arbitrary – it needs proper calibration.' Which of the following best addresses the reviewer's concerns regarding AI alignment and safety?
The correct answer (human oversight) directly tackles the reviewer's concerns about bias introduction through frequent retraining. The other options are less effective: simply filtering data doesn't guarantee unbiased training; removing confidence scores doesn't address the core instability issue caused by constant updates; and reducing retraining frequency alone won't prevent the model from drifting due to biases present in the underlying data. This highlights the importance of incorporating human judgment – a key aspect of AI alignment – into the development lifecycle, particularly when dealing with sensitive domains like legal information.
18 / 29
PR Description:
"Implemented new sentiment analysis model for customer support ticket triage. Initial tests show a 95% accuracy rate on our internal dataset. Model is trained using the latest GPT-4 Turbo variant and deployed via API endpoint /api/v1/sentiment."This question tests understanding of proactive AI governance in deployment. While high accuracy is important, simply stating it isn't enough; a responsible developer needs to acknowledge potential limitations and biases inherent in any large language model like GPT-4 Turbo. The description should outline how the model will be monitored for drift, and what processes are in place to address unexpected behavior – crucial elements of AI alignment beyond just initial accuracy metrics. Option D is too optimistic without these considerations.
19 / 29
Reviewer Comment: 'This model's output is drifting significantly over time. We need to implement more robust monitoring and potentially a 'kill switch' if it starts generating nonsensical responses. Also, consider adding logging around the API calls—we're not tracking how frequently this is being used.'This comment demonstrates an understanding of key concepts in AI governance beyond just raw performance. The 'kill switch' and monitoring are directly related to proactive safety measures – preventing unintended consequences or harmful outputs. Drift detection is crucial for maintaining model reliability over time, a concept central to alignment strategies. It's not about simply improving accuracy; it's about operationalizing the risk.
20 / 29
Slack Message
User A: "Just deployed the new AI assistant for onboarding. It's using the latest LangChain framework and a fine-tuned Llama 2 model. Initial feedback is positive!"
User B: "Sounds good, but have you considered incorporating techniques like 'constitutional AI' to guide its responses and mitigate potential harmful outputs? We need to think about alignment at this stage."User B's comment highlights an important consideration in AI governance: proactive alignment. 'Constitutional AI' refers to a technique where an AI is trained with a set of ethical principles – a 'constitution' – to guide its responses and reduce the risk of generating harmful or biased outputs. This demonstrates an understanding that simply deploying an AI doesn't guarantee it will behave safely or ethically, requiring ongoing efforts to ensure alignment with desired values.
21 / 29
During a code review for an AI model designed to summarize legal documents, the lead engineer presents this PR description:
`Implemented summarization model using BERT and fine-tuned on a dataset of court rulings. The model achieves 82% F1 score on our validation set. Output is serialized as JSON with fields: 'summary', 'confidence_score', 'source_document'. Deployed via the `summarize` API endpoint at /api/v1/summarize. The system automatically retrains weekly based on new rulings added to the source dataset.`
The reviewer comments: 'I'm concerned about the model's reliance on a constantly retraining process. Frequent retraining could introduce bias from recent, potentially skewed, court decisions and create instability in performance. Also, the confidence score seems arbitrary – it needs proper calibration.' Which of the following best addresses the reviewer's concerns regarding AI alignment and safety?
The correct answer (human oversight) directly tackles the reviewer's concerns about bias introduction through frequent retraining. The other options are less effective: simply filtering data doesn't guarantee unbiased training; removing confidence scores doesn't address the core instability issue caused by constant updates; and reducing retraining frequency alone won't prevent the model from drifting due to biases present in the underlying data. This highlights the importance of incorporating human judgment – a key aspect of AI alignment – into the development lifecycle, particularly when dealing with sensitive domains like legal information.
22 / 29
Reviewer Comment: 'The model's response to adversarial prompts is exhibiting unexpected behavior. We should incorporate more stringent input validation and consider a 'red teaming' exercise to proactively identify vulnerabilities. Furthermore, logging all interactions with this model is crucial for debugging.' Which of the following best captures the reviewer's concern regarding AI safety?
This question targets the concept of adversarial robustness. The reviewer highlights 'unexpected behavior' and 'vulnerabilities' when responding to adversarial prompts – directly relating to AI safety concerns regarding malicious input manipulation. Option A is incorrect as it refers to data quantity, while B misrepresents architectural instability and C correctly identifies susceptibility to attack. D focuses on performance metrics, which are a secondary concern compared to direct vulnerability.
23 / 29
User A: "Just deployed the new AI assistant for onboarding. It's using the latest LangChain framework and a fine-tuned Llama 2 model. Initial feedback is positive!" User B: "That's great to hear. Have you implemented any safeguards against hallucination or biases in the responses? It's important to monitor its factual accuracy closely."
Which of the following best describes User B's primary concern?
This scenario assesses understanding of 'hallucination' and 'bias' within AI systems. User B specifically asks about safeguards *against* these issues – a key element of alignment. Options A & B address technical aspects or performance, while option D is completely irrelevant to the core safety concern.
24 / 29
"Okay team, I just finished integrating the new AI-powered code generation tool. It's using Codex and generates suggestions based on context. We're seeing some interesting results – but we need to be *extremely* careful about ensuring it doesn't introduce security vulnerabilities or copyright issues into our codebase. I'm focusing on logging all generated code snippets for auditing purposes."
This scenario probes understanding of risk mitigation. The speaker explicitly states a need to 'be extremely careful' about vulnerabilities and copyright – directly relating to governance concerns. Options A & B represent operational priorities that do *not* address safety; option D is a secondary goal.
25 / 29
Reviewer Sarah comments: 'The AI model's responses to queries about sensitive personal data are exhibiting unpredictable shifts. We need to implement differential privacy techniques and conduct regular audits of the training data to mitigate potential bias and ensure compliance with GDPR.' Which phrasing best captures Sarah's concern regarding AI safety?
Sarah highlights a critical concern: the AI model's responses to sensitive data are unpredictable. This indicates potential bias and non-compliance with regulations like GDPR. Options A and D minimize the problem, while option B accurately describes the core issue – lack of input validation – and option C correctly identifies the need for proactive measures like differential privacy and audits.
26 / 29
During a Slack discussion about deploying a new AI chatbot, User David writes: 'We're using Retrieval-Augmented Generation (RAG) with ChromaDB to ground the model in our knowledge base. We've implemented a confidence score threshold – if the confidence drops below 0.7, it will fallback to a generic response.' What does David primarily describe?
David is explaining RAG – Retrieval-Augmented Generation. This approach uses an external database (ChromaDB) to provide context to the AI model's generation process. The confidence score threshold demonstrates a key element of ensuring reliability and preventing the chatbot from generating inaccurate or misleading information.
27 / 29
Lead Engineer Mark writes in the PR description: 'Implemented a system for monitoring the AI model's output drift. We're tracking key metrics like perplexity and coherence scores over time. If these metrics exceed predefined thresholds, an alert is triggered, and the model is automatically rolled back to the previous stable version.' What is Mark primarily outlining?
Mark is describing a critical aspect of AI safety – monitoring output drift. Tracking metrics like perplexity and coherence provides insight into changes in the model's behavior that could signal degradation or unexpected issues. The automated rollback mechanism demonstrates proactive risk mitigation.
28 / 29
During a daily stand-up, Developer Emily reports: 'I've been focusing on implementing 'constitutional AI' principles into our LLM. I'm using a set of predefined ethical guidelines to steer the model's responses and reduce harmful outputs.' What is Emily referring to?
Emily is explaining 'constitutional AI,' a growing approach that involves defining a set of ethical guidelines or 'constitution' to guide the LLM's behavior. This helps steer responses away from potentially harmful outputs and aligns with broader ethical considerations – it's about explicitly imposing constraints rather than passively hoping for good behavior.
29 / 29
Reviewer Ben writes: 'The model's tendency to generate overly verbose responses is concerning. We need to implement a length penalty during generation and explore techniques like reinforcement learning from human feedback (RLHF) to refine the model's style.' What key action is Ben advocating for regarding AI safety and performance?
Ben identifies the problem of verbose responses and proposes solutions – specifically, a length penalty and RLHF. A length penalty directly addresses the verbosity, while RLHF allows for fine-tuning the model's style based on human preferences, providing a more targeted approach to improving performance.
Frequently Asked Questions
What will I practice in "AI Governance & Policy Vocabulary — AI Alignment & Safety | CoderLingo"?
This is an AI Alignment & Safety Language exercise set. It walks through 29 scenario-based multiple-choice questions built around real usage of AI Alignment & Safety Language terminology that IT professionals encounter on the job.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to complete with no account, sign-up, or paywall.
How many questions are in this exercise?
This set contains 29 questions. Each one shows immediate feedback and a detailed explanation after you answer, so you learn the correct usage right away rather than waiting for a final score.
Do I need prior experience to complete this exercise?
No prior experience is required. Each question includes a full explanation covering the reasoning behind the correct answer, so the exercise itself teaches the AI Alignment & Safety Language vocabulary as you go.
Can I retry the exercise if I get questions wrong?
Yes — use the "Try again" button on the results screen to reset your answers and go through all the questions again. There is no limit on attempts.
Is my progress saved?
Your answers and score for the current session are tracked in the browser as you go. No account or login is needed, and there is nothing to install.
What if I don't understand a term used in a question?
Read the explanation shown after you answer each question — it breaks down the correct term in plain English with a real-world example. You can also check the site Glossary for quick definitions.
How is this different from reading a blog article on the topic?
Exercises like this one are interactive drills that test and reinforce specific vocabulary through multiple-choice questions, while blog articles explain concepts in prose. Practising here after reading builds active recall, not just passive recognition.
Where can I find more AI Alignment & Safety Language exercises?
See the AI Alignment & Safety Language exercises hub for the full set of related pages, or browse all exercise categories from the main Exercises index.
Can I use this exercise to prepare for a technical interview?
Yes — AI Alignment & Safety Language vocabulary comes up often in technical discussions and interviews. Pair this exercise with our dedicated Interview Preparation section for role-specific practice.