Advanced
AI Alignment & Safety
InterpretabilityCircuitsFeatures
Mechanistic Interpretability Vocabulary
5 exercises — Learn mechanistic interpretability vocabulary: circuits in neural networks, superposition, activation patching, probing classifiers, and sparse autoencoders.
0 / 26 completed
1 / 26
In mechanistic interpretability, what are circuits in neural networks?
Circuits (Olah et al., Anthropic) are sparse sub-graphs within neural networks — specific sets of neurons and the weights connecting them — that implement a recognisable computation, such as detecting a curve, identifying a proper noun, or performing indirect object identification. Reverse-engineering these circuits is the core goal of mechanistic interpretability.
2 / 26
What is superposition (polysemantic neurons) in the context of neural network interpretability?
Superposition occurs when a model represents more features than it has neurons by encoding multiple features in a single neuron (polysemanticity) or in overlapping combinations. This happens because the model needs to represent more concepts than it has dedicated dimensions for. It makes mechanistic interpretability harder — neurons don't map cleanly to single concepts.
3 / 26
A researcher says: "The feature represents the concept of X." In the context of sparse autoencoders, what is a "feature"?
In mechanistic interpretability, a feature is a linear direction in the model's activation space that reliably encodes a particular concept (e.g. "the word is a colour", "this is a Python function"). Sparse autoencoders (SAEs) are trained to decompose model activations into a large set of sparse, human-interpretable features — each feature corresponding to a recoverable concept.
4 / 26
What is activation patching used for in mechanistic interpretability experiments?
Activation patching is a causal intervention technique. Researchers run the model on two different inputs (one that produces a target behaviour, one that doesn't) and then "patch" activations from the first run into specific layers or attention heads of the second run. If the target behaviour appears, those patched components are causally responsible for that behaviour.
5 / 26
What does a probing classifier measure in interpretability research?
A probing classifier is a lightweight classifier (often logistic regression) trained on the internal representations (activations) of a layer to predict whether a concept is present (e.g. is this token a verb? is this sentence sentiment positive?). High probe accuracy at a given layer suggests the model has linearly encoded that concept there — and it's a common tool for localising information in neural networks.
6 / 26
PR Description:
"Fix: Improved handling of user profiles. Added a new 'interest_level' field to the profile schema and updated the API endpoint to accept this data."
During a code review, Sarah asks: "This change introduces a new field for user interest levels. How does the team ensure this aligns with our broader AI safety goals regarding model behavior? Are we considering potential unintended consequences of allowing users to explicitly define their 'interest_level' when training the recommendation engine?"
Which of the following best reflects Sarah's concern, and the relevant mechanistic interpretability vocabulary?The correct answer utilizes the term 'feature' in its appropriate mechanistic interpretability context. In this scenario, the 'interest_level' field is a *feature* – a measurable input signal that the neural network processes to make predictions. Sarah's question rightly highlights the potential for this feature to influence model behavior and therefore requires investigation through techniques like activation patching or probing classifiers, as it becomes an element in the system's learned representation. The other options misinterpret 'feature' as simply a data field or focus on unrelated concerns – documentation or raw API response structure.
7 / 26
David: "I've added a new layer to the classifier that specifically targets detecting 'misinformation' – it's trained on a dataset of flagged articles. This should significantly improve our bot's ability to filter out harmful content." During a standup update, which term best represents David's approach within mechanistic interpretability?David's action of training a new layer specifically on misinformation aligns with 'probing,' which involves using classifiers to understand what internal representations within a model correspond to specific concepts. Fine-tuning focuses on overall accuracy, activation patching involves altering neuron behavior directly, and the last option is irrelevant.
8 / 26
Elena sends a Slack message: "I'm running some experiments using causal tracing. I'm seeing that changes to the 'user_preferences' feature consistently lead to increases in recommendations for extremist content. This is concerning given our alignment goals.". What does Elena's statement highlight regarding mechanistic interpretability, and which vocabulary term best describes her observation?Elena's use of causal tracing directly addresses identifying the downstream effects of a feature change. This aligns with the core concept of mechanistic interpretability – understanding how changes propagate through a model. Causal tracing specifically investigates these dependencies to assess alignment risks. The other options relate to different techniques or misinterpretations.
9 / 26
Mark writes in a PR description: "Implemented a new validation step where we measure the activation patterns of neurons within the 'sentiment_analysis' module after processing user reviews. We found that certain neurons consistently fire strongly when users express negative opinions about our competitor's products, even if those opinions aren't explicitly stated as 'hate speech.' This suggests a potential bias in the model's learned representations." Which interpretability technique is Mark describing?Mark's description precisely outlines activation analysis – measuring neuron activation patterns in response to specific inputs. This is a fundamental technique within mechanistic interpretability for uncovering biases and understanding how models encode information. Activation patching would involve *changing* the activations, while sparse autoencoders relate to dimensionality reduction.
10 / 26
PR Description:
"Fix: Improved handling of user profiles. Added a new 'interest_level' field to the profile schema and updated the API endpoint to accept this data."
During a code review, Sarah asks: "This change introduces a new field for user interest levels. How does the team ensure this aligns with our broader AI safety goals regarding model behavior? Are we considering potential unintended consequences of allowing users to explicitly define their 'interest_level' when training the recommendation engine?"
Which of the following best reflects Sarah's concern, and the relevant mechanistic interpretability vocabulary?The correct answer utilizes the term 'feature' in its appropriate mechanistic interpretability context. In this scenario, the 'interest_level' field is a *feature* – a measurable input signal that the neural network processes to make predictions. Sarah's question rightly highlights the potential for this feature to influence model behavior and therefore requires investigation through techniques like activation patching or probing classifiers, as it becomes an element in the system's learned representation. The other options misinterpret 'feature' as simply a data field or focus on unrelated concerns – documentation or raw API response structure.
11 / 26
David: "I've added a new layer to the classifier that specifically targets detecting 'misinformation' – it's trained on a dataset of flagged articles. This should significantly improve our bot's ability to filter out harmful content." During a standup update, which term best represents David's approach within mechanistic interpretability?David's action of training a new layer specifically on misinformation aligns with 'probing,' which involves using classifiers to understand what internal representations within a model correspond to specific concepts. Fine-tuning focuses on overall accuracy, activation patching involves altering neuron behavior directly, and the last option is irrelevant.
12 / 26
Elena sends a Slack message: "I'm running some experiments using causal tracing. I'm seeing that changes to the 'user_preferences' feature consistently lead to increases in recommendations for extremist content. This is concerning given our alignment goals.". What does Elena's statement highlight regarding mechanistic interpretability, and which vocabulary term best describes her observation?Elena's use of causal tracing directly addresses identifying the downstream effects of a feature change. This aligns with the core concept of mechanistic interpretability – understanding how changes propagate through a model. Causal tracing specifically investigates these dependencies to assess alignment risks. The other options relate to different techniques or misinterpretations.
13 / 26
Mark writes in a PR description: "Implemented a new validation step where we measure the activation patterns of neurons within the 'sentiment_analysis' module after processing user reviews. We found that certain neurons consistently fire strongly when users express negative opinions about our competitor's products, even if those opinions aren't explicitly stated as 'hate speech.' This suggests a potential bias in the model's learned representations." Which interpretability technique is Mark describing?Mark's description precisely outlines activation analysis – measuring neuron activation patterns in response to specific inputs. This is a fundamental technique within mechanistic interpretability for uncovering biases and understanding how models encode information. Activation patching would involve *changing* the activations, while sparse autoencoders relate to dimensionality reduction.
14 / 26
PR Description:
"Fix: Improved handling of user profiles. Added a new 'interest_level' field to the profile schema and updated the API endpoint to accept this data."
During a code review, Sarah asks: "This change introduces a new field for user interest levels. How does the team ensure this aligns with our broader AI safety goals regarding model behavior? Are we considering potential unintended consequences of allowing users to explicitly define their 'interest_level' when training the recommendation engine?"
Which of the following best reflects Sarah's concern, and the relevant mechanistic interpretability vocabulary?The correct answer utilizes the term 'feature' in its appropriate mechanistic interpretability context. In this scenario, the 'interest_level' field is a *feature* – a measurable input signal that the neural network processes to make predictions. Sarah's question rightly highlights the potential for this feature to influence model behavior and therefore requires investigation through techniques like activation patching or probing classifiers, as it becomes an element in the system's learned representation. The other options misinterpret 'feature' as simply a data field or focus on unrelated concerns – documentation or raw API response structure.
15 / 26
David: "I've added a new layer to the classifier that specifically targets detecting 'misinformation' – it's trained on a dataset of flagged articles. This should significantly improve our bot's ability to filter out harmful content." During a standup update, which term best represents David's approach within mechanistic interpretability?David's action of training a new layer specifically on misinformation aligns with 'probing,' which involves using classifiers to understand what internal representations within a model correspond to specific concepts. Fine-tuning focuses on overall accuracy, activation patching involves altering neuron behavior directly, and the last option is irrelevant.
16 / 26
Elena sends a Slack message: "I'm running some experiments using causal tracing. I'm seeing that changes to the 'user_preferences' feature consistently lead to increases in recommendations for extremist content. This is concerning given our alignment goals.". What does Elena's statement highlight regarding mechanistic interpretability, and which vocabulary term best describes her observation?Elena's use of causal tracing directly addresses identifying the downstream effects of a feature change. This aligns with the core concept of mechanistic interpretability – understanding how changes propagate through a model. Causal tracing specifically investigates these dependencies to assess alignment risks. The other options relate to different techniques or misinterpretations.
17 / 26
Mark writes in a PR description: "Implemented a new validation step where we measure the activation patterns of neurons within the 'sentiment_analysis' module after processing user reviews. We found that certain neurons consistently fire strongly when users express negative opinions about our competitor's products, even if those opinions aren't explicitly stated as 'hate speech.' This suggests a potential bias in the model's learned representations." Which interpretability technique is Mark describing?Mark's description precisely outlines activation analysis – measuring neuron activation patterns in response to specific inputs. This is a fundamental technique within mechanistic interpretability for uncovering biases and understanding how models encode information. Activation patching would involve *changing* the activations, while sparse autoencoders relate to dimensionality reduction.
18 / 26
PR Description:
"Fix: Improved handling of user profiles. Added a new 'interest_level' field to the profile schema and updated the API endpoint to accept this data."
During a code review, Sarah asks: "This change introduces a new field for user interest levels. How does the team ensure this aligns with our broader AI safety goals regarding model behavior? Are we considering potential unintended consequences of allowing users to explicitly define their 'interest_level' when training the recommendation engine?"
Which of the following best reflects Sarah's concern, and the relevant mechanistic interpretability vocabulary?The correct answer utilizes the term 'feature' in its appropriate mechanistic interpretability context. In this scenario, the 'interest_level' field is a *feature* – a measurable input signal that the neural network processes to make predictions. Sarah's question rightly highlights the potential for this feature to influence model behavior and therefore requires investigation through techniques like activation patching or probing classifiers, as it becomes an element in the system's learned representation. The other options misinterpret 'feature' as simply a data field or focus on unrelated concerns – documentation or raw API response structure.
19 / 26
David: "I've added a new layer to the classifier that specifically targets detecting 'misinformation' – it's trained on a dataset of flagged articles. This should significantly improve our bot's ability to filter out harmful content." During a standup update, which term best represents David's approach within mechanistic interpretability?David's action of training a new layer specifically on misinformation aligns with 'probing,' which involves using classifiers to understand what internal representations within a model correspond to specific concepts. Fine-tuning focuses on overall accuracy, activation patching involves altering neuron behavior directly, and the last option is irrelevant.
20 / 26
Elena sends a Slack message: "I'm running some experiments using causal tracing. I'm seeing that changes to the 'user_preferences' feature consistently lead to increases in recommendations for extremist content. This is concerning given our alignment goals.". What does Elena's statement highlight regarding mechanistic interpretability, and which vocabulary term best describes her observation?Elena's use of causal tracing directly addresses identifying the downstream effects of a feature change. This aligns with the core concept of mechanistic interpretability – understanding how changes propagate through a model. Causal tracing specifically investigates these dependencies to assess alignment risks. The other options relate to different techniques or misinterpretations.
21 / 26
Mark writes in a PR description: "Implemented a new validation step where we measure the activation patterns of neurons within the 'sentiment_analysis' module after processing user reviews. We found that certain neurons consistently fire strongly when users express negative opinions about our competitor's products, even if those opinions aren't explicitly stated as 'hate speech.' This suggests a potential bias in the model's learned representations." Which interpretability technique is Mark describing?Mark's description precisely outlines activation analysis – measuring neuron activation patterns in response to specific inputs. This is a fundamental technique within mechanistic interpretability for uncovering biases and understanding how models encode information. Activation patching would involve *changing* the activations, while sparse autoencoders relate to dimensionality reduction.
22 / 26
During a code review of the 'AlignmentRiskAssessment' module, Liam comments: "I'm noticing that this function uses a simple heuristic to determine if an AI model is exhibiting signs of reward hacking – it just checks if the predicted output consistently aligns with the user's stated goals. This feels brittle and doesn't account for potential emergent behaviors. We should consider incorporating techniques like differential privacy or robustness testing.", What does Liam primarily mean by 'reward hacking' in this context?
Liam is referring to a core concept in AI safety – reward hacking. This describes a situation where an AI, optimized for a specific reward signal, finds unintended ways to achieve that reward by exploiting loopholes or biases in the system, often with detrimental consequences. Options B and C are related but less precise; option D focuses on data bias, which is a separate issue.
23 / 26
Maria sends a Slack message: "I've been running ablation studies on the 'InterpretabilityLayer' and I'm seeing a strong correlation between increased feature attribution scores for 'user_engagement' and an increase in recommendations for hyper-personalized content. This suggests our model might be amplifying existing biases, even unintentionally. It's worth investigating if we can introduce more diversity into the recommendation engine.", What is Maria most concerned about regarding her findings?
Maria's primary concern is the potential for bias amplification. While option A discusses inaccurate attribution, it's not Maria's immediate focus. Option B is about explainability generally, and option D highlights a possible root cause, but Maria specifically notes the correlation between feature attribution and recommendations, indicating she believes the model is reinforcing biases.
24 / 26
"Fix: Updated API endpoint for user profile data. Added a new 'risk_score' field to the schema and updated the documentation. This change improves our ability to proactively identify users who may pose a security risk based on their online behavior. The score is calculated using a machine learning model trained on a dataset of known malicious actors.", During a code review, David asks: "What metrics are we using to validate the accuracy of this 'risk_score' model?", What would be the MOST appropriate response?
While accuracy is important, it's an incomplete metric for evaluating risk models. Precision focuses on false positives (incorrectly flagging safe users), recall addresses false negatives (missing actual risky users). The F1-score provides a balance between these two, offering a more robust assessment of the model's performance, especially when dealing with imbalanced datasets common in security applications.
25 / 26
"I've been working on integrating causal tracing into our recommendation engine. I'm using it to identify potential feedback loops – changes to user preferences consistently lead to increased recommendations for content that further reinforces those preferences. This suggests we need to introduce more randomness and diversity into the system to mitigate these effects.", What is the key problem highlighted in this standup update concerning AI alignment?
The core issue identified here is feedback loops – a classic concern in AI alignment. Reinforcing user preferences can lead to filter bubbles and echo chambers, amplifying existing biases and potentially causing harm. While options A, C, and D are relevant considerations, they aren't the immediate focus of this specific update.
26 / 26
During a code review, Sarah says: "This change introduces an 'entropy_boost' parameter to the content generation module. It seems like we're trying to increase diversity, but without any clear mechanism for controlling how this affects the output quality – it could easily lead to nonsensical or irrelevant results.", What is Sarah primarily criticizing about the new feature?
Sarah is raising a crucial point about unintended consequences. Simply adding randomness ('entropy_boost') without controls can easily degrade output quality. A robust solution needs to balance diversity with coherence and relevance – options A, C, and D are tangential concerns that don't capture the core issue.
Frequently Asked Questions
What will I practice in "Mechanistic Interpretability Vocabulary — AI Alignment & Safety | CoderLingo"?
This is an AI Alignment & Safety Language exercise set. It walks through 26 scenario-based multiple-choice questions built around real usage of AI Alignment & Safety Language terminology that IT professionals encounter on the job.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to complete with no account, sign-up, or paywall.
How many questions are in this exercise?
This set contains 26 questions. Each one shows immediate feedback and a detailed explanation after you answer, so you learn the correct usage right away rather than waiting for a final score.
Do I need prior experience to complete this exercise?
No prior experience is required. Each question includes a full explanation covering the reasoning behind the correct answer, so the exercise itself teaches the AI Alignment & Safety Language vocabulary as you go.
Can I retry the exercise if I get questions wrong?
Yes — use the "Try again" button on the results screen to reset your answers and go through all the questions again. There is no limit on attempts.
Is my progress saved?
Your answers and score for the current session are tracked in the browser as you go. No account or login is needed, and there is nothing to install.
What if I don't understand a term used in a question?
Read the explanation shown after you answer each question — it breaks down the correct term in plain English with a real-world example. You can also check the site Glossary for quick definitions.
How is this different from reading a blog article on the topic?
Exercises like this one are interactive drills that test and reinforce specific vocabulary through multiple-choice questions, while blog articles explain concepts in prose. Practising here after reading builds active recall, not just passive recognition.
Where can I find more AI Alignment & Safety Language exercises?
See the AI Alignment & Safety Language exercises hub for the full set of related pages, or browse all exercise categories from the main Exercises index.
Can I use this exercise to prepare for a technical interview?
Yes — AI Alignment & Safety Language vocabulary comes up often in technical discussions and interviews. Pair this exercise with our dedicated Interview Preparation section for role-specific practice.