Practice English vocabulary for LLM observability: traces, token counts, TTFT latency, spans for retrieval and generation, and LLM-specific telemetry.
0 / 10 completed
1 / 10
What does 'the LLM trace shows the full prompt and response' mean?
LLM traces (in tools like LangSmith, Weights & Biases Prompts, or OpenTelemetry with LLM extensions) capture the full prompt and response for each call. This is essential for debugging: you can see whether hallucinations came from missing context or poor instructions.
2 / 10
Why does 'token count per call' matter for LLM observability?
Monitoring token counts reveals cost anomalies (a single call consuming 50K tokens), prompt engineering opportunities (prompts that can be shortened without quality loss), and helps with capacity planning for high-volume LLM applications.
3 / 10
What is 'TTFT (Time to First Token)' and why is it important?
In streaming LLM applications, TTFT determines how quickly the user starts seeing output. A long TTFT feels slow even if total generation time is reasonable. TTFT is affected by queue time, context length, and model warm-up — tracking it separately from total latency is essential.
4 / 10
What does 'the span captures the retrieval and generation separately' mean?
In a RAG pipeline, separate spans for retrieval (vector DB query) and generation (LLM call) allow profiling each step independently. If P99 latency is 5s, spans reveal whether it's slow vector search (0.2s), slow LLM generation (4.5s), or both — directing optimization effort precisely.
5 / 10
What is 'LLM-specific telemetry' beyond standard application metrics?
Standard application telemetry (requests/sec, error rate, p99 latency) is necessary but not sufficient for LLM applications. LLM-specific telemetry adds insight into why a call was expensive (long prompt), why it stopped (reached max_tokens), and whether prompt caching is working (cache hit rate).
6 / 10
Code Review Comment: Sarah comments on a PR for the 'GenerateProductDescription' function:
'I'm seeing high latency here. The LLM trace shows the full prompt and response, but the processing time is significantly longer than expected. Can you investigate whether the prompt complexity is contributing to this?'
This question tests understanding of a core observability concept: detailed traces provide insights into the entire workflow. The phrase 'the full prompt and response' highlights that the trace captures not just the output but also the input, allowing for analysis of potential bottlenecks related to prompt complexity. The incorrect options misinterpret the purpose of a trace or focus on unrelated causes of latency.
7 / 10
Slack Message: Alex from DevOps sends a message to the team:
'We're hitting our rate limit for the LLM API. Token count per call is approaching its maximum, and we need to optimize our queries to avoid errors.'
This question focuses on a practical concern: API rate limiting. 'Token count per call' is crucial because LLM APIs often have limits based on the number of tokens processed in each request (prompt + response). Monitoring this allows proactive management to avoid service disruptions and control costs. The other options misrepresent the role of token counts or suggest ineffective solutions.
8 / 10
PR Description: You're writing a description for a PR that adds LLM observability to your application. You want to explain 'TTFT':
'We've implemented metrics to track TTFT (Time To First Token) for the LLM-powered chatbot. This metric is important because it directly reflects user experience and helps us identify slow response times.'
This question assesses understanding of a critical metric in LLM observability. 'TTFT' (Time To First Token) is directly tied to the initial user experience – it's the time until the first token of the response arrives. Tracking this provides immediate feedback on responsiveness and highlights potential issues with prompt parsing or generation speed, far beyond just overall response time.
9 / 10
Standup Update: Mark is updating the team:
'We've been adding span captures to our LLM pipeline. The span captures the retrieval and generation separately, allowing us to pinpoint exactly where time is being spent within the LLM workflow.'
This question tests understanding of distributed tracing with LLMs. 'Span captures' break down complex workflows into smaller units (spans), enabling detailed timing analysis. Separating retrieval and generation allows pinpointing bottlenecks – is the prompt parsing slow? Is the model inference itself taking too long? This level of detail isn't possible with traditional metrics.
10 / 10
API Response: You receive an API response from your LLM monitoring system:
{
"model": "gpt-4",
"prompt_tokens": 123,
"completion_tokens": 876,
"latency_ms": 550,
"error": null
}
'The 'completion_tokens' field is a key component of LLM-specific telemetry, going beyond standard application metrics.'
This question addresses the nuanced nature of LLM telemetry. 'Completion tokens' are vital because they represent the *actual* amount of text generated by the model – a critical factor in both cost calculations (many APIs charge per token) and understanding prompt effectiveness. It's distinct from simple request counts, offering deeper insights into the model's output.
What will I practise in "LLM Observability Vocabulary"?
Practice English vocabulary for LLM observability: traces, token counts, TTFT latency, spans for retrieval and generation, and LLM-specific telemetry.
How many exercises are in this module?
This module has 10 multiple-choice exercises, each with instant feedback and a full explanation of the correct answer.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free to use with no account, sign-up, or paywall.
Do I need to create an account to do these exercises?
No account is required. Just click an option to answer — your score for this session is tracked automatically in the progress bar above.
What happens if I choose the wrong answer?
You'll immediately see which answer was correct, plus a full explanation covering the vocabulary and reasoning behind it — mistakes are where most of the learning happens.
Can I retry the exercises if I want a higher score?
Yes — use the "Try again" button on the results screen to reset and go through all the questions again.
Is my progress saved if I close the page?
No. Progress is tracked only for your current visit; reloading or leaving the page resets the counter. This keeps the exercise simple and account-free.
Where can I find more LLM App Development exercises?
Browse the full LLM App Development hub for related drills, or check the "Next up" link below to continue with a connected topic.
How is this different from reading an article on the same topic?
Articles explain vocabulary and concepts in prose; this exercise tests and reinforces that vocabulary through active recall with immediate feedback — the two work best together.
Who writes these exercises?
Every exercise is written by the CoderSlingo team, drawing on real workplace English used in IT roles, then reviewed for accuracy and clarity.