Build fluency in the terms behind Groq's high-speed LPU inference platform.
0 / 5 completed
1 / 5
At standup, someone asks why Groq advertises such high tokens/s. What hardware underpins this?
Groq's speed comes from its LPU (Language Processing Unit), a deterministic, single-core streaming processor purpose-built for sequential inference. Unlike a GPU, the LPU avoids dynamic scheduling overhead, which yields very high and predictable tokens/s. This architectural choice is the headline differentiator of GroqCloud.
2 / 5
During a PR review, a teammate measures latency before the first token arrives. What is this metric called?
TTFT (time to first token) measures the delay from sending a request to receiving the very first token of the response. It is distinct from tokens/s, which measures steady-state generation speed. Groq optimizes both, but TTFT is what users feel as initial responsiveness.
3 / 5
In a design review, you need to call a specific model on Groq. Where do you point requests?
GroqCloud exposes model endpoints via an OpenAI-compatible API, where the model is selected by the model field in the request body. You authenticate with a Groq API key and the platform routes to the appropriate served model. This compatibility makes migration from other providers straightforward.
4 / 5
An incident report shows degraded throughput. Which metric most directly captures generation speed?
tokens/s is the per-request generation rate and the headline metric Groq markets. A drop in tokens/s indicates slower decoding, whereas a rise in TTFT would indicate slower startup. Separating these two helps diagnose whether the bottleneck is queueing or decoding.
5 / 5
During a code review, a dev hardcodes one model name everywhere. What does GroqCloud recommend instead?
GroqCloud serves a catalog of model endpoints identified by stable model ids that should be passed in the model field, ideally sourced from configuration. Hardcoding aliases risks breakage when models are rotated or deprecated. Treating the model id as config keeps the GroqCloud integration maintainable.