LLMOps Workflow Language
5 exercises — Master the vocabulary of running LLM applications in production: prompt versioning, model registry, observability, A/B testing, and quality drift monitoring.
0 / 5 completed
Quick reference: LLMOps core concepts
- Prompt registry — versioned, centralised store for named prompt templates separate from code
- Model alias — mutable label (e.g.,
production) pointing to a specific model version in the registry - TTFT — Time To First Token; the streaming latency metric users feel most directly
- Prompt A/B test — experiment splitting traffic between control (current) and treatment (new) prompts
- Quality drift — gradual decline in output quality detected by continuous evaluation of live traffic
1 / 5
A team lead proposes: "We need to stop hardcoding prompts in the application repository. Let's move them to a prompt registry and version-control them separately so we can roll back bad prompts without a code deployment."
What does a prompt registry provide?
A prompt registry decouples prompt management from code deployments.
A prompt registry (also called a prompt store or prompt management platform) stores prompt templates as versioned, named artifacts — independently of the application codebase. This lets teams roll back to a previous prompt version when quality degrades, audit who changed a prompt and why, and run side-by-side experiments, all without touching application code. Tools such as LangSmith, Langfuse, PromptLayer, and MLflow Prompt Registry provide this capability.
Embedding storage (A) is for vector search, not prompt versioning. Injection linting (B) is a validation step, not a registry. Feature flags (C) control availability but don't version or store prompt content.
Key vocabulary:
• prompt registry — a versioned, centralised store for named prompt templates, separate from application code
• prompt versioning — tracking every change to a prompt with a version identifier and metadata
• rollback — reverting a prompt to a previous version when a newer one causes quality regression
• audit trail — a log of who changed a prompt, when, and why
A prompt registry (also called a prompt store or prompt management platform) stores prompt templates as versioned, named artifacts — independently of the application codebase. This lets teams roll back to a previous prompt version when quality degrades, audit who changed a prompt and why, and run side-by-side experiments, all without touching application code. Tools such as LangSmith, Langfuse, PromptLayer, and MLflow Prompt Registry provide this capability.
Embedding storage (A) is for vector search, not prompt versioning. Injection linting (B) is a validation step, not a registry. Feature flags (C) control availability but don't version or store prompt content.
Key vocabulary:
• prompt registry — a versioned, centralised store for named prompt templates, separate from application code
• prompt versioning — tracking every change to a prompt with a version identifier and metadata
• rollback — reverting a prompt to a previous version when a newer one causes quality regression
• audit trail — a log of who changed a prompt, when, and why