English Vocabulary for Data Engineering Discussions

Data engineering has its own dense vocabulary that can make conversations feel impenetrable if you are not familiar with the terms. Whether you are joining a discussion about pipeline reliability, attending a data platform review, or writing documentation, understanding these concepts — and being able to talk about them naturally in English — will help you contribute confidently to data engineering teams.

Key Vocabulary

Medallion architecture — A data organisation pattern with three layers: bronze (raw, unprocessed data), silver (cleaned and validated data), and gold (aggregated, business-ready data). Example: “We store all raw event logs in the bronze layer and run transformation jobs to promote them to silver.”

Data lineage — The ability to trace where data came from, how it was transformed, and where it went. Example: “Our data lineage tooling lets us see exactly which upstream source introduced that null value.”

Schema evolution — The process of changing a data schema over time without breaking downstream consumers. Example: “We use Avro with schema evolution support so adding new fields does not break existing pipelines.”

Partition pruning — A query optimisation technique where the database skips scanning irrelevant partitions based on query filters. Example: “Querying by date activates partition pruning and reduces the scan from terabytes to gigabytes.”

Late-arriving data — Data that arrives in a system after the time window it belongs to has already been processed. Example: “We need to handle late-arriving data because mobile app events can be delayed by up to 48 hours.”

Watermark — In stream processing, a watermark is a threshold that tells the system how far behind real time the data is allowed to be before a window is considered complete. Example: “We set a watermark of 10 minutes to allow for late-arriving data before closing the aggregation window.”

Exactly-once semantics — A processing guarantee ensuring that each piece of data is processed exactly one time — not zero times, not twice. Example: “Kafka Streams supports exactly-once semantics for critical financial transaction pipelines.”

Data contract — A formal agreement between data producers and consumers defining the schema, format, SLA, and quality expectations of a dataset. Example: “The payments team publishes a data contract that guarantees the event schema will not break without a 30-day deprecation notice.”

SLA vs SLO for pipelines — An SLA (Service Level Agreement) is a contractual commitment, usually with penalties for breach. An SLO (Service Level Objective) is an internal target. Example: “Our internal SLO is to process events within five minutes, but the SLA we signed with the client guarantees one-hour freshness.”

CDC (Change Data Capture) — A technique for capturing row-level changes in a database and streaming them to downstream systems. Example: “We use CDC to replicate changes from our production Postgres database into the data warehouse in near-real time.”

How to Use This in Practice

In data engineering discussions, these terms often appear in combinations. Practise hearing them together:

• “We need to handle late-arriving data in the silver layer by extending the watermark.”
• “The data contract specifies the schema evolution rules — you cannot remove fields without a deprecation period.”
• “Exactly-once semantics are critical for this pipeline because we are processing financial events.”

When discussing pipeline reliability, distinguish between SLAs (external, contractual) and SLOs (internal targets). Teams often talk about “missing SLOs” as an early warning before they risk “breaching SLAs.”

Data lineage discussions often use verbs like “trace”, “track”, “audit”, and “surface”: “Can we surface the lineage of this column in the dashboard?” or “We need to audit the lineage before decommissioning that source.”

Example Conversation

Data Engineer (Anastasia): “We’re seeing stale data in the gold layer. The numbers are off by about 12 hours.”

Pipeline Lead: “Is this a late-arriving data issue or a pipeline failure?”

Anastasia: “It’s late-arriving data. Mobile events from offline users are arriving outside our current watermark. I’d suggest extending the watermark from 10 minutes to 4 hours for this specific source.”

Pipeline Lead: “That will delay window closure. Will we still meet our SLO of one-hour freshness for the business dashboard?”

Anastasia: “Not for mobile events. We should update the data contract for that source to reflect a 4-hour freshness guarantee and notify the analytics team.”

Practice Tips

1. Map a pipeline in English: Draw a simple data pipeline for a project you know, then describe each stage aloud in English using the vocabulary from this post. For example: “Raw events land in the bronze layer via CDC from our Postgres database. A Spark job validates and cleans them before promoting to silver…”

2. Explain medallion layers to a non-engineer: Try explaining bronze, silver, and gold layers to someone who is not a data engineer, using an analogy (for example: raw ore, refined metal, finished product). This forces you to connect the technical vocabulary to plain English.

3. Read a streaming architecture blog post: The Confluent blog, the Databricks engineering blog, and the Airflow documentation use all of these terms naturally. Read one article and highlight every term from this post that you find. Try to infer the meaning from context before checking the definition.

In Practice: Navigating Nuance in Collaborative Discussions

Let’s be honest – professional English can feel incredibly nuanced. It’s not just about knowing the definitions of terms like “schema evolution” or “watermarks”; it’s about how you communicate those concepts effectively within a team, especially when dealing with diverse backgrounds and levels of technical understanding. Many non-native speakers find themselves translating directly from their native language, leading to misunderstandings and friction. The key is recognizing that data engineering discussions are often highly context-dependent. What constitutes “acceptable latency” in one environment might be completely unacceptable in another.

Consider a scenario: you’re reviewing a pull request for a new data pipeline. A junior engineer writes the PR description: “Implemented CDC to ingest data from source system. Improved throughput.” While technically accurate, it lacks crucial detail. A more effective phrasing would acknowledge potential downstream impacts. For example, “Implemented Change Data Capture (CDC) from the legacy system into the star schema. Initial testing indicates a 20% improvement in ingestion rate, but further monitoring is required to assess potential impact on downstream reporting latency – specifically, we need to ensure it doesn’t exceed our SLA of 5 minutes for report generation.” Notice the addition of phrases like “potential impact,” “further monitoring,” and referencing specific metrics (“latency” and “SLA”). These subtle additions demonstrate a proactive understanding of data engineering principles and a commitment to mitigating risks.

Another common challenge arises in Slack conversations discussing data lineage. A senior engineer might say, “Let’s trace the flow of this data back to its source.” While understandable, it begs the question: how are we tracing it? The conversation needs more precision. Suggesting the use of a specific tool – perhaps a data catalog or lineage tracking system – and explicitly stating what information you’re looking for (“We need to see all transformations applied to this field, including the source table and any calculated columns”) dramatically improves clarity. It moves beyond simply asking what needs to be found to specifying how that search should be conducted. This avoids the frustrating cycle of vague requests and rework.

Finally, remember that documentation – PR descriptions, meeting notes, data contracts – is a critical element in ensuring everyone understands the system’s behavior. Clear, concise language, coupled with specific examples, significantly reduces ambiguity and promotes collaboration. Focusing on outcomes rather than just technical details can also be incredibly helpful.

# Example: Using Dbt to document a transformation's lineage

dbt run --select "my_transformation" --profile  --output-json lineage.json

This command, using the popular data transformation tool dbt, demonstrates how you might automatically generate JSON output containing information about a specific transformation’s lineage – connecting it back to its source tables and any intermediate steps. This kind of automated documentation is invaluable for understanding dependencies and troubleshooting issues. The lineage.json file would then be referenced in PR descriptions or discussions, providing immediate context for the team.