Compiler Engineering Vocabulary: From Lexer to Code Generation

English in the Compiler Pipeline

Compiler engineering has a rich and precise vocabulary that has accumulated over decades of research and implementation. For engineers working on compilers — whether writing passes in LLVM, contributing to GCC, or building a language front-end — communicating in this vocabulary with fluency and precision is essential for code reviews, design discussions, and documentation.

This guide walks through the compiler pipeline from front-end to back-end, covering the English terms used at each stage.

---

Front-End Vocabulary: Lexing and Parsing

The front-end of a compiler transforms source text into a structured representation.

Term	Definition
Lexer (tokeniser)	The component that reads source characters and produces a stream of tokens
Token	A categorised unit of source text, such as an identifier, literal, or keyword
Lexeme	The actual character sequence that matched a token pattern
Parser	The component that reads a token stream and constructs a parse tree or AST
Grammar	A formal description of the syntactic rules of a language
Production rule	A single rule in a grammar defining how a non-terminal can be expanded
Lookahead	The number of tokens the parser examines ahead of the current position to make a decision
Ambiguity	A property of a grammar where a single input can be parsed in multiple ways
Abstract Syntax Tree (AST)	A tree representation of the syntactic structure of source code, with implementation detail removed

When reviewing a parser implementation, a common observation is: “This production rule is ambiguous — we need to document the precedence convention or restructure the grammar.”

---

Middle-End Vocabulary: IR and Optimisation

The middle-end operates on an Intermediate Representation (IR), which is the compiler’s internal language for analysis and optimisation.

Term	Definition
Intermediate Representation (IR)	A language-independent, architecture-independent representation of program semantics
Static Single Assignment (SSA) form	An IR property where every variable is assigned exactly once and use-def chains are explicit
Phi node (φ-node)	An SSA construct that selects a value based on which predecessor basic block was executed
Basic block	A straight-line sequence of instructions with a single entry and single exit
Control flow graph (CFG)	A graph where nodes are basic blocks and edges represent possible transfers of control
Data flow analysis	A technique for computing properties of values at each program point
Dominator	A node D dominates node N if every path to N passes through D
Alias analysis	Analysis that determines whether two pointers could refer to the same memory location
Inlining	Replacing a function call with a copy of the called function’s body
Loop invariant code motion (LICM)	Moving computations out of a loop when their result does not change across iterations

---

LLVM-Specific Vocabulary

LLVM has its own set of terms and conventions used in discussions, documentation, and code review.

Term	Definition
Pass	A transformation or analysis that operates on the IR; LLVM programs are optimised by a sequence of passes
Analysis pass	A pass that computes information about the IR without modifying it
Transform pass	A pass that modifies the IR to improve it in some way
Pass manager	The infrastructure that schedules and runs passes efficiently
Canonicalise	To transform an IR construct into a standard, normalised form that is easier to reason about
Emit	To produce output — either a new IR form or target machine code
Legalization	The process of transforming IR operations into forms the target machine supports
Instruction selection	The mapping of IR instructions to target machine instructions
Register allocation	The assignment of IR virtual registers to physical machine registers
Spill	To move a value from a register to memory because there are insufficient registers available

In LLVM code reviews, “canonicalise” is a frequently used instruction: “Before optimising this pattern, canonicalise it to the form the middle-end expects.”

---

Back-End Vocabulary: Register Allocation and Code Generation

Term	Definition
Virtual register	An unlimited, abstract register used in IR before register allocation
Calling convention	The protocol governing how arguments, return values, and registers are managed across function calls
Prologue / Epilogue	The function entry/exit code that sets up and tears down the stack frame
Peephole optimisation	A local optimisation that examines a small window of instructions and replaces them with more efficient equivalents
Instruction scheduling	Reordering instructions to improve performance, typically to avoid pipeline stalls
Relocation	A reference in object code that must be resolved to an address at link time
Object file	The output of the assembler, containing machine code and metadata for the linker

---

Example Sentences

1. “The current lexer does not handle Unicode identifiers correctly — we need to update the token classification rules before the next release.”
2. “After SSA construction, all phi nodes in the entry block are trivial and can be eliminated — the mem2reg pass should handle this automatically.”
3. “This pass canonicalises all GEP instructions to a normalised form, which makes subsequent alias analysis more precise.”
4. “The register allocator is spilling aggressively in this hot loop; we should examine the register pressure and consider splitting the loop into two phases.”
5. “The LLVM backend emits a function prologue that is unnecessarily large for leaf functions — we can apply the leaf function optimisation to eliminate the frame pointer setup.”

---

Communication Tips for Compiler Engineers

When discussing a compiler bug, distinguish between a miscompilation (the compiler generates incorrect code) and a compiler crash (the compiler fails to produce output). These have different debugging approaches and different levels of severity.

When writing pass documentation or code comments, prefer the active voice and imperative mood: “This pass transforms… The analysis computes… Call this pass after…” — not “This pass is used to transform…”

The verb “lower” has a specific meaning in compiler vocabulary: to replace a high-level IR construct with a more concrete one closer to the target machine. “We lower intrinsics to platform-specific builtins in the legalization phase.” Avoid using it in its everyday English sense when writing compiler documentation.

Navigating the Nuances: Practical Application & Common Phrases

Let’s be honest – “lexical analysis” doesn’t roll off the tongue. As a compiler engineer, you’re not just understanding these terms; you need to communicate them effectively, both in documentation and during discussions with colleagues. Often, the biggest hurdle isn’t the technical detail but the phrasing itself. It’s about conveying precision without sounding overly academic or jargon-laden.

A common phrase you’ll hear – particularly when reviewing another engineer’s work – is “the generated code needs more aggressive optimisation.” It’s not simply “bad code.” It’s a specific request tied to the outcome – performance. Similarly, saying “the IR could benefit from loop unrolling” isn’t criticizing the design; it’s suggesting an improvement that aligns with the intermediate representation’s purpose. Focusing on the impact of decisions, rather than just the process itself, is crucial for clear communication. Think about framing changes in terms of measurable results – “reducing memory footprint by X%” or “improving instruction-level parallelism.”

Another area where nuance matters is when discussing errors. Instead of saying “the lexer failed,” a more helpful and professional phrasing would be: “The lexer encountered an unexpected character sequence, leading to a parsing error.” This detail provides context without immediately sounding like a technical problem statement. Similarly, during code reviews, instead of simply stating “this needs fixing,” consider something like, “This section’s performance could be improved by refactoring the data structure.” It’s about offering a constructive suggestion with clear reasoning.

Finally, don’t shy away from using active voice – it makes your explanations clearer and more direct. Instead of “The code was generated…”, say “We generated the code…” This subtly shifts responsibility and emphasizes action.

Here’s an example demonstrating how to use clang-format to enforce a consistent style:

clang-format -i my_source.c

This command will automatically reformat the contents of my_source.c, ensuring that it adheres to the specified formatting rules – a practical application of understanding code generation and its associated tooling. It’s about taking control, not just passively accepting a result.