Technical Vocabulary for Linux Kernel Engineers

Linux kernel engineering sits at the lowest level of the software stack, and its vocabulary reflects that precision. Whether you are contributing to the kernel upstream, writing kernel modules, or tuning system performance at the kernel level, using this vocabulary correctly signals expertise and enables precise technical communication. This guide covers the core vocabulary across scheduling, containerisation primitives, observability, memory management, and performance analysis.

Scheduling and Process Management

Scheduler — the kernel component responsible for deciding which process or thread runs on which CPU core at any given time. The Linux kernel uses the Completely Fair Scheduler (CFS) as its default scheduler. “The task is experiencing high scheduling latency because the scheduler is deprioritising it in favour of higher-priority processes competing on the same core.”

CFS (Completely Fair Scheduler) — the default process scheduler in the Linux kernel. It maintains a virtual runtime for each task and always picks the task with the smallest virtual runtime to run next, aiming to give each process a fair share of CPU time. “CFS provides fairness across competing processes but can introduce jitter in latency-sensitive workloads; consider using a real-time scheduler class for those tasks.”

Context switch — the kernel operation of saving the state of the currently running process and restoring the state of another process so that a different process can run. Frequent context switches impose CPU overhead. “We observed over 100,000 context switches per second on this host; that level of switching is adding measurable CPU overhead.”

CPU affinity — the binding of a process or thread to a specific CPU core or set of cores, preventing the scheduler from migrating it. “We set CPU affinity for the real-time audio processing thread to isolate it from interference by other workloads.”

NUMA (Non-Uniform Memory Access) — a computer memory design where the access time depends on the memory location relative to the CPU. NUMA-aware scheduling places processes on CPUs whose memory bank holds the relevant data. “The process is experiencing NUMA latency because it is running on a core in Node 1 but its memory is allocated in Node 0.”

Containerisation Primitives

cgroup (Control Group) — a Linux kernel mechanism that limits, accounts for, and isolates the resource usage (CPU, memory, I/O, network) of a collection of processes. cgroups are the foundation of container resource management in Docker and Kubernetes. “The container is being throttled because it has exceeded its cgroup CPU quota; we need to increase the CPU limit or optimise the workload.”

cgroup v2 — the current generation of cgroups, introduced in kernel 4.5, with a unified hierarchy and improved resource distribution semantics compared to cgroup v1. “We migrated to cgroup v2 to take advantage of the new PSI (Pressure Stall Information) metrics for memory pressure monitoring.”

Namespace — a Linux kernel feature that isolates process views of system resources. Types include PID namespaces (process IDs), network namespaces, mount namespaces, UTS namespaces (hostname), IPC namespaces, and user namespaces. “Each container runs in its own PID namespace, which is why processes inside the container see PID 1 as their init process regardless of the host’s PID space.”

seccomp (Secure Computing Mode) — a Linux kernel security facility that filters the system calls a process is allowed to make. Used by container runtimes to limit the attack surface of containerised workloads. “The container runtime applies a default seccomp profile that blocks over 300 system calls; if your application needs one of them, you must explicitly allow it.”

System Calls and Kernel Interfaces

Syscall (System Call) — the mechanism by which a user-space process requests a service from the kernel, such as reading a file, allocating memory, or creating a socket. Each syscall involves a context switch from user mode to kernel mode. “The profiling data shows the process is spending 40% of its time in read() syscalls; the application is making many small reads when it should be buffering.”

VDSO (Virtual Dynamic Shared Object) — a kernel mechanism that maps a piece of kernel space into user space, allowing certain syscalls (such as gettimeofday) to be executed entirely in user space without the overhead of a kernel mode transition. “After switching from gettimeofday() to clock_gettime() with the monotonic clock, the timestamps are served from the VDSO and add almost no overhead.”

kprobe — a Linux dynamic tracing mechanism that allows you to insert handlers at nearly any instruction in the kernel (or a kernel module) at runtime, without recompilation. “We used a kprobe on the tcp_retransmit_skb function to capture network retransmission events without instrumenting the application itself.”

eBPF (Extended Berkeley Packet Filter) — a kernel technology that allows sandboxed programmes to run in the kernel in response to events (system calls, network events, tracepoints) without modifying kernel source code or loading kernel modules. eBPF is the foundation of modern observability tools (Cilium, Falco, bpftrace). “We instrument all syscall latency with an eBPF programme that attaches to tracepoints; it adds under 1% overhead compared to strace’s prohibitive cost.”

Memory Management

Hugepages — a memory management feature that uses larger page sizes (2 MB or 1 GB) rather than the default 4 KB pages, reducing the overhead of the Translation Lookaside Buffer (TLB). Important for latency-sensitive applications that manage large memory spaces. “Enabling transparent hugepages improved throughput by 15% for the in-memory database by reducing TLB misses.”

TLB (Translation Lookaside Buffer) — a cache in the CPU that stores recent virtual-to-physical address translations. TLB misses require a page table walk, which is expensive. “The application’s access pattern is causing frequent TLB misses; migrating the hot data structures to hugepages should improve this.”

OOM killer (Out-of-Memory killer) — the kernel mechanism that terminates processes when the system runs out of memory. Processes can be assigned an OOM score to influence which process is killed first. “The container was OOM-killed last night because it hit its memory limit; we need to increase the limit or investigate the memory leak.”

Page fault — an interrupt raised by the hardware when a process accesses a memory page that is not in RAM, requiring the kernel to load it from swap or from disk. “The spike in page faults on startup indicates the application is loading large files on demand rather than pre-loading them.”

Performance Vocabulary

Jitter — variability in latency. A system with low average latency but high jitter is unpredictable, which is often worse than a system with consistently higher but predictable latency. “The real-time audio processing thread is experiencing jitter caused by scheduler preemption; we need to investigate kernel preemption settings.”

Latency — the time taken for an operation to complete. In kernel contexts, measured in microseconds or nanoseconds. “The p99 kernel scheduling latency is 250 µs; for our low-latency trading application, we need to get that below 50 µs.”

Throughput — the volume of operations or data processed per unit of time. “After tuning the I/O scheduler from CFQ to deadline, disk throughput improved from 800 MB/s to 1.1 GB/s.”

Cache pressure — a situation where the workload demands more cache capacity than is available, causing frequent evictions. “High cache pressure from the bulk data ingestion job is evicting hot pages used by the query engine; we need to isolate the two workloads on separate NUMA nodes.”

Example Sentences in Context

1. “The latency spike at the 99th percentile is caused by scheduler jitter; the offending process is not pinned to a specific core, so the scheduler occasionally migrates it, causing cache cold-start overhead.”

2. “We use eBPF to attach a tracepoint on sys_enter_openat and log every file open event with the calling PID and latency; this gives us production observability with negligible overhead.”

3. “The cgroup memory limit was set to 512 MB based on average usage, but the OOM killer fired under peak load — we need to base the limit on the p99.9 memory consumption, not the mean.”

4. “After enabling hugepages for the Redis instance, TLB miss rate dropped by 60% and p99 GET latency improved from 1.2 ms to 0.4 ms — the impact was larger than expected.”

5. “The namespace isolation means that a process inside the container cannot see or send signals to processes in other containers or on the host, even if they share the same underlying kernel.”