Machine Learning Language Exercises

Model training is described as "fitting a model to data" — the process of adjusting model parameters (weights) to minimise a loss function over the training dataset. Engineers use phrases like "training loop," "forward pass," "backward pass," and "gradient descent." They say a model "converges" when the loss stabilises, and describe training duration in "epochs" (complete passes through the training data) or "steps."

Overfitting means a model "memorises the training data" but "fails to generalise" to unseen examples — characterised by low training loss and high validation loss. Underfitting means the model is "too simple to capture the underlying pattern," showing high loss on both training and validation sets. Engineers address overfitting with "regularisation," "dropout," or "early stopping," and underfitting by "increasing model capacity" or "adding features."

Hyperparameter tuning is described as "searching the hyperparameter space" to find the configuration that maximises model performance on the validation set. Engineers use terms like "grid search," "random search," "Bayesian optimisation," and "automated ML (AutoML)." Common hyperparameters discussed include "learning rate," "batch size," "number of layers," and "regularisation strength." The process is often described as "expensive" because each trial requires a full training run.

Feature engineering is the process of creating, selecting, or transforming input variables to improve model performance. Engineers describe it as "domain knowledge encoded into the data" — for example, "extracting day-of-week from a timestamp" or "normalising skewed distributions." Phrases include "feature importance," "feature selection," "dimensionality reduction," and "feature store" (a centralised repository for reusable features).

Experiment tracking is the practice of recording hyperparameters, metrics, code versions, and artefacts for each training run to enable reproducibility and comparison. Engineers use tools like MLflow, Weights and Biases, or Neptune, and say they "log runs," "compare experiments," and "register the best model." Key vocabulary includes "run," "experiment," "artifact," "metric curve," and "model registry."

The dataset is "split" into three subsets: training (used to fit the model), validation (used to tune hyperparameters and detect overfitting during development), and test (held out and used only once to report final performance). Engineers warn against "data leakage" — the test set influencing model design — and use "cross-validation" as an alternative when data is scarce, averaging performance over multiple folds.

The bias-variance trade-off describes the tension between a model being too simple (high bias, underfits) and too complex (high variance, overfits). Engineers say "increasing model complexity reduces bias but increases variance." The goal is to find the "sweet spot" that minimises total error on unseen data. This trade-off is used to explain why ensemble methods like "random forests" or "gradient boosting" often outperform a single model.

Evaluation vocabulary depends on the task: classification uses "accuracy," "precision," "recall," "F1-score," "AUC-ROC," and "confusion matrix"; regression uses "mean absolute error (MAE)," "root mean squared error (RMSE)," and "R-squared." Engineers emphasise choosing metrics aligned with business goals, noting that "accuracy is misleading on imbalanced datasets" and that "precision-recall trade-offs" depend on whether false positives or false negatives are costlier.

Model deployment vocabulary includes "model serving," "inference endpoint," "REST API," "batch prediction," and "real-time scoring." Engineers describe "packaging a model" into a container, "registering it" in a model registry, and "deploying it" behind a load balancer. Key concerns include "latency SLAs," "throughput," and "model versioning" — the ability to "roll back" to a previous version if a new model regresses in production.

Data preprocessing is described as "cleaning," "normalising," "encoding," and "imputing" raw data before it enters a model. Engineers say they "build a preprocessing pipeline" using tools like scikit-learn's Pipeline or Apache Beam, ensuring the same transformations are applied at training and inference (avoiding "training-serving skew"). Terms include "one-hot encoding," "label encoding," "min-max scaling," and "missing value imputation."