Supervised Learning

Supervised learning is the most widely used paradigm in machine learning. The model receives a dataset of examples where each input is paired with a known correct output (the label). During training, the model adjusts its parameters to minimize the difference between its predictions and the actual labels. Once trained, it generalizes to new inputs it has never seen before.

Supervised learning divides into two main task types:

Classification assigns inputs to discrete categories. Given an email, predict whether it is spam or not. Given a medical image, predict whether a tumor is malignant or benign. The output is a category label.

Regression predicts continuous numerical values. Given a house's features, predict its price. Given historical sales data, predict next quarter's revenue. The output is a number.

The quality of a supervised learning model depends heavily on the quality and quantity of labeled data. Labeling is often the most expensive and time-consuming part of the pipeline. A model trained on biased or mislabeled data will learn those biases. This labeling bottleneck is one of the main reasons alternative approaches like self-supervised learning and unsupervised learning have gained traction.

Common supervised learning algorithms include linear regression, logistic regression, decision trees, random forests, gradient boosting (XGBoost, LightGBM), support vector machines, and neural networks. Modern large language models are pre-trained with self-supervised learning but are then fine-tuned with supervised learning using human-labeled preference data (RLHF).