Maximum A Posteriori

Maximum A Posteriori (MAP) estimation is a statistical technique for finding the most probable value of unknown parameters given observed data. It extends Maximum Likelihood Estimation (MLE) by incorporating a prior distribution over parameters -- a mathematical encoding of beliefs about what parameter values are plausible before seeing any data.

Formally, MAP finds the parameter value θ that maximizes P(θ | data) = P(data | θ) × P(θ) / P(data). Since P(data) is constant with respect to θ, this reduces to maximizing the product of the likelihood P(data | θ) and the prior P(θ).

In practice, the prior acts as a regularizer. A Gaussian prior on model weights, for example, penalizes large weight values -- which is mathematically equivalent to L2 regularization (weight decay) in neural network training. This connection makes MAP estimation directly relevant to how modern deep learning models are trained: many regularization techniques can be interpreted as imposing implicit prior distributions on the model's parameters.

MAP differs from full Bayesian inference in that it produces a point estimate (a single best guess) rather than a full posterior distribution over parameters. This makes it computationally tractable for large models where computing the full posterior is intractable. The trade-off is that a point estimate discards uncertainty information -- you know the most probable value but not how confident to be in it.

In the context of LLMs and fine-tuning, MAP intuitions appear in techniques like RLHF, where prior policy distributions constrain how far the model can shift from its pretrained state, and in Bayesian approaches to prompt optimization and uncertainty quantification.