XGBoost (Boosting)
FoundationsA high-performance gradient boosting library that builds accurate tabular models by adding regularized decision trees to correct an ensemble’s remaining errors.
It is an ensemble that edits its own draft: each tree reads the previous model’s mistakes and writes a targeted correction.
XGBoost, short for eXtreme Gradient Boosting, is an implementation of gradient-boosted decision trees released by Tianqi Chen in 2016. It builds an additive model one tree at a time. Each new tree concentrates on the errors, or more precisely the negative gradient of the loss, left by the current ensemble.
Its advantage comes from a dense stack of practical improvements: L1 and L2 regularization, second-order optimization, sparsity-aware handling of missing values, quantile-based split finding, and parallel tree construction. Together they make the method fast, accurate, and resistant to overfitting on structured data.
XGBoost remains a strong first model for fraud detection, credit risk, ranking, pricing, and other tabular problems. It often beats more elaborate neural systems when the data is mostly columns and rows. The tradeoff is tuning effort: tree depth, learning rate, subsampling, regularization, and the number of boosting rounds all shape the final model.
Related Terms
Last updated: March 9, 2026