Decision Tree
FundamentalsA supervised learning algorithm that splits data into branches based on feature values, forming a tree-like structure of if-then decisions that leads to a prediction at each leaf.
Like a game of 20 questions — each question narrows down the possibilities until you arrive at the answer.
A decision tree is a flowchart-like model that makes predictions by learning a series of binary splits on input features. Starting at the root, each internal node tests a condition — such as 'is age > 30?' or 'is income < $50k?' — and branches left or right based on the answer. This process repeats until the data reaches a leaf node, which contains the final prediction.
The tree is built top-down by selecting the split at each node that best separates the data according to a criterion like Gini impurity (for classification) or mean squared error (for regression). The most informative feature is chosen first, creating a natural ranking of feature importance. This makes decision trees one of the most interpretable models in machine learning — you can trace any prediction through the tree and explain exactly why it was made.
The main weakness of a single decision tree is that it overfits easily. A tree grown without constraints will memorize the training data by creating increasingly specific splits until every training example is classified perfectly. Pruning techniques and depth limits help, but the real breakthrough came from ensemble methods: random forests average many decorrelated trees, and boosting methods like AdaBoost, XGBoost, and LightGBM build trees sequentially to correct each other's mistakes. Decision trees are rarely used alone in production, but they are the building block of some of the most powerful algorithms in machine learning.
Last updated: March 9, 2026