Decision Trees

Decision trees create a flowchart-like structure that makes predictions by asking a series of yes/no questions about input features. Each internal node represents a test on a feature, branches represent outcomes, and leaf nodes contain final predictions. The algorithm recursively splits the data based on features that best separate different classes or reduce variance in regression problems.

Trees are highly interpretable since you can follow the exact path of decisions leading to any prediction. They handle both numerical and categorical data naturally and can capture non-linear relationships. Popular algorithms include ID3, C4.5, and CART. However, individual trees tend to overfit training data and can be unstable, where small changes in data lead to very different tree structures.