Support Vector Machines (SVM)

Support vector machines find the optimal boundary (hyperplane) that separates different classes with the maximum possible margin. The "support vectors" are the data points closest to this boundary, which are the only points that matter for defining the decision boundary. For non-linearly separable data, SVMs use kernel functions to transform data into higher-dimensional spaces where linear separation becomes possible.

Common kernels include polynomial, radial basis function (RBF), and sigmoid kernels. SVMs work well with high-dimensional data and are effective even when the number of features exceeds the number of samples. They're particularly popular in text classification, image recognition, and bioinformatics. However, they can be slow on large datasets and require careful parameter tuning. The kernel trick allows SVMs to handle complex, non-linear relationships without explicitly computing high-dimensional transformations.