5.10 Support vector machines and kernels

Work through these

• Maximum margin and support vectors

  Rather than any separating surface, choose the one furthest from the nearest points of each class, and only those nearest points matter. That is the whole geometric idea.

• Soft margin and the C parameter

  Real data overlaps, so the method allows some violations at a price, and one setting controls how much. Tuning it is the main practical decision.

• Kernels: linear, polynomial, RBF

  Replacing dot products with a different similarity lets a linear method draw curved boundaries without ever computing coordinates in the larger space. This is one of the most elegant ideas in the subject.

• SVR for regression

  The same machinery adapted to predicting a number rather than a class, using a tolerance band instead of a margin. It is less commonly used and worth recognising.