Week 46: Support Vector Machines and Project 3
Contents
Overview of week 46
Eventual mini-workshop on project 3, Friday November 18
Workshop topics 2021 (partly online)
Support Vector Machines, overarching aims
Hyperplanes and all that
What is a hyperplane?
A \( p \)-dimensional space of features
The two-dimensional case
Getting into the details
First attempt at a minimization approach
Solving the equations
Code Example
Problems with the Simpler Approach
A better approach
A quick Reminder on Lagrangian Multipliers
Adding the Multiplier
Setting up the Problem
The problem to solve
The last steps
A soft classifier
Soft optmization problem
Kernels and non-linearity
The equations
The problem to solve
Different kernels and Mercer's theorem
The moons example
Mathematical optimization of convex functions
How do we solve these problems?
A simple example
Back to the more realistic cases
Overview of week 46
Thursday
: Summary of Gradient Boosting. Start support Vector Machines
Friday
: Eventual Workshop on project 3. Discussion of project 3 and support vector machines
Reading recommendations:
See lecture notes for week 46 at
https://compphysics.github.io/MachineLearning/doc/web/course.html.
Hastie et al chapter 12
Bishop chapter 7.1 and 7.2
Overview video on Support Vector Machines
See also
this video
.
Excellent videos on Gradient Boosting
«
1
2
3
4
5
6
7
8
9
10
11
...
31
»