Week 46: Support Vector Machines and Project 3. Start Principal Component Analysis
Contents
Overview of week 46
Support Vector Machines, overarching aims
Hyperplanes and all that
What is a hyperplane?
A \( p \)-dimensional space of features
The two-dimensional case
Getting into the details
First attempt at a minimization approach
Solving the equations
Code Example
Problems with the Simpler Approach
A better approach
A quick Reminder on Lagrangian Multipliers
Adding the Multiplier
Setting up the Problem
The problem to solve
The last steps
A soft classifier
Soft optmization problem
Kernels and non-linearity
The equations
The problem to solve
Different kernels and Mercer's theorem
The moons example
Mathematical optimization of convex functions
How do we solve these problems?
A simple example
Back to the more realistic cases
Basic ideas of the Principal Component Analysis (PCA)
Introducing the Covariance and Correlation functions
More on the covariance
Reminding ourselves about Linear Regression
Simple Example
The Correlation Matrix
Numpy Functionality
Correlation Matrix again
Using Pandas
And then the Franke Function
Links with the Design Matrix
Computing the Expectation Values
Towards the PCA theorem
More on the PCA Theorem
Writing our own PCA code
Implementing it
First Step
Scaling
Centered Data
Exploring
Diagonalize the sample covariance matrix to obtain the principal components
Collecting all Steps
Classical PCA Theorem
The PCA Theorem
Geometric Interpretation and link with Singular Value Decomposition
PCA and scikit-learn
Back to the Cancer Data
Incremental PCA
Randomized PCA
Kernel PCA
Other techniques
Overview of week 46
Thursday
: Support Vector Machines, discussion of project 3
Video of lecture
Friday
: Support vector machines and start Principal Component Analysis
Reading recommendations:
See lecture notes for week 46 at
https://compphysics.github.io/MachineLearning/doc/web/course.html.
Hastie et al chapter 12
Bishop chapter 7.1 and 7.2
Overview video on Support Vector Machines
See also
this video
.
«
1
2
3
4
5
6
7
8
9
10
11
...
58
»