Week 36: Statistical interpretation of Linear Regression and Resampling techniques
Contents
Plans for week 36
Thursday September 9
Summary from last Week and Examples
Linear Regression and the SVD
What does it mean?
And finally \( \boldsymbol{X}\boldsymbol{X}^T \)
Code for SVD and Inversion of Matrices
Inverse of Rectangular Matrix
Ridge and LASSO Regression
From OLS to Ridge and Lasso
Deriving the Ridge Regression Equations
Note on Scikit-Learn
Comparison with OLS
SVD analysis
Interpreting the Ridge results
More interpretations
Deriving the Lasso Regression Equations
Simple example to illustrate Ordinary Least Squares, Ridge and Lasso Regression
Ridge Regression
Lasso Regression
Yet another Example
The OLS case
The Ridge case
Writing the Cost Function
Lasso case
The first Case
Simple code for solving the above problem
With Lasso Regression
Another Example, now with a polynomial fit
Using CVXOPT
The simpler Example
Friday September 10
Linking the regression analysis with a statistical interpretation
Assumptions made
Expectation value and variance
Expectation value and variance for \( \boldsymbol{\beta} \)
Deriving OLS from a probability distribution
Independent and Identically Distrubuted (iid)
Maximum Likelihood Estimation (MLE)
A new Cost Function
More basic Statistics and Bayes' theorem
Marginal Probability
Conditional Probability
Bayes' Theorem
Interpretations of Bayes' Theorem
Example of Usage of Bayes' theorem
Doing it correctly
Bayes' Theorem and Ridge and Lasso Regression
Test Function for what happens with OLS, Ridge and Lasso
Invoking Bayes' theorem
Ridge and Bayes
Lasso and Bayes
Friday September 10
«
1
...
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
...
53
»