From 7dc0d0b9829574122db7237a4e2d5f200212fdb5 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Tue, 15 Sep 2020 06:39:02 +0200 Subject: [PATCH] updating regression slides --- doc/src/Regression/Regression.do.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/src/Regression/Regression.do.txt b/doc/src/Regression/Regression.do.txt index 49576bf37..4e92ea3d4 100644 --- a/doc/src/Regression/Regression.do.txt +++ b/doc/src/Regression/Regression.do.txt @@ -1606,6 +1606,15 @@ Going back to our OLS equation we have We will come back to this expression when we discuss Ridge regression. +$$ \tilde{y}^{OLS}={\bf X}\hat{\beta}^{OLS}=\sum_{j=1}^p {\bf u}_j{\bf u}_j^T{\bf y}$$ and for Ridge we have  + +$$ \tilde{y}^{Ridge}={\bf X}\hat{\beta}^{Ridge}=\sum_{j=1}^p {\bf u}_j\frac{\sigma_j^2}{\sigma_j^2+\lambda}{\bf u}_j^T{\bf y}$$ .  + +It is indeed the economy-sized SVD, note the summation runs up tp $$p$$ only and not $$n$$.  + +Here we have that $${\bf X} = {\bf U}{\bf \Sigma}{\bf V}^T$$, with $$\Sigma$$ being an $$ n\times p$$ matrix and $${\bf V}$$ being a $$ p\times p$$ matrix. We also have assumed here that $$ n > p$$.  + + !split ===== Ridge and LASSO Regression =====