From ee42fa66f343e7e4010b96d6adbed7f9f00fb89e Mon Sep 17 00:00:00 2001
From: Morten Hjorth-Jensen <morten.hjorth-jensen@fys.uio.no>
Date: Wed, 29 Aug 2018 08:24:34 +0200
Subject: [PATCH] started writing about ridge and lasso in slides

---
 doc/src/Regression/Regression.do.txt | 41 ++++++++++++++++------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/doc/src/Regression/Regression.do.txt b/doc/src/Regression/Regression.do.txt
index 1cac2287d..83c4f6bc9 100644
--- a/doc/src/Regression/Regression.do.txt
+++ b/doc/src/Regression/Regression.do.txt
@@ -933,6 +933,29 @@ plt.title(r'Random numbers ')
 plt.show()
 
 !ec
+
+
+!split
+===== The singular value decompostion  =====
+!bblock
+How can we use the singular value decomposition to find the parameters $\beta_j$? More details will come. We first note that a general $m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal matrix $\hat{\Sigma}$ of dimensionality $n\times n$ and two orthognal matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality $m \times n$ and the last dimensionality $n\times n$. We have then
+!bt
+\[
+\hat{A} = \hat{U}\hat{\Sigma}\hat{V}
+\]
+!et 
+!eblock
+
+
+Add codes and discuss this in connection with lasso and ridge, show example where the standard inversion of a matrix fails and where SVD comes to rescue
+
+
+!split
+===== Lasso and Ridge regression =====
+
+Discuss the mathematics here
+
+
 !split
 ===== Ridge and Lasso Regression =====
 
@@ -1013,26 +1036,8 @@ plt.show()
 
 
 
-!split
-===== The singular value decompostion  =====
-!bblock
-How can we use the singular value decomposition to find the parameters $\beta_j$? More details will come. We first note that a general $m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal matrix $\hat{\Sigma}$ of dimensionality $n\times n$ and two orthognal matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality $m \times n$ and the last dimensionality $n\times n$. We have then
-!bt
-\[
-\hat{A} = \hat{U}\hat{\Sigma}\hat{V}
-\]
-!et 
-!eblock
 
 
-Add codes and discuss this in connection with lasso and ridge, show example where the standard inversion of a matrix fails and where SVD comes to rescue
-
-
-!split
-===== Lasso and Ridge regression =====
-
-Discuss the mathematics here
-
 !split
 =====  Logistic regression =====
 Add discussion about classification versus regression, show examples of more than two cases and why regression is not the best approach. Motivate for k-nearest neighbors