started writing about ridge and lasso in slides

doc/src/Regression/Regression.do.txt

@@ -933,6 +933,29 @@ plt.title(r'Random numbers ')
 plt.show()
 
 !ec
+
+
+!split
+===== The singular value decompostion  =====
+!bblock
+How can we use the singular value decomposition to find the parameters $\beta_j$? More details will come. We first note that a general $m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal matrix $\hat{\Sigma}$ of dimensionality $n\times n$ and two orthognal matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality $m \times n$ and the last dimensionality $n\times n$. We have then
+!bt
+\[
+\hat{A} = \hat{U}\hat{\Sigma}\hat{V}
+\]
+!et 
+!eblock
+
+
+Add codes and discuss this in connection with lasso and ridge, show example where the standard inversion of a matrix fails and where SVD comes to rescue
+
+
+!split
+===== Lasso and Ridge regression =====
+
+Discuss the mathematics here
+
+
 !split
 ===== Ridge and Lasso Regression =====
 
@@ -1013,26 +1036,8 @@ plt.show()
 
 
 
-!split
-===== The singular value decompostion  =====
-!bblock
-How can we use the singular value decomposition to find the parameters $\beta_j$? More details will come. We first note that a general $m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal matrix $\hat{\Sigma}$ of dimensionality $n\times n$ and two orthognal matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality $m \times n$ and the last dimensionality $n\times n$. We have then
-!bt
-\[
-\hat{A} = \hat{U}\hat{\Sigma}\hat{V}
-\]
-!et 
-!eblock
 
 
-Add codes and discuss this in connection with lasso and ridge, show example where the standard inversion of a matrix fails and where SVD comes to rescue
-
-
-!split
-===== Lasso and Ridge regression =====
-
-Discuss the mathematics here
-
 !split
 =====  Logistic regression =====
 Add discussion about classification versus regression, show examples of more than two cases and why regression is not the best approach. Motivate for k-nearest neighbors