updating week 36

doc/src/week36/week36.do.txt

@@ -18,9 +18,9 @@ DATE: today
 ===== Summary from last Week and Examples =====
 
 !split
-===== Linking with the SVD =====
+===== Linear Regression and  the SVD =====
 
-We saw earlier that 
+We used the SVD to analyse the matrix to invert in ordinary lineat regression
 !bt
 \[
 \bm{X}^T\bm{X}=\bm{V}\bm{\Sigma}^T\bm{U}^T\bm{U}\bm{\Sigma}\bm{V}^T=\bm{V}\bm{\Sigma}^T\bm{\Sigma}\bm{V}^T. 
@@ -163,6 +163,9 @@ where we have used the definition of  a norm-2 vector, that is
 \]
 !et
 
+!split
+===== From OLS to Ridge and Lasso =====
+
 By minimizing the above equation with respect to the parameters
 $\bm{\beta}$ we could then obtain an analytical expression for the
 parameters $\bm{\beta}$.  We can add a regularization parameter $\lambda$ by
@@ -325,7 +328,7 @@ Similarly, "Mehta et al's article":"https://arxiv.org/abs/1803.08823" is also re
 !split
 ===== Deriving the  Lasso Regression Equations =====
 
-Using the matrix-vector expression for Lasso regression and dropping the parameter $1/n$ in front of the standard means squared error equation, we have the following _cost_ function
+Using the matrix-vector expression for Lasso regression and dropping the parameter $1/n$ in front of the standard mean squared error equation, we have the following _cost_ function
 
 !bt
 \[
@@ -352,7 +355,7 @@ and reordering we have
 \bm{X}^T\bm{X}\bm{\beta})+\lambda sgn(\bm{\beta})=2\bm{X}^T(\bm{y}.
 \]
 !et
-This equation does not lead to a nice analytical equation as in either Ridge regression or ordinary least squares. This equation can however be solved by using standard convex optimization algorithms using for example the Python package "CVXOPT":"https://cvxopt.org/". We will discuss this later. 
+This equation does not lead to a nice analytical equation as in Ridge regression or ordinary least squares. This equation can however be solved by using standard convex optimization algorithms using for example the Python package "CVXOPT":"https://cvxopt.org/". We will discuss this later. 
 
 
 
@@ -393,6 +396,16 @@ and minimizing we have that
 !et
 
 
+!split
+===== Lasso Rgeression =====
+
+For Ridge regression our cost function is
+!bt
+\[
+C(\bm{\beta})=\sum_{i=0}^{p-1}(y_i-\beta_i)^2+\lambda\sum_{i=0}^{p-1}\beta_i^2,,
+\]
+!et
+and minimizing we have that
 
 
 !split