updating week 36

doc/src/week36/week36.do.txt

@@ -454,17 +454,43 @@ For ordinary least squares (OLS) we know that the optimal solution is
 
 !bt
 \[
-\hat{\bm{\beta}}=\left( \bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
+\hat{\bm{\beta}}^{\mathrm{OLS}}=\left( \bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 Inserting the above values we obtain that 
 
 !bt
 \[
-\hat{\bm{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
+\hat{\bm{\beta}}^{\mathrm{OLS}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
 \]
 !et
 
+Computing the mean squared error we obtian a value of $0.27$. 
+
+The code which implements this simpler case is presented after the discussion of Ridge and Lasso.
+
+!split
+===== The Ridge case =====
+
+For Ridge regression we have
+
+!bt
+\[
+\hat{\bm{\beta}}^{\mathrm{Ridge}}=\left( \bm{X}^T\bm{X}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}.
+\]
+!et
+Inserting the above values we obtain that 
+
+!bt
+\[
+\hat{\bm{\beta}}^{\mathrm{Ridge}}=\begin{matrix}\frac{11}{5+\lambda} \\ \frac{2}{1+\lambda}\end{bmatrix},
+\]
+!et
+
+There is normally a constraint on the value of $\vert\vert \bm{\beta}\vert\vert_2 via the parameter $\lambda$.
+Let us for simplicity assume now that $\beta_0^2+\beta_1^2=1$ as constraint. This will allow us to find an expression with the optimal values of $\beta$ and $\lambda$.
+
+To see this, let us write the cost function for Ridge regression.  
 
 
 !split