updating weel 36

doc/src/week36/week36.do.txt

@@ -429,6 +429,43 @@ We will now couple the discussions of ordinary least squares, Ridge and Lasso re
 We will amongst other things show that the regularization parameter can reduce considerably the variance of the parameters $\beta$.
 
 
+!split
+===== Yet another Example =====
+
+Let us assume we have a data set with outputs/targets given by the vector
+
+!bt
+\[
+\bm{y}=\begin{matrix}4 \\ 2 \\3\end{bmatrix},
+\]
+!et
+and our inputs as a $3\times 2$ design matrix
+!bt
+\[
+\bm{X}=\begin{matrix}2 & 0\\ 0 & 1 \\ 1 & 0\end{bmatrix},
+\]
+!et
+meaning that we have two features and two unknown parameters $\beta_0$ and $\beta_1$ to be determined either by ordinary least squares, Ridge or Lasso regression.
+
+!split
+===== The OLS case =====
+
+For ordinary least squares (OLS) we know that the optimal solution is
+
+!bt
+\[
+\hat{\bm{\beta}}=\left( \bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
+\]
+!et
+Inserting the above values we obtain that 
+
+!bt
+\[
+\hat{\bm{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
+\]
+!et
+
+
 
 !split 
 ===== Linking the regression analysis with a statistical interpretation =====