hw 2 update

doc/src/Projects/2019/Exercises/hw2.do.txt

@@ -65,3 +65,50 @@ $\hat{z}_i=\hat{X}\hat{v}_i=\hat{u}_1d_1$   is equal to (equation (3.49) of Hast
 where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie *et al*, the matrix elements of $X$ are centered. The consequence is that the mean values of for example $\hat{u}_i$ are zero. 
 
 Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.   
+
+
+===== Solution to the last exercise =====
+
+A possible way to show why $\left \langle \hat u_i \right \rangle = 0$
+given that the columns of $\hat X$ is centered is by considering
+$\left \langle \hat X \hat v_i \right \rangle$:
+
+!bt 
+
+\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
+\end{align*}
+!et
+
+where $x_{jk}$ being the element of $\hat X$ at row $j$ and column
+$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X
+\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and
+$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$.
+
+
+
+Since the columns of $\hat X$ are assumed to be centered, $\left
+\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that
+$\left \langle \hat X \hat v_i \right \rangle = 0$.
+
+
+
+But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
+u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $.
+
+Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i
+\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i
+\neq 0$ (otherwise the variance in the exercise would just be zero),
+gives that $\left \langle \hat u_i \right \rangle = 0$.
+
+
+
+Regarding $\hat V$ and using the similar approach as above by
+computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left
+\langle \hat v_i \right \rangle$, we have
+
+!bt
+\[
+\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\  &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\  &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ 
+\end{align*}
+!et
+