lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

hw 2 update

Browse Source
This commit is contained in:
mhjensen
2019-09-05 05:10:18 +02:00
parent 3a6a64dc4c
commit f56c59cf34
9 changed files with 329 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/Projects/2019/hw2/html/._hw2-bs000.html
+50 -3
View File
@@ -42,7 +42,8 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('Exercise 4', 2, None, '___sec0'),
('Exercise 5', 2, None, '___sec1')]}
('Exercise 5', 2, None, '___sec1'),
('Solution to the last exercise', 2, None, '___sec2')]}
end of tocinfo -->
<body>
@@ -82,6 +83,7 @@ MathJax.Hub.Config({
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Exercise 4</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Exercise 5</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">Solution to the last exercise</a></li>
</ul>
</li>
@@ -115,7 +117,7 @@ MathJax.Hub.Config({
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 2, 2019</h4></center> <!-- date -->
<center><h4>Sep 5, 2019</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
@@ -183,7 +185,52 @@ $$
where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie <em>et al</em>, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
<p>
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
<h2 id="___sec2" class="anchor">Solution to the last exercise </h2>
<p>
A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \)
given that the columns of \( \hat X \) is centered is by considering
\( \left \langle \hat X \hat v_i \right \rangle \):
$$
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
$$
<p>
where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column
\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X
\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and
\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \).
<p>
Since the columns of \( \hat X \) are assumed to be centered, \( \left
\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that
\( \left \langle \hat X \hat v_i \right \rangle = 0 \).
<p>
But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \).
<p>
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i
\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i
\neq 0 \) (otherwise the variance in the exercise would just be zero),
gives that \( \left \langle \hat u_i \right \rangle = 0 \).
<p>
Regarding \( \hat V \) and using the similar approach as above by
computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle \), we have
$$
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
$$
<p>
<!-- navigation buttons at the bottom of the page -->
doc/Projects/2019/hw2/html/hw2-bs.html
+50 -3
View File
@@ -42,7 +42,8 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('Exercise 4', 2, None, '___sec0'),
('Exercise 5', 2, None, '___sec1')]}
('Exercise 5', 2, None, '___sec1'),
('Solution to the last exercise', 2, None, '___sec2')]}
end of tocinfo -->
<body>
@@ -82,6 +83,7 @@ MathJax.Hub.Config({
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Exercise 4</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Exercise 5</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">Solution to the last exercise</a></li>
</ul>
</li>
@@ -115,7 +117,7 @@ MathJax.Hub.Config({
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 2, 2019</h4></center> <!-- date -->
<center><h4>Sep 5, 2019</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
@@ -183,7 +185,52 @@ $$
where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie <em>et al</em>, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
<p>
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
<h2 id="___sec2" class="anchor">Solution to the last exercise </h2>
<p>
A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \)
given that the columns of \( \hat X \) is centered is by considering
\( \left \langle \hat X \hat v_i \right \rangle \):
$$
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
$$
<p>
where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column
\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X
\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and
\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \).
<p>
Since the columns of \( \hat X \) are assumed to be centered, \( \left
\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that
\( \left \langle \hat X \hat v_i \right \rangle = 0 \).
<p>
But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \).
<p>
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i
\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i
\neq 0 \) (otherwise the variance in the exercise would just be zero),
gives that \( \left \langle \hat u_i \right \rangle = 0 \).
<p>
Regarding \( \hat V \) and using the similar approach as above by
computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle \), we have
$$
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
$$
<p>
<!-- navigation buttons at the bottom of the page -->
doc/Projects/2019/hw2/html/hw2.html
+49 -3
View File
@@ -41,7 +41,8 @@ div { text-align: justify; text-justify: inter-word; }
<!-- tocinfo
{'highest level': 2,
'sections': [('Exercise 4', 2, None, '___sec0'),
('Exercise 5', 2, None, '___sec1')]}
('Exercise 5', 2, None, '___sec1'),
('Solution to the last exercise', 2, None, '___sec2')]}
end of tocinfo -->
<body>
@@ -82,7 +83,7 @@ MathJax.Hub.Config({
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 2, 2019</h4></center> <!-- date -->
<center><h4>Sep 5, 2019</h4></center> <!-- date -->
<br>
<h2 id="___sec0">Exercise 4 </h2>
@@ -148,7 +149,52 @@ $$
where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie <em>et al</em>, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
<p>
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
<h2 id="___sec2">Solution to the last exercise </h2>
<p>
A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \)
given that the columns of \( \hat X \) is centered is by considering
\( \left \langle \hat X \hat v_i \right \rangle \):
$$
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
$$
<p>
where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column
\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X
\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and
\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \).
<p>
Since the columns of \( \hat X \) are assumed to be centered, \( \left
\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that
\( \left \langle \hat X \hat v_i \right \rangle = 0 \).
<p>
But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \).
<p>
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i
\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i
\neq 0 \) (otherwise the variance in the exercise would just be zero),
gives that \( \left \langle \hat u_i \right \rangle = 0 \).
<p>
Regarding \( \hat V \) and using the similar approach as above by
computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle \), we have
$$
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
$$
<!-- ------------------- end of main content --------------- -->
doc/Projects/2019/hw2/ipynb/ipynb-hw2-src.tar.gz
BIN
View File
Binary file not shown.
doc/Projects/2019/hw2/pdf/hw2.p.tex
+44 -1
View File
@@ -155,7 +155,7 @@ Homework 2
% --- begin date ---
\begin{center}
Sep 2, 2019
Sep 5, 2019
\end{center}
% --- end date ---
@@ -221,6 +221,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
\subsection{Solution to the last exercise}
A possible way to show why $\left \langle \hat u_i \right \rangle = 0$
given that the columns of $\hat X$ is centered is by considering
$\left \langle \hat X \hat v_i \right \rangle$:
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
where $x_{jk}$ being the element of $\hat X$ at row $j$ and column
$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X
\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and
$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$.
Since the columns of $\hat X$ are assumed to be centered, $\left
\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that
$\left \langle \hat X \hat v_i \right \rangle = 0$.
But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $.
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i
\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i
\neq 0$ (otherwise the variance in the exercise would just be zero),
gives that $\left \langle \hat u_i \right \rangle = 0$.
Regarding $\hat V$ and using the similar approach as above by
computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle$, we have
\[
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
% ------------------- end of main content ---------------
% #ifdef PREAMBLE
doc/Projects/2019/hw2/pdf/hw2.pdf
BIN
View File
Binary file not shown.
doc/Projects/2019/hw2/pdf/hw2.tex
+45 -2
View File
@@ -125,7 +125,7 @@ Homework 2
% --- begin date ---
\begin{center}
Sep 2, 2019
Sep 5, 2019
\end{center}
% --- end date ---
@@ -143,7 +143,7 @@ regression methods. You can use the code under the Regression as an example on h
We will thus again generate our own dataset for a function $y(x)$ where
$x \in [0,1]$ and defined by random numbers computed with the uniform
distribution. The function $y$ is a quadratic polynomial in $x$ with
added stochastic noise according to the normal distribution $N(\mu=0,\sigma^2=1)$.
added stochastic noise according to the normal distribution $\cal{N}(0,1)$.
The following simple Python instructions define our $x$ and $y$ values (with 100 data points).
\begin{verbatim}
@@ -191,6 +191,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
\subsection*{Solution to the last exercise}
A possible way to show why $\left \langle \hat u_i \right \rangle = 0$
given that the columns of $\hat X$ is centered is by considering
$\left \langle \hat X \hat v_i \right \rangle$:
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
where $x_{jk}$ being the element of $\hat X$ at row $j$ and column
$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X
\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and
$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$.
Since the columns of $\hat X$ are assumed to be centered, $\left
\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that
$\left \langle \hat X \hat v_i \right \rangle = 0$.
But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $.
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i
\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i
\neq 0$ (otherwise the variance in the exercise would just be zero),
gives that $\left \langle \hat u_i \right \rangle = 0$.
Regarding $\hat V$ and using the similar approach as above by
computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle$, we have
\begin{align}
\left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align}
% ------------------- end of main content ---------------
\end{document}
doc/Projects/2019/hw2/pdf/hw2.tex~
+44 -1
View File
@@ -125,7 +125,7 @@ Homework 2
% --- begin date ---
\begin{center}
Sep 2, 2019
Sep 5, 2019
\end{center}
% --- end date ---
@@ -191,6 +191,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
\subsection*{Solution to the last exercise}
A possible way to show why $\left \langle \hat u_i \right \rangle = 0$
given that the columns of $\hat X$ is centered is by considering
$\left \langle \hat X \hat v_i \right \rangle$:
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
where $x_{jk}$ being the element of $\hat X$ at row $j$ and column
$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X
\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and
$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$.
Since the columns of $\hat X$ are assumed to be centered, $\left
\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that
$\left \langle \hat X \hat v_i \right \rangle = 0$.
But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $.
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i
\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i
\neq 0$ (otherwise the variance in the exercise would just be zero),
gives that $\left \langle \hat u_i \right \rangle = 0$.
Regarding $\hat V$ and using the similar approach as above by
computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle$, we have
\[
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
% ------------------- end of main content ---------------
\end{document}
doc/src/Projects/2019/Exercises/hw2.do.txt
+47
View File
@@ -65,3 +65,50 @@ $\hat{z}_i=\hat{X}\hat{v}_i=\hat{u}_1d_1$ is equal to (equation (3.49) of Hast
where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie *et al*, the matrix elements of $X$ are centered. The consequence is that the mean values of for example $\hat{u}_i$ are zero.
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
===== Solution to the last exercise =====
A possible way to show why $\left \langle \hat u_i \right \rangle = 0$
given that the columns of $\hat X$ is centered is by considering
$\left \langle \hat X \hat v_i \right \rangle$:
!bt
\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle
\end{align*}
!et
where $x_{jk}$ being the element of $\hat X$ at row $j$ and column
$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X
\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and
$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$.
Since the columns of $\hat X$ are assumed to be centered, $\left
\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that
$\left \langle \hat X \hat v_i \right \rangle = 0$.
But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat
u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $.
Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i
\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i
\neq 0$ (otherwise the variance in the exercise would just be zero),
gives that $\left \langle \hat u_i \right \rangle = 0$.
Regarding $\hat V$ and using the similar approach as above by
computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left
\langle \hat v_i \right \rangle$, we have
!bt
\[
\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\
\end{align*}
!et