-
@@ -183,7 +185,52 @@ $$ where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie et al, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
-Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. +Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +
+A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \) +given that the columns of \( \hat X \) is centered is by considering +\( \left \langle \hat X \hat v_i \right \rangle \): + +$$ + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} +$$ + +
+where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column +\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X +\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and +\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \). + +
+Since the columns of \( \hat X \) are assumed to be centered, \( \left +\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that +\( \left \langle \hat X \hat v_i \right \rangle = 0 \). + +
+But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \). + +
+Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i +\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i +\neq 0 \) (otherwise the variance in the exercise would just be zero), +gives that \( \left \langle \hat u_i \right \rangle = 0 \). + +
+Regarding \( \hat V \) and using the similar approach as above by +computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle \), we have + +$$ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} +$$ +
diff --git a/doc/Projects/2019/hw2/html/hw2-bs.html b/doc/Projects/2019/hw2/html/hw2-bs.html index a7fbc548e..c89bd57cb 100644 --- a/doc/Projects/2019/hw2/html/hw2-bs.html +++ b/doc/Projects/2019/hw2/html/hw2-bs.html @@ -42,7 +42,8 @@ Automatically generated HTML file from DocOnce source
@@ -82,6 +83,7 @@ MathJax.Hub.Config({ @@ -115,7 +117,7 @@ MathJax.Hub.Config({-
@@ -183,7 +185,52 @@ $$ where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie et al, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
-Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. +Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +
+A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \) +given that the columns of \( \hat X \) is centered is by considering +\( \left \langle \hat X \hat v_i \right \rangle \): + +$$ + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} +$$ + +
+where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column +\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X +\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and +\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \). + +
+Since the columns of \( \hat X \) are assumed to be centered, \( \left +\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that +\( \left \langle \hat X \hat v_i \right \rangle = 0 \). + +
+But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \). + +
+Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i +\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i +\neq 0 \) (otherwise the variance in the exercise would just be zero), +gives that \( \left \langle \hat u_i \right \rangle = 0 \). + +
+Regarding \( \hat V \) and using the similar approach as above by +computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle \), we have + +$$ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} +$$ +
diff --git a/doc/Projects/2019/hw2/html/hw2.html b/doc/Projects/2019/hw2/html/hw2.html index 183b95c13..8205d616c 100644 --- a/doc/Projects/2019/hw2/html/hw2.html +++ b/doc/Projects/2019/hw2/html/hw2.html @@ -41,7 +41,8 @@ div { text-align: justify; text-justify: inter-word; }
@@ -82,7 +83,7 @@ MathJax.Hub.Config({-
-Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. +Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +
+A possible way to show why \( \left \langle \hat u_i \right \rangle = 0 \) +given that the columns of \( \hat X \) is centered is by considering +\( \left \langle \hat X \hat v_i \right \rangle \): + +$$ + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} +$$ + +
+where \( x_{jk} \) being the element of \( \hat X \) at row \( j \) and column +\( k \), \( ( \hat X \hat v_i )_j \) the \( j \)-th element of the vector \( \hat X +\hat v_i \), \( \hat x_k \) being the \( k \)-th column vector of \( \hat X \), and +\( \hat v_i(k) \) the \( k \)-th element of the vector \( \hat v_i \). + +
+Since the columns of \( \hat X \) are assumed to be centered, \( \left +\langle \hat x_k \right \rangle = 0 \) for all \( k \). This gives that +\( \left \langle \hat X \hat v_i \right \rangle = 0 \). + +
+But \( \left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle \). + +
+Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then \( d_i +\left \langle \hat u_i \right \rangle = 0 \) also. Assuming that \( d_i +\neq 0 \) (otherwise the variance in the exercise would just be zero), +gives that \( \left \langle \hat u_i \right \rangle = 0 \). + +
+Regarding \( \hat V \) and using the similar approach as above by +computing \( \left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle \), we have + +$$ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} +$$ + diff --git a/doc/Projects/2019/hw2/ipynb/ipynb-hw2-src.tar.gz b/doc/Projects/2019/hw2/ipynb/ipynb-hw2-src.tar.gz index 5a9ba93bc..35a1d074a 100644 Binary files a/doc/Projects/2019/hw2/ipynb/ipynb-hw2-src.tar.gz and b/doc/Projects/2019/hw2/ipynb/ipynb-hw2-src.tar.gz differ diff --git a/doc/Projects/2019/hw2/pdf/hw2.p.tex b/doc/Projects/2019/hw2/pdf/hw2.p.tex index 97bbfc181..a6b42f37b 100644 --- a/doc/Projects/2019/hw2/pdf/hw2.p.tex +++ b/doc/Projects/2019/hw2/pdf/hw2.p.tex @@ -155,7 +155,7 @@ Homework 2 % --- begin date --- \begin{center} -Sep 2, 2019 +Sep 5, 2019 \end{center} % --- end date --- @@ -221,6 +221,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +\subsection{Solution to the last exercise} + +A possible way to show why $\left \langle \hat u_i \right \rangle = 0$ +given that the columns of $\hat X$ is centered is by considering +$\left \langle \hat X \hat v_i \right \rangle$: + + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} + +where $x_{jk}$ being the element of $\hat X$ at row $j$ and column +$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X +\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and +$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$. + + + +Since the columns of $\hat X$ are assumed to be centered, $\left +\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that +$\left \langle \hat X \hat v_i \right \rangle = 0$. + + + +But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $. + +Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i +\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i +\neq 0$ (otherwise the variance in the exercise would just be zero), +gives that $\left \langle \hat u_i \right \rangle = 0$. + + + +Regarding $\hat V$ and using the similar approach as above by +computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle$, we have + +\[ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} + + % ------------------- end of main content --------------- % #ifdef PREAMBLE diff --git a/doc/Projects/2019/hw2/pdf/hw2.pdf b/doc/Projects/2019/hw2/pdf/hw2.pdf index e96546e65..94125847b 100644 Binary files a/doc/Projects/2019/hw2/pdf/hw2.pdf and b/doc/Projects/2019/hw2/pdf/hw2.pdf differ diff --git a/doc/Projects/2019/hw2/pdf/hw2.tex b/doc/Projects/2019/hw2/pdf/hw2.tex index 3b5d38f94..5fe8320af 100644 --- a/doc/Projects/2019/hw2/pdf/hw2.tex +++ b/doc/Projects/2019/hw2/pdf/hw2.tex @@ -125,7 +125,7 @@ Homework 2 % --- begin date --- \begin{center} -Sep 2, 2019 +Sep 5, 2019 \end{center} % --- end date --- @@ -143,7 +143,7 @@ regression methods. You can use the code under the Regression as an example on h We will thus again generate our own dataset for a function $y(x)$ where $x \in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with -added stochastic noise according to the normal distribution $N(\mu=0,\sigma^2=1)$. +added stochastic noise according to the normal distribution $\cal{N}(0,1)$. The following simple Python instructions define our $x$ and $y$ values (with 100 data points). \begin{verbatim} @@ -191,6 +191,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +\subsection*{Solution to the last exercise} + +A possible way to show why $\left \langle \hat u_i \right \rangle = 0$ +given that the columns of $\hat X$ is centered is by considering +$\left \langle \hat X \hat v_i \right \rangle$: + + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} + +where $x_{jk}$ being the element of $\hat X$ at row $j$ and column +$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X +\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and +$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$. + + + +Since the columns of $\hat X$ are assumed to be centered, $\left +\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that +$\left \langle \hat X \hat v_i \right \rangle = 0$. + + + +But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $. + +Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i +\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i +\neq 0$ (otherwise the variance in the exercise would just be zero), +gives that $\left \langle \hat u_i \right \rangle = 0$. + + + +Regarding $\hat V$ and using the similar approach as above by +computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle$, we have + +\begin{align} +\left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align} + + % ------------------- end of main content --------------- \end{document} diff --git a/doc/Projects/2019/hw2/pdf/hw2.tex~ b/doc/Projects/2019/hw2/pdf/hw2.tex~ index b95841e79..6d748ce12 100644 --- a/doc/Projects/2019/hw2/pdf/hw2.tex~ +++ b/doc/Projects/2019/hw2/pdf/hw2.tex~ @@ -125,7 +125,7 @@ Homework 2 % --- begin date --- \begin{center} -Sep 2, 2019 +Sep 5, 2019 \end{center} % --- end date --- @@ -191,6 +191,49 @@ where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie \emph{et Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + +\subsection*{Solution to the last exercise} + +A possible way to show why $\left \langle \hat u_i \right \rangle = 0$ +given that the columns of $\hat X$ is centered is by considering +$\left \langle \hat X \hat v_i \right \rangle$: + + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} + +where $x_{jk}$ being the element of $\hat X$ at row $j$ and column +$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X +\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and +$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$. + + + +Since the columns of $\hat X$ are assumed to be centered, $\left +\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that +$\left \langle \hat X \hat v_i \right \rangle = 0$. + + + +But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $. + +Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i +\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i +\neq 0$ (otherwise the variance in the exercise would just be zero), +gives that $\left \langle \hat u_i \right \rangle = 0$. + + + +Regarding $\hat V$ and using the similar approach as above by +computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle$, we have + +\[ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} + + % ------------------- end of main content --------------- \end{document} diff --git a/doc/src/Projects/2019/Exercises/hw2.do.txt b/doc/src/Projects/2019/Exercises/hw2.do.txt index ffd52ba39..904d0f1e7 100644 --- a/doc/src/Projects/2019/Exercises/hw2.do.txt +++ b/doc/src/Projects/2019/Exercises/hw2.do.txt @@ -65,3 +65,50 @@ $\hat{z}_i=\hat{X}\hat{v}_i=\hat{u}_1d_1$ is equal to (equation (3.49) of Hast where $d_i$ are the singular values of the matrix $\hat{X}$. In Hastie *et al*, the matrix elements of $X$ are centered. The consequence is that the mean values of for example $\hat{u}_i$ are zero. Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise. + + +===== Solution to the last exercise ===== + +A possible way to show why $\left \langle \hat u_i \right \rangle = 0$ +given that the columns of $\hat X$ is centered is by considering +$\left \langle \hat X \hat v_i \right \rangle$: + +!bt + +\begin{align*} \left \langle \hat X \hat v_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X \hat v_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{jk}\hat v_i(k)\\ &= \frac{1}{N}\sum_k \hat v_i(k) \sum_j x_{jk} \\ &= \sum_k \hat v_i(k)\left( \frac{1}{N}\sum_j x_{jk} \right) \\ &= \sum_k \hat v_i(k) \left \langle \hat x_k \right \rangle +\end{align*} +!et + +where $x_{jk}$ being the element of $\hat X$ at row $j$ and column +$k$, $( \hat X \hat v_i )_j $ the $j$-th element of the vector $\hat X +\hat v_i $, $\hat x_k$ being the $k$-th column vector of $\hat X$, and +$\hat v_i(k)$ the $k$-th element of the vector $\hat v_i$. + + + +Since the columns of $\hat X$ are assumed to be centered, $\left +\langle \hat x_k \right \rangle = 0$ for all $k$. This gives that +$\left \langle \hat X \hat v_i \right \rangle = 0$. + + + +But $\left \langle \hat X \hat v_i \right \rangle = \left \langle \hat +u_i d_i \right \rangle = d_i \left \langle \hat u_i \right \rangle $. + +Since $ \left \langle \hat X \hat v_i \right \rangle = 0$, then $d_i +\left \langle \hat u_i \right \rangle = 0$ also. Assuming that $d_i +\neq 0$ (otherwise the variance in the exercise would just be zero), +gives that $\left \langle \hat u_i \right \rangle = 0$. + + + +Regarding $\hat V$ and using the similar approach as above by +computing $\left \langle \hat X^T \hat u_i \right \rangle = d_i \left +\langle \hat v_i \right \rangle$, we have + +!bt +\[ +\begin{align*} \left \langle \hat X^T \hat u_i \right \rangle &= \frac{1}{N}\sum_j ( \hat X^T \hat u_i )_j \\ &= \frac{1}{N}\sum_j \sum_k x_{kj} \hat u_i(k)\\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \frac{1}{N}\sum_k \hat u_i(k) \sum_j x_{kj} \\ &= \sum_k \hat u_i(k) \left( \frac{1}{N} \sum_j x_{kj} \right)\\ +\end{align*} +!et +