diff --git a/doc/pub/week35/html/week35-bs.html b/doc/pub/week35/html/week35-bs.html index e22a4cc7b..b5c92388f 100644 --- a/doc/pub/week35/html/week35-bs.html +++ b/doc/pub/week35/html/week35-bs.html @@ -183,10 +183,10 @@ Automatically generated HTML file from DocOnce source None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -350,7 +350,7 @@ MathJax.Hub.Config({
  • Codes for the SVD
  • Note about SVD Calculations
  • Friday September 3
    • -
  • Matheamtics of the SVD and implications
    • +
  • Mathematics of the SVD and implications
  • Example Matrix
  • Setting up the Matrix to be inverted
  • Further properties (important for our analyses later)
    • diff --git a/doc/pub/week35/html/week35-reveal.html b/doc/pub/week35/html/week35-reveal.html index f14fc4647..43a84655c 100644 --- a/doc/pub/week35/html/week35-reveal.html +++ b/doc/pub/week35/html/week35-reveal.html @@ -1615,7 +1615,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re We have defined the residual error as

     
    $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

     
    @@ -1998,7 +1998,7 @@ More material will be added here, see handwritten notes also. Note that this m

    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2007,14 +2007,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \)

     
    $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

     
    @@ -2138,7 +2138,7 @@ $$ $$

     
    -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2267,7 +2267,7 @@ function, that is we have

     
    $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$

     
    @@ -2706,7 +2706,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/html/week35-solarized.html b/doc/pub/week35/html/week35-solarized.html index 02bae429b..bd108cb8e 100644 --- a/doc/pub/week35/html/week35-solarized.html +++ b/doc/pub/week35/html/week35-solarized.html @@ -203,10 +203,10 @@ div { text-align: justify; text-justify: inter-word; } None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -1726,7 +1726,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re

    We have defined the residual error as $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

    @@ -2085,7 +2085,7 @@ More material will be added here, see handwritten notes also. Note that this m











    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2093,14 +2093,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat

    Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \) $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

    @@ -2205,7 +2205,7 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2311,7 +2311,7 @@ least squares is proportional to the second derivative of the cost function, that is we have $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$ This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2707,7 +2707,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/html/week35.html b/doc/pub/week35/html/week35.html index 2fbf430b4..ba815752f 100644 --- a/doc/pub/week35/html/week35.html +++ b/doc/pub/week35/html/week35.html @@ -208,10 +208,10 @@ div { text-align: justify; text-justify: inter-word; } None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -1731,7 +1731,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re

    We have defined the residual error as $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

    @@ -2090,7 +2090,7 @@ More material will be added here, see handwritten notes also. Note that this m











    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2098,14 +2098,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat

    Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \) $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

    @@ -2210,7 +2210,7 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2316,7 +2316,7 @@ least squares is proportional to the second derivative of the cost function, that is we have $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$ This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2712,7 +2712,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz index f1916750f..706923914 100644 Binary files a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz and b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz differ diff --git a/doc/pub/week35/ipynb/week35.ipynb b/doc/pub/week35/ipynb/week35.ipynb index 55177c1d4..7db8aacb8 100644 --- a/doc/pub/week35/ipynb/week35.ipynb +++ b/doc/pub/week35/ipynb/week35.ipynb @@ -1992,7 +1992,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{\\epsilon}=\\boldsymbol{y}-\\tilde{\\boldsymbol{y}}=(\\boldsymbol{I}-\\boldsymbol{X}\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right^{-1}\\boldsymbol{X}^T)\\boldsymbol{y}.\n", + "\\boldsymbol{\\epsilon}=\\boldsymbol{y}-\\tilde{\\boldsymbol{y}}=\\left[\\boldsymbol{I}-\\boldsymbol{X}\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\right]\\boldsymbol{y}.\n", "$$" ] }, @@ -2457,7 +2457,7 @@ "More material will be added here, see handwritten notes also. Note that this material will be cleaned up after the lecture of Friday September 3. See the handwritten notes from Friday's lecture at .\n", "\n", "\n", - "## Matheamtics of the SVD and implications\n", + "## Mathematics of the SVD and implications\n", "\n", "Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies.\n", "\n", @@ -2469,14 +2469,14 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{X}=\n", - "\\begin{bmatrix}\n", - "x_{0,0} & x_{0,1} & x_{0,2}& \\dots & \\dots & x_{0,p-1}\\\\\n", - "x_{1,0} & x_{1,1} & x_{1,2& \\dots & \\dots & x_{1,p-1}\\\\\n", - "x_{2,0} & x_{2,1} & x_{2,2}& \\dots & \\dots & x_{2,p-1}\\\\\n", - "\\dots& \\dots &\\dots & \\dots & \\dots &\\dots\\\\\n", - "x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \\dots & \\dots & x_{n-1,p-1}\\\\\n", - "\\end{bmatrix}\n", + "\\boldsymbol{X}=\\begin{bmatrix}\n", + "x_{0,0} & x_{0,1} & x_{0,2}& \\dots & \\dots x_{0,p-1}\\\\\n", + "x_{1,0} & x_{1,1} & x_{1,2}& \\dots & \\dots x_{1,p-1}\\\\\n", + "x_{2,0} & x_{2,1} & x_{2,2}& \\dots & \\dots x_{2,p-1}\\\\\n", + "\\dots & \\dots & \\dots & \\dots \\dots & \\dots \\\\\n", + "x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \\dots & \\dots x_{n-2,p-1}\\\\\n", + "x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \\dots & \\dots x_{n-1,p-1}\\\\\n", + "\\end{bmatrix}.\n", "$$" ] }, @@ -2665,7 +2665,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\sigma}=\\boldsymbol{D}^2, where $\\boldsymbol{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $\\p \\times p$.\n", + "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\sigma}=\\boldsymbol{D}^2, where $\\boldsymbol{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \\times p$.\n", "\n", "This means, using the orthogonality of $\\boldsymbol{V}$, that we get" ] @@ -2877,7 +2877,7 @@ "metadata": {}, "source": [ "$$\n", - "\\frac{\\partial}{\\partial \\boldsymbol{\\beta}^T}\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n", + "\\frac{\\partial^2 C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T\\partial \\boldsymbol{\\beta}} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n", "$$" ] }, @@ -3488,7 +3488,7 @@ "$$\n", "\\tilde{\\boldsymbol{\\Sigma}}=\\begin{bmatrix} \\sigma_0 & 0 & 0 & \\dots & 0 & 0 \\\\\n", " 0 & \\sigma_1 & 0 & \\dots & 0 & 0 \\\\\n", - "\t\t\t\t 0 & 0 & \\sigma_2 & \\dots & 0 & 0 \\\n", + "\t\t\t\t 0 & 0 & \\sigma_2 & \\dots & 0 & 0 \\\\\n", "\t\t\t\t 0 & 0 & 0 & \\dots & \\sigma_{p-2} & 0 \\\\\n", "\t\t\t\t 0 & 0 & 0 & \\dots & 0 & \\sigma_{p-1} \\\\\n", "\\end{bmatrix},\n", diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt index 5d31e1f9a..afd08ed6f 100644 --- a/doc/src/week35/week35.do.txt +++ b/doc/src/week35/week35.do.txt @@ -1242,7 +1242,7 @@ We can then interpret our optimal model $\tilde{\bm{y}}$ as being represented b We have defined the residual error as !bt \[ -\bm{\epsilon}=\bm{y}-\tilde{\bm{y}}=(\bm{I}-\bm{X}\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T)\bm{y}. +\bm{\epsilon}=\bm{y}-\tilde{\bm{y}}=\left[\bm{I}-\bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\right]\bm{y}. \] !et @@ -1574,21 +1574,21 @@ More material will be added here, see handwritten notes also. Note that this m !split -===== Matheamtics of the SVD and implications ===== +===== Mathematics of the SVD and implications ===== Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. Our starting point is our design matrix $\bm{X}$ of dimension $n\times p$ !bt \[ -\bm{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\bm{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. \] !et @@ -1695,7 +1695,7 @@ and using the orthogonality of the matrix $\bm{U}$ we have \bm{X}^T\bm{X}=\bm{V}\bm{\Sigma}^T\bm{\Sigma}\bm{V}^T. \] !et -We define $\bm{\Sigma}^T\bm{\sigma}=\bm{D}^2, where $\bm{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $\p \times p$. +We define $\bm{\Sigma}^T\bm{\sigma}=\bm{D}^2, where $\bm{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \times p$. This means, using the orthogonality of $\bm{V}$, that we get @@ -1805,7 +1805,7 @@ function, that is we have !bt \[ -\frac{\partial}{\partial \bm{\beta}^T}\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}} =\frac{2}{n}\bm{X}^T\bm{X}. +\frac{\partial^2 C(\bm{\beta})}{\partial \bm{\beta}^T\partial \bm{\beta}} =\frac{2}{n}\bm{X}^T\bm{X}. \] !et This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2190,7 +2190,7 @@ where the tilde-matrix $\tilde{\bm{\Sigma}}$ is a matrix of dimension $p\times p \[ \tilde{\bm{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix},