From 35db449d6cffc6328b53e9a2e0e6cfc0add0f579 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Mon, 6 Sep 2021 17:39:41 +0200 Subject: [PATCH] update of week 35 --- doc/pub/week35/html/week35-bs.html | 6 ++-- doc/pub/week35/html/week35-reveal.html | 26 ++++++++-------- doc/pub/week35/html/week35-solarized.html | 30 +++++++++---------- doc/pub/week35/html/week35.html | 30 +++++++++---------- doc/pub/week35/ipynb/ipynb-week35-src.tar.gz | Bin 192 -> 192 bytes doc/pub/week35/ipynb/week35.ipynb | 26 ++++++++-------- doc/src/week35/week35.do.txt | 26 ++++++++-------- 7 files changed, 72 insertions(+), 72 deletions(-) diff --git a/doc/pub/week35/html/week35-bs.html b/doc/pub/week35/html/week35-bs.html index e22a4cc7b..b5c92388f 100644 --- a/doc/pub/week35/html/week35-bs.html +++ b/doc/pub/week35/html/week35-bs.html @@ -183,10 +183,10 @@ Automatically generated HTML file from DocOnce source None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -350,7 +350,7 @@ MathJax.Hub.Config({
  • Codes for the SVD
  • Note about SVD Calculations
  • Friday September 3
    • -
  • Matheamtics of the SVD and implications
    • +
  • Mathematics of the SVD and implications
  • Example Matrix
  • Setting up the Matrix to be inverted
  • Further properties (important for our analyses later)
    • diff --git a/doc/pub/week35/html/week35-reveal.html b/doc/pub/week35/html/week35-reveal.html index f14fc4647..43a84655c 100644 --- a/doc/pub/week35/html/week35-reveal.html +++ b/doc/pub/week35/html/week35-reveal.html @@ -1615,7 +1615,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re We have defined the residual error as

     
    $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

     
    @@ -1998,7 +1998,7 @@ More material will be added here, see handwritten notes also. Note that this m

    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2007,14 +2007,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \)

     
    $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

     
    @@ -2138,7 +2138,7 @@ $$ $$

     
    -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2267,7 +2267,7 @@ function, that is we have

     
    $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$

     
    @@ -2706,7 +2706,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/html/week35-solarized.html b/doc/pub/week35/html/week35-solarized.html index 02bae429b..bd108cb8e 100644 --- a/doc/pub/week35/html/week35-solarized.html +++ b/doc/pub/week35/html/week35-solarized.html @@ -203,10 +203,10 @@ div { text-align: justify; text-justify: inter-word; } None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -1726,7 +1726,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re

    We have defined the residual error as $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

    @@ -2085,7 +2085,7 @@ More material will be added here, see handwritten notes also. Note that this m











    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2093,14 +2093,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat

    Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \) $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

    @@ -2205,7 +2205,7 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2311,7 +2311,7 @@ least squares is proportional to the second derivative of the cost function, that is we have $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$ This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2707,7 +2707,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/html/week35.html b/doc/pub/week35/html/week35.html index 2fbf430b4..ba815752f 100644 --- a/doc/pub/week35/html/week35.html +++ b/doc/pub/week35/html/week35.html @@ -208,10 +208,10 @@ div { text-align: justify; text-justify: inter-word; } None, 'note-about-svd-calculations'), ('Friday September 3', 2, None, 'friday-september-3'), - ('Matheamtics of the SVD and implications', + ('Mathematics of the SVD and implications', 2, None, - 'matheamtics-of-the-svd-and-implications'), + 'mathematics-of-the-svd-and-implications'), ('Example Matrix', 2, None, 'example-matrix'), ('Setting up the Matrix to be inverted', 2, @@ -1731,7 +1731,7 @@ We can then interpret our optimal model \( \tilde{\boldsymbol{y}} \) as being re

    We have defined the residual error as $$ -\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=(\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T)\boldsymbol{y}. +\boldsymbol{\epsilon}=\boldsymbol{y}-\tilde{\boldsymbol{y}}=\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\right]\boldsymbol{y}. $$

    @@ -2090,7 +2090,7 @@ More material will be added here, see handwritten notes also. Note that this m











    -

    Matheamtics of the SVD and implications

    +

    Mathematics of the SVD and implications

    Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. @@ -2098,14 +2098,14 @@ Let us take a closer look at the mathematics of the SVD and the various implicat

    Our starting point is our design matrix \( \boldsymbol{X} \) of dimension \( n\times p \) $$ -\boldsymbol{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\boldsymbol{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. $$

    @@ -2210,7 +2210,7 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( \p \times p \). +We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).

    This means, using the orthogonality of \( \boldsymbol{V} \), that we get @@ -2316,7 +2316,7 @@ least squares is proportional to the second derivative of the cost function, that is we have $$ -\frac{\partial}{\partial \boldsymbol{\beta}^T}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. +\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T\partial \boldsymbol{\beta}} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}. $$ This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2712,7 +2712,7 @@ where the tilde-matrix \( \tilde{\boldsymbol{\Sigma}} \) is a matrix of dimensio $$ \tilde{\boldsymbol{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix}, diff --git a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz index f1916750fc61205132376517d589bc66e79fb0c0..70692391492588e3664c6c314bbc2e3b459c04c2 100644 GIT binary patch delta 169 zcmV;a09OCN0l)zzABzY8Fg7+}00ZsMOA5j;5C-6`dx|+hnKnsX3wGf`5b**jO_3!a zdqhkrg)vttn-Y`xJ&9umF&!!;1XBcz2}>{lTyCY8&N^;}RhsH7loje--_TZ;ALh)b zz%&2Ev62?H``%SrfznRoTGwzBq9!boZLe}Dw6NnB7;){S5x`XsyeOoTTJcNR8htXn Xu~GQzV?59Eystd~Xh=CZ00;m8rU*u1 delta 169 zcmV;a09OCN0l)zzABzY8qAWIH00ZsMOA5j;5C-6`dx|+hNzv0}`ILmt?@1hUh}lpXO(~;5;)GKc09kIOm(Dt2hEV*Eh74<%c=* zDe%lcajc|;?Y?)FR-m-QT.\n", "\n", "\n", - "## Matheamtics of the SVD and implications\n", + "## Mathematics of the SVD and implications\n", "\n", "Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies.\n", "\n", @@ -2469,14 +2469,14 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{X}=\n", - "\\begin{bmatrix}\n", - "x_{0,0} & x_{0,1} & x_{0,2}& \\dots & \\dots & x_{0,p-1}\\\\\n", - "x_{1,0} & x_{1,1} & x_{1,2& \\dots & \\dots & x_{1,p-1}\\\\\n", - "x_{2,0} & x_{2,1} & x_{2,2}& \\dots & \\dots & x_{2,p-1}\\\\\n", - "\\dots& \\dots &\\dots & \\dots & \\dots &\\dots\\\\\n", - "x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \\dots & \\dots & x_{n-1,p-1}\\\\\n", - "\\end{bmatrix}\n", + "\\boldsymbol{X}=\\begin{bmatrix}\n", + "x_{0,0} & x_{0,1} & x_{0,2}& \\dots & \\dots x_{0,p-1}\\\\\n", + "x_{1,0} & x_{1,1} & x_{1,2}& \\dots & \\dots x_{1,p-1}\\\\\n", + "x_{2,0} & x_{2,1} & x_{2,2}& \\dots & \\dots x_{2,p-1}\\\\\n", + "\\dots & \\dots & \\dots & \\dots \\dots & \\dots \\\\\n", + "x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \\dots & \\dots x_{n-2,p-1}\\\\\n", + "x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \\dots & \\dots x_{n-1,p-1}\\\\\n", + "\\end{bmatrix}.\n", "$$" ] }, @@ -2665,7 +2665,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\sigma}=\\boldsymbol{D}^2, where $\\boldsymbol{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $\\p \\times p$.\n", + "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\sigma}=\\boldsymbol{D}^2, where $\\boldsymbol{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \\times p$.\n", "\n", "This means, using the orthogonality of $\\boldsymbol{V}$, that we get" ] @@ -2877,7 +2877,7 @@ "metadata": {}, "source": [ "$$\n", - "\\frac{\\partial}{\\partial \\boldsymbol{\\beta}^T}\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n", + "\\frac{\\partial^2 C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T\\partial \\boldsymbol{\\beta}} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n", "$$" ] }, @@ -3488,7 +3488,7 @@ "$$\n", "\\tilde{\\boldsymbol{\\Sigma}}=\\begin{bmatrix} \\sigma_0 & 0 & 0 & \\dots & 0 & 0 \\\\\n", " 0 & \\sigma_1 & 0 & \\dots & 0 & 0 \\\\\n", - "\t\t\t\t 0 & 0 & \\sigma_2 & \\dots & 0 & 0 \\\n", + "\t\t\t\t 0 & 0 & \\sigma_2 & \\dots & 0 & 0 \\\\\n", "\t\t\t\t 0 & 0 & 0 & \\dots & \\sigma_{p-2} & 0 \\\\\n", "\t\t\t\t 0 & 0 & 0 & \\dots & 0 & \\sigma_{p-1} \\\\\n", "\\end{bmatrix},\n", diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt index 5d31e1f9a..afd08ed6f 100644 --- a/doc/src/week35/week35.do.txt +++ b/doc/src/week35/week35.do.txt @@ -1242,7 +1242,7 @@ We can then interpret our optimal model $\tilde{\bm{y}}$ as being represented b We have defined the residual error as !bt \[ -\bm{\epsilon}=\bm{y}-\tilde{\bm{y}}=(\bm{I}-\bm{X}\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T)\bm{y}. +\bm{\epsilon}=\bm{y}-\tilde{\bm{y}}=\left[\bm{I}-\bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\right]\bm{y}. \] !et @@ -1574,21 +1574,21 @@ More material will be added here, see handwritten notes also. Note that this m !split -===== Matheamtics of the SVD and implications ===== +===== Mathematics of the SVD and implications ===== Let us take a closer look at the mathematics of the SVD and the various implications for machine learning studies. Our starting point is our design matrix $\bm{X}$ of dimension $n\times p$ !bt \[ -\bm{X}= -\begin{bmatrix} -x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots & x_{0,p-1}\\ -x_{1,0} & x_{1,1} & x_{1,2& \dots & \dots & x_{1,p-1}\\ -x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots & x_{2,p-1}\\ -\dots& \dots &\dots & \dots & \dots &\dots\\ -x_{n-1,0} & x_{n-1,1} & x_{n-1,2} & \dots & \dots & x_{n-1,p-1}\\ -\end{bmatrix} +\bm{X}=\begin{bmatrix} +x_{0,0} & x_{0,1} & x_{0,2}& \dots & \dots x_{0,p-1}\\ +x_{1,0} & x_{1,1} & x_{1,2}& \dots & \dots x_{1,p-1}\\ +x_{2,0} & x_{2,1} & x_{2,2}& \dots & \dots x_{2,p-1}\\ +\dots & \dots & \dots & \dots \dots & \dots \\ +x_{n-2,0} & x_{n-2,1} & x_{n-2,2}& \dots & \dots x_{n-2,p-1}\\ +x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\ +\end{bmatrix}. \] !et @@ -1695,7 +1695,7 @@ and using the orthogonality of the matrix $\bm{U}$ we have \bm{X}^T\bm{X}=\bm{V}\bm{\Sigma}^T\bm{\Sigma}\bm{V}^T. \] !et -We define $\bm{\Sigma}^T\bm{\sigma}=\bm{D}^2, where $\bm{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $\p \times p$. +We define $\bm{\Sigma}^T\bm{\sigma}=\bm{D}^2, where $\bm{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \times p$. This means, using the orthogonality of $\bm{V}$, that we get @@ -1805,7 +1805,7 @@ function, that is we have !bt \[ -\frac{\partial}{\partial \bm{\beta}^T}\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}} =\frac{2}{n}\bm{X}^T\bm{X}. +\frac{\partial^2 C(\bm{\beta})}{\partial \bm{\beta}^T\partial \bm{\beta}} =\frac{2}{n}\bm{X}^T\bm{X}. \] !et This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize). @@ -2190,7 +2190,7 @@ where the tilde-matrix $\tilde{\bm{\Sigma}}$ is a matrix of dimension $p\times p \[ \tilde{\bm{\Sigma}}=\begin{bmatrix} \sigma_0 & 0 & 0 & \dots & 0 & 0 \\ 0 & \sigma_1 & 0 & \dots & 0 & 0 \\ - 0 & 0 & \sigma_2 & \dots & 0 & 0 \ + 0 & 0 & \sigma_2 & \dots & 0 & 0 \\ 0 & 0 & 0 & \dots & \sigma_{p-2} & 0 \\ 0 & 0 & 0 & \dots & 0 & \sigma_{p-1} \\ \end{bmatrix},