lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating typos

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-09-11 21:07:47 +02:00
parent b1a5156248
commit 8ef5665705
17 changed files with 6112 additions and 2653 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/exercisesweek37.html
+16 -7
View File
@@ -438,11 +438,20 @@ doconce format html exercisesweek37.do.txt -->
<p>Date: <strong>Deadline is Sunday September 17 at midnight</strong></p>
<div class="section" id="overarching-aims-of-the-exercises-this-week">
<h2>Overarching aims of the exercises this week<a class="headerlink" href="#overarching-aims-of-the-exercises-this-week" title="Permalink to this headline"></a></h2>
<p>This exercise deals with various mean values and variances in linear regression method (here it may be useful to look up chapter 3, equation (3.8) of <a class="reference external" href="https://www.springer.com/gp/book/9780387848570">Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer</a>). The exercise is also a part of project 1 and can be reused in the theory part of the project.</p>
<p>For more discussions on Ridge regression and calculation of expectation values, <a class="reference external" href="https://arxiv.org/abs/1509.09169">Wessel van Wieringen’s</a> article is highly recommended.</p>
<p>The assumption we have made is
that there exists a continuous function <span class="math notranslate nohighlight">\(f(\boldsymbol{x})\)</span> and a normal distributed error <span class="math notranslate nohighlight">\(\boldsymbol{\varepsilon}\sim N(0, \sigma^2)\)</span>
which describes our data</p>
<p>This exercise deals with various mean values and variances in linear
regression method (here it may be useful to look up chapter 3,
equation (3.8) of <a class="reference external" href="https://www.springer.com/gp/book/9780387848570">Trevor Hastie, Robert Tibshirani, Jerome
H. Friedman, The Elements of Statistical Learning,
Springer</a>). The
exercise is also a part of project 1 and can be reused in the theory
part of the project.</p>
<p>For more discussions on Ridge regression and calculation of
expectation values, <a class="reference external" href="https://arxiv.org/abs/1509.09169">Wessel van
Wieringen’s</a> article is highly
recommended.</p>
<p>The assumption we have made is that there exists a continuous function
<span class="math notranslate nohighlight">\(f(\boldsymbol{x})\)</span> and a normal distributed error <span class="math notranslate nohighlight">\(\boldsymbol{\varepsilon}\sim N(0,
\sigma^2)\)</span> which describes our data</p>
<div class="math notranslate nohighlight">
\[
\boldsymbol{y} = f(\boldsymbol{x})+\boldsymbol{\varepsilon}
@@ -488,7 +497,7 @@ A given parameter <span class="math notranslate nohighlight">\(\beta_j\)</span>
<p>Show that</p>
<div class="math notranslate nohighlight">
\[
\mathbb{E} \big[ \boldsymbol{\beta}^{\mathrm{Ridge}} \big]=(\mathbf{X}^{T} \mathbf{X} + \lambda \mathbf{I}_{pp})^{-1} (\mathbf{X}^{\top} \mathbf{X})\
\mathbb{E} \big[ \hat{\boldsymbol{\beta}}^{\mathrm{Ridge}} \big]=(\mathbf{X}^{T} \mathbf{X} + \lambda \mathbf{I}_{pp})^{-1} (\mathbf{X}^{\top} \mathbf{X})\
\boldsymbol{\beta}^{\mathrm{OLS}}.
\]</div>
<p>We see clearly that
@@ -496,7 +505,7 @@ A given parameter <span class="math notranslate nohighlight">\(\beta_j\)</span>
<p>Show also that the variance is</p>
<div class="math notranslate nohighlight">
\[
\mbox{Var}[\boldsymbol{\beta}^{\mathrm{Ridge}}]=\sigma^2[ \mathbf{X}^{T} \mathbf{X} + \lambda \mathbf{I} ]^{-1} \mathbf{X}^{T}\mathbf{X} \{ [ \mathbf{X}^{\top} \mathbf{X} + \lambda \mathbf{I} ]^{-1}\}^{T},
\mbox{Var}[\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}]=\sigma^2[ \mathbf{X}^{T} \mathbf{X} + \lambda \mathbf{I} ]^{-1} \mathbf{X}^{T}\mathbf{X} \{ [ \mathbf{X}^{\top} \mathbf{X} + \lambda \mathbf{I} ]^{-1}\}^{T},
\]</div>
<p>and it is easy to see that if the parameter <span class="math notranslate nohighlight">\(\lambda\)</span> goes to infinity then the variance of Ridge parameters <span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span> goes to zero.</p>
</div>