more update
This commit is contained in:
Morten Hjorth-Jensen
@@ -1255,7 +1255,7 @@ For Ridge regression this becomes
|
||||
\]
|
||||
!et
|
||||
|
||||
with the vectors $\bm{u}_j$ being the columns of $\bm{U}$ from the SVD of the matrix $\bm{X}$.
|
||||
with the vectors $\bm{u}_j$ being the columns of $\bm{U}$ from the SVD of the matrix $\bm{X}$. Note that the sums goes to $p-1$ since, by definition, $\sigma_j=0$ for $j > p-1$.
|
||||
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
Binary file not shown.
File diff suppressed because it is too large
Load Diff
@@ -1204,10 +1204,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.04279651270127165
|
||||
4.090014704675496
|
||||
[[0.77618224 2.31644071]
|
||||
[2.31644071 8.00233155]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.024023115996453476
|
||||
4.231224729143838
|
||||
[[ 0.86171505 2.958476 ]
|
||||
[ 2.958476 11.14451625]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1244,10 +1244,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08322642264994606
|
||||
1.7484669886413582
|
||||
[[1. 0.6960326]
|
||||
[0.6960326 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07897647347778382
|
||||
1.7546383166870465
|
||||
[[1. 0.61406871]
|
||||
[0.61406871 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1277,30 +1277,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.84886051 -1.52171908]
|
||||
[-0.91616374 -3.74126425]
|
||||
[ 1.4399427 4.17240522]
|
||||
[-0.993865 -2.87927126]
|
||||
[ 0.51782322 0.94077605]
|
||||
[ 0.80279641 3.93037171]
|
||||
[ 0.53952479 0.67970864]
|
||||
[ 1.40829683 5.9212905 ]
|
||||
[ 0.03326365 -0.11962537]
|
||||
[-1.98275836 -7.38267217]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 1.4224801 5.74297504]
|
||||
[-0.01053024 -1.16804444]
|
||||
[ 0.01257962 -1.29082851]
|
||||
[ 0.56500639 2.63754524]
|
||||
[-0.33680101 -1.01023891]
|
||||
[-0.38906684 -0.89335182]
|
||||
[ 0.56437897 2.06942103]
|
||||
[-1.51131471 -4.53479276]
|
||||
[ 0.55518724 0.75623206]
|
||||
[-0.87191952 -2.30891693]]
|
||||
0 1
|
||||
0 -0.848861 -1.521719
|
||||
1 -0.916164 -3.741264
|
||||
2 1.439943 4.172405
|
||||
3 -0.993865 -2.879271
|
||||
4 0.517823 0.940776
|
||||
5 0.802796 3.930372
|
||||
6 0.539525 0.679709
|
||||
7 1.408297 5.921291
|
||||
8 0.033264 -0.119625
|
||||
9 -1.982758 -7.382672
|
||||
0 1
|
||||
0 1.00000 0.97202
|
||||
1 0.97202 1.00000
|
||||
0 1.422480 5.742975
|
||||
1 -0.010530 -1.168044
|
||||
2 0.012580 -1.290829
|
||||
3 0.565006 2.637545
|
||||
4 -0.336801 -1.010239
|
||||
5 -0.389067 -0.893352
|
||||
6 0.564379 2.069421
|
||||
7 -1.511315 -4.534793
|
||||
8 0.555187 0.756232
|
||||
9 -0.871920 -2.308917
|
||||
0 1
|
||||
0 1.000000 0.954988
|
||||
1 0.954988 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1357,37 +1357,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.088476 0.091031 0.086154 0.087387 0.088765 0.076329 0.077009
|
||||
2 0.0 0.091031 0.094722 0.088119 0.090039 0.092126 0.077467 0.078609
|
||||
3 0.0 0.086154 0.088119 0.089551 0.090358 0.091266 0.082786 0.083087
|
||||
4 0.0 0.087387 0.090039 0.090358 0.091620 0.092998 0.083026 0.083648
|
||||
5 0.0 0.088765 0.092126 0.091266 0.092998 0.094867 0.083320 0.084277
|
||||
6 0.0 0.076329 0.077467 0.082786 0.083026 0.083320 0.078845 0.078710
|
||||
7 0.0 0.077009 0.078609 0.083087 0.083648 0.084277 0.078710 0.078812
|
||||
8 0.0 0.077822 0.079901 0.083495 0.084391 0.085368 0.078651 0.078999
|
||||
9 0.0 0.078776 0.081357 0.084018 0.085264 0.086608 0.078672 0.079278
|
||||
10 0.0 0.067141 0.067560 0.075017 0.074773 0.074552 0.073013 0.072523
|
||||
11 0.0 0.067449 0.068197 0.074992 0.074986 0.075012 0.072651 0.072342
|
||||
12 0.0 0.067864 0.068953 0.075061 0.075302 0.075586 0.072361 0.072240
|
||||
13 0.0 0.068391 0.069838 0.075228 0.075728 0.076282 0.072147 0.072222
|
||||
14 0.0 0.069036 0.070860 0.075498 0.076270 0.077108 0.072011 0.072290
|
||||
1 0.0 0.082939 0.082189 0.090270 0.084728 0.079489 0.086174 0.080200
|
||||
2 0.0 0.082189 0.084434 0.087894 0.084147 0.080455 0.082852 0.078167
|
||||
3 0.0 0.090270 0.087894 0.103822 0.096894 0.090337 0.102449 0.095171
|
||||
4 0.0 0.084728 0.084147 0.096894 0.091477 0.086249 0.095313 0.089306
|
||||
5 0.0 0.079489 0.080455 0.090337 0.086249 0.082204 0.088560 0.083681
|
||||
6 0.0 0.086174 0.082852 0.102449 0.095313 0.088560 0.103257 0.095902
|
||||
7 0.0 0.080200 0.078167 0.095171 0.089306 0.083681 0.095902 0.089683
|
||||
8 0.0 0.074838 0.073925 0.088606 0.083849 0.079214 0.089246 0.084019
|
||||
9 0.0 0.070034 0.070105 0.082683 0.078899 0.075138 0.083217 0.078860
|
||||
10 0.0 0.080407 0.076760 0.097602 0.090730 0.084223 0.099777 0.092769
|
||||
11 0.0 0.074763 0.072145 0.090755 0.084979 0.079447 0.092879 0.086877
|
||||
12 0.0 0.069748 0.068019 0.084645 0.079819 0.075136 0.086705 0.081576
|
||||
13 0.0 0.065282 0.064329 0.079179 0.075181 0.071243 0.081163 0.076795
|
||||
14 0.0 0.061297 0.061028 0.074276 0.071003 0.067720 0.076173 0.072471
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.077822 0.078776 0.067141 0.067449 0.067864 0.068391 0.069036
|
||||
2 0.079901 0.081357 0.067560 0.068197 0.068953 0.069838 0.070860
|
||||
3 0.083495 0.084018 0.075017 0.074992 0.075061 0.075228 0.075498
|
||||
4 0.084391 0.085264 0.074773 0.074986 0.075302 0.075728 0.076270
|
||||
5 0.085368 0.086608 0.074552 0.075012 0.075586 0.076282 0.077108
|
||||
6 0.078651 0.078672 0.073013 0.072651 0.072361 0.072147 0.072011
|
||||
7 0.078999 0.079278 0.072523 0.072342 0.072240 0.072222 0.072290
|
||||
8 0.079443 0.079991 0.072084 0.072091 0.072184 0.072370 0.072652
|
||||
9 0.079991 0.080821 0.071698 0.071901 0.072198 0.072598 0.073105
|
||||
10 0.072084 0.071698 0.068724 0.068098 0.067527 0.067011 0.066553
|
||||
11 0.072091 0.071901 0.068098 0.067615 0.067191 0.066828 0.066528
|
||||
12 0.072184 0.072198 0.067527 0.067191 0.066919 0.066714 0.066580
|
||||
13 0.072370 0.072598 0.067011 0.066828 0.066714 0.066675 0.066714
|
||||
14 0.072652 0.073105 0.066553 0.066528 0.066580 0.066714 0.066934
|
||||
1 0.074838 0.070034 0.080407 0.074763 0.069748 0.065282 0.061297
|
||||
2 0.073925 0.070105 0.076760 0.072145 0.068019 0.064329 0.061028
|
||||
3 0.088606 0.082683 0.097602 0.090755 0.084645 0.079179 0.074276
|
||||
4 0.083849 0.078899 0.090730 0.084979 0.079819 0.075181 0.071003
|
||||
5 0.079214 0.075138 0.084223 0.079447 0.075136 0.071243 0.067720
|
||||
6 0.089246 0.083217 0.099777 0.092879 0.086705 0.081163 0.076173
|
||||
7 0.084019 0.078860 0.092769 0.086877 0.081576 0.076795 0.072471
|
||||
8 0.079225 0.074833 0.086409 0.081396 0.076862 0.072751 0.069015
|
||||
9 0.074833 0.071120 0.080630 0.076389 0.072529 0.069010 0.065796
|
||||
10 0.086409 0.080630 0.097396 0.090830 0.084936 0.079631 0.074839
|
||||
11 0.081396 0.076389 0.090830 0.085165 0.080056 0.075433 0.071239
|
||||
12 0.076862 0.072529 0.084936 0.080056 0.075629 0.071604 0.067934
|
||||
13 0.072751 0.069010 0.079631 0.075433 0.071604 0.068104 0.064896
|
||||
14 0.069015 0.065796 0.074839 0.071239 0.067934 0.064896 0.062097
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1611,7 +1611,7 @@ We have already analyzed the OLS solutions in terms of the eigenvectors (the col
|
||||
\[
|
||||
\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
|
||||
\]</div>
|
||||
<p>with the vectors <span class="math notranslate nohighlight">\(\boldsymbol{u}_j\)</span> being the columns of <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> from the SVD of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span>.</p>
|
||||
<p>with the vectors <span class="math notranslate nohighlight">\(\boldsymbol{u}_j\)</span> being the columns of <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> from the SVD of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span>. Note that the sums goes to <span class="math notranslate nohighlight">\(p-1\)</span> since, by definition, <span class="math notranslate nohighlight">\(\sigma_j=0\)</span> for <span class="math notranslate nohighlight">\(j > p-1\)</span>.</p>
|
||||
<p>Since <span class="math notranslate nohighlight">\(\lambda \geq 0\)</span>, it means that compared to OLS, we have</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
@@ -1139,7 +1139,7 @@ print(covariance_matrix)
|
||||
# \tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
|
||||
# $$
|
||||
|
||||
# with the vectors $\boldsymbol{u}_j$ being the columns of $\boldsymbol{U}$ from the SVD of the matrix $\boldsymbol{X}$.
|
||||
# with the vectors $\boldsymbol{u}_j$ being the columns of $\boldsymbol{U}$ from the SVD of the matrix $\boldsymbol{X}$. Note that the sums goes to $p-1$ since, by definition, $\sigma_j=0$ for $j > p-1$.
|
||||
#
|
||||
# Since $\lambda \geq 0$, it means that compared to OLS, we have
|
||||
|
||||
|
||||
+352
-352
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user