lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added files

Browse Source
This commit is contained in:
mhjensen
2020-12-23 23:19:14 +01:00
parent 274dc10623
commit 89e1c3637f
116 changed files with 45130 additions and 516 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/LectureNotes/_build/html/chapter8.html
+109 -93
View File
@@ -125,6 +125,26 @@
8. Convolutional Neural Networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter10.html">
9. Recurrent Neural Networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter10.html#solving-odes-with-deep-learning">
10. Solving ODEs with Deep Learning
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter11.html">
11. Data Analysis and Machine Learning:
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter11.html#elements-of-bayesian-theory-and-bayesian-neural-networks">
12. Elements of Bayesian theory and Bayesian Neural Networks
</a>
</li>
</ul>
</nav>
@@ -548,27 +568,27 @@ techniques.</p>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>MSE before scaling: 0.01
R2 score before scaling 0.94
Feature min values before scaling:
[1.00000000e+00 2.52277631e-04 1.58998839e-03 6.36440033e-08
4.01118504e-07 2.52806307e-06 1.60559584e-11 1.01193226e-10
6.37773764e-10 4.01959093e-09 4.05055916e-15 2.55287874e-14
1.60896055e-13 1.01405288e-12 6.39110289e-12 1.02186547e-18
6.44034203e-18 4.05904756e-17 2.55822858e-16 1.61233230e-15
1.01617794e-14]
[1.00000000e+00 3.86972479e-03 3.66528972e-03 1.49747700e-05
1.41836625e-05 1.34343487e-05 5.79482386e-08 5.48868704e-08
5.19872323e-08 4.92407803e-08 2.24243736e-10 2.12397083e-10
2.01176282e-10 1.90548268e-10 1.80481726e-10 8.67761543e-13
8.21918258e-13 7.78496845e-13 7.37369358e-13 6.98414609e-13
6.61517814e-13]
Feature max values before scaling:
[1. 0.99869632 0.999475 0.99739435 0.998172 0.99895027
0.99609407 0.99687071 0.99764796 0.99842581 0.99479548 0.99557111
0.99634735 0.99712419 0.99790164 0.99349859 0.99427321 0.99504844
0.99582426 0.99660069 0.99737773]
[1. 0.99791107 0.99418827 0.99582651 0.99211148 0.98841032
0.9937463 0.99003903 0.9863456 0.98266595 0.99167043 0.98797091
0.9842852 0.98061323 0.97695496 0.9895989 0.98590711 0.98222909
0.9785648 0.97491417 0.97127716]
Feature min values after scaling:
[ 0. -1.70327945 -1.70487638 -1.10269333 -1.11031464 -1.11779074
-0.8675412 -0.87387845 -0.88025384 -0.88666051 -0.73621263 -0.74081963
-0.74548023 -0.75019465 -0.75496285 -0.65064103 -0.65385832 -0.65711132
-0.66040159 -0.66373067 -0.66710007]
[ 0. -1.66745992 -1.72823447 -1.09228638 -1.1114811 -1.1313648
-0.86506245 -0.87590574 -0.88698187 -0.89829265 -0.73687981 -0.74440021
-0.75205856 -0.75985475 -0.76778823 -0.65190491 -0.65755533 -0.66330693
-0.66916042 -0.67511627 -0.68117476]
Feature max values after scaling:
[0. 1.7648157 1.69590827 2.29614193 2.24159129 2.18630067
2.72674862 2.67767959 2.62822832 2.57838874 3.10180456 3.05384136
3.0057178 2.95743601 2.90899738 3.44157349 3.39262419 3.3436382
3.2946235 3.24558749 3.196537 ]
[0. 1.76330702 1.70952132 2.29506297 2.25330559 2.21033632
2.73427311 2.69911557 2.66307432 2.6261167 3.11835914 3.08796041
3.05688927 3.02512212 2.99263434 3.46490261 3.4379033 3.41038453
3.38232999 3.35372259 3.32454443]
MSE after scaling: 0.00
R2 score for scaled data: 0.97
</pre></div>
@@ -683,9 +703,7 @@ Test set accuracy scaled data with Standar Scaler: 0.96
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy: 0.95
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy scaled data: 0.96
Test set accuracy scaled data: 0.96
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
@@ -953,10 +971,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07206099142503472
4.125937788943556
[[ 1.03889255 3.1254397 ]
[ 3.1254397 10.77395298]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.11455861677725063
4.414311328416008
[[1.04460586 2.98508511]
[2.98508511 9.60711152]]
</pre></div>
</div>
</div>
@@ -996,10 +1014,10 @@ a more brute force way. Here we scale the mean values for each column of the des
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07865526085644693
1.4346285243728323
[[1. 0.63566281]
[0.63566281 1. ]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07650519724946082
1.5077307406836722
[[1. 0.63582252]
[0.63582252 1. ]]
</pre></div>
</div>
</div>
@@ -1031,30 +1049,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.53680052 2.50858953]
[-0.62753667 -1.74895806]
[ 0.16570918 0.64706639]
[-0.27228985 -0.08345099]
[ 0.85643181 1.55737697]
[-0.329196 -2.18391809]
[ 0.92272798 2.91370357]
[-0.37864859 -0.25719203]
[-0.55226917 -1.4496257 ]
[-0.32172921 -1.9035916 ]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.21054401 -1.67179046]
[-0.27924545 -1.98111498]
[-1.24089109 -3.51965915]
[ 2.17042981 6.77708985]
[-0.36489938 -0.63821317]
[-1.61222872 -5.67668711]
[-0.67184873 -0.62798419]
[ 1.21860807 4.36876959]
[ 0.0351748 -0.93789716]
[ 0.95544469 3.90748678]]
0 1
0 0.536801 2.508590
1 -0.627537 -1.748958
2 0.165709 0.647066
3 -0.272290 -0.083451
4 0.856432 1.557377
5 -0.329196 -2.183918
6 0.922728 2.913704
7 -0.378649 -0.257192
8 -0.552269 -1.449626
9 -0.321729 -1.903592
0 -0.210544 -1.671790
1 -0.279245 -1.981115
2 -1.240891 -3.519659
3 2.170430 6.777090
4 -0.364899 -0.638213
5 -1.612229 -5.676687
6 -0.671849 -0.627984
7 1.218608 4.368770
8 0.035175 -0.937897
9 0.955445 3.907487
0 1
0 1.000000 0.907143
1 0.907143 1.000000
0 1.000000 0.972249
1 0.972249 1.000000
</pre></div>
</div>
</div>
@@ -1114,37 +1132,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.076168 0.080935 0.078268 0.081921 0.085682 0.070727 0.073798
2 0.0 0.080935 0.086925 0.081766 0.086005 0.090410 0.072900 0.076302
3 0.0 0.078268 0.081766 0.085075 0.088305 0.091557 0.079623 0.082640
4 0.0 0.081921 0.086005 0.088305 0.091884 0.095514 0.082116 0.085368
5 0.0 0.085682 0.090410 0.091557 0.095514 0.099557 0.084565 0.088068
6 0.0 0.070727 0.072900 0.079623 0.082116 0.084565 0.076371 0.078937
7 0.0 0.073798 0.076302 0.082640 0.085368 0.088068 0.078937 0.081684
8 0.0 0.077019 0.079892 0.085767 0.088751 0.091725 0.081567 0.084509
9 0.0 0.080401 0.083686 0.089005 0.092271 0.095547 0.084258 0.087409
10 0.0 0.062426 0.063699 0.072024 0.073913 0.075721 0.070361 0.072486
11 0.0 0.065031 0.066507 0.074745 0.076802 0.078786 0.072797 0.075066
12 0.0 0.067773 0.069474 0.077585 0.079825 0.082002 0.075324 0.077748
13 0.0 0.070658 0.072612 0.080551 0.082990 0.085376 0.077944 0.080532
14 0.0 0.073697 0.075932 0.083645 0.086302 0.088919 0.080656 0.083422
1 0.0 0.062851 0.073533 0.065769 0.068538 0.069947 0.059538 0.060320
2 0.0 0.073533 0.087625 0.078561 0.082559 0.084744 0.071705 0.072983
3 0.0 0.065769 0.078561 0.072769 0.076696 0.079022 0.068407 0.069803
4 0.0 0.068538 0.082559 0.076696 0.081211 0.083977 0.072511 0.074206
5 0.0 0.069947 0.084744 0.079022 0.083977 0.087102 0.075127 0.077070
6 0.0 0.059538 0.071705 0.068407 0.072511 0.075127 0.066156 0.067782
7 0.0 0.060320 0.072983 0.069803 0.074206 0.077070 0.067782 0.069583
8 0.0 0.060735 0.073737 0.070715 0.075353 0.078424 0.068928 0.070878
9 0.0 0.060967 0.074217 0.071359 0.076188 0.079434 0.069792 0.071870
10 0.0 0.052371 0.063245 0.061909 0.065807 0.068404 0.061247 0.062897
11 0.0 0.052613 0.063714 0.062491 0.066554 0.069297 0.062003 0.063759
12 0.0 0.052756 0.064027 0.062920 0.067118 0.069987 0.062592 0.064441
13 0.0 0.052853 0.064260 0.063261 0.067574 0.070552 0.063079 0.065009
14 0.0 0.052933 0.064453 0.063553 0.067966 0.071040 0.063502 0.065504
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.077019 0.080401 0.062426 0.065031 0.067773 0.070658 0.073697
2 0.079892 0.083686 0.063699 0.066507 0.069474 0.072612 0.075932
3 0.085767 0.089005 0.072024 0.074745 0.077585 0.080551 0.083645
4 0.088751 0.092271 0.073913 0.076802 0.079825 0.082990 0.086302
5 0.091725 0.095547 0.075721 0.078786 0.082002 0.085376 0.088919
6 0.081567 0.084258 0.070361 0.072797 0.075324 0.077944 0.080656
7 0.084509 0.087409 0.072486 0.075066 0.077748 0.080532 0.083422
8 0.087542 0.090669 0.074644 0.077376 0.080221 0.083181 0.086261
9 0.090669 0.094040 0.076827 0.079721 0.082739 0.085887 0.089170
10 0.074644 0.076827 0.065759 0.067867 0.070042 0.072284 0.074590
11 0.077376 0.079721 0.067867 0.070098 0.072403 0.074782 0.077234
12 0.080221 0.082739 0.070042 0.072403 0.074845 0.077371 0.079979
13 0.083181 0.085887 0.072284 0.074782 0.077371 0.080053 0.082828
14 0.086261 0.089170 0.074590 0.077234 0.079979 0.082828 0.085781
1 0.060735 0.060967 0.052371 0.052613 0.052756 0.052853 0.052933
2 0.073737 0.074217 0.063245 0.063714 0.064027 0.064260 0.064453
3 0.070715 0.071359 0.061909 0.062491 0.062920 0.063261 0.063553
4 0.075353 0.076188 0.065807 0.066554 0.067118 0.067574 0.067966
5 0.078424 0.079434 0.068404 0.069297 0.069987 0.070552 0.071040
6 0.068928 0.069792 0.061247 0.062003 0.062592 0.063079 0.063502
7 0.070878 0.071870 0.062897 0.063759 0.064441 0.065009 0.065504
8 0.072303 0.073408 0.064109 0.065065 0.065830 0.066473 0.067035
9 0.073408 0.074615 0.065054 0.066094 0.066934 0.067645 0.068268
10 0.064109 0.065054 0.057780 0.058594 0.059248 0.059799 0.060281
11 0.065065 0.066094 0.058594 0.059478 0.060193 0.060800 0.061333
12 0.065830 0.066934 0.059248 0.060193 0.060965 0.061622 0.062202
13 0.066473 0.067645 0.059799 0.060800 0.061622 0.062326 0.062949
14 0.067035 0.068268 0.060281 0.061333 0.062202 0.062949 0.063612
</pre></div>
</div>
</div>
@@ -1341,10 +1359,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
0 3.884269 1.937050
1 1.937050 1.951359
[[3.88426936 1.93705016]
[1.93705016 1.95135877]]
0 3.963873 1.979586
1 1.979586 1.976796
[[3.96387325 1.97958566]
[1.97958566 1.97679551]]
</pre></div>
</div>
</div>
@@ -1371,8 +1389,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
[[3.88426936 1.93705016]
[1.93705016 1.95135877]]
[[3.96387325 1.97958566]
[1.97958566 1.97679551]]
</pre></div>
</div>
<img alt="_images/chapter8_77_1.png" src="_images/chapter8_77_1.png" />
@@ -1432,16 +1450,14 @@ questions.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
5.082577137078103
0.7530509884519185
5.185256249406441
0.7554125101643117
First eigenvector
[0.85042593 0.5260948 ]
[0.85104821 0.52508756]
Second eigenvector
[-0.5260948 0.85042593]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
[-0.85042593 -0.5260948 ]
[-0.52508756 0.85104821]
Eigenvector of largest eigenvalue
[-0.85104821 -0.52508756]
</pre></div>
</div>
</div>