lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added files

Browse Source
This commit is contained in:
mhjensen
2020-12-23 23:05:47 +01:00
parent 43d23c1552
commit b63d445452
137 changed files with 54440 additions and 2342 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/LectureNotes/_build/html/chapter8.html
+102 -93
View File
@@ -38,6 +38,7 @@
<script async="async" src="_static/sphinx-thebe.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="8. Convolutional Neural Networks" href="chapter9.html" />
<link rel="prev" title="6. Support Vector Machines, overarching aims" href="chapter7.html" />
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -119,6 +120,11 @@
7. Dimensionality Reduction
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter9.html">
8. Convolutional Neural Networks
</a>
</li>
</ul>
</nav>
@@ -540,29 +546,29 @@ techniques.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>MSE before scaling: 0.01
R2 score before scaling 0.93
R2 score before scaling 0.94
Feature min values before scaling:
[1.00000000e+00 7.43297505e-04 1.67887686e-04 5.52491181e-07
1.24790498e-07 2.81862750e-08 4.10665316e-10 9.27564656e-11
2.09507879e-11 4.73212847e-12 3.05246505e-13 6.89456494e-14
1.55726683e-14 3.51737928e-15 7.94466097e-16 2.26888965e-16
5.12471292e-17 1.15751255e-17 2.61445925e-18 5.90524667e-19
1.33381074e-19]
[1.00000000e+00 2.52277631e-04 1.58998839e-03 6.36440033e-08
4.01118504e-07 2.52806307e-06 1.60559584e-11 1.01193226e-10
6.37773764e-10 4.01959093e-09 4.05055916e-15 2.55287874e-14
1.60896055e-13 1.01405288e-12 6.39110289e-12 1.02186547e-18
6.44034203e-18 4.05904756e-17 2.55822858e-16 1.61233230e-15
1.01617794e-14]
Feature max values before scaling:
[1. 0.99422559 0.99481826 0.98848453 0.98907377 0.98966337
0.98277662 0.98336246 0.98394865 0.98453519 0.97710167 0.97768412
0.97826693 0.97885008 0.97943358 0.97145948 0.97203858 0.97261802
0.9731978 0.97377793 0.97435841]
[1. 0.99869632 0.999475 0.99739435 0.998172 0.99895027
0.99609407 0.99687071 0.99764796 0.99842581 0.99479548 0.99557111
0.99634735 0.99712419 0.99790164 0.99349859 0.99427321 0.99504844
0.99582426 0.99660069 0.99737773]
Feature min values after scaling:
[ 0. -1.7697784 -1.72415276 -1.14649434 -1.12953807 -1.11320874
-0.90535485 -0.8968819 -0.88864254 -0.88062733 -0.77069333 -0.76574003
-0.76089806 -0.7561637 -0.75153337 -0.68210821 -0.6789547 -0.67586334
-0.67283231 -0.66985986 -0.66694428]
[ 0. -1.70327945 -1.70487638 -1.10269333 -1.11031464 -1.11779074
-0.8675412 -0.87387845 -0.88025384 -0.88666051 -0.73621263 -0.74081963
-0.74548023 -0.75019465 -0.75496285 -0.65064103 -0.65385832 -0.65711132
-0.66040159 -0.66373067 -0.66710007]
Feature max values after scaling:
[0. 1.68453745 1.71039356 2.1618639 2.17707405 2.19168422
2.54536814 2.55730765 2.56887875 2.58009023 2.87398064 2.88442648
2.89461285 2.90454384 2.91422324 3.16472629 3.17449812 3.18407693
3.19346461 3.20266288 3.21167335]
[0. 1.7648157 1.69590827 2.29614193 2.24159129 2.18630067
2.72674862 2.67767959 2.62822832 2.57838874 3.10180456 3.05384136
3.0057178 2.95743601 2.90899738 3.44157349 3.39262419 3.3436382
3.2946235 3.24558749 3.196537 ]
MSE after scaling: 0.00
R2 score for scaled data: 0.97
</pre></div>
@@ -637,9 +643,7 @@ Feature max values before scaling:
Feature min values before scaling:
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Feature max values before scaling:
Feature max values before scaling:
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1.]
Test set accuracy scaled data with Min-Max scaling: 0.97
@@ -679,7 +683,9 @@ Test set accuracy scaled data with Standar Scaler: 0.96
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy: 0.95
Test set accuracy scaled data: 0.96
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy scaled data: 0.96
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
@@ -947,10 +953,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.039184456674535545
4.128680426693387
[[ 1.12297057 3.2186233 ]
[ 3.2186233 10.11517976]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07206099142503472
4.125937788943556
[[ 1.03889255 3.1254397 ]
[ 3.1254397 10.77395298]]
</pre></div>
</div>
</div>
@@ -990,10 +996,10 @@ a more brute force way. Here we scale the mean values for each column of the des
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08474873038505544
1.7239002399681738
[[1. 0.62968416]
[0.62968416 1. ]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07865526085644693
1.4346285243728323
[[1. 0.63566281]
[0.63566281 1. ]]
</pre></div>
</div>
</div>
@@ -1025,30 +1031,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.29439863 1.90169666]
[ 1.67736015 3.10465335]
[-0.55604865 -0.71001501]
[-0.86385893 -3.79843757]
[-0.32231733 0.0602184 ]
[-1.30134141 -3.01442901]
[-0.46095356 0.49571687]
[ 0.83524925 1.94721304]
[-0.41069583 -1.37424606]
[ 1.10820768 1.38762934]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.53680052 2.50858953]
[-0.62753667 -1.74895806]
[ 0.16570918 0.64706639]
[-0.27228985 -0.08345099]
[ 0.85643181 1.55737697]
[-0.329196 -2.18391809]
[ 0.92272798 2.91370357]
[-0.37864859 -0.25719203]
[-0.55226917 -1.4496257 ]
[-0.32172921 -1.9035916 ]]
0 1
0 0.294399 1.901697
1 1.677360 3.104653
2 -0.556049 -0.710015
3 -0.863859 -3.798438
4 -0.322317 0.060218
5 -1.301341 -3.014429
6 -0.460954 0.495717
7 0.835249 1.947213
8 -0.410696 -1.374246
9 1.108208 1.387629
0 0.536801 2.508590
1 -0.627537 -1.748958
2 0.165709 0.647066
3 -0.272290 -0.083451
4 0.856432 1.557377
5 -0.329196 -2.183918
6 0.922728 2.913704
7 -0.378649 -0.257192
8 -0.552269 -1.449626
9 -0.321729 -1.903592
0 1
0 1.000000 0.883341
1 0.883341 1.000000
0 1.000000 0.907143
1 0.907143 1.000000
</pre></div>
</div>
</div>
@@ -1108,37 +1114,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.088096 0.086427 0.083845 0.082640 0.081425 0.073689 0.072606
2 0.0 0.086427 0.085149 0.082400 0.081356 0.080291 0.072444 0.071452
3 0.0 0.083845 0.082400 0.085937 0.084688 0.083422 0.079032 0.077855
4 0.0 0.082640 0.081356 0.084688 0.083531 0.082352 0.077865 0.076753
5 0.0 0.081425 0.080291 0.083422 0.082352 0.081256 0.076682 0.075632
6 0.0 0.073689 0.072444 0.079032 0.077865 0.076682 0.074903 0.073780
7 0.0 0.072606 0.071452 0.077855 0.076753 0.075632 0.073780 0.072708
8 0.0 0.071547 0.070480 0.076701 0.075660 0.074600 0.072677 0.071655
9 0.0 0.070513 0.069529 0.075570 0.074589 0.073587 0.071595 0.070622
10 0.0 0.064543 0.063450 0.071358 0.070294 0.069218 0.069100 0.068064
11 0.0 0.063581 0.062551 0.070288 0.069275 0.068248 0.068065 0.067072
12 0.0 0.062646 0.061677 0.069247 0.068283 0.067304 0.067057 0.066105
13 0.0 0.061738 0.060828 0.068235 0.067318 0.066385 0.066075 0.065164
14 0.0 0.060857 0.060003 0.067250 0.066378 0.065490 0.065119 0.064247
1 0.0 0.076168 0.080935 0.078268 0.081921 0.085682 0.070727 0.073798
2 0.0 0.080935 0.086925 0.081766 0.086005 0.090410 0.072900 0.076302
3 0.0 0.078268 0.081766 0.085075 0.088305 0.091557 0.079623 0.082640
4 0.0 0.081921 0.086005 0.088305 0.091884 0.095514 0.082116 0.085368
5 0.0 0.085682 0.090410 0.091557 0.095514 0.099557 0.084565 0.088068
6 0.0 0.070727 0.072900 0.079623 0.082116 0.084565 0.076371 0.078937
7 0.0 0.073798 0.076302 0.082640 0.085368 0.088068 0.078937 0.081684
8 0.0 0.077019 0.079892 0.085767 0.088751 0.091725 0.081567 0.084509
9 0.0 0.080401 0.083686 0.089005 0.092271 0.095547 0.084258 0.087409
10 0.0 0.062426 0.063699 0.072024 0.073913 0.075721 0.070361 0.072486
11 0.0 0.065031 0.066507 0.074745 0.076802 0.078786 0.072797 0.075066
12 0.0 0.067773 0.069474 0.077585 0.079825 0.082002 0.075324 0.077748
13 0.0 0.070658 0.072612 0.080551 0.082990 0.085376 0.077944 0.080532
14 0.0 0.073697 0.075932 0.083645 0.086302 0.088919 0.080656 0.083422
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.071547 0.070513 0.064543 0.063581 0.062646 0.061738 0.060857
2 0.070480 0.069529 0.063450 0.062551 0.061677 0.060828 0.060003
3 0.076701 0.075570 0.071358 0.070288 0.069247 0.068235 0.067250
4 0.075660 0.074589 0.070294 0.069275 0.068283 0.067318 0.066378
5 0.074600 0.073587 0.069218 0.068248 0.067304 0.066385 0.065490
6 0.072677 0.071595 0.069100 0.068065 0.067057 0.066075 0.065119
7 0.071655 0.070622 0.068064 0.067072 0.066105 0.065164 0.064247
8 0.070650 0.069664 0.067046 0.066096 0.065170 0.064268 0.063389
9 0.069664 0.068722 0.066047 0.065138 0.064251 0.063387 0.062545
10 0.067046 0.066047 0.064786 0.063821 0.062880 0.061964 0.061071
11 0.066096 0.065138 0.063821 0.062893 0.061989 0.061108 0.060249
12 0.065170 0.064251 0.062880 0.061989 0.061120 0.060273 0.059447
13 0.064268 0.063387 0.061964 0.061108 0.060273 0.059458 0.058665
14 0.063389 0.062545 0.061071 0.060249 0.059447 0.058665 0.057901
1 0.077019 0.080401 0.062426 0.065031 0.067773 0.070658 0.073697
2 0.079892 0.083686 0.063699 0.066507 0.069474 0.072612 0.075932
3 0.085767 0.089005 0.072024 0.074745 0.077585 0.080551 0.083645
4 0.088751 0.092271 0.073913 0.076802 0.079825 0.082990 0.086302
5 0.091725 0.095547 0.075721 0.078786 0.082002 0.085376 0.088919
6 0.081567 0.084258 0.070361 0.072797 0.075324 0.077944 0.080656
7 0.084509 0.087409 0.072486 0.075066 0.077748 0.080532 0.083422
8 0.087542 0.090669 0.074644 0.077376 0.080221 0.083181 0.086261
9 0.090669 0.094040 0.076827 0.079721 0.082739 0.085887 0.089170
10 0.074644 0.076827 0.065759 0.067867 0.070042 0.072284 0.074590
11 0.077376 0.079721 0.067867 0.070098 0.072403 0.074782 0.077234
12 0.080221 0.082739 0.070042 0.072403 0.074845 0.077371 0.079979
13 0.083181 0.085887 0.072284 0.074782 0.077371 0.080053 0.082828
14 0.086261 0.089170 0.074590 0.077234 0.079979 0.082828 0.085781
</pre></div>
</div>
</div>
@@ -1335,10 +1341,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
0 4.050693 2.010827
1 2.010827 1.974163
[[4.050693 2.01082738]
[2.01082738 1.97416255]]
0 3.884269 1.937050
1 1.937050 1.951359
[[3.88426936 1.93705016]
[1.93705016 1.95135877]]
</pre></div>
</div>
</div>
@@ -1365,8 +1371,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
[[4.050693 2.01082738]
[2.01082738 1.97416255]]
[[3.88426936 1.93705016]
[1.93705016 1.95135877]]
</pre></div>
</div>
<img alt="_images/chapter8_77_1.png" src="_images/chapter8_77_1.png" />
@@ -1426,14 +1432,16 @@ questions.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
5.275483542917728
0.7493720026008108
5.082577137078103
0.7530509884519185
First eigenvector
[0.85404598 0.52019753]
[0.85042593 0.5260948 ]
Second eigenvector
[-0.52019753 0.85404598]
Eigenvector of largest eigenvalue
[0.85404598 0.52019753]
[-0.5260948 0.85042593]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
[-0.85042593 -0.5260948 ]
</pre></div>
</div>
</div>
@@ -1955,6 +1963,7 @@ these local relationships are best preserved (more details shortly).</p>
<div class='prev-next-bottom'>
<a class='left-prev' id="prev-link" href="chapter7.html" title="previous page"><span class="section-number">6. </span>Support Vector Machines, overarching aims</a>
<a class='right-next' id="next-link" href="chapter9.html" title="next page"><span class="section-number">8. </span>Convolutional Neural Networks</a>
</div>
<footer class="footer mt-5 mt-md-0">