lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated book

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-09-11 05:36:59 +02:00
parent 19a064b629
commit 3b762e5606
42 changed files with 580 additions and 524 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/.doctrees/environment.pickle
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/statistics.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/week34.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/week35.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/week37.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/_images/chapter1_17_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 13 KiB

doc/LectureNotes/_build/html/_images/chapter1_19_1.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 19 KiB

doc/LectureNotes/_build/html/_images/chapter1_33_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 24 KiB

doc/LectureNotes/_build/html/_images/chapter1_9_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 18 KiB

doc/LectureNotes/_build/html/_images/chapter3_51_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 21 KiB

doc/LectureNotes/_build/html/_images/chapter3_66_4.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

doc/LectureNotes/_build/html/_images/chapter6_1_1.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 48 KiB

After

Width:  |  Height:  |  Size: 44 KiB

doc/LectureNotes/_build/html/_images/chapter6_1_2.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 36 KiB

doc/LectureNotes/_build/html/_images/chapter8_65_1.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 39 KiB

doc/LectureNotes/_build/html/_images/statistics_181_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

doc/LectureNotes/_build/html/_images/statistics_188_1.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 18 KiB

doc/LectureNotes/_build/html/_images/week37_121_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 22 KiB

doc/LectureNotes/_build/html/_images/week37_139_6.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

doc/LectureNotes/_build/html/_images/week37_148_11.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

doc/LectureNotes/_build/html/chapter1.html
+4 -4
View File
@@ -1021,10 +1021,10 @@ example of the functionality of <strong>Scikit-Learn</strong>.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The intercept alpha:
[2.00955125]
[1.99194201]
Coefficient beta :
[[4.94419718]]
Mean squared error: 0.31
[[4.85108001]]
Mean squared error: 0.28
Variance score: 0.87
Mean squared log error: 0.01
Mean absolute error: 0.43
@@ -1127,7 +1127,7 @@ a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we s
</div>
<div class="cell_output docutils container">
<img alt="_images/chapter1_33_0.png" src="_images/chapter1_33_0.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.005000000000000001
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.005
</pre></div>
</div>
</div>
doc/LectureNotes/_build/html/chapter10.html
+62 -35
View File
@@ -1159,8 +1159,9 @@ probability that image 0 is in category 0,1,2,...,9 =
1.10378326e-04 5.08318298e-09 2.03256632e-04 1.92507116e-03
9.84443254e-01 3.11507992e-04]
probabilities sum up to: 1.0
predictions = (n_inputs) = (1437,)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>predictions = (n_inputs) = (1437,)
prediction for image 0: 8
correct label for image 0: 6
</pre></div>
@@ -1338,7 +1339,7 @@ the <em>Hadamard product</em>, meaning element-wise multiplication.</p>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Old accuracy on training data: 0.1440501043841336
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1672,7 +1673,7 @@ Lambda = 10.0
Accuracy score on test set: 0.19166666666666668
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1681,7 +1682,7 @@ Lambda = 1e-05
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1690,7 +1691,7 @@ Lambda = 0.0001
Accuracy score on test set: 0.08611111111111111
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1699,7 +1700,7 @@ Lambda = 0.001
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1708,7 +1709,7 @@ Lambda = 0.01
Accuracy score on test set: 0.08888888888888889
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1717,7 +1718,7 @@ Lambda = 0.1
Accuracy score on test set: 0.08611111111111111
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1726,7 +1727,7 @@ Lambda = 1.0
Accuracy score on test set: 0.08888888888888889
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1735,11 +1736,11 @@ Lambda = 10.0
Accuracy score on test set: 0.09166666666666666
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1748,11 +1749,11 @@ Lambda = 1e-05
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1761,11 +1762,11 @@ Lambda = 0.0001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1774,11 +1775,11 @@ Lambda = 0.001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1787,11 +1788,11 @@ Lambda = 0.01
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1800,7 +1801,7 @@ Lambda = 0.1
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1809,11 +1810,11 @@ Lambda = 1.0
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1822,11 +1823,11 @@ Lambda = 10.0
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1835,11 +1836,37 @@ Lambda = 1e-05
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1323/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.0001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6316/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
doc/LectureNotes/_build/html/chapter11.html
+36 -29
View File
@@ -2706,20 +2706,19 @@ Using TensorFlow results in a much better execution time. Try it!</p>
<span class="g g-Whitespace"> </span><span class="mi">60</span> <span class="n">x</span><span class="p">,</span><span class="n">t</span> <span class="o">=</span> <span class="n">point</span>
<span class="ne">---&gt; </span><span class="mi">61</span> <span class="k">return</span> <span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="n">t</span><span class="p">)</span><span class="o">*</span><span class="n">u</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">+</span> <span class="n">x</span><span class="o">*</span><span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="n">x</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="o">*</span><span class="n">deep_neural_network</span><span class="p">(</span><span class="n">P</span><span class="p">,</span><span class="n">point</span><span class="p">)</span>
<span class="nn">Cell In[9], line 48,</span> in <span class="ni">deep_neural_network</span><span class="nt">(deep_params, x)</span>
<span class="g g-Whitespace"> </span><span class="mi">45</span> <span class="n">w_output</span> <span class="o">=</span> <span class="n">deep_params</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="g g-Whitespace"> </span><span class="mi">47</span> <span class="c1"># Include bias:</span>
<span class="ne">---&gt; </span><span class="mi">48</span> <span class="n">x_prev</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">concatenate</span><span class="p">((</span><span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span><span class="n">num_points</span><span class="p">)),</span> <span class="n">x_prev</span><span class="p">),</span> <span class="n">axis</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">50</span> <span class="n">z_output</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="n">w_output</span><span class="p">,</span> <span class="n">x_prev</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">51</span> <span class="n">x_output</span> <span class="o">=</span> <span class="n">z_output</span>
<span class="nn">Cell In[9], line 37,</span> in <span class="ni">deep_neural_network</span><span class="nt">(deep_params, x)</span>
<span class="g g-Whitespace"> </span><span class="mi">34</span> <span class="n">x_prev</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">concatenate</span><span class="p">((</span><span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span><span class="n">num_points</span><span class="p">)),</span> <span class="n">x_prev</span> <span class="p">),</span> <span class="n">axis</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="n">z_hidden</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="n">w_hidden</span><span class="p">,</span> <span class="n">x_prev</span><span class="p">)</span>
<span class="ne">---&gt; </span><span class="mi">37</span> <span class="n">x_hidden</span> <span class="o">=</span> <span class="n">sigmoid</span><span class="p">(</span><span class="n">z_hidden</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="c1"># Update x_prev such that next layer can use the output from this layer</span>
<span class="g g-Whitespace"> </span><span class="mi">40</span> <span class="n">x_prev</span> <span class="o">=</span> <span class="n">x_hidden</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:38,</span> in <span class="ni">&lt;lambda&gt;</span><span class="nt">(arr_list, axis)</span>
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="nd">@primitive</span>
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="k">def</span> <span class="nf">concatenate_args</span><span class="p">(</span><span class="n">axis</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="g g-Whitespace"> </span><span class="mi">37</span> <span class="k">return</span> <span class="n">_np</span><span class="o">.</span><span class="n">concatenate</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">axis</span><span class="p">)</span><span class="o">.</span><span class="n">view</span><span class="p">(</span><span class="n">ndarray</span><span class="p">)</span>
<span class="ne">---&gt; </span><span class="mi">38</span> <span class="n">concatenate</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">arr_list</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span> <span class="p">:</span> <span class="n">concatenate_args</span><span class="p">(</span><span class="n">axis</span><span class="p">,</span> <span class="o">*</span><span class="n">arr_list</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="n">vstack</span> <span class="o">=</span> <span class="n">row_stack</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">tup</span><span class="p">:</span> <span class="n">concatenate</span><span class="p">([</span><span class="n">atleast_2d</span><span class="p">(</span><span class="n">_m</span><span class="p">)</span> <span class="k">for</span> <span class="n">_m</span> <span class="ow">in</span> <span class="n">tup</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">40</span> <span class="k">def</span> <span class="nf">hstack</span><span class="p">(</span><span class="n">tup</span><span class="p">):</span>
<span class="nn">Cell In[9], line 11,</span> in <span class="ni">sigmoid</span><span class="nt">(z)</span>
<span class="g g-Whitespace"> </span><span class="mi">10</span> <span class="k">def</span> <span class="nf">sigmoid</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
<span class="ne">---&gt; </span><span class="mi">11</span> <span class="k">return</span> <span class="mi">1</span><span class="o">/</span><span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">z</span><span class="p">))</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:39,</span> in <span class="ni">ArrayBox.__rtruediv__</span><span class="nt">(self, other)</span>
<span class="ne">---&gt; </span><span class="mi">39</span> <span class="k">def</span> <span class="fm">__rtruediv__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span> <span class="k">return</span> <span class="n">anp</span><span class="o">.</span><span class="n">true_divide</span><span class="p">(</span><span class="n">other</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:45,</span> in <span class="ni">primitive.&lt;locals&gt;.f_wrapped</span><span class="nt">(*args, **kwargs)</span>
<span class="g g-Whitespace"> </span><span class="mi">43</span> <span class="n">argnums</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">argnum</span> <span class="k">for</span> <span class="n">argnum</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">boxed_args</span><span class="p">)</span>
@@ -2734,24 +2733,32 @@ Using TensorFlow results in a much better execution time. Try it!</p>
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun_name</span><span class="p">,</span> <span class="n">parent_argnums</span><span class="p">))</span>
<span class="ne">---&gt; </span><span class="mi">36</span> <span class="bp">self</span><span class="o">.</span><span class="n">vjp</span> <span class="o">=</span> <span class="n">vjpmaker</span><span class="p">(</span><span class="n">parent_argnums</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">)</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:48,</span> in <span class="ni">defvjp_argnum.&lt;locals&gt;.vjp_argnums</span><span class="nt">(argnums, *args)</span>
<span class="g g-Whitespace"> </span><span class="mi">47</span> <span class="k">def</span> <span class="nf">vjp_argnums</span><span class="p">(</span><span class="n">argnums</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="ne">---&gt; </span><span class="mi">48</span> <span class="n">vjps</span> <span class="o">=</span> <span class="p">[</span><span class="n">vjpmaker</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="k">for</span> <span class="n">argnum</span> <span class="ow">in</span> <span class="n">argnums</span><span class="p">]</span>
<span class="g g-Whitespace"> </span><span class="mi">49</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">)</span> <span class="k">for</span> <span class="n">vjp</span> <span class="ow">in</span> <span class="n">vjps</span><span class="p">)</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:66,</span> in <span class="ni">defvjp.&lt;locals&gt;.vjp_argnums</span><span class="nt">(argnums, ans, args, kwargs)</span>
<span class="g g-Whitespace"> </span><span class="mi">63</span> <span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
<span class="g g-Whitespace"> </span><span class="mi">64</span> <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">(</span>
<span class="g g-Whitespace"> </span><span class="mi">65</span> <span class="s2">&quot;VJP of </span><span class="si">{}</span><span class="s2"> wrt argnum 0 not defined&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun</span><span class="o">.</span><span class="vm">__name__</span><span class="p">))</span>
<span class="ne">---&gt; </span><span class="mi">66</span> <span class="n">vjp</span> <span class="o">=</span> <span class="n">vjpfun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">67</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">),)</span>
<span class="g g-Whitespace"> </span><span class="mi">68</span> <span class="k">elif</span> <span class="n">L</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:48,</span> in <span class="ni">&lt;listcomp&gt;</span><span class="nt">(.0)</span>
<span class="g g-Whitespace"> </span><span class="mi">47</span> <span class="k">def</span> <span class="nf">vjp_argnums</span><span class="p">(</span><span class="n">argnums</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="ne">---&gt; </span><span class="mi">48</span> <span class="n">vjps</span> <span class="o">=</span> <span class="p">[</span><span class="n">vjpmaker</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="k">for</span> <span class="n">argnum</span> <span class="ow">in</span> <span class="n">argnums</span><span class="p">]</span>
<span class="g g-Whitespace"> </span><span class="mi">49</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">)</span> <span class="k">for</span> <span class="n">vjp</span> <span class="ow">in</span> <span class="n">vjps</span><span class="p">)</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:53,</span> in <span class="ni">&lt;lambda&gt;</span><span class="nt">(ans, x, y)</span>
<span class="g g-Whitespace"> </span><span class="mi">48</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">logaddexp</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">x</span><span class="o">-</span><span class="n">ans</span><span class="p">)),</span>
<span class="g g-Whitespace"> </span><span class="mi">49</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">y</span><span class="o">-</span><span class="n">ans</span><span class="p">)))</span>
<span class="g g-Whitespace"> </span><span class="mi">50</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">logaddexp2</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="mi">2</span><span class="o">**</span><span class="p">(</span><span class="n">x</span><span class="o">-</span><span class="n">ans</span><span class="p">)),</span>
<span class="g g-Whitespace"> </span><span class="mi">51</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="mi">2</span><span class="o">**</span><span class="p">(</span><span class="n">y</span><span class="o">-</span><span class="n">ans</span><span class="p">)))</span>
<span class="g g-Whitespace"> </span><span class="mi">52</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">true_divide</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">y</span><span class="p">),</span>
<span class="ne">---&gt; </span><span class="mi">53</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span> <span class="n">g</span> <span class="o">*</span> <span class="n">x</span> <span class="o">/</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
<span class="g g-Whitespace"> </span><span class="mi">54</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">mod</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
<span class="g g-Whitespace"> </span><span class="mi">55</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span><span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">floor</span><span class="p">(</span><span class="n">x</span><span class="o">/</span><span class="n">y</span><span class="p">)))</span>
<span class="g g-Whitespace"> </span><span class="mi">56</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">remainder</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
<span class="g g-Whitespace"> </span><span class="mi">57</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span><span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">floor</span><span class="p">(</span><span class="n">x</span><span class="o">/</span><span class="n">y</span><span class="p">)))</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:537,</span> in <span class="ni">grad_concatenate_args</span><span class="nt">(argnum, ans, axis_args, kwargs)</span>
<span class="g g-Whitespace"> </span><span class="mi">532</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">tensordot_adjoint_1</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">A</span><span class="p">,</span> <span class="n">G</span><span class="p">,</span> <span class="n">axes</span><span class="p">,</span> <span class="n">An</span><span class="p">,</span> <span class="n">Bn</span><span class="p">:</span> <span class="k">lambda</span> <span class="n">B</span><span class="p">:</span> <span class="n">match_complex</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">tensordot_adjoint_0</span><span class="p">(</span><span class="n">B</span><span class="p">,</span> <span class="n">G</span><span class="p">,</span> <span class="n">axes</span><span class="p">,</span> <span class="n">An</span><span class="p">,</span> <span class="n">Bn</span><span class="p">)),</span>
<span class="g g-Whitespace"> </span><span class="mi">533</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">A</span><span class="p">,</span> <span class="n">G</span><span class="p">,</span> <span class="n">axes</span><span class="p">,</span> <span class="n">An</span><span class="p">,</span> <span class="n">Bn</span><span class="p">:</span> <span class="k">lambda</span> <span class="n">B</span><span class="p">:</span> <span class="n">match_complex</span><span class="p">(</span><span class="n">G</span><span class="p">,</span> <span class="n">anp</span><span class="o">.</span><span class="n">tensordot</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">,</span> <span class="n">axes</span><span class="p">)))</span>
<span class="g g-Whitespace"> </span><span class="mi">534</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">outer</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">match_complex</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">anp</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">g</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">T</span><span class="p">)),</span>
<span class="g g-Whitespace"> </span><span class="mi">535</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">match_complex</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">anp</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">T</span><span class="p">,</span> <span class="n">g</span><span class="p">)))</span>
<span class="ne">--&gt; </span><span class="mi">537</span> <span class="k">def</span> <span class="nf">grad_concatenate_args</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="n">ans</span><span class="p">,</span> <span class="n">axis_args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">):</span>
<span class="g g-Whitespace"> </span><span class="mi">538</span> <span class="n">axis</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="n">axis_args</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">axis_args</span><span class="p">[</span><span class="mi">1</span><span class="p">:]</span>
<span class="g g-Whitespace"> </span><span class="mi">539</span> <span class="n">sizes</span> <span class="o">=</span> <span class="p">[</span><span class="n">anp</span><span class="o">.</span><span class="n">shape</span><span class="p">(</span><span class="n">a</span><span class="p">)[</span><span class="n">axis</span><span class="p">]</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">args</span><span class="p">[:</span><span class="n">argnum</span><span class="p">]]</span>
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:658,</span> in <span class="ni">unbroadcast_f</span><span class="nt">(target, f)</span>
<span class="g g-Whitespace"> </span><span class="mi">655</span> <span class="n">x</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">real</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">656</span> <span class="k">return</span> <span class="n">x</span>
<span class="ne">--&gt; </span><span class="mi">658</span> <span class="k">def</span> <span class="nf">unbroadcast_f</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">f</span><span class="p">):</span>
<span class="g g-Whitespace"> </span><span class="mi">659</span> <span class="n">target_meta</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">metadata</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">660</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">unbroadcast</span><span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">target_meta</span><span class="p">)</span>
<span class="ne">KeyboardInterrupt</span>:
</pre></div>
doc/LectureNotes/_build/html/chapter2.html
+58 -58
View File
@@ -1270,10 +1270,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.05005634426334421
4.366375489616074
[[ 0.93787605 2.95563211]
[ 2.95563211 10.33025801]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.0369544130635358
3.662836197064178
[[1.058997 3.11439407]
[3.11439407 9.99498272]]
</pre></div>
</div>
</div>
@@ -1310,10 +1310,10 @@ a more brute force way. Here we scale the mean values for each column of the des
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07808426989543932
1.4121966338442804
[[1. 0.70362677]
[0.70362677 1. ]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08826182458028335
1.7026722043092946
[[1. 0.61113781]
[0.61113781 1. ]]
</pre></div>
</div>
</div>
@@ -1343,30 +1343,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.34661376 -1.6809195 ]
[ 0.05792927 0.30915293]
[ 0.65183066 3.00564344]
[ 1.75018686 4.35667342]
[-0.75682834 -1.67875366]
[ 1.16654048 3.9065894 ]
[-1.86267497 -5.53585173]
[ 0.29803738 2.45731144]
[-0.63031478 -2.76157429]
[-0.3280928 -2.37827145]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.7252563 -2.26264849]
[ 1.19052935 3.11261935]
[-0.62158409 -2.99662602]
[-0.06216141 -0.18120973]
[ 1.32065614 3.50269821]
[ 0.83995705 2.80855691]
[ 0.1571284 0.96919021]
[-0.03404758 1.01551815]
[-0.23596934 0.77449804]
[-1.82925222 -6.74259663]]
0 1
0 -0.346614 -1.680920
1 0.057929 0.309153
2 0.651831 3.005643
3 1.750187 4.356673
4 -0.756828 -1.678754
5 1.166540 3.906589
6 -1.862675 -5.535852
7 0.298037 2.457311
8 -0.630315 -2.761574
9 -0.328093 -2.378271
0 -0.725256 -2.262648
1 1.190529 3.112619
2 -0.621584 -2.996626
3 -0.062161 -0.181210
4 1.320656 3.502698
5 0.839957 2.808557
6 0.157128 0.969190
7 -0.034048 1.015518
8 -0.235969 0.774498
9 -1.829252 -6.742597
0 1
0 1.000000 0.959076
1 0.959076 1.000000
0 1.000000 0.963187
1 0.963187 1.000000
</pre></div>
</div>
</div>
@@ -1423,37 +1423,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.084996 0.084434 0.084712 0.085455 0.086212 0.075771 0.076638
2 0.0 0.084434 0.084514 0.084042 0.085120 0.086230 0.075176 0.076259
3 0.0 0.084712 0.084042 0.089718 0.090439 0.091159 0.083479 0.084358
4 0.0 0.085455 0.085120 0.090439 0.091388 0.092347 0.084091 0.085139
5 0.0 0.086212 0.086230 0.091159 0.092347 0.093554 0.084695 0.085918
6 0.0 0.075771 0.075176 0.083479 0.084091 0.084695 0.079903 0.080657
7 0.0 0.076638 0.076259 0.084358 0.085139 0.085918 0.080657 0.081544
8 0.0 0.077567 0.077411 0.085294 0.086250 0.087210 0.081457 0.082482
9 0.0 0.078557 0.078636 0.086286 0.087424 0.088574 0.082303 0.083471
10 0.0 0.066997 0.066458 0.075909 0.076389 0.076857 0.074221 0.074824
11 0.0 0.067764 0.067380 0.076693 0.077304 0.077906 0.074892 0.075602
12 0.0 0.068592 0.068369 0.077539 0.078284 0.079027 0.075615 0.076436
13 0.0 0.069484 0.069427 0.078447 0.079334 0.080222 0.076393 0.077329
14 0.0 0.070441 0.070558 0.079420 0.080453 0.081494 0.077226 0.078282
1 0.0 0.077382 0.076459 0.074114 0.076542 0.079074 0.064978 0.067074
2 0.0 0.076459 0.078118 0.070150 0.073866 0.078123 0.059624 0.062356
3 0.0 0.074114 0.070150 0.076618 0.077248 0.077529 0.070432 0.071572
4 0.0 0.076542 0.073866 0.077248 0.078731 0.080082 0.069802 0.071449
5 0.0 0.079074 0.078123 0.077529 0.080082 0.082803 0.068607 0.070859
6 0.0 0.064978 0.059624 0.070432 0.069802 0.068607 0.066865 0.067168
7 0.0 0.067074 0.062356 0.071572 0.071449 0.070859 0.067168 0.067809
8 0.0 0.069439 0.065556 0.072752 0.073253 0.073422 0.067367 0.068408
9 0.0 0.072092 0.069317 0.073923 0.075198 0.076333 0.067380 0.068897
10 0.0 0.056856 0.051021 0.063637 0.062282 0.060288 0.061848 0.061588
11 0.0 0.058361 0.052856 0.064604 0.063560 0.061921 0.062263 0.062231
12 0.0 0.060085 0.055000 0.065679 0.065007 0.063802 0.062700 0.062932
13 0.0 0.062053 0.057514 0.066855 0.066633 0.065963 0.063133 0.063673
14 0.0 0.064294 0.060470 0.068116 0.068446 0.068446 0.063525 0.064427
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.077567 0.078557 0.066997 0.067764 0.068592 0.069484 0.070441
2 0.077411 0.078636 0.066458 0.067380 0.068369 0.069427 0.070558
3 0.085294 0.086286 0.075909 0.076693 0.077539 0.078447 0.079420
4 0.086250 0.087424 0.076389 0.077304 0.078284 0.079334 0.080453
5 0.087210 0.088574 0.076857 0.077906 0.079027 0.080222 0.081494
6 0.081457 0.082303 0.074221 0.074892 0.075615 0.076393 0.077226
7 0.082482 0.083471 0.074824 0.075602 0.076436 0.077329 0.078282
8 0.083564 0.084702 0.075463 0.076351 0.077301 0.078313 0.079391
9 0.084702 0.085996 0.076138 0.077141 0.078210 0.079347 0.080555
10 0.075463 0.076138 0.070095 0.070630 0.071208 0.071831 0.072500
11 0.076351 0.077141 0.070630 0.071252 0.071921 0.072638 0.073405
12 0.077301 0.078210 0.071208 0.071921 0.072684 0.073499 0.074368
13 0.078313 0.079347 0.071831 0.072638 0.073499 0.074417 0.075392
14 0.079391 0.080555 0.072500 0.073405 0.074368 0.075392 0.076479
1 0.069439 0.072092 0.056856 0.058361 0.060085 0.062053 0.064294
2 0.065556 0.069317 0.051021 0.052856 0.055000 0.057514 0.060470
3 0.072752 0.073923 0.063637 0.064604 0.065679 0.066855 0.068116
4 0.073253 0.075198 0.062282 0.063560 0.065007 0.066633 0.068446
5 0.073422 0.076333 0.060288 0.061921 0.063802 0.065963 0.068446
6 0.067367 0.067380 0.061848 0.062263 0.062700 0.063133 0.063525
7 0.068408 0.068897 0.061588 0.062231 0.062932 0.063673 0.064427
8 0.069486 0.070556 0.061154 0.062058 0.063064 0.064166 0.065352
9 0.070556 0.072344 0.060453 0.061656 0.063017 0.064547 0.066256
10 0.061154 0.060453 0.058245 0.058254 0.058234 0.058152 0.057961
11 0.062058 0.061656 0.058254 0.058427 0.058593 0.058721 0.058770
12 0.063064 0.063017 0.058234 0.058593 0.058970 0.059340 0.059669
13 0.064166 0.064547 0.058152 0.058721 0.059340 0.059992 0.060652
14 0.065352 0.066256 0.057961 0.058770 0.059669 0.060652 0.061706
</pre></div>
</div>
</div>
doc/LectureNotes/_build/html/chapter3.html
+34 -32
View File
@@ -824,10 +824,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.14708 sec
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.165272 sec
Jackknife Statistics :
original bias std. error
100.034 100.024 0.147836
99.6801 99.6702 0.149483
</pre></div>
</div>
</div>
@@ -1046,7 +1046,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
100.179 15.0422 100.179 0.151522
100.121 15.022 100.121 0.149904
</pre></div>
</div>
</div>
@@ -1263,14 +1263,14 @@ Error: 0.06547790180152355
Bias^2: 0.06208238634231949
Var: 0.0033955154592040936
0.06547790180152355 &gt;= 0.06208238634231949 + 0.0033955154592040936 = 0.06547790180152359
Polynomial degree: 4
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 4
Error: 0.06844519414009445
Bias^2: 0.06453579006728324
Var: 0.003909404072811226
0.06844519414009445 &gt;= 0.06453579006728324 + 0.003909404072811226 = 0.06844519414009446
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 5
Polynomial degree: 5
Error: 0.05227921801205686
Bias^2: 0.0481872773043029
Var: 0.004091940707753939
@@ -1280,7 +1280,9 @@ Error: 0.037813671417389005
Bias^2: 0.033657685071527665
Var: 0.00415598634586135
0.037813671417389005 &gt;= 0.033657685071527665 + 0.00415598634586135 = 0.03781367141738902
Polynomial degree: 7
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
Error: 0.02760977349102253
Bias^2: 0.022999498260366312
Var: 0.004610275230656212
@@ -1290,9 +1292,7 @@ Error: 0.017355848195593347
Bias^2: 0.010331721306655127
Var: 0.007024126888938232
0.017355848195593347 &gt;= 0.010331721306655127 + 0.007024126888938232 = 0.01735584819559336
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 9
Polynomial degree: 9
Error: 0.02660572763718093
Bias^2: 0.010018312644137363
Var: 0.016587414993043573
@@ -1307,7 +1307,9 @@ Error: 0.07160048164233104
Bias^2: 0.014436800088904942
Var: 0.05716368155342608
0.07160048164233104 &gt;= 0.014436800088904942 + 0.05716368155342608 = 0.07160048164233102
Polynomial degree: 12
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 12
Error: 0.11547777218872497
Bias^2: 0.01628578269596628
Var: 0.09919198949275869
@@ -1319,7 +1321,7 @@ Var: 0.20867052175034223
0.22842468702219465 &gt;= 0.01975416527185249 + 0.20867052175034223 = 0.2284246870221947
</pre></div>
</div>
<img alt="_images/chapter3_66_3.png" src="_images/chapter3_66_3.png" />
<img alt="_images/chapter3_66_4.png" src="_images/chapter3_66_4.png" />
</div>
</div>
<p>The bias-variance tradeoff summarizes the fundamental tension in
@@ -1634,9 +1636,9 @@ Mean squared error on training data: 0.00060704
Mean squared error on test data: 3262.26814548
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(trainingerror), label=&#39;Training Error&#39;)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(testerror), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -1870,7 +1872,7 @@ cross-validation (LOOCV).</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -2759,7 +2761,7 @@ linear system as an equation would reduce this down to
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2903,7 +2905,7 @@ with the form utilized in linear regression, viz.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2943,7 +2945,7 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2978,7 +2980,7 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1390/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6403/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -3031,43 +3033,43 @@ constant as opposed to ridge and OLS. We get a sparse solution with
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:628: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
model = cd_fast.enet_coordinate_descent(
10%|███████████████████▏ | 1/10 [00:00&lt;00:07, 1.14it/s]
10%|██████████ | 1/10 [00:00&lt;00:07, 1.15it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██████████████████████████████████████▍ | 2/10 [00:01&lt;00:07, 1.13it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|████████████████████ | 2/10 [00:01&lt;00:07, 1.12it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|█████████████████████████████████████████████████████████▌ | 3/10 [00:02&lt;00:05, 1.24it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|██████████████████████████████ | 3/10 [00:02&lt;00:05, 1.32it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████████████████████████████████████████████████████████████████████████████▊ | 4/10 [00:03&lt;00:04, 1.31it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████████████████████████████████████████ | 4/10 [00:02&lt;00:03, 1.50it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|████████████████████████████████████████████████████████████████████████████████████████████████ | 5/10 [00:03&lt;00:03, 1.33it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|██████████████████████████████████████████████████ | 5/10 [00:03&lt;00:02, 1.69it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 6/10 [00:04&lt;00:03, 1.30it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████ | 6/10 [00:03&lt;00:02, 1.69it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 7/10 [00:05&lt;00:02, 1.28it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████ | 7/10 [00:04&lt;00:01, 1.65it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 8/10 [00:06&lt;00:01, 1.33it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████████████████████████████████████████████████████████████████████████████ | 8/10 [00:05&lt;00:01, 1.67it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 9/10 [00:07&lt;00:00, 1.30it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|██████████████████████████████████████████████████████████████████████████████████████████ | 9/10 [00:05&lt;00:00, 1.56it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:07&lt;00:00, 1.35it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06&lt;00:00, 1.54it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:07&lt;00:00, 1.30it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06&lt;00:00, 1.52it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
doc/LectureNotes/_build/html/chapter6.html
+4 -6
View File
@@ -752,9 +752,9 @@ predicting the target features of query instances is as follows:</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2nd degree coefficients:
zero power: 2.1810415856976313
first power: -0.2546817701709956
second power: 0.0008297120772365539
zero power: -2.767367275553824
first power: 0.024011020121022356
second power: -0.00021270681344726395
</pre></div>
</div>
<img alt="_images/chapter6_1_1.png" src="_images/chapter6_1_1.png" />
@@ -1621,9 +1621,7 @@ Test set accuracy with Logistic Regression: 0.94
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy with SVM: 0.63
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy with Decision Trees: 0.90
Test set accuracy with Decision Trees: 0.90
Test set accuracy Logistic Regression with scaled data: 0.96
Test set accuracy SVM with scaled data: 0.96
Test set accuracy with Decision Trees and scaled data: 0.89
doc/LectureNotes/_build/html/chapter8.html
+69 -69
View File
@@ -706,10 +706,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.1477190177681485
3.5426270409877345
[[1.01393496 3.02432309]
[3.02432309 9.86643649]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.10541723644166373
4.575870409023631
[[0.84972787 2.5321613 ]
[2.5321613 8.59875207]]
</pre></div>
</div>
</div>
@@ -749,10 +749,10 @@ a more brute force way. Here we scale the mean values for each column of the des
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08793554992813543
1.9271707090281667
[[1. 0.6690108]
[0.6690108 1. ]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0768805855280187
1.6568154596723088
[[1. 0.69438869]
[0.69438869 1. ]]
</pre></div>
</div>
</div>
@@ -781,30 +781,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.95395895 -3.11535632]
[ 1.05641352 3.6533977 ]
[ 0.801356 4.56475921]
[-0.69136414 -1.7642448 ]
[ 0.68822559 0.63896182]
[ 0.30916988 1.25233253]
[ 0.10008326 0.10539984]
[-0.52155823 -1.85777073]
[ 0.24377554 0.94616709]
[-1.03214247 -4.42364634]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-1.6629598 -6.60625144]
[-1.59424119 -4.17676247]
[ 0.13699574 -1.26680052]
[ 1.67275915 7.04206048]
[ 1.48931464 4.73718419]
[ 0.82341746 3.16411163]
[ 0.56141009 1.13137881]
[ 0.38616125 0.98338288]
[-1.28000355 -3.69734609]
[-0.53285379 -1.31095745]]
0 1
0 -0.953959 -3.115356
1 1.056414 3.653398
2 0.801356 4.564759
3 -0.691364 -1.764245
4 0.688226 0.638962
5 0.309170 1.252333
6 0.100083 0.105400
7 -0.521558 -1.857771
8 0.243776 0.946167
9 -1.032142 -4.423646
0 -1.662960 -6.606251
1 -1.594241 -4.176762
2 0.136996 -1.266801
3 1.672759 7.042060
4 1.489315 4.737184
5 0.823417 3.164112
6 0.561410 1.131379
7 0.386161 0.983383
8 -1.280004 -3.697346
9 -0.532854 -1.310957
0 1
0 1.000000 0.947607
1 0.947607 1.000000
0 1.000000 0.972149
1 0.972149 1.000000
</pre></div>
</div>
</div>
@@ -861,37 +861,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.079977 0.079947 0.079510 0.081431 0.083259 0.070891 0.072689
2 0.0 0.079947 0.081195 0.081235 0.083734 0.086125 0.073415 0.075557
3 0.0 0.079510 0.081235 0.084255 0.086970 0.089578 0.078324 0.080630
4 0.0 0.081431 0.083734 0.086970 0.090033 0.092982 0.081221 0.083765
5 0.0 0.083259 0.086125 0.089578 0.092982 0.096270 0.084018 0.086799
6 0.0 0.070891 0.073415 0.078324 0.081221 0.084018 0.074971 0.077341
7 0.0 0.072689 0.075557 0.080630 0.083765 0.086799 0.077341 0.079887
8 0.0 0.074531 0.077736 0.082971 0.086346 0.089619 0.079739 0.082464
9 0.0 0.076418 0.079959 0.085353 0.088970 0.092486 0.082173 0.085079
10 0.0 0.062300 0.065028 0.070886 0.073685 0.076396 0.069322 0.071578
11 0.0 0.063862 0.066824 0.072817 0.075794 0.078684 0.071279 0.073672
12 0.0 0.065482 0.068680 0.074807 0.077965 0.081038 0.073288 0.075822
13 0.0 0.067164 0.070600 0.076859 0.080205 0.083465 0.075356 0.078035
14 0.0 0.068912 0.072590 0.078981 0.082519 0.085973 0.077486 0.080316
1 0.0 0.083793 0.077516 0.081955 0.073791 0.066763 0.071803 0.064695
2 0.0 0.077516 0.074112 0.078363 0.072304 0.066838 0.070556 0.064873
3 0.0 0.081955 0.078363 0.084619 0.077941 0.071937 0.076833 0.070462
4 0.0 0.073791 0.072304 0.077941 0.073102 0.068533 0.072132 0.067141
5 0.0 0.066763 0.066838 0.071937 0.068533 0.065108 0.067671 0.063791
6 0.0 0.071803 0.070556 0.076833 0.072132 0.067671 0.071608 0.066653
7 0.0 0.064695 0.064873 0.070462 0.067141 0.063791 0.066653 0.062797
8 0.0 0.058641 0.059876 0.064878 0.062637 0.060171 0.062173 0.059201
9 0.0 0.053462 0.055477 0.059977 0.058582 0.056826 0.058133 0.055876
10 0.0 0.061862 0.062199 0.067948 0.064820 0.061637 0.064615 0.060898
11 0.0 0.056026 0.057316 0.062429 0.060312 0.057964 0.060087 0.057216
12 0.0 0.051042 0.053036 0.057609 0.056287 0.054609 0.056041 0.053854
13 0.0 0.046764 0.049276 0.053387 0.052692 0.051554 0.052427 0.050796
14 0.0 0.043072 0.045963 0.049677 0.049481 0.048781 0.049196 0.048020
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.074531 0.076418 0.062300 0.063862 0.065482 0.067164 0.068912
2 0.077736 0.079959 0.065028 0.066824 0.068680 0.070600 0.072590
3 0.082971 0.085353 0.070886 0.072817 0.074807 0.076859 0.078981
4 0.086346 0.088970 0.073685 0.075794 0.077965 0.080205 0.082519
5 0.089619 0.092486 0.076396 0.078684 0.081038 0.083465 0.085973
6 0.079739 0.082173 0.069322 0.071279 0.073288 0.075356 0.077486
7 0.082464 0.085079 0.071578 0.073672 0.075822 0.078035 0.080316
8 0.085222 0.088023 0.073858 0.076091 0.078386 0.080747 0.083182
9 0.088023 0.091015 0.076167 0.078544 0.080987 0.083501 0.086095
10 0.073858 0.076167 0.065149 0.067003 0.068903 0.070855 0.072863
11 0.076091 0.078544 0.067003 0.068967 0.070980 0.073048 0.075177
12 0.078386 0.080987 0.068903 0.070980 0.073109 0.075298 0.077553
13 0.080747 0.083501 0.070855 0.073048 0.075298 0.077613 0.079998
14 0.083182 0.086095 0.072863 0.075177 0.077553 0.079998 0.082518
1 0.058641 0.053462 0.061862 0.056026 0.051042 0.046764 0.043072
2 0.059876 0.055477 0.062199 0.057316 0.053036 0.049276 0.045963
3 0.064878 0.059977 0.067948 0.062429 0.057609 0.053387 0.049677
4 0.062637 0.058582 0.064820 0.060312 0.056287 0.052692 0.049481
5 0.060171 0.056826 0.061637 0.057964 0.054609 0.051554 0.048781
6 0.062173 0.058133 0.064615 0.060087 0.056041 0.052427 0.049196
7 0.059201 0.055876 0.060898 0.057216 0.053854 0.050796 0.048020
8 0.056328 0.053599 0.057420 0.054434 0.051645 0.049060 0.046678
9 0.053599 0.051368 0.054197 0.051787 0.049479 0.047299 0.045258
10 0.057420 0.054197 0.059243 0.055653 0.052370 0.049380 0.046663
11 0.054434 0.051787 0.055653 0.052738 0.050015 0.047489 0.045161
12 0.051645 0.049479 0.052370 0.050015 0.047760 0.045630 0.043637
13 0.049060 0.047299 0.049380 0.047489 0.045630 0.043839 0.042136
14 0.046678 0.045258 0.046663 0.045161 0.043637 0.042136 0.040684
</pre></div>
</div>
</div>
@@ -1080,10 +1080,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
0 3.970827 1.972533
1 1.972533 1.968650
[[3.97082748 1.97253307]
[1.97253307 1.96865004]]
0 3.986362 1.994474
1 1.994474 2.001468
[[3.98636199 1.99447418]
[1.99447418 2.00146807]]
</pre></div>
</div>
</div>
@@ -1110,8 +1110,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
[[3.97082748 1.97253307]
[1.97253307 1.96865004]]
[[3.98636199 1.99447418]
[1.99447418 2.00146807]]
</pre></div>
</div>
<img alt="_images/chapter8_65_1.png" src="_images/chapter8_65_1.png" />
@@ -1171,16 +1171,16 @@ questions.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
5.181766185664273
0.7577113351177733
5.221666864828611
0.766163196245293
First eigenvector
[0.85222243 0.52317963]
[0.85014487 0.52654886]
Second eigenvector
[-0.52317963 0.85222243]
[-0.52654886 0.85014487]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
[0.85222243 0.52317963]
[-0.85014487 -0.52654886]
</pre></div>
</div>
</div>
doc/LectureNotes/_build/html/linalg.html
+39 -29
View File
@@ -608,8 +608,8 @@ matrices and vectors.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.02808229 1.3194467 -1.8476874 -0.00537955 -0.47991892 -1.54490887
-0.04110474 0.70857635 -1.39855569 -0.11081083]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 0.93937564 0.92308867 0.34427423 -1.37685349 2.80413696 0.25555619
1.121292 -0.42392359 -0.13913033 -0.7228852 ]
</pre></div>
</div>
</div>
@@ -830,26 +830,36 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.04413243 0.8917148 0.26113912 0.87399194 0.30400113 0.35432563
0.05060379 0.65641173 0.77507653 0.83369175]
[0.71261809 0.93968601 0.91961463 0.79866484 0.81919129 0.73062648
0.1488598 0.24254071 0.39922082 0.24398826]
[0.09171886 0.59348521 0.078588 0.74613334 0.18094575 0.61883807
0.89408972 0.86978877 0.82802004 0.75433448]
[0.26715191 0.8905826 0.19852045 0.06432267 0.72771857 0.63030526
0.97272223 0.66289235 0.41744203 0.6663569 ]
[0.91114704 0.01530321 0.55020649 0.40140374 0.67100236 0.5847256
0.80410179 0.37055062 0.4729218 0.26775644]
[0.34271514 0.45193407 0.55542568 0.82242798 0.40266454 0.64713979
0.03873507 0.81506255 0.72848736 0.16118615]
[0.72602818 0.13825388 0.03701105 0.76807288 0.58493493 0.1441031
0.72744372 0.20755569 0.0317606 0.67313212]
[0.85005989 0.9485531 0.81622636 0.32003025 0.57914918 0.36482524
0.17934801 0.84726382 0.52397611 0.14829228]
[0.18510775 0.22528536 0.56977352 0.53105728 0.43962226 0.06444224
0.01772779 0.20912557 0.08839544 0.06984502]
[0.71469351 0.70474145 0.97836358 0.65475653 0.14213876 0.6816947
0.65082441 0.01573768 0.06410638 0.64425744]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[7.39090680e-02 6.22755839e-01 6.49123502e-01 2.88135577e-01
8.94704941e-01 3.03201179e-01 3.18840034e-01 8.41366128e-01
8.95033155e-01 5.99812668e-01]
[9.16312434e-01 9.43984553e-02 9.29253213e-01 5.26464303e-01
2.21390371e-01 1.24224776e-01 6.89710385e-01 4.27861634e-01
7.43779698e-03 8.17834506e-01]
[1.51914183e-01 5.02447063e-01 5.99647992e-01 8.36235052e-01
8.14046224e-01 1.06662069e-01 2.91416944e-01 5.89463761e-01
7.07712197e-01 4.57819238e-01]
[6.21192602e-01 1.27597777e-01 9.92527681e-01 3.95743466e-01
2.66757105e-01 7.72309979e-01 1.18019623e-01 9.40415979e-03
9.27352535e-01 9.26378176e-02]
[3.52989287e-02 1.01080465e-01 7.27602210e-01 4.01826811e-01
2.33826500e-01 8.47478922e-01 4.73646130e-01 4.59491404e-01
1.56857739e-01 8.65889641e-01]
[1.29100624e-01 1.30387411e-01 2.49156266e-01 1.86123808e-01
3.06942115e-01 7.77873845e-01 6.61979664e-01 7.08699295e-01
4.47297587e-02 5.88738394e-01]
[5.17405911e-01 9.84789579e-01 7.30334592e-01 5.61602014e-01
9.95053694e-01 5.73305640e-01 9.36545697e-01 5.29033075e-01
1.00989834e-04 6.96379806e-01]
[8.23093598e-01 9.99201684e-01 9.42197297e-01 8.48268005e-01
1.65900052e-01 2.60605083e-01 3.83884553e-01 5.91410559e-02
6.89766337e-01 7.91434750e-01]
[6.39621036e-03 4.53512750e-01 2.85259666e-01 6.82623709e-01
4.62905281e-01 9.88236598e-01 6.74272285e-02 5.17547294e-01
5.67238764e-01 5.67819487e-01]
[5.16172219e-01 1.42083463e-01 2.01779091e-01 8.29541992e-02
8.23994188e-01 1.28183460e-01 4.64564368e-01 4.61106937e-01
7.93891335e-01 6.25458506e-01]]
</pre></div>
</div>
</div>
@@ -909,13 +919,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.030645975175292262
4.057913350391706
0.6380763200092493
[[ 0.87584434 2.75131415 3.30124108]
[ 2.75131415 9.73994248 10.78758569]
[ 3.30124108 10.78758569 24.86174943]]
[31.05888195 0.08758013 4.33107417]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.2160073466353432
4.720889407643293
1.0362483569261072
[[ 1.17260271 3.60999067 4.15283463]
[ 3.60999067 12.23423987 13.18824338]
[ 4.15283463 13.18824338 21.31571995]]
[31.70587399 0.09156055 2.92512799]
</pre></div>
</div>
</div>
doc/LectureNotes/_build/html/searchindex.js
+1 -1
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/html/statistics.html
+31 -41
View File
@@ -980,37 +980,27 @@ uncorrelated.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>3.1732863504708044
[[8.55460327e+00 4.59630232e+00 7.71507714e+00 3.56832496e-01
1.62620041e+01 3.84661230e+00 8.38865681e+00 1.18285689e+01
1.78876155e+01 4.77352152e+00]
[4.59630232e+00 2.46954702e+00 4.14523337e+00 1.91722512e-01
8.73741131e+00 2.06674611e+00 4.50714096e+00 6.35537114e+00
9.61083596e+00 2.56476512e+00]
[7.71507714e+00 4.14523337e+00 6.95793988e+00 3.21813899e-01
1.46660941e+01 3.46911594e+00 7.56541622e+00 1.06677444e+01
1.61321722e+01 4.30506075e+00]
[3.56832496e-01 1.91722512e-01 3.21813899e-01 1.48843174e-02
6.78326199e-01 1.60451189e-01 3.49910481e-01 4.93397254e-01
7.46134248e-01 1.99114739e-01]
[1.62620041e+01 8.73741131e+00 1.46660941e+01 6.78326199e-01
3.09135059e+01 7.31227657e+00 1.59465457e+01 2.24856992e+01
3.40037367e+01 9.07429886e+00]
[3.84661230e+00 2.06674611e+00 3.46911594e+00 1.60451189e-01
7.31227657e+00 1.72964493e+00 3.77199379e+00 5.31876431e+00
8.04323936e+00 2.14643345e+00]
[8.38865681e+00 4.50714096e+00 7.56541622e+00 3.49910481e-01
1.59465457e+01 3.77199379e+00 8.22592946e+00 1.15991124e+01
1.75406226e+01 4.68092237e+00]
[1.18285689e+01 6.35537114e+00 1.06677444e+01 4.93397254e-01
2.24856992e+01 5.31876431e+00 1.15991124e+01 1.63555267e+01
2.47334547e+01 6.60041459e+00]
[1.78876155e+01 9.61083596e+00 1.61321722e+01 7.46134248e-01
3.40037367e+01 8.04323936e+00 1.75406226e+01 2.47334547e+01
3.74028787e+01 9.98140006e+00]
[4.77352152e+00 2.56476512e+00 4.30506075e+00 1.99114739e-01
9.07429886e+00 2.14643345e+00 4.68092237e+00 6.60041459e+00
9.98140006e+00 2.66365453e+00]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2.076776262573027
[[ 5.24692014 11.86416941 1.87818331 3.59971727 6.93989887 8.39223923
1.96991192 6.69391797 5.22667819 3.39929428]
[11.86416941 26.82688363 4.24688853 8.13956654 15.69227923 18.97626519
4.45430236 15.13607502 11.81839897 7.68637642]
[ 1.87818331 4.24688853 0.67231298 1.2885519 2.48420061 3.0040792
0.70514808 2.39614949 1.87093752 1.21680864]
[ 3.59971727 8.13956654 1.2885519 2.46963249 4.76120718 5.75760403
1.3514835 4.59244881 3.58583003 2.33212971]
[ 6.93989887 15.69227923 2.48420061 4.76120718 9.17913653 11.1000911
2.60552651 8.85378708 6.91312563 4.49611541]
[ 8.39223923 18.97626519 3.0040792 5.75760403 11.1000911 13.42305151
3.15079545 10.70665448 8.35986305 5.43703545]
[ 1.96991192 4.45430236 0.70514808 1.3514835 2.60552651 3.15079545
0.73958682 2.51317506 1.96231225 1.27623637]
[ 6.69391797 15.13607502 2.39614949 4.59244881 8.85378708 10.70665448
2.51317506 8.53996947 6.66809369 4.33675308]
[ 5.22667819 11.81839897 1.87093752 3.58583003 6.91312563 8.35986305
1.96231225 6.66809369 5.20651434 3.38618024]
[ 3.39929428 7.68637642 1.21680864 2.33212971 4.49611541 5.43703545
1.27623637 4.33675308 3.38618024 2.20228273]]
</pre></div>
</div>
</div>
@@ -1278,15 +1268,15 @@ more practically oriented methods like the blocking technique.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.03872795610436844
3.8505072899794293
0.03015288036252618
0.8896094720230128 9.323184775146844 7.56423957069263
2.7487814387207385 2.000626681519836 6.325228546249804
[[0.88960947 2.74878144 2.00062668]
[2.74878144 9.32318478 6.32522855]
[2.00062668 6.32522855 7.56423957]]
[15.60940356 0.06849372 2.09913654]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09790216282081063
4.184866696796263
0.2373286546935638
0.9722865317690382 10.417644792969273 7.499394932852416
3.055091049054481 2.2630723514841553 7.22977659964799
[[ 0.97228653 3.05509105 2.26307235]
[ 3.05509105 10.41764479 7.2297766 ]
[ 2.26307235 7.2297766 7.49939493]]
[17.22169087 0.0637712 1.6038642 ]
</pre></div>
</div>
</div>
@@ -1616,7 +1606,7 @@ assumption for approximating <span class="math notranslate nohighlight">\(\sigma
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.011156605304609659 0.9767506308987675
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.023295599127611474 0.9904314810719695
</pre></div>
</div>
<img alt="_images/statistics_188_1.png" src="_images/statistics_188_1.png" />
doc/LectureNotes/_build/html/week34.html
+30 -30
View File
@@ -1668,8 +1668,8 @@ developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly he
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 0.27600259 -1.0831367 -2.92599642 -0.36091273 -0.39398622 -1.03979291
0.93237862 1.4635149 0.01552874 0.4761747 ]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.93289669 0.72236654 0.6671767 1.31936237 -0.39865452 -0.86247117
0.64411666 -1.56072658 1.17367225 0.96391888]
</pre></div>
</div>
</div>
@@ -1894,26 +1894,26 @@ lowercase letters for vectors and uppercase letters for matrices)</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.89184351 0.97435908 0.73122034 0.58084123 0.86891741 0.74957448
0.65593984 0.06020264 0.48306473 0.47729832]
[0.86445334 0.75532041 0.31162122 0.57261202 0.2257449 0.31160886
0.68591105 0.01084359 0.57192197 0.83466807]
[0.39525059 0.32684035 0.91648657 0.34904819 0.47052809 0.7029975
0.0057065 0.1074792 0.6389681 0.16567361]
[0.46984559 0.3241988 0.77559615 0.16431956 0.92604454 0.08624193
0.17567503 0.96919836 0.00859693 0.39692815]
[0.26817703 0.49224801 0.05240397 0.14260706 0.91365263 0.2876045
0.38381781 0.23526911 0.28109198 0.01024875]
[0.47892661 0.88377332 0.51858116 0.02345858 0.52087499 0.63309898
0.589841 0.19433318 0.24071032 0.76262775]
[0.67464946 0.69903406 0.44838546 0.72952595 0.41987877 0.0907958
0.21998973 0.54856099 0.54725976 0.29390668]
[0.75600095 0.3882273 0.13024288 0.19677112 0.6401576 0.34599598
0.62351789 0.49211524 0.6267112 0.82121282]
[0.77959897 0.35714377 0.08946029 0.70039784 0.60418171 0.618746
0.46107913 0.41636775 0.94625725 0.85079059]
[0.0525733 0.16201555 0.18147009 0.41408221 0.15173932 0.88501323
0.01376216 0.8030191 0.61192203 0.64399916]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.84159992 0.17563274 0.40632663 0.78278086 0.22919456 0.7510197
0.57584612 0.46342934 0.4745474 0.65543115]
[0.59730078 0.77482712 0.70301068 0.29223521 0.15202654 0.74883972
0.48827791 0.09346106 0.58890784 0.7766443 ]
[0.59184268 0.65554958 0.91546928 0.87340054 0.48200792 0.14824254
0.36185989 0.98654811 0.0473916 0.24032019]
[0.87788573 0.61774362 0.82914126 0.23139242 0.32651488 0.61621902
0.59908884 0.49381549 0.97716508 0.21531156]
[0.01654574 0.32393078 0.91854134 0.93909866 0.75300068 0.53942728
0.66063786 0.48867802 0.53149078 0.6831505 ]
[0.57847325 0.42774546 0.24433117 0.07531349 0.98190064 0.68879472
0.18485685 0.85422602 0.58493681 0.00348246]
[0.8517571 0.29620357 0.3096154 0.18409254 0.54880148 0.29881308
0.7509571 0.46891823 0.42124182 0.38203725]
[0.59963873 0.51154388 0.28399125 0.60026673 0.49074536 0.32906581
0.4069157 0.89724282 0.48326853 0.43373107]
[0.28415767 0.95518112 0.68257097 0.59215613 0.64373221 0.81283649
0.04262217 0.80979265 0.73337355 0.2077068 ]
[0.87696461 0.09529067 0.39540235 0.68352799 0.1598058 0.03648711
0.73018893 0.60921896 0.33220123 0.50887105]]
</pre></div>
</div>
</div>
@@ -1968,13 +1968,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.028638258927275215
4.036527092051294
0.19009085621304586
[[ 1.07405556 3.2781456 2.97935648]
[ 3.2781456 11.21805426 9.35966688]
[ 2.97935648 9.35966688 14.77152924]]
[23.38440469 0.10036694 3.57886743]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.03681479262838276
3.936877889972962
-0.43976975777097144
[[ 1.18742521 3.6407249 4.14122256]
[ 3.6407249 12.06804775 12.20775115]
[ 4.14122256 12.20775115 20.86009093]]
[30.46593404 0.0604888 3.58914105]
</pre></div>
</div>
</div>
@@ -2199,7 +2199,7 @@ Name: Aragorn, dtype: object
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">AttributeError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1539/1326197715.py</span> in <span class="ni">?</span><span class="nt">()</span>
<span class="nn">/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6690/1326197715.py</span> in <span class="ni">?</span><span class="nt">()</span>
<span class="ne">----&gt; </span><span class="mi">6</span> <span class="n">new_hobbit</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;First Name&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Peregrin&quot;</span><span class="p">],</span>
<span class="g g-Whitespace"> </span><span class="mi">7</span> <span class="s1">&#39;Last Name&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Took&quot;</span><span class="p">],</span>
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="s1">&#39;Place of birth&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Shire&quot;</span><span class="p">],</span>
doc/LectureNotes/_build/html/week35.html
+24 -24
View File
@@ -1636,7 +1636,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9952537939995855
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9958983289118531
</pre></div>
</div>
</div>
@@ -1653,7 +1653,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.011208613520466846
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.010348289064969316
</pre></div>
</div>
</div>
@@ -1668,23 +1668,23 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.05040878 0.02601643 0.01922269 0.05006037 0.02572685 0.10595991
0.04487298 0.00334047 0.00330046 0.00606382 0.02500488 0.03247316
0.01897462 0.0241039 0.0606958 0.00472276 0.01756114 0.06536971
0.02809972 0.04955942 0.00956827 0.00667611 0.02576358 0.04216532
0.04808723 0.01625794 0.00282226 0.00220013 0.00017733 0.0211429
0.02207054 0.02156196 0.0694226 0.01119738 0.0041148 0.01783096
0.0062202 0.03317599 0.02032056 0.00798909 0.06901081 0.01353638
0.01863203 0.01179128 0.01178857 0.00634299 0.01793261 0.00018053
0.13055762 0.02441422 0.05029018 0.0253208 0.01979808 0.02693015
0.05336637 0.01373484 0.09291806 0.00168745 0.04588592 0.01013849
0.04018985 0.03887801 0.03033791 0.01811279 0.02540212 0.02980537
0.02784266 0.03158013 0.01060492 0.01620955 0.00942574 0.0043587
0.02651857 0.00053001 0.0337609 0.01131771 0.00023813 0.02091662
0.01315875 0.00434043 0.04161572 0.05045 0.0121289 0.01532738
0.02334754 0.01206221 0.00930146 0.03244944 0.00702721 0.02576685
0.05224117 0.0262517 0.02946852 0.09604976 0.01406777 0.02183817
0.0164974 0.02322594 0.04238763 0.00647029]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.01250964 0.00679013 0.06416459 0.02126205 0.01247909 0.00080497
0.07373479 0.00940611 0.04794409 0.02361665 0.03311432 0.00114401
0.06392846 0.0813156 0.03578526 0.00691012 0.05978512 0.02928194
0.00209403 0.02014577 0.13600596 0.0178462 0.02016185 0.00574735
0.029003 0.04877931 0.03796359 0.02374496 0.06622369 0.02965159
0.01655384 0.00290613 0.00560098 0.00641423 0.05549529 0.03916813
0.01288787 0.02472936 0.02564828 0.03365012 0.03642173 0.02102403
0.00633289 0.02631942 0.02164667 0.04899367 0.00593362 0.03664879
0.00250787 0.00791263 0.01480256 0.01329403 0.02151521 0.00133638
0.01717549 0.01681612 0.03358833 0.01003519 0.00659604 0.02026756
0.01234261 0.058967 0.0126711 0.0039259 0.00254973 0.04315855
0.02909574 0.02386932 0.02404706 0.02769274 0.05804539 0.00732665
0.03868372 0.00513051 0.00172865 0.00353938 0.03076019 0.01643795
0.00458631 0.01817048 0.08667591 0.01706166 0.00637188 0.0930438
0.01506354 0.02787184 0.0002926 0.09398153 0.00570321 0.03511387
0.01075018 0.00170809 0.00943959 0.02654585 0.00399972 0.03057995
0.00211418 0.00685287 0.02330799 0.04191323]
</pre></div>
</div>
</div>
@@ -1753,15 +1753,15 @@ but now splitting the data into a training set and a test set.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.82079885 2.45560415 -4.73595198 14.38102552 -7.04838148]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.04641529 -0.77354301 7.7511273 -3.70241548 1.69515531]
Training R2
0.9952183728736417
0.9958589197366403
Training MSE
0.009338082195270294
0.008560581831215528
Test R2
0.9969461173312454
0.997252717901263
Test MSE
0.008043811612683473
0.00683875021199284
</pre></div>
</div>
</div>
doc/LectureNotes/_build/html/week37.html
+38 -30
View File
@@ -1624,7 +1624,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
100.211 14.8834 100.212 0.149388
100.213 14.98 100.211 0.149466
</pre></div>
</div>
</div>
@@ -1844,7 +1844,9 @@ Error: 0.08426840630693411
Bias^2: 0.0796891867672603
Var: 0.004579219539673834
0.08426840630693411 &gt;= 0.0796891867672603 + 0.004579219539673834 = 0.08426840630693413
Polynomial degree: 2
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 2
Error: 0.10398646080125035
Bias^2: 0.1007711427354898
Var: 0.0032153180657605116
@@ -1866,14 +1868,14 @@ Error: 0.05227921801205686
Bias^2: 0.0481872773043029
Var: 0.004091940707753939
0.05227921801205686 &gt;= 0.0481872773043029 + 0.004091940707753939 = 0.052279218012056844
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 6
Polynomial degree: 6
Error: 0.037813671417389005
Bias^2: 0.033657685071527665
Var: 0.00415598634586135
0.037813671417389005 &gt;= 0.033657685071527665 + 0.00415598634586135 = 0.03781367141738902
Polynomial degree: 7
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
Error: 0.02760977349102253
Bias^2: 0.022999498260366312
Var: 0.004610275230656212
@@ -1895,7 +1897,9 @@ Error: 0.021592704588025025
Bias^2: 0.010516485576645508
Var: 0.011076219011379514
0.021592704588025025 &gt;= 0.010516485576645508 + 0.011076219011379514 = 0.021592704588025022
Polynomial degree: 11
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
Error: 0.07160048164233104
Bias^2: 0.014436800088904942
Var: 0.05716368155342608
@@ -1912,7 +1916,7 @@ Var: 0.20867052175034223
0.22842468702219465 &gt;= 0.01975416527185249 + 0.20867052175034223 = 0.2284246870221947
</pre></div>
</div>
<img alt="_images/week37_139_4.png" src="_images/week37_139_4.png" />
<img alt="_images/week37_139_6.png" src="_images/week37_139_6.png" />
</div>
</div>
</div>
@@ -2263,29 +2267,31 @@ Mean squared error on test data: 10.50427787
Degree of polynomial: 7
Mean squared error on training data: 0.47313680
Mean squared error on test data: 1.53738247
Degree of polynomial: 8
Mean squared error on training data: 0.04926746
Mean squared error on test data: 0.14629156
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 9
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 8
Mean squared error on training data: 0.04926746
Mean squared error on test data: 0.14629156
Degree of polynomial: 9
Mean squared error on training data: 0.02546675
Mean squared error on test data: 0.11202337
Degree of polynomial: 10
Mean squared error on training data: 0.02424794
Mean squared error on test data: 0.22467274
Degree of polynomial: 11
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 11
Mean squared error on training data: 0.01594452
Mean squared error on test data: 1.07641937
Degree of polynomial: 12
Mean squared error on training data: 0.00805074
Mean squared error on test data: 0.04295757
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 13
Degree of polynomial: 13
Mean squared error on training data: 0.00781918
Mean squared error on test data: 0.56965674
Degree of polynomial: 14
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 14
Mean squared error on training data: 0.00465099
Mean squared error on test data: 0.28443039
Degree of polynomial: 15
@@ -2305,29 +2311,31 @@ Mean squared error on test data: 429.25695398
Degree of polynomial: 19
Mean squared error on training data: 0.00154853
Mean squared error on test data: 239.97065359
Degree of polynomial: 20
Mean squared error on training data: 0.00140846
Mean squared error on test data: 1350.24493666
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 21
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 20
Mean squared error on training data: 0.00140846
Mean squared error on test data: 1350.24493666
Degree of polynomial: 21
Mean squared error on training data: 0.00119688
Mean squared error on test data: 1840.50530832
Degree of polynomial: 22
Mean squared error on training data: 0.00092898
Mean squared error on test data: 1184.60929685
Degree of polynomial: 23
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 23
Mean squared error on training data: 0.00089193
Mean squared error on test data: 3892.17483760
Degree of polynomial: 24
Mean squared error on training data: 0.00083355
Mean squared error on test data: 1332.46736215
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
Degree of polynomial: 25
Mean squared error on training data: 0.00079904
Mean squared error on test data: 7577.76690383
Degree of polynomial: 26
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 26
Mean squared error on training data: 0.00075590
Mean squared error on test data: 1079.36895644
Degree of polynomial: 27
@@ -2343,13 +2351,13 @@ Mean squared error on training data: 0.00063866
Mean squared error on test data: 3099.60342978
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1579/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6729/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(trainingerror), label=&#39;Training Error&#39;)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1579/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6729/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(testerror), label=&#39;Test Error&#39;)
</pre></div>
</div>
<img alt="_images/week37_148_9.png" src="_images/week37_148_9.png" />
<img alt="_images/week37_148_11.png" src="_images/week37_148_11.png" />
</div>
</div>
<p>Note that we kept the intercept column in the fitting here. This means that we need to set the <strong>intercept</strong> in the call to the <strong>Scikit-Learn</strong> function as <strong>False</strong>. Alternatively, we could have set up the design matrix <span class="math notranslate nohighlight">\(X\)</span> without the first column of ones.</p>
@@ -2430,7 +2438,7 @@ Mean squared error on test data: 3099.60342978
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1579/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6729/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label=&#39;Test Error&#39;)
</pre></div>
</div>
doc/LectureNotes/_build/jupyter_execute/statistics.ipynb
+33 -43
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/jupyter_execute/statistics_181_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

doc/LectureNotes/_build/jupyter_execute/statistics_188_1.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 18 KiB

doc/LectureNotes/_build/jupyter_execute/week34.ipynb
+30 -30
View File
@@ -874,8 +874,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[ 0.27600259 -1.0831367 -2.92599642 -0.36091273 -0.39398622 -1.03979291\n",
" 0.93237862 1.4635149 0.01552874 0.4761747 ]\n"
"[ 2.93289669 0.72236654 0.6671767 1.31936237 -0.39865452 -0.86247117\n",
" 0.64411666 -1.56072658 1.17367225 0.96391888]\n"
]
}
],
@@ -1313,26 +1313,26 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[[0.89184351 0.97435908 0.73122034 0.58084123 0.86891741 0.74957448\n",
" 0.65593984 0.06020264 0.48306473 0.47729832]\n",
" [0.86445334 0.75532041 0.31162122 0.57261202 0.2257449 0.31160886\n",
" 0.68591105 0.01084359 0.57192197 0.83466807]\n",
" [0.39525059 0.32684035 0.91648657 0.34904819 0.47052809 0.7029975\n",
" 0.0057065 0.1074792 0.6389681 0.16567361]\n",
" [0.46984559 0.3241988 0.77559615 0.16431956 0.92604454 0.08624193\n",
" 0.17567503 0.96919836 0.00859693 0.39692815]\n",
" [0.26817703 0.49224801 0.05240397 0.14260706 0.91365263 0.2876045\n",
" 0.38381781 0.23526911 0.28109198 0.01024875]\n",
" [0.47892661 0.88377332 0.51858116 0.02345858 0.52087499 0.63309898\n",
" 0.589841 0.19433318 0.24071032 0.76262775]\n",
" [0.67464946 0.69903406 0.44838546 0.72952595 0.41987877 0.0907958\n",
" 0.21998973 0.54856099 0.54725976 0.29390668]\n",
" [0.75600095 0.3882273 0.13024288 0.19677112 0.6401576 0.34599598\n",
" 0.62351789 0.49211524 0.6267112 0.82121282]\n",
" [0.77959897 0.35714377 0.08946029 0.70039784 0.60418171 0.618746\n",
" 0.46107913 0.41636775 0.94625725 0.85079059]\n",
" [0.0525733 0.16201555 0.18147009 0.41408221 0.15173932 0.88501323\n",
" 0.01376216 0.8030191 0.61192203 0.64399916]]\n"
"[[0.84159992 0.17563274 0.40632663 0.78278086 0.22919456 0.7510197\n",
" 0.57584612 0.46342934 0.4745474 0.65543115]\n",
" [0.59730078 0.77482712 0.70301068 0.29223521 0.15202654 0.74883972\n",
" 0.48827791 0.09346106 0.58890784 0.7766443 ]\n",
" [0.59184268 0.65554958 0.91546928 0.87340054 0.48200792 0.14824254\n",
" 0.36185989 0.98654811 0.0473916 0.24032019]\n",
" [0.87788573 0.61774362 0.82914126 0.23139242 0.32651488 0.61621902\n",
" 0.59908884 0.49381549 0.97716508 0.21531156]\n",
" [0.01654574 0.32393078 0.91854134 0.93909866 0.75300068 0.53942728\n",
" 0.66063786 0.48867802 0.53149078 0.6831505 ]\n",
" [0.57847325 0.42774546 0.24433117 0.07531349 0.98190064 0.68879472\n",
" 0.18485685 0.85422602 0.58493681 0.00348246]\n",
" [0.8517571 0.29620357 0.3096154 0.18409254 0.54880148 0.29881308\n",
" 0.7509571 0.46891823 0.42124182 0.38203725]\n",
" [0.59963873 0.51154388 0.28399125 0.60026673 0.49074536 0.32906581\n",
" 0.4069157 0.89724282 0.48326853 0.43373107]\n",
" [0.28415767 0.95518112 0.68257097 0.59215613 0.64373221 0.81283649\n",
" 0.04262217 0.80979265 0.73337355 0.2077068 ]\n",
" [0.87696461 0.09529067 0.39540235 0.68352799 0.1598058 0.03648711\n",
" 0.73018893 0.60921896 0.33220123 0.50887105]]\n"
]
}
],
@@ -1446,13 +1446,13 @@
"name": "stdout",
"output_type": "stream",
"text": [
"0.028638258927275215\n",
"4.036527092051294\n",
"0.19009085621304586\n",
"[[ 1.07405556 3.2781456 2.97935648]\n",
" [ 3.2781456 11.21805426 9.35966688]\n",
" [ 2.97935648 9.35966688 14.77152924]]\n",
"[23.38440469 0.10036694 3.57886743]\n"
"-0.03681479262838276\n",
"3.936877889972962\n",
"-0.43976975777097144\n",
"[[ 1.18742521 3.6407249 4.14122256]\n",
" [ 3.6407249 12.06804775 12.20775115]\n",
" [ 4.14122256 12.20775115 20.86009093]]\n",
"[30.46593404 0.0604888 3.58914105]\n"
]
}
],
@@ -1808,7 +1808,7 @@
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_1539/1326197715.py\u001b[0m in \u001b[0;36m?\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m new_hobbit = {'First Name': [\"Peregrin\"],\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;34m'Last Name'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"Took\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;34m'Place of birth'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"Shire\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;34m'Date of Birth T.A.'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m2990\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_6690/1326197715.py\u001b[0m in \u001b[0;36m?\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m new_hobbit = {'First Name': [\"Peregrin\"],\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;34m'Last Name'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"Took\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;34m'Place of birth'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"Shire\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;34m'Date of Birth T.A.'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m2990\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/pandas/core/generic.py\u001b[0m in \u001b[0;36m?\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m 6200\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_accessors\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6201\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_info_axis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_can_hold_identifiers_and_holds_name\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6202\u001b[0m ):\n\u001b[1;32m 6203\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6204\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mobject\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__getattribute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mAttributeError\u001b[0m: 'DataFrame' object has no attribute 'append'"
]
doc/LectureNotes/_build/jupyter_execute/week35.ipynb
+24 -24
View File
@@ -1533,7 +1533,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"0.9952537939995855\n"
"0.9958983289118531\n"
]
}
],
@@ -1564,7 +1564,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"0.011208613520466846\n"
"0.010348289064969316\n"
]
}
],
@@ -1599,23 +1599,23 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[0.05040878 0.02601643 0.01922269 0.05006037 0.02572685 0.10595991\n",
" 0.04487298 0.00334047 0.00330046 0.00606382 0.02500488 0.03247316\n",
" 0.01897462 0.0241039 0.0606958 0.00472276 0.01756114 0.06536971\n",
" 0.02809972 0.04955942 0.00956827 0.00667611 0.02576358 0.04216532\n",
" 0.04808723 0.01625794 0.00282226 0.00220013 0.00017733 0.0211429\n",
" 0.02207054 0.02156196 0.0694226 0.01119738 0.0041148 0.01783096\n",
" 0.0062202 0.03317599 0.02032056 0.00798909 0.06901081 0.01353638\n",
" 0.01863203 0.01179128 0.01178857 0.00634299 0.01793261 0.00018053\n",
" 0.13055762 0.02441422 0.05029018 0.0253208 0.01979808 0.02693015\n",
" 0.05336637 0.01373484 0.09291806 0.00168745 0.04588592 0.01013849\n",
" 0.04018985 0.03887801 0.03033791 0.01811279 0.02540212 0.02980537\n",
" 0.02784266 0.03158013 0.01060492 0.01620955 0.00942574 0.0043587\n",
" 0.02651857 0.00053001 0.0337609 0.01131771 0.00023813 0.02091662\n",
" 0.01315875 0.00434043 0.04161572 0.05045 0.0121289 0.01532738\n",
" 0.02334754 0.01206221 0.00930146 0.03244944 0.00702721 0.02576685\n",
" 0.05224117 0.0262517 0.02946852 0.09604976 0.01406777 0.02183817\n",
" 0.0164974 0.02322594 0.04238763 0.00647029]\n"
"[0.01250964 0.00679013 0.06416459 0.02126205 0.01247909 0.00080497\n",
" 0.07373479 0.00940611 0.04794409 0.02361665 0.03311432 0.00114401\n",
" 0.06392846 0.0813156 0.03578526 0.00691012 0.05978512 0.02928194\n",
" 0.00209403 0.02014577 0.13600596 0.0178462 0.02016185 0.00574735\n",
" 0.029003 0.04877931 0.03796359 0.02374496 0.06622369 0.02965159\n",
" 0.01655384 0.00290613 0.00560098 0.00641423 0.05549529 0.03916813\n",
" 0.01288787 0.02472936 0.02564828 0.03365012 0.03642173 0.02102403\n",
" 0.00633289 0.02631942 0.02164667 0.04899367 0.00593362 0.03664879\n",
" 0.00250787 0.00791263 0.01480256 0.01329403 0.02151521 0.00133638\n",
" 0.01717549 0.01681612 0.03358833 0.01003519 0.00659604 0.02026756\n",
" 0.01234261 0.058967 0.0126711 0.0039259 0.00254973 0.04315855\n",
" 0.02909574 0.02386932 0.02404706 0.02769274 0.05804539 0.00732665\n",
" 0.03868372 0.00513051 0.00172865 0.00353938 0.03076019 0.01643795\n",
" 0.00458631 0.01817048 0.08667591 0.01706166 0.00637188 0.0930438\n",
" 0.01506354 0.02787184 0.0002926 0.09398153 0.00570321 0.03511387\n",
" 0.01075018 0.00170809 0.00943959 0.02654585 0.00399972 0.03057995\n",
" 0.00211418 0.00685287 0.02330799 0.04191323]\n"
]
}
],
@@ -1669,15 +1669,15 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[ 1.82079885 2.45560415 -4.73595198 14.38102552 -7.04838148]\n",
"[ 2.04641529 -0.77354301 7.7511273 -3.70241548 1.69515531]\n",
"Training R2\n",
"0.9952183728736417\n",
"0.9958589197366403\n",
"Training MSE\n",
"0.009338082195270294\n",
"0.008560581831215528\n",
"Test R2\n",
"0.9969461173312454\n",
"0.997252717901263\n",
"Test MSE\n",
"0.008043811612683473\n"
"0.00683875021199284\n"
]
}
],
doc/LectureNotes/_build/jupyter_execute/week37.ipynb
+63 -39
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/jupyter_execute/week37_121_0.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 22 KiB

doc/LectureNotes/_build/jupyter_execute/week37_139_6.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

doc/LectureNotes/_build/jupyter_execute/week37_148_11.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

doc/LectureNotes/gaussian.pdf
BIN
View File
Binary file not shown.