lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-10-08 17:04:26 +02:00
parent 87d1e4455c
commit dbbebedb06
114 changed files with 3736 additions and 1751 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter10.html
+476 -56
View File
@@ -308,6 +308,21 @@ const thebe_selector_output = ".output, .cell_output"
Week 39: Optimization and Gradient Methods
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week40.html">
Week 40: Gradient descent methods (continued) and start Neural networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek41.html">
Exercises week 41
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week41.html">
Week 41 Neural networks and constructing a neural network code
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -320,6 +335,11 @@ const thebe_selector_output = ".output, .cell_output"
Project 1 on Machine Learning, deadline October 7 (midnight), 2024
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="project2.html">
Project 2 on Machine Learning, deadline November 4 (Midnight)
</a>
</li>
</ul>
</div>
@@ -1353,7 +1373,7 @@ the <em>Hadamard product</em>, meaning element-wise multiplication.</p>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Old accuracy on training data: 0.1440501043841336
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1687,7 +1707,7 @@ Lambda = 10.0
Accuracy score on test set: 0.19166666666666668
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1696,7 +1716,7 @@ Lambda = 1e-05
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1705,7 +1725,7 @@ Lambda = 0.0001
Accuracy score on test set: 0.08611111111111111
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1714,7 +1734,7 @@ Lambda = 0.001
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1723,7 +1743,7 @@ Lambda = 0.01
Accuracy score on test set: 0.08888888888888889
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1732,7 +1752,7 @@ Lambda = 0.1
Accuracy score on test set: 0.08611111111111111
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1741,7 +1761,7 @@ Lambda = 1.0
Accuracy score on test set: 0.08888888888888889
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1750,11 +1770,11 @@ Lambda = 10.0
Accuracy score on test set: 0.09166666666666666
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1763,11 +1783,11 @@ Lambda = 1e-05
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1776,11 +1796,11 @@ Lambda = 0.0001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1789,11 +1809,11 @@ Lambda = 0.001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1802,11 +1822,11 @@ Lambda = 0.01
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1815,7 +1835,7 @@ Lambda = 0.1
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
@@ -1824,11 +1844,11 @@ Lambda = 1.0
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1837,11 +1857,11 @@ Lambda = 10.0
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
@@ -1850,37 +1870,82 @@ Lambda = 1e-05
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/953065564.py:4: RuntimeWarning: overflow encountered in exp
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:43: RuntimeWarning: overflow encountered in exp
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_95354/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">KeyboardInterrupt</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="n">Cell</span> <span class="n">In</span><span class="p">[</span><span class="mi">8</span><span class="p">],</span> <span class="n">line</span> <span class="mi">11</span>
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">for</span> <span class="n">j</span><span class="p">,</span> <span class="n">lmbd</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">lmbd_vals</span><span class="p">):</span>
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">dnn</span> <span class="o">=</span> <span class="n">NeuralNetwork</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">Y_train_onehot</span><span class="p">,</span> <span class="n">eta</span><span class="o">=</span><span class="n">eta</span><span class="p">,</span> <span class="n">lmbd</span><span class="o">=</span><span class="n">lmbd</span><span class="p">,</span> <span class="n">epochs</span><span class="o">=</span><span class="n">epochs</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span>
<span class="g g-Whitespace"> </span><span class="mi">10</span> <span class="n">n_hidden_neurons</span><span class="o">=</span><span class="n">n_hidden_neurons</span><span class="p">,</span> <span class="n">n_categories</span><span class="o">=</span><span class="n">n_categories</span><span class="p">)</span>
<span class="ne">---&gt; </span><span class="mi">11</span> <span class="n">dnn</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>
<span class="g g-Whitespace"> </span><span class="mi">13</span> <span class="n">DNN_numpy</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="n">dnn</span>
<span class="g g-Whitespace"> </span><span class="mi">15</span> <span class="n">test_predict</span> <span class="o">=</span> <span class="n">dnn</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="nn">Cell In[6], line 99,</span> in <span class="ni">NeuralNetwork.train</span><span class="nt">(self)</span>
<span class="g g-Whitespace"> </span><span class="mi">96</span> <span class="bp">self</span><span class="o">.</span><span class="n">Y_data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">Y_data_full</span><span class="p">[</span><span class="n">chosen_datapoints</span><span class="p">]</span>
<span class="g g-Whitespace"> </span><span class="mi">98</span> <span class="bp">self</span><span class="o">.</span><span class="n">feed_forward</span><span class="p">()</span>
<span class="ne">---&gt; </span><span class="mi">99</span> <span class="bp">self</span><span class="o">.</span><span class="n">backpropagation</span><span class="p">()</span>
<span class="nn">Cell In[6], line 64,</span> in <span class="ni">NeuralNetwork.backpropagation</span><span class="nt">(self)</span>
<span class="g g-Whitespace"> </span><span class="mi">61</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_weights_gradient</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">a_h</span><span class="o">.</span><span class="n">T</span><span class="p">,</span> <span class="n">error_output</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">62</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_bias_gradient</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">error_output</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="ne">---&gt; </span><span class="mi">64</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_weights_gradient</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">X_data</span><span class="o">.</span><span class="n">T</span><span class="p">,</span> <span class="n">error_hidden</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">65</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_bias_gradient</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">error_hidden</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">67</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">lmbd</span> <span class="o">&gt;</span> <span class="mf">0.0</span><span class="p">:</span>
<span class="ne">KeyboardInterrupt</span>:
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.0001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.001
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.01
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.1
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 1.0
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:43: RuntimeWarning: overflow encountered in exp
exp_term = np.exp(self.z_o)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 10.0
Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
</div>
@@ -1926,6 +1991,22 @@ Accuracy score on test set: 0.07777777777777778
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_22411/953065564.py:4: RuntimeWarning: overflow encountered in exp
return 1/(1 + np.exp(-x))
</pre></div>
</div>
<img alt="_images/chapter10_59_1.png" src="_images/chapter10_59_1.png" />
<img alt="_images/chapter10_59_2.png" src="_images/chapter10_59_2.png" />
</div>
</div>
</div>
<div class="section" id="scikit-learn-implementation">
@@ -1961,6 +2042,333 @@ performance overall.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 1e-05
Accuracy score on test set: 0.18333333333333332
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 0.0001
Accuracy score on test set: 0.18611111111111112
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 0.001
Accuracy score on test set: 0.13055555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 0.01
Accuracy score on test set: 0.24444444444444444
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 0.1
Accuracy score on test set: 0.23333333333333334
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 1.0
Accuracy score on test set: 0.12777777777777777
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
Lambda = 10.0
Accuracy score on test set: 0.1527777777777778
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 1e-05
Accuracy score on test set: 0.9111111111111111
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 0.0001
Accuracy score on test set: 0.8888888888888888
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 0.001
Accuracy score on test set: 0.8722222222222222
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 0.01
Accuracy score on test set: 0.8305555555555556
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 0.1
Accuracy score on test set: 0.8888888888888888
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 1.0
Accuracy score on test set: 0.8805555555555555
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
Lambda = 10.0
Accuracy score on test set: 0.8944444444444445
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 1e-05
Accuracy score on test set: 0.975
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 0.0001
Accuracy score on test set: 0.9777777777777777
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 0.001
Accuracy score on test set: 0.9805555555555555
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 0.01
Accuracy score on test set: 0.9861111111111112
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 0.1
Accuracy score on test set: 0.9805555555555555
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 1.0
Accuracy score on test set: 0.9777777777777777
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
Lambda = 10.0
Accuracy score on test set: 0.9444444444444444
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn&#39;t converged yet.
warnings.warn(
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 1e-05
Accuracy score on test set: 0.9861111111111112
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 0.0001
Accuracy score on test set: 0.9888888888888889
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 0.001
Accuracy score on test set: 0.9888888888888889
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 0.01
Accuracy score on test set: 0.9861111111111112
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 0.1
Accuracy score on test set: 0.9888888888888889
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
Lambda = 1.0
Accuracy score on test set: 0.9722222222222222
Learning rate = 0.01
Lambda = 10.0
Accuracy score on test set: 0.9527777777777777
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 1e-05
Accuracy score on test set: 0.9027777777777778
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 0.0001
Accuracy score on test set: 0.8583333333333333
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 0.001
Accuracy score on test set: 0.8722222222222222
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 0.01
Accuracy score on test set: 0.9055555555555556
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 0.1
Accuracy score on test set: 0.8805555555555555
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 1.0
Accuracy score on test set: 0.8722222222222222
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
Lambda = 10.0
Accuracy score on test set: 0.8666666666666667
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
Lambda = 1e-05
Accuracy score on test set: 0.08611111111111111
Learning rate = 1.0
Lambda = 0.0001
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
Lambda = 0.001
Accuracy score on test set: 0.10555555555555556
Learning rate = 1.0
Lambda = 0.01
Accuracy score on test set: 0.17777777777777778
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
Lambda = 0.1
Accuracy score on test set: 0.08333333333333333
Learning rate = 1.0
Lambda = 1.0
Accuracy score on test set: 0.08888888888888889
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
Lambda = 10.0
Accuracy score on test set: 0.09444444444444444
Learning rate = 10.0
Lambda = 1e-05
Accuracy score on test set: 0.17222222222222222
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.0001
Accuracy score on test set: 0.11666666666666667
Learning rate = 10.0
Lambda = 0.001
Accuracy score on test set: 0.10555555555555556
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 0.01
Accuracy score on test set: 0.1388888888888889
Learning rate = 10.0
Lambda = 0.1
Accuracy score on test set: 0.11388888888888889
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
Lambda = 1.0
Accuracy score on test set: 0.10555555555555556
Learning rate = 10.0
Lambda = 10.0
Accuracy score on test set: 0.09444444444444444
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="id1">
@@ -2004,6 +2412,10 @@ performance overall.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/chapter10_63_0.png" src="_images/chapter10_63_0.png" />
<img alt="_images/chapter10_63_1.png" src="_images/chapter10_63_1.png" />
</div>
</div>
</div>
<div class="section" id="building-neural-networks-in-tensorflow-and-keras">
@@ -2042,6 +2454,14 @@ and/or if you use <strong>anaconda</strong>, just write (or install from the gra
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span> <span class="n">Cell</span> <span class="n">In</span><span class="p">[</span><span class="mi">12</span><span class="p">],</span> <span class="n">line</span> <span class="mi">1</span>
<span class="n">conda</span> <span class="n">create</span> <span class="o">-</span><span class="n">n</span> <span class="n">tf</span> <span class="n">tensorflow</span>
<span class="o">^</span>
<span class="ne">SyntaxError</span>: invalid syntax
</pre></div>
</div>
</div>
</div>
<p>To install the current release of GPU TensorFlow</p>
<div class="cell docutils container">