last update, perhaps
This commit is contained in:
Morten Hjorth-Jensen
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 49 KiB After Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 35 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 34 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.9 KiB After Width: | Height: | Size: 9.9 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 19 KiB After Width: | Height: | Size: 19 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
File diff suppressed because it is too large
Load Diff
@@ -1066,13 +1066,13 @@ example of the functionality of <strong>Scikit-Learn</strong>.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The intercept alpha:
|
||||
[2.07549007]
|
||||
[2.06336262]
|
||||
Coefficient beta :
|
||||
[[5.14029264]]
|
||||
Mean squared error: 0.23
|
||||
Variance score: 0.89
|
||||
[[4.80453713]]
|
||||
Mean squared error: 0.19
|
||||
Variance score: 0.90
|
||||
Mean squared log error: 0.01
|
||||
Mean absolute error: 0.38
|
||||
Mean absolute error: 0.35
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_19_1.png" src="_images/chapter1_19_1.png" />
|
||||
@@ -1172,7 +1172,7 @@ a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we s
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter1_33_0.png" src="_images/chapter1_33_0.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.004999999999999987
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.004999999999999996
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -1383,7 +1383,7 @@ the <em>Hadamard product</em>, meaning element-wise multiplication.</p>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Old accuracy on training data: 0.1440501043841336
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1717,7 +1717,7 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.19166666666666668
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1726,7 +1726,7 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1735,7 +1735,7 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1744,7 +1744,7 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1753,7 +1753,7 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1762,7 +1762,7 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1771,34 +1771,191 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57122/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
|
||||
<span class="ne">KeyboardInterrupt</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
|
||||
<span class="n">Cell</span> <span class="n">In</span><span class="p">[</span><span class="mi">8</span><span class="p">],</span> <span class="n">line</span> <span class="mi">11</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">for</span> <span class="n">j</span><span class="p">,</span> <span class="n">lmbd</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">lmbd_vals</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">dnn</span> <span class="o">=</span> <span class="n">NeuralNetwork</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">Y_train_onehot</span><span class="p">,</span> <span class="n">eta</span><span class="o">=</span><span class="n">eta</span><span class="p">,</span> <span class="n">lmbd</span><span class="o">=</span><span class="n">lmbd</span><span class="p">,</span> <span class="n">epochs</span><span class="o">=</span><span class="n">epochs</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">10</span> <span class="n">n_hidden_neurons</span><span class="o">=</span><span class="n">n_hidden_neurons</span><span class="p">,</span> <span class="n">n_categories</span><span class="o">=</span><span class="n">n_categories</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">11</span> <span class="n">dnn</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">13</span> <span class="n">DNN_numpy</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="n">dnn</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">15</span> <span class="n">test_predict</span> <span class="o">=</span> <span class="n">dnn</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">Cell In[6], line 98,</span> in <span class="ni">NeuralNetwork.train</span><span class="nt">(self)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">95</span> <span class="bp">self</span><span class="o">.</span><span class="n">X_data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">X_data_full</span><span class="p">[</span><span class="n">chosen_datapoints</span><span class="p">]</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">96</span> <span class="bp">self</span><span class="o">.</span><span class="n">Y_data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">Y_data_full</span><span class="p">[</span><span class="n">chosen_datapoints</span><span class="p">]</span>
|
||||
<span class="ne">---> </span><span class="mi">98</span> <span class="bp">self</span><span class="o">.</span><span class="n">feed_forward</span><span class="p">()</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">99</span> <span class="bp">self</span><span class="o">.</span><span class="n">backpropagation</span><span class="p">()</span>
|
||||
|
||||
<span class="nn">Cell In[6], line 38,</span> in <span class="ni">NeuralNetwork.feed_forward</span><span class="nt">(self)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="k">def</span> <span class="nf">feed_forward</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">37</span> <span class="c1"># feed-forward for training</span>
|
||||
<span class="ne">---> </span><span class="mi">38</span> <span class="bp">self</span><span class="o">.</span><span class="n">z_h</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">X_data</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_weights</span><span class="p">)</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_bias</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="bp">self</span><span class="o">.</span><span class="n">a_h</span> <span class="o">=</span> <span class="n">sigmoid</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">z_h</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">41</span> <span class="bp">self</span><span class="o">.</span><span class="n">z_o</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">a_h</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_weights</span><span class="p">)</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_bias</span>
|
||||
|
||||
<span class="ne">KeyboardInterrupt</span>:
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.09166666666666666
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1844,6 +2001,22 @@ Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59478/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter10_59_1.png" src="_images/chapter10_59_1.png" />
|
||||
<img alt="_images/chapter10_59_2.png" src="_images/chapter10_59_2.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="scikit-learn-implementation">
|
||||
@@ -1879,6 +2052,327 @@ performance overall.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.18333333333333332
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.18611111111111112
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.13055555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.24444444444444444
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.23333333333333334
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.12777777777777777
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1e-05
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.1527777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.9111111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.8888888888888888
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.8305555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.8888888888888888
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8805555555555555
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.0001
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.8944444444444445
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.975
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.9777777777777777
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.9805555555555555
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.9861111111111112
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.9805555555555555
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.9777777777777777
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.001
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.9444444444444444
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.9861111111111112
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.9888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.9888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.9861111111111112
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.9888888888888889
|
||||
|
||||
Learning rate = 0.01
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.9722222222222222
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.9527777777777777
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.9027777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.8583333333333333
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.9055555555555556
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.8805555555555555
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.8666666666666667
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.17777777777777778
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.08333333333333333
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.09444444444444444
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.17222222222222222
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.11666666666666667
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.1388888888888889
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.11388888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.09444444444444444
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="id1">
|
||||
@@ -1922,6 +2416,10 @@ performance overall.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter10_63_0.png" src="_images/chapter10_63_0.png" />
|
||||
<img alt="_images/chapter10_63_1.png" src="_images/chapter10_63_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="building-neural-networks-in-tensorflow-and-keras">
|
||||
@@ -1960,6 +2458,14 @@ and/or if you use <strong>anaconda</strong>, just write (or install from the gra
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span> <span class="n">Cell</span> <span class="n">In</span><span class="p">[</span><span class="mi">12</span><span class="p">],</span> <span class="n">line</span> <span class="mi">1</span>
|
||||
<span class="n">conda</span> <span class="n">create</span> <span class="o">-</span><span class="n">n</span> <span class="n">tf</span> <span class="n">tensorflow</span>
|
||||
<span class="o">^</span>
|
||||
<span class="ne">SyntaxError</span>: invalid syntax
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>To install the current release of GPU TensorFlow</p>
|
||||
<div class="cell docutils container">
|
||||
|
||||
@@ -2709,63 +2709,50 @@ Using TensorFlow results in a much better execution time. Try it!</p>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">19</span> <span class="n">x</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">args</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">argnum</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">20</span> <span class="k">return</span> <span class="n">unary_operator</span><span class="p">(</span><span class="n">unary_f</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="o">*</span><span class="n">nary_op_args</span><span class="p">,</span> <span class="o">**</span><span class="n">nary_op_kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:64,</span> in <span class="ni">jacobian</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">62</span> <span class="n">jacobian_shape</span> <span class="o">=</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">shape</span> <span class="o">+</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">shape</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">63</span> <span class="n">grads</span> <span class="o">=</span> <span class="nb">map</span><span class="p">(</span><span class="n">vjp</span><span class="p">,</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">standard_basis</span><span class="p">())</span>
|
||||
<span class="ne">---> </span><span class="mi">64</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">stack</span><span class="p">(</span><span class="n">grads</span><span class="p">),</span> <span class="n">jacobian_shape</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:60,</span> in <span class="ni">jacobian</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">50</span> <span class="nd">@unary_to_nary</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">51</span> <span class="k">def</span> <span class="nf">jacobian</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">52</span><span class="w"> </span><span class="sd">"""</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">53</span><span class="sd"> Returns a function which computes the Jacobian of `fun` with respect to</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">54</span><span class="sd"> positional argument number `argnum`, which must be a scalar or array. Unlike</span>
|
||||
<span class="sd"> (...)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">58</span><span class="sd"> (out1, out2, ...) then the Jacobian has shape (out1, out2, ..., in1, in2, ...).</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">59</span><span class="sd"> """</span>
|
||||
<span class="ne">---> </span><span class="mi">60</span> <span class="n">vjp</span><span class="p">,</span> <span class="n">ans</span> <span class="o">=</span> <span class="n">_make_vjp</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">61</span> <span class="n">ans_vspace</span> <span class="o">=</span> <span class="n">vspace</span><span class="p">(</span><span class="n">ans</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">62</span> <span class="n">jacobian_shape</span> <span class="o">=</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">shape</span> <span class="o">+</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">shape</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:88,</span> in <span class="ni">stack</span><span class="nt">(arrays, axis)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">83</span> <span class="k">def</span> <span class="nf">stack</span><span class="p">(</span><span class="n">arrays</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">84</span> <span class="c1"># this code is basically copied from numpy/core/shape_base.py's stack</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">85</span> <span class="c1"># we need it here because we want to re-implement stack in terms of the</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">86</span> <span class="c1"># primitives defined in this file</span>
|
||||
<span class="ne">---> </span><span class="mi">88</span> <span class="n">arrays</span> <span class="o">=</span> <span class="p">[</span><span class="n">array</span><span class="p">(</span><span class="n">arr</span><span class="p">)</span> <span class="k">for</span> <span class="n">arr</span> <span class="ow">in</span> <span class="n">arrays</span><span class="p">]</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">89</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">arrays</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">90</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'need at least one array to stack'</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:10,</span> in <span class="ni">make_vjp</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">def</span> <span class="nf">make_vjp</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">start_node</span> <span class="o">=</span> <span class="n">VJPNode</span><span class="o">.</span><span class="n">new_root</span><span class="p">()</span>
|
||||
<span class="ne">---> </span><span class="mi">10</span> <span class="n">end_value</span><span class="p">,</span> <span class="n">end_node</span> <span class="o">=</span> <span class="n">trace</span><span class="p">(</span><span class="n">start_node</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">11</span> <span class="k">if</span> <span class="n">end_node</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">12</span> <span class="k">def</span> <span class="nf">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">):</span> <span class="k">return</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">zeros</span><span class="p">()</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:88,</span> in <span class="ni"><listcomp></span><span class="nt">(.0)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">83</span> <span class="k">def</span> <span class="nf">stack</span><span class="p">(</span><span class="n">arrays</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">84</span> <span class="c1"># this code is basically copied from numpy/core/shape_base.py's stack</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">85</span> <span class="c1"># we need it here because we want to re-implement stack in terms of the</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">86</span> <span class="c1"># primitives defined in this file</span>
|
||||
<span class="ne">---> </span><span class="mi">88</span> <span class="n">arrays</span> <span class="o">=</span> <span class="p">[</span><span class="n">array</span><span class="p">(</span><span class="n">arr</span><span class="p">)</span> <span class="k">for</span> <span class="n">arr</span> <span class="ow">in</span> <span class="n">arrays</span><span class="p">]</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">89</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">arrays</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">90</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'need at least one array to stack'</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:10,</span> in <span class="ni">trace</span><span class="nt">(start_node, fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">with</span> <span class="n">trace_stack</span><span class="o">.</span><span class="n">new_trace</span><span class="p">()</span> <span class="k">as</span> <span class="n">t</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">start_box</span> <span class="o">=</span> <span class="n">new_box</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">start_node</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">10</span> <span class="n">end_box</span> <span class="o">=</span> <span class="n">fun</span><span class="p">(</span><span class="n">start_box</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">11</span> <span class="k">if</span> <span class="n">isbox</span><span class="p">(</span><span class="n">end_box</span><span class="p">)</span> <span class="ow">and</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_trace</span> <span class="o">==</span> <span class="n">start_box</span><span class="o">.</span><span class="n">_trace</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">12</span> <span class="k">return</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_value</span><span class="p">,</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_node</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:14,</span> in <span class="ni">make_vjp.<locals>.vjp</span><span class="nt">(g)</span>
|
||||
<span class="ne">---> </span><span class="mi">14</span> <span class="k">def</span> <span class="nf">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">):</span> <span class="k">return</span> <span class="n">backward_pass</span><span class="p">(</span><span class="n">g</span><span class="p">,</span> <span class="n">end_node</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:15,</span> in <span class="ni">unary_to_nary.<locals>.nary_operator.<locals>.nary_f.<locals>.unary_f</span><span class="nt">(x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">13</span> <span class="k">else</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">14</span> <span class="n">subargs</span> <span class="o">=</span> <span class="n">subvals</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="nb">zip</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="n">x</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">15</span> <span class="k">return</span> <span class="n">fun</span><span class="p">(</span><span class="o">*</span><span class="n">subargs</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:21,</span> in <span class="ni">backward_pass</span><span class="nt">(g, end_node)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">19</span> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">toposort</span><span class="p">(</span><span class="n">end_node</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">20</span> <span class="n">outgrad</span> <span class="o">=</span> <span class="n">outgrads</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">21</span> <span class="n">ingrads</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">vjp</span><span class="p">(</span><span class="n">outgrad</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">22</span> <span class="k">for</span> <span class="n">parent</span><span class="p">,</span> <span class="n">ingrad</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parents</span><span class="p">,</span> <span class="n">ingrads</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">23</span> <span class="n">outgrads</span><span class="p">[</span><span class="n">parent</span><span class="p">]</span> <span class="o">=</span> <span class="n">add_outgrads</span><span class="p">(</span><span class="n">outgrads</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">parent</span><span class="p">),</span> <span class="n">ingrad</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:15,</span> in <span class="ni">unary_to_nary.<locals>.nary_operator.<locals>.nary_f.<locals>.unary_f</span><span class="nt">(x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">13</span> <span class="k">else</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">14</span> <span class="n">subargs</span> <span class="o">=</span> <span class="n">subvals</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="nb">zip</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="n">x</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">15</span> <span class="k">return</span> <span class="n">fun</span><span class="p">(</span><span class="o">*</span><span class="n">subargs</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:67,</span> in <span class="ni">defvjp.<locals>.vjp_argnums.<locals>.<lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">64</span> <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">(</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">65</span> <span class="s2">"VJP of </span><span class="si">{}</span><span class="s2"> wrt argnum 0 not defined"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun</span><span class="o">.</span><span class="vm">__name__</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">66</span> <span class="n">vjp</span> <span class="o">=</span> <span class="n">vjpfun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">67</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">),)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">68</span> <span class="k">elif</span> <span class="n">L</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">69</span> <span class="n">argnum_0</span><span class="p">,</span> <span class="n">argnum_1</span> <span class="o">=</span> <span class="n">argnums</span>
|
||||
<span class="nn">Cell In[9], line 61,</span> in <span class="ni">g_trial</span><span class="nt">(point, P)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">59</span> <span class="k">def</span> <span class="nf">g_trial</span><span class="p">(</span><span class="n">point</span><span class="p">,</span><span class="n">P</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">60</span> <span class="n">x</span><span class="p">,</span><span class="n">t</span> <span class="o">=</span> <span class="n">point</span>
|
||||
<span class="ne">---> </span><span class="mi">61</span> <span class="k">return</span> <span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="n">t</span><span class="p">)</span><span class="o">*</span><span class="n">u</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">+</span> <span class="n">x</span><span class="o">*</span><span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="n">x</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="o">*</span><span class="n">deep_neural_network</span><span class="p">(</span><span class="n">P</span><span class="p">,</span><span class="n">point</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:423,</span> in <span class="ni">matmul_vjp_1.<locals>.<lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">421</span> <span class="n">A_ndim</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">ndim</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">422</span> <span class="n">B_meta</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">metadata</span><span class="p">(</span><span class="n">B</span><span class="p">)</span>
|
||||
<span class="ne">--> </span><span class="mi">423</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">matmul_adjoint_1</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">g</span><span class="p">,</span> <span class="n">A_ndim</span><span class="p">,</span> <span class="n">B_meta</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:413,</span> in <span class="ni">matmul_adjoint_1</span><span class="nt">(A, G, A_ndim, B_meta)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">411</span> <span class="k">if</span> <span class="n">B_is_vec</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">412</span> <span class="n">result</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">squeeze</span><span class="p">(</span><span class="n">result</span><span class="p">,</span> <span class="n">anp</span><span class="o">.</span><span class="n">ndim</span><span class="p">(</span><span class="n">G</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="ne">--> </span><span class="mi">413</span> <span class="k">return</span> <span class="n">unbroadcast</span><span class="p">(</span><span class="n">result</span><span class="p">,</span> <span class="n">B_meta</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:653,</span> in <span class="ni">unbroadcast</span><span class="nt">(x, target_meta, broadcast_idx)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">651</span> <span class="k">for</span> <span class="n">axis</span><span class="p">,</span> <span class="n">size</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">target_shape</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">652</span> <span class="k">if</span> <span class="n">size</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
<span class="ne">--> </span><span class="mi">653</span> <span class="n">x</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="n">axis</span><span class="p">,</span> <span class="n">keepdims</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">654</span> <span class="k">if</span> <span class="n">anp</span><span class="o">.</span><span class="n">iscomplexobj</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">target_iscomplex</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">655</span> <span class="n">x</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">real</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:34,</span> in <span class="ni">ArrayBox.__rsub__</span><span class="nt">(self, other)</span>
|
||||
<span class="ne">---> </span><span class="mi">34</span> <span class="k">def</span> <span class="fm">__rsub__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span> <span class="k">return</span> <span class="n">anp</span><span class="o">.</span><span class="n">subtract</span><span class="p">(</span><span class="n">other</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:45,</span> in <span class="ni">primitive.<locals>.f_wrapped</span><span class="nt">(*args, **kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">43</span> <span class="n">argnums</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">argnum</span> <span class="k">for</span> <span class="n">argnum</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">boxed_args</span><span class="p">)</span>
|
||||
@@ -2788,12 +2775,33 @@ Using TensorFlow results in a much better execution time. Try it!</p>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">67</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">),)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">68</span> <span class="k">elif</span> <span class="n">L</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:297,</span> in <span class="ni">grad_np_sum</span><span class="nt">(ans, x, axis, keepdims, dtype)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">294</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">anp</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">g</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="n">broadcast_axes</span><span class="p">,</span> <span class="n">keepdims</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">295</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">broadcast_to</span><span class="p">,</span> <span class="n">grad_broadcast_to</span><span class="p">)</span>
|
||||
<span class="ne">--> </span><span class="mi">297</span> <span class="k">def</span> <span class="nf">grad_np_sum</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">keepdims</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">298</span> <span class="n">shape</span><span class="p">,</span> <span class="n">dtype</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">shape</span><span class="p">(</span><span class="n">x</span><span class="p">),</span> <span class="n">anp</span><span class="o">.</span><span class="n">result_type</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">299</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">repeat_to_match_shape</span><span class="p">(</span><span class="n">g</span><span class="p">,</span> <span class="n">shape</span><span class="p">,</span> <span class="n">dtype</span><span class="p">,</span> <span class="n">axis</span><span class="p">,</span> <span class="n">keepdims</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:37,</span> in <span class="ni"><lambda></span><span class="nt">(ans, x, y)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">32</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">add</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">33</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">34</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">multiply</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">y</span> <span class="o">*</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">subtract</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="ne">---> </span><span class="mi">37</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span><span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">38</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">divide</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">y</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span> <span class="n">g</span> <span class="o">*</span> <span class="n">x</span> <span class="o">/</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">40</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">maximum</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="n">balanced_eq</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ans</span><span class="p">,</span> <span class="n">y</span><span class="p">)),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">41</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="n">balanced_eq</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">)))</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:659,</span> in <span class="ni">unbroadcast_f</span><span class="nt">(target, f)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">658</span> <span class="k">def</span> <span class="nf">unbroadcast_f</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">f</span><span class="p">):</span>
|
||||
<span class="ne">--> </span><span class="mi">659</span> <span class="n">target_meta</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">metadata</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">660</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">unbroadcast</span><span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">target_meta</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:61,</span> in <span class="ni">notrace_primitive.<locals>.f_wrapped</span><span class="nt">(*args, **kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">58</span> <span class="nd">@wraps</span><span class="p">(</span><span class="n">f_raw</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">59</span> <span class="k">def</span> <span class="nf">f_wrapped</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">60</span> <span class="n">argvals</span> <span class="o">=</span> <span class="nb">map</span><span class="p">(</span><span class="n">getval</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">61</span> <span class="k">return</span> <span class="n">f_raw</span><span class="p">(</span><span class="o">*</span><span class="n">argvals</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:148,</span> in <span class="ni">metadata</span><span class="nt">(A)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">146</span> <span class="nd">@notrace_primitive</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">147</span> <span class="k">def</span> <span class="nf">metadata</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
|
||||
<span class="ne">--> </span><span class="mi">148</span> <span class="k">return</span> <span class="n">_np</span><span class="o">.</span><span class="n">shape</span><span class="p">(</span><span class="n">A</span><span class="p">),</span> <span class="n">_np</span><span class="o">.</span><span class="n">ndim</span><span class="p">(</span><span class="n">A</span><span class="p">),</span> <span class="n">_np</span><span class="o">.</span><span class="n">result_type</span><span class="p">(</span><span class="n">A</span><span class="p">),</span> <span class="n">_np</span><span class="o">.</span><span class="n">iscomplexobj</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
|
||||
|
||||
<span class="ne">KeyboardInterrupt</span>:
|
||||
</pre></div>
|
||||
|
||||
@@ -1315,10 +1315,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.04718566894028431
|
||||
4.11080997912276
|
||||
[[ 1.10517643 3.48455788]
|
||||
[ 3.48455788 12.00216162]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.014394967608286841
|
||||
4.011594819155615
|
||||
[[ 1.24508783 3.8595836 ]
|
||||
[ 3.8595836 12.92663007]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1355,10 +1355,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07836997022107646
|
||||
1.1378267322316808
|
||||
[[1. 0.63980097]
|
||||
[0.63980097 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07630326327869198
|
||||
1.6893421391051477
|
||||
[[1. 0.6373454]
|
||||
[0.6373454 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1388,30 +1388,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 1.34931214 3.06139439]
|
||||
[-0.44476964 -2.60794187]
|
||||
[ 0.02225493 0.16388664]
|
||||
[-1.91193672 -3.82324216]
|
||||
[-0.2044881 -1.56027537]
|
||||
[-1.15572395 -3.25982474]
|
||||
[ 0.94217756 1.49888671]
|
||||
[ 0.28472162 2.92474572]
|
||||
[ 2.38943 7.14118216]
|
||||
[-1.27097785 -3.5388115 ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.20396326 0.93053605]
|
||||
[ 0.51936974 2.03440431]
|
||||
[-0.53851084 -1.2527027 ]
|
||||
[-0.50429483 0.72966563]
|
||||
[ 0.71314288 1.60421343]
|
||||
[-0.29995377 -2.31356849]
|
||||
[-0.14484831 -2.19932442]
|
||||
[-0.00570826 -0.32643011]
|
||||
[-0.22821607 0.56775345]
|
||||
[ 0.28505619 0.22545286]]
|
||||
0 1
|
||||
0 1.349312 3.061394
|
||||
1 -0.444770 -2.607942
|
||||
2 0.022255 0.163887
|
||||
3 -1.911937 -3.823242
|
||||
4 -0.204488 -1.560275
|
||||
5 -1.155724 -3.259825
|
||||
6 0.942178 1.498887
|
||||
7 0.284722 2.924746
|
||||
8 2.389430 7.141182
|
||||
9 -1.270978 -3.538811
|
||||
0 1
|
||||
0 1.000000 0.950873
|
||||
1 0.950873 1.000000
|
||||
0 0.203963 0.930536
|
||||
1 0.519370 2.034404
|
||||
2 -0.538511 -1.252703
|
||||
3 -0.504295 0.729666
|
||||
4 0.713143 1.604213
|
||||
5 -0.299954 -2.313568
|
||||
6 -0.144848 -2.199324
|
||||
7 -0.005708 -0.326430
|
||||
8 -0.228216 0.567753
|
||||
9 0.285056 0.225453
|
||||
0 1
|
||||
0 1.0000 0.6373
|
||||
1 0.6373 1.0000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1468,37 +1468,40 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.074334 0.080585 0.077061 0.078751 0.080220 0.070657 0.071406
|
||||
2 0.0 0.080585 0.088425 0.082009 0.084289 0.086338 0.074009 0.075052
|
||||
3 0.0 0.077061 0.082009 0.085147 0.086339 0.087297 0.081324 0.081796
|
||||
4 0.0 0.078751 0.084289 0.086339 0.087789 0.089007 0.081926 0.082537
|
||||
5 0.0 0.080220 0.086338 0.087297 0.089007 0.090492 0.082307 0.083061
|
||||
6 0.0 0.070657 0.074009 0.081324 0.081926 0.082307 0.079874 0.080032
|
||||
7 0.0 0.071406 0.075052 0.081796 0.082537 0.083061 0.080032 0.080271
|
||||
8 0.0 0.072148 0.076101 0.082240 0.083128 0.083801 0.080150 0.080474
|
||||
9 0.0 0.072902 0.077180 0.082670 0.083714 0.084548 0.080237 0.080651
|
||||
10 0.0 0.063646 0.065859 0.075320 0.075498 0.075472 0.075478 0.075409
|
||||
11 0.0 0.064071 0.066452 0.075576 0.075838 0.075897 0.075542 0.075525
|
||||
12 0.0 0.064514 0.067074 0.075838 0.076189 0.076340 0.075602 0.075639
|
||||
13 0.0 0.064980 0.067731 0.076108 0.076555 0.076803 0.075658 0.075753
|
||||
14 0.0 0.065472 0.068429 0.076389 0.076938 0.077292 0.075711 0.075868
|
||||
1 0.0 0.072835 0.076616 0.071253 0.074193 0.077337 0.063283 0.065305
|
||||
2 0.0 0.076616 0.082149 0.073794 0.077643 0.081811 0.064373 0.066929
|
||||
3 0.0 0.071253 0.073794 0.075423 0.077534 0.079745 0.070414 0.071978
|
||||
4 0.0 0.074193 0.077643 0.077534 0.080199 0.083034 0.071575 0.073483
|
||||
5 0.0 0.077337 0.081811 0.079745 0.083034 0.086576 0.072733 0.075028
|
||||
6 0.0 0.063283 0.064373 0.070414 0.071575 0.072733 0.068008 0.068984
|
||||
7 0.0 0.065305 0.066929 0.071978 0.073483 0.075028 0.068984 0.070181
|
||||
8 0.0 0.067523 0.069754 0.073667 0.075563 0.077550 0.070010 0.071459
|
||||
9 0.0 0.069964 0.072888 0.075496 0.077839 0.080330 0.071092 0.072826
|
||||
10 0.0 0.055866 0.055925 0.064324 0.064790 0.065183 0.063660 0.064182
|
||||
11 0.0 0.057298 0.057682 0.065508 0.066192 0.066830 0.064471 0.065139
|
||||
12 0.0 0.058866 0.059624 0.066786 0.067719 0.068637 0.065330 0.066162
|
||||
13 0.0 0.060590 0.061775 0.068171 0.069388 0.070625 0.066241 0.067260
|
||||
14 0.0 0.062489 0.064164 0.069675 0.071217 0.072820 0.067209 0.068440
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.072148 0.072902 0.063646 0.064071 0.064514 0.064980 0.065472
|
||||
2 0.076101 0.077180 0.065859 0.066452 0.067074 0.067731 0.068429
|
||||
3 0.082240 0.082670 0.075320 0.075576 0.075838 0.076108 0.076389
|
||||
4 0.083128 0.083714 0.075498 0.075838 0.076189 0.076555 0.076938
|
||||
5 0.083801 0.084548 0.075472 0.075897 0.076340 0.076803 0.077292
|
||||
6 0.080150 0.080237 0.075478 0.075542 0.075602 0.075658 0.075711
|
||||
7 0.080474 0.080651 0.075409 0.075525 0.075639 0.075753 0.075868
|
||||
8 0.080766 0.081038 0.075293 0.075463 0.075634 0.075809 0.075988
|
||||
9 0.081038 0.081411 0.075136 0.075363 0.075595 0.075834 0.076082
|
||||
10 0.075293 0.075136 0.072406 0.072329 0.072240 0.072140 0.072028
|
||||
11 0.075463 0.075363 0.072329 0.072286 0.072234 0.072173 0.072101
|
||||
12 0.075634 0.075595 0.072240 0.072234 0.072220 0.072199 0.072171
|
||||
13 0.075809 0.075834 0.072140 0.072173 0.072199 0.072221 0.072238
|
||||
14 0.075988 0.076082 0.072028 0.072101 0.072171 0.072238 0.072303
|
||||
1 0.067523 0.069964 0.055866 0.057298 0.058866 0.060590 0.062489
|
||||
2 0.069754 0.072888 0.055925 0.057682 0.059624 0.061775 0.064164
|
||||
3 0.073667 0.075496 0.064324 0.065508 0.066786 0.068171 0.069675
|
||||
4 0.075563 0.077839 0.064790 0.066192 0.067719 0.069388 0.071217
|
||||
5 0.077550 0.080330 0.065183 0.066830 0.068637 0.070625 0.072820
|
||||
6 0.070010 0.071092 0.063660 0.064471 0.065330 0.066241 0.067209
|
||||
7 0.071459 0.072826 0.064182 0.065139 0.066162 0.067260 0.068440
|
||||
8 0.073021 0.074713 0.064703 0.065823 0.067032 0.068341 0.069761
|
||||
9 0.074713 0.076776 0.065219 0.066524 0.067943 0.069492 0.071186
|
||||
10 0.064703 0.065219 0.060678 0.061187 0.061707 0.062241 0.062786
|
||||
11 0.065823 0.066524 0.061187 0.061793 0.062424 0.063082 0.063768
|
||||
12 0.067032 0.067943 0.061707 0.062424 0.063180 0.063978 0.064823
|
||||
13 0.068341 0.069492 0.062241 0.063082 0.063978 0.064935 0.065960
|
||||
14 0.069761 0.071186 0.062786 0.063768 0.064823 0.065960 0.067192
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1979,13 +1982,11 @@ We select values of the hyperparameter <span class="math notranslate nohighlight
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2. 2.]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Training MSE for OLS
|
||||
Training MSE for OLS
|
||||
3.0
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter2_252_2.png" src="_images/chapter2_252_2.png" />
|
||||
<img alt="_images/chapter2_252_1.png" src="_images/chapter2_252_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>We see here that we reach a plateau for the Ridge results. Writing out the coefficients <span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span>, we observe that they are getting smaller and smaller and our error stabilizes since the predicted values of <span class="math notranslate nohighlight">\(\tilde{\boldsymbol{y}}\)</span> approach zero.</p>
|
||||
|
||||
@@ -869,10 +869,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.154751 sec
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.136066 sec
|
||||
Jackknife Statistics :
|
||||
original bias std. error
|
||||
99.9896 99.9796 0.149524
|
||||
99.8762 99.8662 0.150735
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1091,7 +1091,7 @@ theorem.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
|
||||
original bias std. error
|
||||
100.307 14.9693 100.309 0.149416
|
||||
99.9623 14.9594 99.9606 0.14934
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1298,9 +1298,7 @@ Error: 0.08426840630693411
|
||||
Bias^2: 0.0796891867672603
|
||||
Var: 0.004579219539673834
|
||||
0.08426840630693411 >= 0.0796891867672603 + 0.004579219539673834 = 0.08426840630693413
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 2
|
||||
Polynomial degree: 2
|
||||
Error: 0.10398646080125035
|
||||
Bias^2: 0.1007711427354898
|
||||
Var: 0.0032153180657605116
|
||||
@@ -1327,9 +1325,7 @@ Error: 0.037813671417389005
|
||||
Bias^2: 0.033657685071527665
|
||||
Var: 0.00415598634586135
|
||||
0.037813671417389005 >= 0.033657685071527665 + 0.00415598634586135 = 0.03781367141738902
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
|
||||
Polynomial degree: 7
|
||||
Error: 0.02760977349102253
|
||||
Bias^2: 0.022999498260366312
|
||||
Var: 0.004610275230656212
|
||||
@@ -1356,9 +1352,7 @@ Error: 0.07160048164233104
|
||||
Bias^2: 0.014436800088904942
|
||||
Var: 0.05716368155342608
|
||||
0.07160048164233104 >= 0.014436800088904942 + 0.05716368155342608 = 0.07160048164233102
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 12
|
||||
Polynomial degree: 12
|
||||
Error: 0.11547777218872497
|
||||
Bias^2: 0.01628578269596628
|
||||
Var: 0.09919198949275869
|
||||
@@ -1370,7 +1364,7 @@ Var: 0.20867052175034223
|
||||
0.22842468702219465 >= 0.01975416527185249 + 0.20867052175034223 = 0.2284246870221947
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_66_6.png" src="_images/chapter3_66_6.png" />
|
||||
<img alt="_images/chapter3_66_3.png" src="_images/chapter3_66_3.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The bias-variance tradeoff summarizes the fundamental tension in
|
||||
@@ -1661,12 +1655,12 @@ Mean squared error on test data: 877.21517262
|
||||
Degree of polynomial: 23
|
||||
Mean squared error on training data: 0.00085892
|
||||
Mean squared error on test data: 5567.04664255
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 24
|
||||
Degree of polynomial: 24
|
||||
Mean squared error on training data: 0.00084707
|
||||
Mean squared error on test data: 1325.26124692
|
||||
Degree of polynomial: 25
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
|
||||
Mean squared error on training data: 0.00079125
|
||||
Mean squared error on test data: 129012.83870189
|
||||
Degree of polynomial: 26
|
||||
@@ -1675,19 +1669,19 @@ Mean squared error on test data: 18388.59354079
|
||||
Degree of polynomial: 27
|
||||
Mean squared error on training data: 0.00069123
|
||||
Mean squared error on test data: 2351.97979891
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 28
|
||||
Degree of polynomial: 28
|
||||
Mean squared error on training data: 0.00062592
|
||||
Mean squared error on test data: 3983.63037846
|
||||
Degree of polynomial: 29
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 29
|
||||
Mean squared error on training data: 0.00060704
|
||||
Mean squared error on test data: 3262.26814548
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(trainingerror), label='Training Error')
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(testerror), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1921,7 +1915,7 @@ cross-validation (LOOCV).</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2810,7 +2804,7 @@ linear system as an equation would reduce this down to
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2954,7 +2948,7 @@ with the form utilized in linear regression, viz.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2994,7 +2988,7 @@ cost function is given by</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3029,7 +3023,7 @@ cost function is given by</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57183/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59625/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3082,43 +3076,43 @@ constant as opposed to ridge and OLS. We get a sparse solution with
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00<?, ?it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00<?, ?it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:628: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
|
||||
model = cd_fast.enet_coordinate_descent(
|
||||
|
||||
10%|█████████████▍ | 1/10 [00:00<00:07, 1.14it/s]
|
||||
10%|██████████▋ | 1/10 [00:00<00:07, 1.23it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██████████████████████████▊ | 2/10 [00:01<00:06, 1.16it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|█████████████████████▍ | 2/10 [00:01<00:06, 1.15it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████▏ | 3/10 [00:02<00:05, 1.31it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████ | 3/10 [00:02<00:05, 1.34it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|█████████████████████████████████████████████████████▌ | 4/10 [00:03<00:04, 1.39it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|██████████████████████████████████████████▊ | 4/10 [00:02<00:04, 1.43it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|███████████████████████████████████████████████████████████████████ | 5/10 [00:03<00:03, 1.38it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████████████████████████████████████████████████████▌ | 5/10 [00:03<00:03, 1.54it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████▍ | 6/10 [00:04<00:02, 1.44it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████▏ | 6/10 [00:04<00:02, 1.59it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|█████████████████████████████████████████████████████████████████████████████████████████████▊ | 7/10 [00:05<00:02, 1.45it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████████▉ | 7/10 [00:04<00:01, 1.64it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 8/10 [00:05<00:01, 1.55it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|█████████████████████████████████████████████████████████████████████████████████████▌ | 8/10 [00:05<00:01, 1.64it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 9/10 [00:06<00:00, 1.54it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████▎ | 9/10 [00:06<00:00, 1.53it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:07<00:00, 1.48it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06<00:00, 1.66it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:07<00:00, 1.43it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06<00:00, 1.54it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
|
||||
|
||||
@@ -797,9 +797,9 @@ predicting the target features of query instances is as follows:</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2nd degree coefficients:
|
||||
zero power: -0.22158725223474995
|
||||
first power: 0.24121476598003452
|
||||
second power: -0.0009532583857747255
|
||||
zero power: 4.58641507967355
|
||||
first power: -0.2107546970521115
|
||||
second power: 0.0003919793592273513
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter6_1_1.png" src="_images/chapter6_1_1.png" />
|
||||
@@ -1662,11 +1662,13 @@ attributes at each step while growing the tree.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>(426, 30)
|
||||
(143, 30)
|
||||
Test set accuracy with Logistic Regression: 0.94
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy with Logistic Regression: 0.94
|
||||
Test set accuracy with SVM: 0.63
|
||||
Test set accuracy with Decision Trees: 0.90
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy with SVM: 0.63
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Test set accuracy with Decision Trees: 0.90
|
||||
Test set accuracy Logistic Regression with scaled data: 0.96
|
||||
Test set accuracy SVM with scaled data: 0.96
|
||||
Test set accuracy with Decision Trees and scaled data: 0.89
|
||||
|
||||
@@ -751,10 +751,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.1001408041761458
|
||||
4.2807716628772665
|
||||
[[ 1.15654145 3.54867722]
|
||||
[ 3.54867722 11.70485195]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.05665875086534638
|
||||
4.128393685824704
|
||||
[[ 0.93987367 2.98650457]
|
||||
[ 2.98650457 10.47544463]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -794,10 +794,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09543871010617433
|
||||
1.6888043337746685
|
||||
[[1. 0.7167077]
|
||||
[0.7167077 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.07617734331359052
|
||||
1.6957182489166325
|
||||
[[1. 0.68029423]
|
||||
[0.68029423 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -826,30 +826,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.20575734 0.01384583]
|
||||
[-0.89876098 -3.04065686]
|
||||
[-0.76289128 -3.17080691]
|
||||
[-0.0334136 0.16124569]
|
||||
[ 2.73970542 9.28885103]
|
||||
[ 0.75413023 2.98474769]
|
||||
[-1.87894459 -5.48121459]
|
||||
[-1.26814205 -2.4848097 ]
|
||||
[ 0.18114057 -0.9889962 ]
|
||||
[ 1.37293361 2.71779401]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.29087396 0.13244119]
|
||||
[-1.21617146 -2.69678073]
|
||||
[-1.37024276 -3.76728511]
|
||||
[ 0.49342785 1.50638863]
|
||||
[ 0.4155974 -0.07435812]
|
||||
[ 0.64813145 1.94455739]
|
||||
[-0.48364163 -2.62178739]
|
||||
[-0.3807176 -1.25007671]
|
||||
[ 1.73036439 5.00709577]
|
||||
[ 0.45412633 1.81980509]]
|
||||
0 1
|
||||
0 -0.205757 0.013846
|
||||
1 -0.898761 -3.040657
|
||||
2 -0.762891 -3.170807
|
||||
3 -0.033414 0.161246
|
||||
4 2.739705 9.288851
|
||||
5 0.754130 2.984748
|
||||
6 -1.878945 -5.481215
|
||||
7 -1.268142 -2.484810
|
||||
8 0.181141 -0.988996
|
||||
9 1.372934 2.717794
|
||||
0 -0.290874 0.132441
|
||||
1 -1.216171 -2.696781
|
||||
2 -1.370243 -3.767285
|
||||
3 0.493428 1.506389
|
||||
4 0.415597 -0.074358
|
||||
5 0.648131 1.944557
|
||||
6 -0.483642 -2.621787
|
||||
7 -0.380718 -1.250077
|
||||
8 1.730364 5.007096
|
||||
9 0.454126 1.819805
|
||||
0 1
|
||||
0 1.000000 0.970965
|
||||
1 0.970965 1.000000
|
||||
0 1.000000 0.961042
|
||||
1 0.961042 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -906,37 +906,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.090565 0.089065 0.093226 0.091674 0.090154 0.086287 0.084847
|
||||
2 0.0 0.089065 0.087946 0.092421 0.091068 0.089735 0.085988 0.084674
|
||||
3 0.0 0.093226 0.092421 0.102147 0.100816 0.099494 0.098144 0.096731
|
||||
4 0.0 0.091674 0.091068 0.100816 0.099622 0.098431 0.097115 0.095803
|
||||
5 0.0 0.090154 0.089735 0.099494 0.098431 0.097365 0.096077 0.094862
|
||||
6 0.0 0.086287 0.085988 0.098144 0.097115 0.096077 0.096630 0.095395
|
||||
7 0.0 0.084847 0.084674 0.096731 0.095803 0.094862 0.095395 0.094243
|
||||
8 0.0 0.083459 0.083405 0.095358 0.094527 0.093680 0.094189 0.093115
|
||||
9 0.0 0.082121 0.082180 0.094027 0.093288 0.092530 0.093011 0.092013
|
||||
10 0.0 0.078708 0.078711 0.091732 0.090935 0.090118 0.091871 0.090804
|
||||
11 0.0 0.077431 0.077523 0.090387 0.089668 0.088926 0.090626 0.089626
|
||||
12 0.0 0.076203 0.076378 0.089086 0.088441 0.087772 0.089417 0.088481
|
||||
13 0.0 0.075021 0.075274 0.087828 0.087255 0.086655 0.088242 0.087369
|
||||
14 0.0 0.073883 0.074212 0.086611 0.086107 0.085573 0.087102 0.086289
|
||||
1 0.0 0.068131 0.073423 0.069509 0.072262 0.074935 0.062568 0.064540
|
||||
2 0.0 0.073423 0.079638 0.075544 0.078741 0.081836 0.068222 0.070480
|
||||
3 0.0 0.069509 0.075544 0.075689 0.079013 0.082264 0.070991 0.073416
|
||||
4 0.0 0.072262 0.078741 0.079013 0.082596 0.086101 0.074265 0.076876
|
||||
5 0.0 0.074935 0.081836 0.082264 0.086101 0.089859 0.077488 0.080287
|
||||
6 0.0 0.062568 0.068222 0.070991 0.074265 0.077488 0.068479 0.070924
|
||||
7 0.0 0.064540 0.070480 0.073416 0.076876 0.080287 0.070924 0.073514
|
||||
8 0.0 0.066545 0.072773 0.075883 0.079535 0.083138 0.073418 0.076157
|
||||
9 0.0 0.068597 0.075114 0.078409 0.082256 0.086058 0.075974 0.078867
|
||||
10 0.0 0.055105 0.060160 0.064322 0.067367 0.070380 0.063327 0.065650
|
||||
11 0.0 0.056730 0.062004 0.066330 0.069526 0.072694 0.065375 0.067819
|
||||
12 0.0 0.058409 0.063910 0.068406 0.071759 0.075086 0.067492 0.070063
|
||||
13 0.0 0.060149 0.065884 0.070556 0.074073 0.077566 0.069686 0.072388
|
||||
14 0.0 0.061956 0.067932 0.072787 0.076473 0.080139 0.071962 0.074802
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.083459 0.082121 0.078708 0.077431 0.076203 0.075021 0.073883
|
||||
2 0.083405 0.082180 0.078711 0.077523 0.076378 0.075274 0.074212
|
||||
3 0.095358 0.094027 0.091732 0.090387 0.089086 0.087828 0.086611
|
||||
4 0.094527 0.093288 0.090935 0.089668 0.088441 0.087255 0.086107
|
||||
5 0.093680 0.092530 0.090118 0.088926 0.087772 0.086655 0.085573
|
||||
6 0.094189 0.093011 0.091871 0.090626 0.089417 0.088242 0.087102
|
||||
7 0.093115 0.092013 0.090804 0.089626 0.088481 0.087369 0.086289
|
||||
8 0.092064 0.091034 0.089755 0.088642 0.087560 0.086508 0.085486
|
||||
9 0.091034 0.090075 0.088726 0.087675 0.086653 0.085659 0.084694
|
||||
10 0.089755 0.088726 0.088455 0.087327 0.086227 0.085155 0.084112
|
||||
11 0.088642 0.087675 0.087327 0.086256 0.085212 0.084195 0.083203
|
||||
12 0.087560 0.086653 0.086227 0.085212 0.084222 0.083257 0.082316
|
||||
13 0.086508 0.085659 0.085155 0.084195 0.083257 0.082342 0.081450
|
||||
14 0.085486 0.084694 0.084112 0.083203 0.082316 0.081450 0.080605
|
||||
1 0.066545 0.068597 0.055105 0.056730 0.058409 0.060149 0.061956
|
||||
2 0.072773 0.075114 0.060160 0.062004 0.063910 0.065884 0.067932
|
||||
3 0.075883 0.078409 0.064322 0.066330 0.068406 0.070556 0.072787
|
||||
4 0.079535 0.082256 0.067367 0.069526 0.071759 0.074073 0.076473
|
||||
5 0.083138 0.086058 0.070380 0.072694 0.075086 0.077566 0.080139
|
||||
6 0.073418 0.075974 0.063327 0.065375 0.067492 0.069686 0.071962
|
||||
7 0.076157 0.078867 0.065650 0.067819 0.070063 0.072388 0.074802
|
||||
8 0.078953 0.081823 0.068024 0.070318 0.072693 0.075155 0.077711
|
||||
9 0.081823 0.084858 0.070460 0.072885 0.075396 0.078000 0.080704
|
||||
10 0.068024 0.070460 0.059480 0.061447 0.063482 0.065591 0.067780
|
||||
11 0.070318 0.072885 0.061447 0.063517 0.065661 0.067883 0.070190
|
||||
12 0.072693 0.075396 0.063482 0.065661 0.067917 0.070258 0.072688
|
||||
13 0.075155 0.078000 0.065591 0.067883 0.070258 0.072721 0.075281
|
||||
14 0.077711 0.080704 0.067780 0.070190 0.072688 0.075281 0.077975
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1125,12 +1125,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
|
||||
0 4.114499 2.071143
|
||||
1 2.071143 2.061388
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.11449851 2.07114326]
|
||||
[2.07114326 2.0613875 ]]
|
||||
0 4.060824 2.038890
|
||||
1 2.038890 2.029664
|
||||
[[4.06082419 2.03888998]
|
||||
[2.03888998 2.02966421]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1157,8 +1155,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
|
||||
[[4.11449851 2.07114326]
|
||||
[2.07114326 2.0613875 ]]
|
||||
[[4.06082419 2.03888998]
|
||||
[2.03888998 2.02966421]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter8_65_1.png" src="_images/chapter8_65_1.png" />
|
||||
@@ -1218,16 +1216,16 @@ questions.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
|
||||
5.399533503407795
|
||||
0.776352510835556
|
||||
5.323066643161066
|
||||
0.7674217625964539
|
||||
First eigenvector
|
||||
[0.84973247 0.52721412]
|
||||
[0.85025162 0.52637646]
|
||||
Second eigenvector
|
||||
[-0.52721412 0.84973247]
|
||||
[-0.52637646 0.85025162]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
|
||||
[-0.84973247 -0.52721412]
|
||||
[-0.85025162 -0.52637646]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -804,15 +804,15 @@ regression.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.1729993]
|
||||
[3.0170097]]
|
||||
Eigenvalues of Hessian Matrix:[0.28192769 4.68753434]
|
||||
[[3.99556258]
|
||||
[2.99418621]]
|
||||
Eigenvalues of Hessian Matrix:[0.29411652 4.64408368]
|
||||
theta from own gd
|
||||
[[4.1729993]
|
||||
[3.0170097]]
|
||||
[[3.99556258]
|
||||
[2.99418621]]
|
||||
theta from own sdg
|
||||
[[4.14043884]
|
||||
[2.99071523]]
|
||||
[[4.03364384]
|
||||
[2.99870462]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/exercisesweek41_5_1.png" src="_images/exercisesweek41_5_1.png" />
|
||||
@@ -934,14 +934,14 @@ first example shows results with ordinary leats squares.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.03696458]
|
||||
[3.0324793 ]]
|
||||
Eigenvalues of Hessian Matrix:[0.30959659 4.4150026 ]
|
||||
[[3.94033034]
|
||||
[3.07147868]]
|
||||
Eigenvalues of Hessian Matrix:[0.26240613 4.76140219]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
|
||||
[[4.03696458]
|
||||
[3.0324793 ]]
|
||||
[[3.94033034]
|
||||
[3.07147868]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/exercisesweek41_16_2.png" src="_images/exercisesweek41_16_2.png" />
|
||||
@@ -1012,73 +1012,73 @@ Eigenvalues of Hessian Matrix:[0.30959659 4.4150026 ]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.]
|
||||
[3.]]
|
||||
Eigenvalues of Hessian Matrix:[0.23469347 4.90407685]
|
||||
0 [-18.02987043] [-22.42501752]
|
||||
1 [-0.32329897] [0.25206371]
|
||||
2 [-0.30782691] [0.24000075]
|
||||
3 [-0.2930953] [0.22851508]
|
||||
4 [-0.27906869] [0.21757908]
|
||||
5 [-0.26571335] [0.20716644]
|
||||
6 [-0.25299716] [0.19725211]
|
||||
7 [-0.24088952] [0.18781225]
|
||||
8 [-0.22936132] [0.17882416]
|
||||
9 [-0.21838482] [0.17026621]
|
||||
10 [-0.20793362] [0.16211781]
|
||||
11 [-0.19798258] [0.15435937]
|
||||
12 [-0.18850776] [0.14697222]
|
||||
13 [-0.17948638] [0.1399386]
|
||||
14 [-0.17089674] [0.13324158]
|
||||
15 [-0.16271817] [0.12686507]
|
||||
16 [-0.15493099] [0.12079371]
|
||||
17 [-0.14751649] [0.11501291]
|
||||
18 [-0.14045682] [0.10950876]
|
||||
19 [-0.13373501] [0.10426802]
|
||||
20 [-0.12733488] [0.09927808]
|
||||
21 [-0.12124104] [0.09452695]
|
||||
22 [-0.11543883] [0.09000319]
|
||||
23 [-0.10991429] [0.08569593]
|
||||
24 [-0.10465415] [0.08159479]
|
||||
25 [-0.09964573] [0.07768993]
|
||||
26 [-0.094877] [0.07397193]
|
||||
27 [-0.09033649] [0.07043187]
|
||||
28 [-0.08601328] [0.06706123]
|
||||
29 [-0.08189696] [0.06385189]
|
||||
Eigenvalues of Hessian Matrix:[0.30332201 4.25820894]
|
||||
0 [-14.184119] [-15.76632434]
|
||||
1 [-0.27465664] [0.23807169]
|
||||
2 [-0.25509222] [0.2211133]
|
||||
3 [-0.23692142] [0.20536289]
|
||||
4 [-0.22004496] [0.19073442]
|
||||
5 [-0.20437065] [0.17714797]
|
||||
6 [-0.18981286] [0.16452931]
|
||||
7 [-0.17629205] [0.15280951]
|
||||
8 [-0.16373437] [0.14192454]
|
||||
9 [-0.15207119] [0.13181493]
|
||||
10 [-0.14123881] [0.12242545]
|
||||
11 [-0.13117805] [0.1137048]
|
||||
12 [-0.12183393] [0.10560535]
|
||||
13 [-0.11315542] [0.09808284]
|
||||
14 [-0.1050951] [0.09109617]
|
||||
15 [-0.09760893] [0.08460718]
|
||||
16 [-0.09065603] [0.07858042]
|
||||
17 [-0.08419839] [0.07298295]
|
||||
18 [-0.07820074] [0.06778421]
|
||||
19 [-0.07263033] [0.06295579]
|
||||
20 [-0.0674567] [0.0584713]
|
||||
21 [-0.06265161] [0.05430625]
|
||||
22 [-0.05818879] [0.0504379]
|
||||
23 [-0.05404387] [0.04684509]
|
||||
24 [-0.0501942] [0.04350821]
|
||||
25 [-0.04661875] [0.04040902]
|
||||
26 [-0.04329799] [0.03753059]
|
||||
27 [-0.04021377] [0.0348572]
|
||||
28 [-0.03734925] [0.03237424]
|
||||
29 [-0.03468878] [0.03006815]
|
||||
theta from own gd
|
||||
[[3.66774691]
|
||||
[3.25904489]]
|
||||
0 [-0.07797763] [0.06079614]
|
||||
1 [-0.07424587] [0.05788664]
|
||||
2 [-0.06957317] [0.05424351]
|
||||
3 [-0.06484181] [0.05055465]
|
||||
4 [-0.06031928] [0.04702861]
|
||||
5 [-0.05607583] [0.04372016]
|
||||
6 [-0.05211919] [0.04063532]
|
||||
7 [-0.04843794] [0.03776519]
|
||||
8 [-0.04501548] [0.03509683]
|
||||
9 [-0.04183444] [0.0326167]
|
||||
10 [-0.03887807] [0.03031173]
|
||||
11 [-0.03613058] [0.02816961]
|
||||
12 [-0.03357723] [0.02617887]
|
||||
13 [-0.03120433] [0.02432881]
|
||||
14 [-0.02899912] [0.02260949]
|
||||
15 [-0.02694975] [0.02101168]
|
||||
16 [-0.02504521] [0.01952679]
|
||||
17 [-0.02327527] [0.01814683]
|
||||
18 [-0.0216304] [0.01686439]
|
||||
19 [-0.02010178] [0.01567258]
|
||||
20 [-0.01868119] [0.014565]
|
||||
21 [-0.01736099] [0.01353569]
|
||||
22 [-0.01613409] [0.01257912]
|
||||
23 [-0.01499389] [0.01169016]
|
||||
24 [-0.01393427] [0.01086401]
|
||||
25 [-0.01294954] [0.01009625]
|
||||
26 [-0.01203439] [0.00938275]
|
||||
27 [-0.01118392] [0.00871967]
|
||||
28 [-0.01039355] [0.00810345]
|
||||
29 [-0.00965904] [0.00753078]
|
||||
[[3.89378344]
|
||||
[3.09206825]]
|
||||
0 [-0.03221782] [0.02792633]
|
||||
1 [-0.02992287] [0.02593707]
|
||||
2 [-0.02710291] [0.02349274]
|
||||
3 [-0.02432632] [0.02108599]
|
||||
4 [-0.02176052] [0.01886197]
|
||||
5 [-0.01944073] [0.01685118]
|
||||
6 [-0.01735999] [0.01504759]
|
||||
7 [-0.01549917] [0.01343464]
|
||||
8 [-0.01383689] [0.01199378]
|
||||
9 [-0.01235257] [0.01070717]
|
||||
10 [-0.01102737] [0.0095585]
|
||||
11 [-0.0098443] [0.00853302]
|
||||
12 [-0.00878815] [0.00761755]
|
||||
13 [-0.00784531] [0.00680029]
|
||||
14 [-0.00700361] [0.00607072]
|
||||
15 [-0.00625222] [0.00541941]
|
||||
16 [-0.00558145] [0.00483798]
|
||||
17 [-0.00498263] [0.00431893]
|
||||
18 [-0.00444806] [0.00385557]
|
||||
19 [-0.00397085] [0.00344192]
|
||||
20 [-0.00354483] [0.00307265]
|
||||
21 [-0.00316452] [0.002743]
|
||||
22 [-0.00282501] [0.00244871]
|
||||
23 [-0.00252192] [0.002186]
|
||||
24 [-0.00225136] [0.00195147]
|
||||
25 [-0.00200982] [0.0017421]
|
||||
26 [-0.00179419] [0.0015552]
|
||||
27 [-0.0016017] [0.00138835]
|
||||
28 [-0.00142986] [0.0012394]
|
||||
29 [-0.00127645] [0.00110643]
|
||||
theta from own gd wth momentum
|
||||
[[3.96175251]
|
||||
[3.02982009]]
|
||||
[[3.99624324]
|
||||
[3.00325635]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1131,17 +1131,17 @@ theta from own gd wth momentum
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.15451852]
|
||||
[2.83230774]]
|
||||
Eigenvalues of Hessian Matrix:[0.30616802 4.24299211]
|
||||
0 [-10.57502449] [-11.57610367]
|
||||
1 [-4.47905601e-15] [-4.07372439e-16]
|
||||
2 [-6.9388939e-16] [-7.21432413e-16]
|
||||
3 [-6.9388939e-16] [-7.21432413e-16]
|
||||
4 [-6.9388939e-16] [-7.21432413e-16]
|
||||
[[3.86346614]
|
||||
[3.11457699]]
|
||||
Eigenvalues of Hessian Matrix:[0.29561686 4.57324367]
|
||||
0 [-14.07510754] [-17.31973052]
|
||||
1 [1.77982629e-15] [1.66047885e-15]
|
||||
2 [1.66533454e-16] [2.95162248e-16]
|
||||
3 [1.66533454e-16] [2.95162248e-16]
|
||||
4 [1.66533454e-16] [2.95162248e-16]
|
||||
beta from own Newton code
|
||||
[[4.15451852]
|
||||
[2.83230774]]
|
||||
[[3.86346614]
|
||||
[3.11457699]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1230,20 +1230,18 @@ beta from own Newton code
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.04601419]
|
||||
[3.12204312]]
|
||||
Eigenvalues of Hessian Matrix:[0.33604208 4.45709724]
|
||||
[[4.08540984]
|
||||
[2.92466997]]
|
||||
Eigenvalues of Hessian Matrix:[0.32128064 4.10407019]
|
||||
theta from own gd
|
||||
[[4.08540984]
|
||||
[2.92466997]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
|
||||
[[4.04601419]
|
||||
[3.12204312]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/exercisesweek41_22_2.png" src="_images/exercisesweek41_22_2.png" />
|
||||
<img alt="_images/exercisesweek41_22_1.png" src="_images/exercisesweek41_22_1.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
|
||||
[[4.02781444]
|
||||
[3.13976073]]
|
||||
[[4.02171224]
|
||||
[2.94008244]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1325,15 +1323,17 @@ Eigenvalues of Hessian Matrix:[0.33604208 4.45709724]
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[3.96417888]
|
||||
[3.06634473]]
|
||||
Eigenvalues of Hessian Matrix:[0.32962444 4.18715465]
|
||||
[[3.72066289]
|
||||
[3.08799729]]
|
||||
Eigenvalues of Hessian Matrix:[0.28726748 4.35313779]
|
||||
theta from own gd
|
||||
[[3.9639885]
|
||||
[3.0665111]]
|
||||
theta from own sdg with momentum
|
||||
[[4.00842216]
|
||||
[3.14285244]]
|
||||
[[3.72055288]
|
||||
[3.08809115]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
|
||||
[[3.72183526]
|
||||
[3.12143852]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1408,9 +1408,9 @@ theta from own sdg with momentum
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
|
||||
[[1.99969895]
|
||||
[3.00167058]
|
||||
[3.99835872]]
|
||||
[[2.00001365]
|
||||
[2.99991971]
|
||||
[4.00007964]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1492,9 +1492,9 @@ theta from own sdg with momentum
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
|
||||
[[1.99852187]
|
||||
[3.03868311]
|
||||
[3.95744254]]
|
||||
[[2.00264795]
|
||||
[3.00057362]
|
||||
[3.99959681]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1580,9 +1580,9 @@ theta from own sdg with momentum
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
|
||||
[[1.99996471]
|
||||
[3.00026784]
|
||||
[3.99973141]]
|
||||
[[1.99992477]
|
||||
[3.00056337]
|
||||
[3.99950283]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1655,7 +1655,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
|
||||
return asarray(x, dtype=self.dtype)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x11892e7f0>]
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x1522ef130>]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/exercisesweek41_39_2.png" src="_images/exercisesweek41_39_2.png" />
|
||||
@@ -1690,7 +1690,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><matplotlib.collections.PathCollection at 0x118995f70>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><matplotlib.collections.PathCollection at 0x11c3bd700>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/exercisesweek41_41_1.png" src="_images/exercisesweek41_41_1.png" />
|
||||
|
||||
@@ -653,8 +653,8 @@ matrices and vectors.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.34851141 0.11232457 -0.59560497 0.79558423 -0.74632311 0.97068015
|
||||
1.45921927 0.39225935 -0.87171492 0.44129766]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.03609183 1.12393064 -0.46049816 0.86567539 2.03530044 0.18872206
|
||||
0.67747547 -1.387672 0.30136959 -0.51164347]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -875,36 +875,26 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[6.95039348e-01 4.77602426e-01 4.58218587e-02 7.93416236e-01
|
||||
9.55643340e-01 7.98621772e-01 4.69806017e-01 1.11371176e-01
|
||||
6.52359414e-01 4.86833907e-01]
|
||||
[6.34549184e-01 4.06225028e-01 6.55208611e-01 1.10909182e-02
|
||||
6.45609936e-01 2.42839645e-01 2.14335140e-01 4.88477745e-01
|
||||
7.33046180e-01 3.04253288e-01]
|
||||
[9.24911588e-01 6.09812323e-01 3.17886464e-01 8.85018556e-01
|
||||
9.82435177e-01 9.43167881e-01 7.43951798e-01 7.31787864e-01
|
||||
4.80369722e-01 8.57475388e-01]
|
||||
[1.16436977e-01 8.00829346e-01 3.64493447e-01 4.94082113e-02
|
||||
5.68163369e-01 4.84993546e-03 2.34121963e-01 2.49849206e-01
|
||||
3.86274590e-01 8.78949699e-01]
|
||||
[4.91568958e-01 9.21047414e-02 7.84745063e-01 6.96054705e-01
|
||||
6.19052323e-02 8.28645331e-01 8.30191327e-01 2.48853875e-01
|
||||
7.24382159e-01 3.97218946e-01]
|
||||
[3.79942527e-01 1.36403018e-02 7.08949703e-02 5.98556486e-01
|
||||
5.10274274e-01 5.86030593e-01 8.96762953e-02 9.27993904e-01
|
||||
5.79873884e-01 4.12085937e-01]
|
||||
[4.75733813e-01 3.83164611e-01 9.64968814e-01 2.10369090e-01
|
||||
6.15338896e-04 5.29509520e-01 4.33492417e-01 9.06496916e-01
|
||||
1.39646454e-01 4.21846767e-01]
|
||||
[1.24848631e-01 9.48875568e-01 8.00598510e-01 4.10923006e-01
|
||||
9.72407267e-01 9.52121734e-01 1.56665044e-01 5.06276815e-01
|
||||
5.30196557e-01 8.40012736e-01]
|
||||
[3.25145374e-01 1.83317854e-01 7.66763324e-02 9.66707072e-01
|
||||
8.23006632e-01 4.30655251e-01 4.69811070e-02 8.72758060e-01
|
||||
6.59088350e-01 7.28365323e-01]
|
||||
[5.96282505e-01 9.69432059e-01 1.10687432e-01 9.36439409e-01
|
||||
1.82212861e-01 9.15905706e-01 4.35826030e-01 3.89219980e-01
|
||||
4.21786054e-01 4.13799037e-02]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.58452753 0.23142852 0.47865692 0.18572822 0.31436277 0.90536337
|
||||
0.4096949 0.68748147 0.94493877 0.97696399]
|
||||
[0.95000259 0.40310129 0.74533876 0.01597958 0.74471533 0.36031586
|
||||
0.12031117 0.72690169 0.49008744 0.43492736]
|
||||
[0.05224627 0.62033782 0.14697641 0.34601775 0.97622642 0.8644215
|
||||
0.24629299 0.5128288 0.24777879 0.43007284]
|
||||
[0.76134837 0.76159011 0.55582327 0.81598832 0.40209956 0.31639624
|
||||
0.20393163 0.87700882 0.11331834 0.20621146]
|
||||
[0.00922905 0.79619113 0.72444668 0.05605667 0.38949027 0.92292668
|
||||
0.84659427 0.10325557 0.55170845 0.63144217]
|
||||
[0.25206089 0.00866766 0.06506624 0.53397188 0.25345918 0.67527216
|
||||
0.6696885 0.29142853 0.78198192 0.10425587]
|
||||
[0.72882234 0.32052031 0.58799372 0.04235439 0.60740312 0.12152902
|
||||
0.17158357 0.79041505 0.22523018 0.63353014]
|
||||
[0.13518746 0.54977724 0.1793636 0.31983943 0.12702643 0.3265097
|
||||
0.81592028 0.53488344 0.51698843 0.81385326]
|
||||
[0.85786373 0.84416298 0.775204 0.84047912 0.21501369 0.66603461
|
||||
0.53869015 0.88147037 0.06512065 0.38535182]
|
||||
[0.60586595 0.55084297 0.9202867 0.39376038 0.50558976 0.09347829
|
||||
0.50861923 0.79443618 0.17728993 0.29414225]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -964,13 +954,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.11406297261341168
|
||||
4.669908227073521
|
||||
0.2806901924685401
|
||||
[[ 0.98292239 2.97664837 2.73565966]
|
||||
[ 2.97664837 10.21905519 8.21690241]
|
||||
[ 2.73565966 8.21690241 13.03354844]]
|
||||
[20.77597019 0.09165112 3.36790471]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0684990216069429
|
||||
4.3391965479837245
|
||||
0.32336220689648504
|
||||
[[0.81091776 2.49361065 1.78904605]
|
||||
[2.49361065 8.68609856 5.49742477]
|
||||
[1.78904605 5.49742477 6.25106936]]
|
||||
[13.8211787 0.07429213 1.85261484]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1025,27 +1025,27 @@ uncorrelated.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.5715150087179857
|
||||
[[ 9.98938878 7.54804438 5.31210861 4.15563816 11.04286013 12.33871003
|
||||
5.34554342 14.48549903 6.87661362 7.26028795]
|
||||
[ 7.54804438 5.70334935 4.01386236 3.14002608 8.3440539 9.32320616
|
||||
4.0391259 10.94533329 5.19601208 5.4859188 ]
|
||||
[ 5.31210861 4.01386236 2.8248473 2.20986506 5.87231849 6.56141925
|
||||
2.8426271 7.70302827 3.65681217 3.86084064]
|
||||
[ 4.15563816 3.14002608 2.20986506 1.72876728 4.59388777 5.13296813
|
||||
2.22377412 6.02604362 2.86070736 3.02031789]
|
||||
[11.04286013 8.3440539 5.87231849 4.59388777 12.20742956 13.63993855
|
||||
5.9092793 16.01312585 7.60181469 8.02595095]
|
||||
[12.33871003 9.32320616 6.56141925 5.13296813 13.63993855 15.24054861
|
||||
6.60271731 17.89222305 8.49386718 8.96777468]
|
||||
[ 5.34554342 4.0391259 2.8426271 2.22377412 5.9092793 6.60271731
|
||||
2.8605188 7.7515117 3.67982841 3.88514106]
|
||||
[14.48549903 10.94533329 7.70302827 6.02604362 16.01312585 17.89222305
|
||||
7.7515117 21.00525734 9.97169918 10.52806096]
|
||||
[ 6.87661362 5.19601208 3.65681217 2.86070736 7.60181469 8.49386718
|
||||
3.67982841 9.97169918 4.73380463 4.99792291]
|
||||
[ 7.26028795 5.4859188 3.86084064 3.02031789 8.02595095 8.96777468
|
||||
3.88514106 10.52806096 4.99792291 5.27677742]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.2551096333979939
|
||||
[[ 8.89676583 14.62475001 12.73614157 6.35956676 11.68962656 15.02535112
|
||||
7.01266855 5.77132083 15.90800497 7.16405708]
|
||||
[14.62475001 24.040569 20.93602214 10.45403192 19.2157318 24.69908819
|
||||
11.52761872 9.48705697 26.15001903 11.77647539]
|
||||
[12.73614157 20.93602214 18.23238973 9.10402095 16.73425394 21.5095016
|
||||
10.03896708 8.26191905 22.77306239 10.25568695]
|
||||
[ 6.35956676 10.45403192 9.10402095 4.54593165 8.35595337 10.74038873
|
||||
5.01278044 4.12544297 11.37132543 5.12099567]
|
||||
[11.68962656 19.2157318 16.73425394 8.35595337 15.35921835 19.74208907
|
||||
9.21407599 7.58304607 20.90182442 9.41298822]
|
||||
[15.02535112 24.69908819 21.5095016 10.74038873 19.74208907 25.37564555
|
||||
11.84338322 9.74692641 26.86632028 12.09905657]
|
||||
[ 7.01266855 11.52761872 10.03896708 5.01278044 9.21407599 11.84338322
|
||||
5.52757272 4.54910928 12.53911457 5.64690121]
|
||||
[ 5.77132083 9.48705697 8.26191905 4.12544297 7.58304607 9.74692641
|
||||
4.54910928 3.74384858 10.31950286 4.64731484]
|
||||
[15.90800497 26.15001903 22.77306239 11.37132543 20.90182442 26.86632028
|
||||
12.53911457 10.31950286 28.44456367 12.80980728]
|
||||
[ 7.16405708 11.77647539 10.25568695 5.12099567 9.41298822 12.09905657
|
||||
5.64690121 4.64731484 12.80980728 5.76880575]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1313,15 +1313,15 @@ more practically oriented methods like the blocking technique.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.05818561278267464
|
||||
3.8056560957923695
|
||||
0.06926151969326948
|
||||
1.1885906477601598 12.229598952352728 16.227738984674172
|
||||
3.6795562921258607 3.451702023099923 10.293665700387852
|
||||
[[ 1.18859065 3.67955629 3.45170202]
|
||||
[ 3.67955629 12.22959895 10.2936657 ]
|
||||
[ 3.45170202 10.2936657 16.22773898]]
|
||||
[25.73550536 0.06057711 3.84984612]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.19350397125914334
|
||||
3.2837242622462304
|
||||
-0.8221979401496963
|
||||
1.227072949395066 12.297476846305301 41.228467858509994
|
||||
3.7100707615589594 5.460688958585373 17.510697484548
|
||||
[[ 1.22707295 3.71007076 5.46068896]
|
||||
[ 3.71007076 12.29747685 17.51069748]
|
||||
[ 5.46068896 17.51069748 41.22846786]]
|
||||
[50.34205649 0.0976598 4.31330137]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1651,7 +1651,7 @@ assumption for approximating <span class="math notranslate nohighlight">\(\sigma
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.03974553487733608 1.0433282860079154
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.006352536528464608 1.0383613556521865
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/statistics_188_1.png" src="_images/statistics_188_1.png" />
|
||||
|
||||
@@ -1713,8 +1713,8 @@ developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly he
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.81737781 0.45847011 0.53332849 1.04937026 0.53235952 2.24033384
|
||||
-0.05605892 -0.02193078 -0.37343957 0.17849836]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.69749114 -0.06437266 0.55440601 -0.59770229 1.83671148 0.02661264
|
||||
-1.45840653 1.96917669 0.32786329 -1.51690815]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1939,26 +1939,26 @@ lowercase letters for vectors and uppercase letters for matrices)</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.15586219 0.68891428 0.38675231 0.63546905 0.97286326 0.423102
|
||||
0.42841423 0.3478095 0.66130478 0.63443371]
|
||||
[0.88685415 0.82561703 0.56766321 0.43258818 0.99057275 0.40058341
|
||||
0.84581375 0.64630711 0.03266152 0.57599494]
|
||||
[0.10145028 0.9074157 0.39386718 0.12831838 0.46242424 0.09280524
|
||||
0.88415819 0.26443391 0.55095316 0.85897711]
|
||||
[0.69243548 0.33745439 0.14452065 0.29030256 0.8063779 0.60720029
|
||||
0.42914251 0.44895662 0.09310479 0.48442601]
|
||||
[0.95152814 0.83294718 0.11005335 0.8758851 0.19375828 0.73888203
|
||||
0.83203197 0.69203997 0.65147818 0.35195241]
|
||||
[0.21506004 0.24874378 0.31370028 0.9525328 0.71672791 0.05879106
|
||||
0.45007578 0.36388542 0.50937003 0.57854115]
|
||||
[0.80033616 0.45273617 0.18038547 0.49557088 0.36209091 0.44512218
|
||||
0.84078641 0.28924386 0.99166852 0.22896144]
|
||||
[0.77579275 0.83519517 0.40640797 0.66272614 0.18234499 0.97628064
|
||||
0.19808709 0.1280526 0.33700495 0.32114535]
|
||||
[0.95462534 0.72047148 0.24512233 0.18474924 0.69169665 0.68763036
|
||||
0.8861811 0.54193001 0.87830277 0.79251831]
|
||||
[0.13092444 0.41452482 0.40447213 0.89714357 0.25360039 0.80373997
|
||||
0.51279028 0.58161787 0.08742496 0.45104086]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.72863433 0.17780089 0.4565147 0.92353194 0.4565685 0.45408759
|
||||
0.55900177 0.02561386 0.63461575 0.70015185]
|
||||
[0.52488859 0.52729975 0.49035757 0.2383703 0.9206617 0.2766892
|
||||
0.89708415 0.52998985 0.45993741 0.74506271]
|
||||
[0.67036507 0.42813351 0.79628257 0.00872006 0.06083072 0.80873785
|
||||
0.74407544 0.32616922 0.81337164 0.97183244]
|
||||
[0.23449217 0.64638876 0.5291335 0.03625417 0.47382705 0.03068149
|
||||
0.72365764 0.53228519 0.63483713 0.01924105]
|
||||
[0.99849105 0.05293216 0.52012715 0.77037707 0.28836035 0.7080469
|
||||
0.91692081 0.50152186 0.08734023 0.69892546]
|
||||
[0.96154578 0.52229337 0.78888045 0.59879372 0.38240257 0.55451651
|
||||
0.01380833 0.4659454 0.51857188 0.98893579]
|
||||
[0.38422061 0.39516594 0.22569872 0.40434293 0.80846898 0.33300231
|
||||
0.45647803 0.60116948 0.55862544 0.20607165]
|
||||
[0.50029371 0.11487977 0.17093974 0.40903389 0.40362686 0.63204599
|
||||
0.99864751 0.7033198 0.61402242 0.72219757]
|
||||
[0.78025829 0.65744898 0.93708805 0.68532803 0.22974904 0.00166889
|
||||
0.33180288 0.26196737 0.62846663 0.4390092 ]
|
||||
[0.23570126 0.11877604 0.81642349 0.76688358 0.82757017 0.92808577
|
||||
0.89790867 0.97165491 0.98806419 0.48333419]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2013,13 +2013,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.0458524213754298
|
||||
3.614161296466206
|
||||
-0.22985907723809532
|
||||
[[0.72589774 2.09219464 1.64672839]
|
||||
[2.09219464 6.9187554 4.62198131]
|
||||
[1.64672839 4.62198131 6.70530438]]
|
||||
[12.05431945 0.07101262 2.22462544]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.15294186382924843
|
||||
4.473730000968826
|
||||
0.7333854378748542
|
||||
[[ 0.89730533 2.66990322 2.88518 ]
|
||||
[ 2.66990322 8.78338459 8.44007188]
|
||||
[ 2.88518 8.44007188 16.31799354]]
|
||||
[22.48904404 0.06826275 3.44137667]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2244,7 +2244,7 @@ Name: Aragorn, dtype: object
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
|
||||
<span class="ne">AttributeError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
|
||||
<span class="nn">/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57294/1326197715.py</span> in <span class="ni">?</span><span class="nt">()</span>
|
||||
<span class="nn">/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59734/1326197715.py</span> in <span class="ni">?</span><span class="nt">()</span>
|
||||
<span class="ne">----> </span><span class="mi">6</span> <span class="n">new_hobbit</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'First Name'</span><span class="p">:</span> <span class="p">[</span><span class="s2">"Peregrin"</span><span class="p">],</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">7</span> <span class="s1">'Last Name'</span><span class="p">:</span> <span class="p">[</span><span class="s2">"Took"</span><span class="p">],</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="s1">'Place of birth'</span><span class="p">:</span> <span class="p">[</span><span class="s2">"Shire"</span><span class="p">],</span>
|
||||
|
||||
@@ -1681,7 +1681,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9953466931203151
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9950865473984227
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1698,7 +1698,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.010175219431920396
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.009998596494598343
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1713,23 +1713,23 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.03699304 0.0218856 0.02021288 0.0334505 0.02960445 0.01369614
|
||||
0.00606112 0.01429853 0.01174937 0.02585059 0.02044223 0.00174884
|
||||
0.06133572 0.03159246 0.04435458 0.00125802 0.00100423 0.00230308
|
||||
0.01055384 0.02352843 0.06971862 0.01300036 0.00547332 0.05554795
|
||||
0.01142354 0.01181458 0.00148216 0.01419341 0.0305221 0.01272102
|
||||
0.01700746 0.01552039 0.01616657 0.10695771 0.00405576 0.02979087
|
||||
0.0529838 0.01420773 0.06220192 0.04104182 0.00653725 0.07170448
|
||||
0.00997215 0.02490769 0.02580654 0.01682317 0.03221473 0.01838531
|
||||
0.02028936 0.0064349 0.04323964 0.02705202 0.03692828 0.01975755
|
||||
0.05636265 0.02880987 0.05692207 0.04085864 0.01085261 0.01457889
|
||||
0.03418643 0.01919791 0.00101555 0.00636951 0.04217103 0.0476266
|
||||
0.01169528 0.04544327 0.00978267 0.04046984 0.0032882 0.02072876
|
||||
0.05526935 0.05461692 0.00877816 0.00724303 0.00045923 0.00059105
|
||||
0.00917881 0.04254787 0.08728977 0.04513394 0.01606644 0.08956994
|
||||
0.02687671 0.07449122 0.04497158 0.01713187 0.02553907 0.0397137
|
||||
0.03313193 0.00738299 0.01743124 0.02953975 0.01131825 0.0864086
|
||||
0.01925934 0.02439287 0.09331672 0.01721277]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.0442242 0.01222848 0.01176967 0.00562142 0.00392853 0.02034593
|
||||
0.01294799 0.07286621 0.00893112 0.00155695 0.03181997 0.00965205
|
||||
0.00523789 0.03510464 0.04098879 0.04626298 0.03733016 0.05632246
|
||||
0.02241952 0.03770824 0.05984726 0.00875286 0.04355106 0.01981665
|
||||
0.06929519 0.03426934 0.00410974 0.0142117 0.00099936 0.04030508
|
||||
0.05247827 0.05227563 0.02499437 0.01459912 0.00154737 0.03099794
|
||||
0.06159762 0.00052051 0.04268493 0.01479672 0.01149099 0.02280634
|
||||
0.04740084 0.00617261 0.00103024 0.00740838 0.00577229 0.00142146
|
||||
0.00140253 0.01112157 0.01180692 0.00039821 0.02257687 0.03196582
|
||||
0.01289266 0.03194307 0.00192165 0.08543112 0.01377529 0.06267966
|
||||
0.10637914 0.00872869 0.00331168 0.03291795 0.08933713 0.00786896
|
||||
0.01299711 0.01870931 0.03870024 0.0089204 0.0180908 0.05893076
|
||||
0.00110867 0.0374535 0.03500569 0.00381349 0.01974315 0.01109955
|
||||
0.0193386 0.01273648 0.00520618 0.00420536 0.02597861 0.01698477
|
||||
0.02535973 0.03901061 0.06862038 0.01811373 0.02478142 0.00293201
|
||||
0.06915429 0.01105668 0.01370129 0.03923714 0.02045223 0.00193326
|
||||
0.01717238 0.01441661 0.04590873 0.00871267]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1798,15 +1798,15 @@ but now splitting the data into a training set and a test set.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.94735263 0.70175778 2.99348646 1.86611894 -0.45576546]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.08019747 -1.5544062 12.12040799 -11.41377873 5.90871271]
|
||||
Training R2
|
||||
0.9959836634296064
|
||||
0.9958244721767004
|
||||
Training MSE
|
||||
0.0085274606925055
|
||||
0.010405448150048726
|
||||
Test R2
|
||||
0.992232777849821
|
||||
0.9929666457835595
|
||||
Test MSE
|
||||
0.010638334964957053
|
||||
0.018365206985554446
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2477,7 +2477,9 @@ Feature min values before scaling:
|
||||
2.89126914e-10 1.00934327e-10 3.52362157e-11 1.65571174e-11
|
||||
5.78009660e-12 2.01783414e-12 7.04426744e-13 2.45915671e-13
|
||||
8.58492636e-14]
|
||||
Feature max values before scaling:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Feature max values before scaling:
|
||||
[1. 0.99970894 0.99978365 0.99941797 0.99949266 0.99956735
|
||||
0.99912709 0.99920175 0.99927642 0.9993511 0.99883628 0.99891093
|
||||
0.99898558 0.99906023 0.99913489 0.99854557 0.99862019 0.99869482
|
||||
|
||||
@@ -1669,7 +1669,7 @@ theorem.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
|
||||
original bias std. error
|
||||
100.091 15.1529 100.089 0.15052
|
||||
100.203 14.9405 100.203 0.148913
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1894,7 +1894,9 @@ Error: 0.10398646080125035
|
||||
Bias^2: 0.1007711427354898
|
||||
Var: 0.0032153180657605116
|
||||
0.10398646080125035 >= 0.1007711427354898 + 0.0032153180657605116 = 0.10398646080125032
|
||||
Polynomial degree: 3
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 3
|
||||
Error: 0.06547790180152355
|
||||
Bias^2: 0.06208238634231949
|
||||
Var: 0.0033955154592040936
|
||||
@@ -1904,14 +1906,14 @@ Error: 0.06844519414009445
|
||||
Bias^2: 0.06453579006728324
|
||||
Var: 0.003909404072811226
|
||||
0.06844519414009445 >= 0.06453579006728324 + 0.003909404072811226 = 0.06844519414009446
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 5
|
||||
Polynomial degree: 5
|
||||
Error: 0.05227921801205686
|
||||
Bias^2: 0.0481872773043029
|
||||
Var: 0.004091940707753939
|
||||
0.05227921801205686 >= 0.0481872773043029 + 0.004091940707753939 = 0.052279218012056844
|
||||
Polynomial degree: 6
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 6
|
||||
Error: 0.037813671417389005
|
||||
Bias^2: 0.033657685071527665
|
||||
Var: 0.00415598634586135
|
||||
@@ -1938,9 +1940,7 @@ Error: 0.021592704588025025
|
||||
Bias^2: 0.010516485576645508
|
||||
Var: 0.011076219011379514
|
||||
0.021592704588025025 >= 0.010516485576645508 + 0.011076219011379514 = 0.021592704588025022
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
|
||||
Polynomial degree: 11
|
||||
Error: 0.07160048164233104
|
||||
Bias^2: 0.014436800088904942
|
||||
Var: 0.05716368155342608
|
||||
@@ -1950,14 +1950,16 @@ Error: 0.11547777218872497
|
||||
Bias^2: 0.01628578269596628
|
||||
Var: 0.09919198949275869
|
||||
0.11547777218872497 >= 0.01628578269596628 + 0.09919198949275869 = 0.11547777218872497
|
||||
Polynomial degree: 13
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 13
|
||||
Error: 0.22842468702219465
|
||||
Bias^2: 0.01975416527185249
|
||||
Var: 0.20867052175034223
|
||||
0.22842468702219465 >= 0.01975416527185249 + 0.20867052175034223 = 0.2284246870221947
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week37_139_4.png" src="_images/week37_139_4.png" />
|
||||
<img alt="_images/week37_139_5.png" src="_images/week37_139_5.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2388,9 +2390,9 @@ Mean squared error on training data: 0.00063866
|
||||
Mean squared error on test data: 3099.60342978
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57311/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59752/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(trainingerror), label='Training Error')
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57311/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59752/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(testerror), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2475,7 +2477,7 @@ Mean squared error on test data: 3099.60342978
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57311/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59752/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
|
||||
@@ -1812,7 +1812,7 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x11ffa0ee0>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x127d8ceb0>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week39_82_1.png" src="_images/week39_82_1.png" />
|
||||
@@ -1870,7 +1870,7 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x12ccea880>]
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x1322a9a00>]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week39_90_1.png" src="_images/week39_90_1.png" />
|
||||
@@ -2164,11 +2164,11 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Hessian Matrix:[0.33433563 4.2051784 ]
|
||||
[[3.94121002]
|
||||
[3.08191754]]
|
||||
[[3.94121002]
|
||||
[3.08191754]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Hessian Matrix:[0.28754232 4.16543524]
|
||||
[[3.78714005]
|
||||
[3.07335575]]
|
||||
[[3.78714005]
|
||||
[3.07335575]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week39_153_1.png" src="_images/week39_153_1.png" />
|
||||
@@ -2199,9 +2199,9 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.91483251]
|
||||
[3.09848937]]
|
||||
[3.87616945] [3.07001431]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.02163476]
|
||||
[2.9518022 ]]
|
||||
[4.06200442] [3.03827385]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2301,11 +2301,11 @@ minimum of this function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Hessian Matrix:[0.26638638 4.40906565]
|
||||
[[4.26969649]
|
||||
[2.78617455]]
|
||||
[[4.26953857]
|
||||
[2.78630865]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Hessian Matrix:[0.36579128 3.8614682 ]
|
||||
[[3.94096731]
|
||||
[3.09895512]]
|
||||
[[3.94091671]
|
||||
[3.09900258]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week39_166_1.png" src="_images/week39_166_1.png" />
|
||||
@@ -3878,16 +3878,12 @@ beta from own Newton code
|
||||
[[4.0586484]
|
||||
[3.0718316]]
|
||||
Eigenvalues of Hessian Matrix:[0.29860173 3.8931686 ]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.0586484]
|
||||
theta from own gd
|
||||
[[4.0586484]
|
||||
[3.0718316]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week39_269_3.png" src="_images/week39_269_3.png" />
|
||||
<img alt="_images/week39_269_1.png" src="_images/week39_269_1.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
|
||||
[[4.02496085]
|
||||
[3.12081773]]
|
||||
|
||||
@@ -1314,17 +1314,17 @@ We summarize some of these here for the methods we hvae studied in project one,
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
|
||||
[[3.8184887 ]
|
||||
[3.47851966]
|
||||
[4.77551387]]
|
||||
[[3.78243922]
|
||||
[3.4825169 ]
|
||||
[4.79102612]]
|
||||
Parameters for Ridge using gradient descent
|
||||
[[3.92021197]
|
||||
[3.11388017]
|
||||
[4.9458396 ]]
|
||||
[[3.94463114]
|
||||
[3.02606605]
|
||||
[4.9912346 ]]
|
||||
Parameters for Lasso using gradient descent
|
||||
[[3.87323528]
|
||||
[3.3008836 ]
|
||||
[4.86284277]]
|
||||
[[3.79673108]
|
||||
[3.54406885]
|
||||
[4.74100159]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1376,11 +1376,11 @@ Parameters for Lasso using gradient descent
|
||||
[[4.]
|
||||
[3.]
|
||||
[5.]]
|
||||
0 [-31.91417133] [-44.48017159]
|
||||
1 [3.48805429e-13] [4.67477123e-13]
|
||||
2 [6.75015599e-16] [1.30675215e-15]
|
||||
3 [-1.17239551e-15] [-1.78477759e-15]
|
||||
4 [6.75015599e-16] [1.30675215e-15]
|
||||
0 [-31.16622624] [-43.76852722]
|
||||
1 [1.2420287e-13] [1.07341744e-13]
|
||||
2 [-6.21724894e-16] [-7.8406741e-16]
|
||||
3 [1.0658141e-15] [1.70040367e-15]
|
||||
4 [4.4408921e-16] [8.75488337e-16]
|
||||
beta from own Newton code
|
||||
[[4.]
|
||||
[3.]
|
||||
@@ -1722,18 +1722,20 @@ function.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[3.90300704]
|
||||
[3.16913489]]
|
||||
Eigenvalues of Hessian Matrix:[0.2964378 4.12443871]
|
||||
theta from own gd
|
||||
[[3.90300704]
|
||||
[3.16913489]]
|
||||
theta from own sdg
|
||||
[[3.93272428]
|
||||
[3.16328315]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.79540402]
|
||||
[3.14383145]]
|
||||
Eigenvalues of Hessian Matrix:[0.29866395 3.79522963]
|
||||
theta from own gd
|
||||
[[3.79540402]
|
||||
[3.14383145]]
|
||||
theta from own sdg
|
||||
[[3.78596158]
|
||||
[3.12461758]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_34_2.png" src="_images/week40_34_2.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2444,12 +2446,12 @@ first example shows results with ordinary leats squares.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.13791264]
|
||||
[2.92552059]]
|
||||
Eigenvalues of Hessian Matrix:[0.27874136 4.16226023]
|
||||
[[4.35914932]
|
||||
[2.77699722]]
|
||||
Eigenvalues of Hessian Matrix:[0.32641558 4.49529361]
|
||||
theta from own gd
|
||||
[[4.13791264]
|
||||
[2.92552059]]
|
||||
[[4.35914932]
|
||||
[2.77699722]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
|
||||
@@ -2520,73 +2522,73 @@ theta from own gd
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.]
|
||||
[3.]]
|
||||
Eigenvalues of Hessian Matrix:[0.32606365 3.80499859]
|
||||
0 [-10.98955596] [-10.8332972]
|
||||
1 [-0.26421616] [0.25444295]
|
||||
2 [-0.24157456] [0.23263884]
|
||||
3 [-0.22087319] [0.2127032]
|
||||
4 [-0.20194579] [0.19447592]
|
||||
5 [-0.18464035] [0.1778106]
|
||||
6 [-0.16881787] [0.16257339]
|
||||
7 [-0.15435127] [0.1486419]
|
||||
8 [-0.14112437] [0.13590426]
|
||||
9 [-0.12903093] [0.12425815]
|
||||
10 [-0.11797382] [0.11361003]
|
||||
11 [-0.10786423] [0.10387439]
|
||||
12 [-0.09862097] [0.09497303]
|
||||
13 [-0.09016979] [0.08683446]
|
||||
14 [-0.08244283] [0.07939331]
|
||||
15 [-0.07537801] [0.07258982]
|
||||
16 [-0.06891861] [0.06636934]
|
||||
17 [-0.06301273] [0.06068192]
|
||||
18 [-0.05761295] [0.05548187]
|
||||
19 [-0.05267589] [0.05072744]
|
||||
20 [-0.04816191] [0.04638043]
|
||||
21 [-0.04403475] [0.04240593]
|
||||
22 [-0.04026126] [0.03877201]
|
||||
23 [-0.03681113] [0.0354495]
|
||||
24 [-0.03365665] [0.03241171]
|
||||
25 [-0.0307725] [0.02963424]
|
||||
26 [-0.02813549] [0.02709478]
|
||||
27 [-0.02572446] [0.02477293]
|
||||
28 [-0.02352005] [0.02265005]
|
||||
29 [-0.02150453] [0.02070909]
|
||||
Eigenvalues of Hessian Matrix:[0.31711533 4.28519494]
|
||||
0 [-8.99246366] [-10.02181163]
|
||||
1 [-0.20915094] [0.17948385]
|
||||
2 [-0.19367324] [0.16620159]
|
||||
3 [-0.17934093] [0.15390225]
|
||||
4 [-0.16606924] [0.14251309]
|
||||
5 [-0.1537797] [0.13196676]
|
||||
6 [-0.14239961] [0.12220088]
|
||||
7 [-0.13186167] [0.11315771]
|
||||
8 [-0.12210357] [0.10478375]
|
||||
9 [-0.1130676] [0.09702948]
|
||||
10 [-0.10470031] [0.08984905]
|
||||
11 [-0.09695222] [0.0832]
|
||||
12 [-0.08977751] [0.07704298]
|
||||
13 [-0.08313374] [0.07134161]
|
||||
14 [-0.07698164] [0.06606215]
|
||||
15 [-0.0712848] [0.06117338]
|
||||
16 [-0.06600954] [0.05664639]
|
||||
17 [-0.06112467] [0.05245442]
|
||||
18 [-0.05660129] [0.04857266]
|
||||
19 [-0.05241265] [0.04497816]
|
||||
20 [-0.04853398] [0.04164966]
|
||||
21 [-0.04494234] [0.03856748]
|
||||
22 [-0.04161649] [0.03571339]
|
||||
23 [-0.03853677] [0.0330705]
|
||||
24 [-0.03568495] [0.0306232]
|
||||
25 [-0.03304417] [0.02835701]
|
||||
26 [-0.03059882] [0.02625852]
|
||||
27 [-0.02833443] [0.02431532]
|
||||
28 [-0.02623761] [0.02251592]
|
||||
29 [-0.02429596] [0.02084969]
|
||||
theta from own gd
|
||||
[[3.93969971]
|
||||
[3.05806981]]
|
||||
0 [-0.01966173] [0.01893445]
|
||||
1 [-0.01797685] [0.0173119]
|
||||
2 [-0.01593089] [0.01534161]
|
||||
3 [-0.01395192] [0.01343585]
|
||||
4 [-0.01216264] [0.01171276]
|
||||
5 [-0.0105836] [0.01019212]
|
||||
6 [-0.00920294] [0.00886253]
|
||||
7 [-0.00800011] [0.00770419]
|
||||
8 [-0.00695371] [0.00669649]
|
||||
9 [-0.0060439] [0.00582034]
|
||||
10 [-0.00525303] [0.00505872]
|
||||
11 [-0.00456562] [0.00439674]
|
||||
12 [-0.00396815] [0.00382137]
|
||||
13 [-0.00344887] [0.0033213]
|
||||
14 [-0.00299754] [0.00288666]
|
||||
15 [-0.00260527] [0.0025089]
|
||||
16 [-0.00226433] [0.00218058]
|
||||
17 [-0.00196801] [0.00189522]
|
||||
18 [-0.00171047] [0.0016472]
|
||||
19 [-0.00148663] [0.00143164]
|
||||
20 [-0.00129209] [0.00124429]
|
||||
21 [-0.001123] [0.00108146]
|
||||
22 [-0.00097604] [0.00093994]
|
||||
23 [-0.00084831] [0.00081693]
|
||||
24 [-0.0007373] [0.00071003]
|
||||
25 [-0.00064081] [0.00061711]
|
||||
26 [-0.00055695] [0.00053635]
|
||||
27 [-0.00048407] [0.00046616]
|
||||
28 [-0.00042072] [0.00040516]
|
||||
29 [-0.00036566] [0.00035214]
|
||||
[[3.92905422]
|
||||
[3.06088245]]
|
||||
0 [-0.022498] [0.01930676]
|
||||
1 [-0.02083309] [0.01787801]
|
||||
2 [-0.01879191] [0.01612637]
|
||||
3 [-0.01678891] [0.01440748]
|
||||
4 [-0.01494559] [0.01282563]
|
||||
5 [-0.01328658] [0.01140194]
|
||||
6 [-0.01180564] [0.01013106]
|
||||
7 [-0.01048771] [0.00900007]
|
||||
8 [-0.00931621] [0.00799475]
|
||||
9 [-0.00827534] [0.00710152]
|
||||
10 [-0.00735068] [0.00630802]
|
||||
11 [-0.00652931] [0.00560316]
|
||||
12 [-0.00579972] [0.00497706]
|
||||
13 [-0.00515165] [0.00442091]
|
||||
14 [-0.00457599] [0.00392691]
|
||||
15 [-0.00406466] [0.0034881]
|
||||
16 [-0.00361046] [0.00309834]
|
||||
17 [-0.00320702] [0.00275212]
|
||||
18 [-0.00284866] [0.00244459]
|
||||
19 [-0.00253034] [0.00217143]
|
||||
20 [-0.0022476] [0.00192879]
|
||||
21 [-0.00199645] [0.00171326]
|
||||
22 [-0.00177336] [0.00152182]
|
||||
23 [-0.0015752] [0.00135176]
|
||||
24 [-0.00139918] [0.00120071]
|
||||
25 [-0.00124283] [0.00106654]
|
||||
26 [-0.00110396] [0.00094737]
|
||||
27 [-0.0009806] [0.0008415]
|
||||
28 [-0.00087102] [0.00074747]
|
||||
29 [-0.00077369] [0.00066395]
|
||||
theta from own gd wth momentum
|
||||
[[3.99902531]
|
||||
[3.00093864]]
|
||||
[[3.99783284]
|
||||
[3.00185976]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2675,18 +2677,18 @@ theta from own gd wth momentum
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.11762444]
|
||||
[3.04098313]]
|
||||
Eigenvalues of Hessian Matrix:[0.29738252 4.51279273]
|
||||
[[4.04829439]
|
||||
[3.02060364]]
|
||||
Eigenvalues of Hessian Matrix:[0.24427624 4.58825417]
|
||||
theta from own gd
|
||||
[[4.11762444]
|
||||
[3.04098313]]
|
||||
[[4.04829439]
|
||||
[3.02060364]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
|
||||
[[4.07058967]
|
||||
[3.024004 ]]
|
||||
[[4.04658881]
|
||||
[3.02962903]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2768,17 +2770,17 @@ theta from own gd
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[4.4252885 ]
|
||||
[2.70944365]]
|
||||
Eigenvalues of Hessian Matrix:[0.28973035 4.32089655]
|
||||
[[4.08777858]
|
||||
[2.93519647]]
|
||||
Eigenvalues of Hessian Matrix:[0.30054056 4.29201128]
|
||||
theta from own gd
|
||||
[[4.42394588]
|
||||
[2.71059619]]
|
||||
[[4.08779325]
|
||||
[2.93518383]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
|
||||
[[4.44845593]
|
||||
[2.72577807]]
|
||||
[[4.03961179]
|
||||
[2.924011 ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2847,9 +2849,9 @@ theta from own gd
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
|
||||
[[2.00036797]
|
||||
[2.99817613]
|
||||
[4.00177485]]
|
||||
[[1.99968483]
|
||||
[3.00127606]
|
||||
[3.9985605 ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2925,9 +2927,9 @@ theta from own gd
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
|
||||
[[2.00119865]
|
||||
[3.01346635]
|
||||
[3.99284588]]
|
||||
[[2.00035708]
|
||||
[2.9930344 ]
|
||||
[3.98966074]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -3007,9 +3009,9 @@ theta from own gd
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
|
||||
[[1.99997244]
|
||||
[3.00018876]
|
||||
[3.99983332]]
|
||||
[[2.00005688]
|
||||
[2.99969642]
|
||||
[4.00028552]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -3130,7 +3132,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
|
||||
return asarray(x, dtype=self.dtype)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x10cc52cd0>]
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x124252b80>]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
|
||||
@@ -3165,7 +3167,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><matplotlib.collections.PathCollection at 0x10fcfaeb0>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><matplotlib.collections.PathCollection at 0x125dbb640>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />
|
||||
|
||||
@@ -1566,10 +1566,8 @@ doconce format html week42.do.txt --no_mako -->
|
||||
<p><strong>Readings and videos.</strong></p>
|
||||
<ol class="simple">
|
||||
<li><p>These lecture notes</p></li>
|
||||
</ol>
|
||||
<!-- * [Video of lecture](https://youtu.be/0q5-PhovchQ) -->
|
||||
<!-- * [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesOct19.pdf) -->
|
||||
<ol class="simple">
|
||||
<li><p><a class="reference external" href="https://youtu.be/7B2F35gNj2Y">Video of lecture</a></p></li>
|
||||
<li><p><a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesOct14.pdf">Whiteboard notes</a></p></li>
|
||||
<li><p>For a more in depth discussion on neural networks we recommend Goodfellow et al chapters 6 and 7.</p></li>
|
||||
<li><p>Neural Networks demystified at <a class="reference external" href="https://www.youtube.com/watch?v=bxe2T-V8XRs&amp;list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&amp;ab_channel=WelchLabs">https://www.youtube.com/watch?v=bxe2T-V8XRs&amp;list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&amp;ab_channel=WelchLabs</a></p></li>
|
||||
<li><p>Building Neural Networks from scratch at <a class="reference external" href="https://www.youtube.com/watch?v=Wo5dMEP_BbI&amp;list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&amp;ab_channel=sentdex">https://www.youtube.com/watch?v=Wo5dMEP_BbI&amp;list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&amp;ab_channel=sentdex</a></p></li>
|
||||
@@ -3378,7 +3376,7 @@ the <em>Hadamard product</em>, meaning element-wise multiplication.</p>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Old accuracy on training data: 0.1440501043841336
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3714,7 +3712,7 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.19166666666666668
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3723,7 +3721,7 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3732,7 +3730,7 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3741,7 +3739,7 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3750,7 +3748,7 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3759,7 +3757,7 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3768,7 +3766,7 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3777,11 +3775,11 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.09166666666666666
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3790,11 +3788,11 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3803,11 +3801,11 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3816,11 +3814,11 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3829,11 +3827,11 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3842,7 +3840,7 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3851,11 +3849,11 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3864,11 +3862,11 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3877,11 +3875,11 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3890,11 +3888,11 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3903,11 +3901,11 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3916,11 +3914,11 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3929,11 +3927,11 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3942,11 +3940,11 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3999,15 +3997,15 @@ Accuracy score on test set: 0.07777777777777778
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_57345/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_59845/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -4289,30 +4287,32 @@ Accuracy score on test set: 0.8583333333333333
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
|
||||
Learning rate = 0.1
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.9055555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.8805555555555555
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.8805555555555555
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.8666666666666667
|
||||
|
||||
Learning rate = 1.0
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
|
||||
Learning rate = 1.0
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
@@ -4337,8 +4337,9 @@ Accuracy score on test set: 0.08888888888888889
|
||||
Learning rate = 1.0
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.09444444444444444
|
||||
|
||||
Learning rate = 10.0
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.17222222222222222
|
||||
</pre></div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
File diff suppressed because it is too large
Load Diff
@@ -16,18 +16,20 @@
|
||||
# **Readings and videos.**
|
||||
#
|
||||
# 1. These lecture notes
|
||||
# <!-- * [Video of lecture](https://youtu.be/0q5-PhovchQ) -->
|
||||
# <!-- * [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesOct19.pdf) -->
|
||||
#
|
||||
# 2. For a more in depth discussion on neural networks we recommend Goodfellow et al chapters 6 and 7.
|
||||
# 2. [Video of lecture](https://youtu.be/7B2F35gNj2Y)
|
||||
#
|
||||
# 3. Neural Networks demystified at <https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabs>
|
||||
# 3. [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesOct14.pdf)
|
||||
#
|
||||
# 4. Building Neural Networks from scratch at <https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdex>
|
||||
# 4. For a more in depth discussion on neural networks we recommend Goodfellow et al chapters 6 and 7.
|
||||
#
|
||||
# 5. Video on Neural Networks at <https://www.youtube.com/watch?v=CqOfi41LfDw>
|
||||
# 5. Neural Networks demystified at <https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabs>
|
||||
#
|
||||
# 6. Video on the back propagation algorithm at <https://www.youtube.com/watch?v=Ilg3gGewQ5U>
|
||||
# 6. Building Neural Networks from scratch at <https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdex>
|
||||
#
|
||||
# 7. Video on Neural Networks at <https://www.youtube.com/watch?v=CqOfi41LfDw>
|
||||
#
|
||||
# 8. Video on the back propagation algorithm at <https://www.youtube.com/watch?v=Ilg3gGewQ5U>
|
||||
#
|
||||
# I also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at <http://neuralnetworksanddeeplearning.com/chap4.html>.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user