lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated typos

Browse Source
This commit is contained in:
mhjensen
2018-10-04 05:35:51 +02:00
parent ff08e04b6e
commit 0c44eedada
64 changed files with 1948 additions and 2013 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/NeuralNet/html/._NeuralNet-bs049.html
+26 -51
View File
@@ -110,20 +110,19 @@ Automatically generated HTML file from DocOnce source
('Improving performance', 2, None, '___sec44'),
('Full object-oriented implementation', 2, None, '___sec45'),
('Evaluate model performance on test data', 2, None, '___sec46'),
('Adjust hyperparameters (if necessary, network architecture',
2,
None,
'___sec47'),
('scikit-learn implementation', 2, None, '___sec48'),
('Adjust hyperparameters', 2, None, '___sec47'),
('Visualization', 2, None, '___sec48'),
('scikit-learn implementation', 2, None, '___sec49'),
('Visualization', 2, None, '___sec50'),
('Building neural networks in Tensorflow and Keras',
2,
None,
'___sec49'),
('Tensorflow', 2, None, '___sec50'),
('Collect and pre-process data', 2, None, '___sec51'),
('Using TensorFlow backend', 2, None, '___sec52'),
('Optimizing and using gradient descent', 2, None, '___sec53'),
('Using Keras', 2, None, '___sec54')]}
'___sec51'),
('Tensorflow', 2, None, '___sec52'),
('Collect and pre-process data', 2, None, '___sec53'),
('Using TensorFlow backend', 2, None, '___sec54'),
('Optimizing and using gradient descent', 2, None, '___sec55'),
('Using Keras', 2, None, '___sec56')]}
end of tocinfo -->
<body>
@@ -208,14 +207,16 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._NeuralNet-bs045.html#___sec44" style="font-size: 80%;"><b>Improving performance</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs046.html#___sec45" style="font-size: 80%;"><b>Full object-oriented implementation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs047.html#___sec46" style="font-size: 80%;"><b>Evaluate model performance on test data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs048.html#___sec47" style="font-size: 80%;"><b>Adjust hyperparameters (if necessary, network architecture</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec48" style="font-size: 80%;"><b>scikit-learn implementation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs050.html#___sec49" style="font-size: 80%;"><b>Building neural networks in Tensorflow and Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs051.html#___sec50" style="font-size: 80%;"><b>Tensorflow</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs052.html#___sec51" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs053.html#___sec52" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs054.html#___sec53" style="font-size: 80%;"><b>Optimizing and using gradient descent</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs055.html#___sec54" style="font-size: 80%;"><b>Using Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs048.html#___sec47" style="font-size: 80%;"><b>Adjust hyperparameters</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec48" style="font-size: 80%;"><b>Visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs050.html#___sec49" style="font-size: 80%;"><b>scikit-learn implementation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs051.html#___sec50" style="font-size: 80%;"><b>Visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs052.html#___sec51" style="font-size: 80%;"><b>Building neural networks in Tensorflow and Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs053.html#___sec52" style="font-size: 80%;"><b>Tensorflow</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs054.html#___sec53" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs055.html#___sec54" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs056.html#___sec55" style="font-size: 80%;"><b>Optimizing and using gradient descent</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs057.html#___sec56" style="font-size: 80%;"><b>Using Keras</b></a></li>
</ul>
</li>
@@ -231,41 +232,13 @@ MathJax.Hub.Config({
<a name="part0049"></a>
<!-- !split -->
<h2 id="___sec48" class="anchor">scikit-learn implementation </h2>
<p>
scikit-learn is a machine learning library for Python. It focuses more on traditional machine learning methods, such as regression, clustering, decision trees, etc. As such, it has only two types of neural networks: Multi Layer Perceptron outputting continuous values, <em>MPLRegressor</em>, and Multi Layer Perceptron outputting labels, <em>MLPClassifier</em>. We will see how simple it is to use these classes.
<p>
scikit-learn implements a few improvements from our neural network, such as early stopping, a varying learning rate, different optimization methods, etc. We would therefore expect a better performance overall.
<h2 id="___sec48" class="anchor">Visualization </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.neural_network</span> <span style="color: #008000; font-weight: bold">import</span> MLPClassifier
<span style="color: #408080; font-style: italic"># store models for later use</span>
DNN_scikit <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros((<span style="color: #008000">len</span>(eta_vals), <span style="color: #008000">len</span>(lmbd_vals)), dtype<span style="color: #666666">=</span><span style="color: #008000">object</span>)
<span style="color: #008000; font-weight: bold">for</span> i, eta <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>(eta_vals):
<span style="color: #008000; font-weight: bold">for</span> j, lmbd <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>(lmbd_vals):
dnn <span style="color: #666666">=</span> MLPClassifier(hidden_layer_sizes<span style="color: #666666">=</span>(n_hidden_neurons), activation<span style="color: #666666">=</span><span style="color: #BA2121">&#39;logistic&#39;</span>,
alpha<span style="color: #666666">=</span>lmbd, learning_rate_init<span style="color: #666666">=</span>eta, max_iter<span style="color: #666666">=</span>epochs)
dnn<span style="color: #666666">.</span>fit(X_train, Y_train)
DNN_scikit[i][j] <span style="color: #666666">=</span> dnn
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Learning rate = &quot;</span>, eta)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Lambda = &quot;</span>, lmbd)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Accuracy score on test set: &quot;</span>, dnn<span style="color: #666666">.</span>score(X_test, Y_test))
<span style="color: #008000; font-weight: bold">print</span>()
</pre></div>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># optional</span>
<span style="color: #408080; font-style: italic"># visual representation of grid search</span>
<span style="color: #408080; font-style: italic"># uses seaborn heatmap, could probably do this in matplotlib</span>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># visual representation of grid search</span>
<span style="color: #408080; font-style: italic"># uses seaborn heatmap, you can also do this with matplotlib imshow</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">seaborn</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">sns</span>
sns<span style="color: #666666">.</span>set()
@@ -275,7 +248,7 @@ test_accuracy <span style="color: #666666">=</span> np<span style="color: #66666
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #008000">len</span>(eta_vals)):
<span style="color: #008000; font-weight: bold">for</span> j <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #008000">len</span>(lmbd_vals)):
dnn <span style="color: #666666">=</span> DNN_scikit[i][j]
dnn <span style="color: #666666">=</span> DNN_numpy[i][j]
train_pred <span style="color: #666666">=</span> dnn<span style="color: #666666">.</span>predict(X_train)
test_pred <span style="color: #666666">=</span> dnn<span style="color: #666666">.</span>predict(X_test)
@@ -320,6 +293,8 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._NeuralNet-bs053.html">54</a></li>
<li><a href="._NeuralNet-bs054.html">55</a></li>
<li><a href="._NeuralNet-bs055.html">56</a></li>
<li><a href="._NeuralNet-bs056.html">57</a></li>
<li><a href="._NeuralNet-bs057.html">58</a></li>
<li><a href="._NeuralNet-bs050.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->