lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4f4c4e69464b1b4bc304ff84c9054e5ffdb7a4ee
FYS-STK4155/doc/pub/NeuralNet/html/._NeuralNet-bs044.html
T
mhjensen 0c44eedada updated typos
2018-10-04 05:35:51 +02:00

401 lines
25 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning">
<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Neural networks', 2, None, '___sec0'),
('Artificial neurons', 2, None, '___sec1'),
('Neural network types', 2, None, '___sec2'),
('Feed-forward neural networks', 2, None, '___sec3'),
('Convolutional Neural Network', 2, None, '___sec4'),
('Recurrent neural networks', 2, None, '___sec5'),
('Other types of networks', 2, None, '___sec6'),
('Multilayer perceptrons', 2, None, '___sec7'),
('Why multilayer perceptrons?', 2, None, '___sec8'),
('Mathematical model', 2, None, '___sec9'),
('Mathematical model', 2, None, '___sec10'),
('Mathematical model', 2, None, '___sec11'),
('Mathematical model', 2, None, '___sec12'),
('Mathematical model', 2, None, '___sec13'),
('Matrix-vector notation', 3, None, '___sec14'),
('Matrix-vector notation and activation', 3, None, '___sec15'),
('Activation functions', 3, None, '___sec16'),
('Activation functions, Logistic and Hyperbolic ones',
3,
None,
'___sec17'),
('Relevance', 3, None, '___sec18'),
('The multilayer perceptron (MLP)', 2, None, '___sec19'),
('From one to many layers, the universal approximation theorem',
2,
None,
'___sec20'),
('Deriving the back propagation code for a multilayer perceptron '
'model',
2,
None,
'___sec21'),
('Definitions', 2, None, '___sec22'),
('Derivatives and the chain rule', 2, None, '___sec23'),
('Derivative of the cost function', 2, None, '___sec24'),
('Bringing it together, first back propagation equation',
2,
None,
'___sec25'),
('Derivatives in terms of $z_j^L$', 2, None, '___sec26'),
('Bringing it together', 2, None, '___sec27'),
('Final back propagating equation', 2, None, '___sec28'),
('Setting up the Back propagation algorithm',
2,
None,
'___sec29'),
('Setting up a Multi-layer perceptron model for classification',
2,
None,
'___sec30'),
('Defining the cost function', 2, None, '___sec31'),
('Developing a code for doing neural networks with back '
'propagation',
2,
None,
'___sec32'),
('Collect and pre-process data', 2, None, '___sec33'),
('Train and test datasets', 2, None, '___sec34'),
('Define model and architecture', 2, None, '___sec35'),
('Layers', 2, None, '___sec36'),
('Weights and biases', 2, None, '___sec37'),
('Feed-forward pass', 2, None, '___sec38'),
('Matrix multiplication', 2, None, '___sec39'),
('Choose cost function and optimizer', 2, None, '___sec40'),
('Optimizing the cost function', 2, None, '___sec41'),
('Regularization', 2, None, '___sec42'),
('Matrix multiplication', 2, None, '___sec43'),
('Improving performance', 2, None, '___sec44'),
('Full object-oriented implementation', 2, None, '___sec45'),
('Evaluate model performance on test data', 2, None, '___sec46'),
('Adjust hyperparameters', 2, None, '___sec47'),
('Visualization', 2, None, '___sec48'),
('scikit-learn implementation', 2, None, '___sec49'),
('Visualization', 2, None, '___sec50'),
('Building neural networks in Tensorflow and Keras',
2,
None,
'___sec51'),
('Tensorflow', 2, None, '___sec52'),
('Collect and pre-process data', 2, None, '___sec53'),
('Using TensorFlow backend', 2, None, '___sec54'),
('Optimizing and using gradient descent', 2, None, '___sec55'),
('Using Keras', 2, None, '___sec56')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._NeuralNet-bs001.html#___sec0" style="font-size: 80%;"><b>Neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs002.html#___sec1" style="font-size: 80%;"><b>Artificial neurons</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs003.html#___sec2" style="font-size: 80%;"><b>Neural network types</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs004.html#___sec3" style="font-size: 80%;"><b>Feed-forward neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs005.html#___sec4" style="font-size: 80%;"><b>Convolutional Neural Network</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs006.html#___sec5" style="font-size: 80%;"><b>Recurrent neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs007.html#___sec6" style="font-size: 80%;"><b>Other types of networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs008.html#___sec7" style="font-size: 80%;"><b>Multilayer perceptrons</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs009.html#___sec8" style="font-size: 80%;"><b>Why multilayer perceptrons?</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs010.html#___sec9" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs011.html#___sec10" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs012.html#___sec11" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs013.html#___sec12" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs014.html#___sec13" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs015.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Matrix-vector notation</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs016.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Matrix-vector notation and activation</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs017.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Activation functions</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs018.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Activation functions, Logistic and Hyperbolic ones</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs019.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Relevance</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs020.html#___sec19" style="font-size: 80%;"><b>The multilayer perceptron (MLP)</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs021.html#___sec20" style="font-size: 80%;"><b>From one to many layers, the universal approximation theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs022.html#___sec21" style="font-size: 80%;"><b>Deriving the back propagation code for a multilayer perceptron model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs023.html#___sec22" style="font-size: 80%;"><b>Definitions</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs024.html#___sec23" style="font-size: 80%;"><b>Derivatives and the chain rule</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs025.html#___sec24" style="font-size: 80%;"><b>Derivative of the cost function</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs026.html#___sec25" style="font-size: 80%;"><b>Bringing it together, first back propagation equation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs027.html#___sec26" style="font-size: 80%;"><b>Derivatives in terms of \( z_j^L \)</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs028.html#___sec27" style="font-size: 80%;"><b>Bringing it together</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs029.html#___sec28" style="font-size: 80%;"><b>Final back propagating equation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs030.html#___sec29" style="font-size: 80%;"><b>Setting up the Back propagation algorithm</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs031.html#___sec30" style="font-size: 80%;"><b>Setting up a Multi-layer perceptron model for classification</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs032.html#___sec31" style="font-size: 80%;"><b>Defining the cost function</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs033.html#___sec32" style="font-size: 80%;"><b>Developing a code for doing neural networks with back propagation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs034.html#___sec33" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs035.html#___sec34" style="font-size: 80%;"><b>Train and test datasets</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs036.html#___sec35" style="font-size: 80%;"><b>Define model and architecture</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs037.html#___sec36" style="font-size: 80%;"><b>Layers</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs038.html#___sec37" style="font-size: 80%;"><b>Weights and biases</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs039.html#___sec38" style="font-size: 80%;"><b>Feed-forward pass</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs040.html#___sec39" style="font-size: 80%;"><b>Matrix multiplication</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs041.html#___sec40" style="font-size: 80%;"><b>Choose cost function and optimizer</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs042.html#___sec41" style="font-size: 80%;"><b>Optimizing the cost function</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs043.html#___sec42" style="font-size: 80%;"><b>Regularization</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec43" style="font-size: 80%;"><b>Matrix multiplication</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs045.html#___sec44" style="font-size: 80%;"><b>Improving performance</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs046.html#___sec45" style="font-size: 80%;"><b>Full object-oriented implementation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs047.html#___sec46" style="font-size: 80%;"><b>Evaluate model performance on test data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs048.html#___sec47" style="font-size: 80%;"><b>Adjust hyperparameters</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs049.html#___sec48" style="font-size: 80%;"><b>Visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs050.html#___sec49" style="font-size: 80%;"><b>scikit-learn implementation</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs051.html#___sec50" style="font-size: 80%;"><b>Visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs052.html#___sec51" style="font-size: 80%;"><b>Building neural networks in Tensorflow and Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs053.html#___sec52" style="font-size: 80%;"><b>Tensorflow</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs054.html#___sec53" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs055.html#___sec54" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs056.html#___sec55" style="font-size: 80%;"><b>Optimizing and using gradient descent</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs057.html#___sec56" style="font-size: 80%;"><b>Using Keras</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0044"></a>
<!-- !split -->
<h2 id="___sec43" class="anchor">Matrix multiplication </h2>
<p>
To more efficently train our network these equations are implemented using matrix operations.
The error in the output layer is calculated simply as
$$ \delta_L = \hat{y} - y = (n_{inputs}, n_{categories}) .$$
<p>
The gradient for the output weights is calculated as
$$ \nabla W_{L} = \hat{a}^T \delta_L = (n_{hidden}, n_{categories}) ,$$
<p>
where \( \hat{a} = (n_{inputs}, n_{hidden}) \). This simply means that we are summing up the gradients for each input.
Since we are going backwards we have to transpose the activation matrix.
<p>
The gradient with respect to the output bias is then
$$ \nabla \hat{b}_{L} = \sum_{i=1}^{n_{inputs}} \delta_L = (n_{categories}) .$$
<p>
The error in the hidden layer is
$$ \Delta_h = \delta_L W_{L}^T \circ f'(z_{h}) = \delta_L W_{L}^T \circ a_{h} \circ (1 - a_{h}) = (n_{inputs}, n_{hidden}) ,$$
<p>
where \( f'(a_{h}) \) is the derivative of the activation in the hidden layer. The matrix products mean
that we are summing up the products for each neuron in the output layer. The symbol \( \circ \) denotes
the <em>Hadamard product</em>, meaning element-wise multiplication.
<p>
This again gives us the gradients in the hidden layer:
$$ \nabla W_{h} = X^T \delta_h = (n_{features}, n_{hidden}) ,$$
$$ \nabla b_{h} = \sum_{i=1}^{n_{inputs}} \delta_h = (n_{hidden}) .$$
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># to categorical turns our integer vector into a onehot representation</span>
<span style="color: #408080; font-style: italic">#from keras.utils import to_categorical</span>
<span style="color: #408080; font-style: italic"># calculate the accuracy score of our model</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.metrics</span> <span style="color: #008000; font-weight: bold">import</span> accuracy_score
<span style="color: #408080; font-style: italic"># one-hot in numpy</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">to_categorical_numpy</span>(integer_vector):
n_inputs <span style="color: #666666">=</span> <span style="color: #008000">len</span>(integer_vector)
n_categories <span style="color: #666666">=</span> np<span style="color: #666666">.</span>max(integer_vector) <span style="color: #666666">+</span> <span style="color: #666666">1</span>
onehot_vector <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros((n_inputs, n_categories))
onehot_vector[<span style="color: #008000">range</span>(n_inputs), integer_vector] <span style="color: #666666">=</span> <span style="color: #666666">1</span>
<span style="color: #008000; font-weight: bold">return</span> onehot_vector
<span style="color: #408080; font-style: italic">#Y_train_onehot, Y_test_onehot = to_categorical(Y_train), to_categorical(Y_test)</span>
Y_train_onehot, Y_test_onehot <span style="color: #666666">=</span> to_categorical_numpy(Y_train), to_categorical_numpy(Y_test)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">feed_forward_train</span>(X):
<span style="color: #408080; font-style: italic"># weighted sum of inputs to the hidden layer</span>
z_h <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(X, hidden_weights) <span style="color: #666666">+</span> hidden_bias
<span style="color: #408080; font-style: italic"># activation in the hidden layer</span>
a_h <span style="color: #666666">=</span> sigmoid(z_h)
<span style="color: #408080; font-style: italic"># weighted sum of inputs to the output layer</span>
z_o <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(a_h, output_weights) <span style="color: #666666">+</span> output_bias
<span style="color: #408080; font-style: italic"># softmax output</span>
<span style="color: #408080; font-style: italic"># axis 0 holds each input and axis 1 the probabilities of each category</span>
exp_term <span style="color: #666666">=</span> np<span style="color: #666666">.</span>exp(z_o)
probabilities <span style="color: #666666">=</span> exp_term <span style="color: #666666">/</span> np<span style="color: #666666">.</span>sum(exp_term, axis<span style="color: #666666">=1</span>, keepdims<span style="color: #666666">=</span><span style="color: #008000">True</span>)
<span style="color: #408080; font-style: italic"># for backpropagation need activations in hidden and output layers</span>
<span style="color: #008000; font-weight: bold">return</span> a_h, probabilities
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">backpropagation</span>(X, Y):
a_h, probabilities <span style="color: #666666">=</span> feed_forward_train(X)
<span style="color: #408080; font-style: italic"># error in the output layer</span>
error_output <span style="color: #666666">=</span> probabilities <span style="color: #666666">-</span> Y
<span style="color: #408080; font-style: italic"># error in the hidden layer</span>
error_hidden <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(error_output, output_weights<span style="color: #666666">.</span>T) <span style="color: #666666">*</span> a_h <span style="color: #666666">*</span> (<span style="color: #666666">1</span> <span style="color: #666666">-</span> a_h)
<span style="color: #408080; font-style: italic"># gradients for the output layer</span>
output_weights_gradient <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(a_h<span style="color: #666666">.</span>T, error_output)
output_bias_gradient <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sum(error_output, axis<span style="color: #666666">=0</span>)
<span style="color: #408080; font-style: italic"># gradient for the hidden layer</span>
hidden_weights_gradient <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(X<span style="color: #666666">.</span>T, error_hidden)
hidden_bias_gradient <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sum(error_hidden, axis<span style="color: #666666">=0</span>)
<span style="color: #008000; font-weight: bold">return</span> output_weights_gradient, output_bias_gradient, hidden_weights_gradient, hidden_bias_gradient
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Old accuracy on training data: &quot;</span> <span style="color: #666666">+</span> <span style="color: #008000">str</span>(accuracy_score(predict(X_train), Y_train)))
eta <span style="color: #666666">=</span> <span style="color: #666666">0.01</span>
lmbd <span style="color: #666666">=</span> <span style="color: #666666">0.01</span>
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">1000</span>):
<span style="color: #408080; font-style: italic"># calculate gradients</span>
dWo, dBo, dWh, dBh <span style="color: #666666">=</span> backpropagation(X_train, Y_train_onehot)
<span style="color: #408080; font-style: italic"># regularization term gradients</span>
dWo <span style="color: #666666">+=</span> lmbd <span style="color: #666666">*</span> output_weights
dWh <span style="color: #666666">+=</span> lmbd <span style="color: #666666">*</span> hidden_weights
<span style="color: #408080; font-style: italic"># update weights and biases</span>
output_weights <span style="color: #666666">-=</span> eta <span style="color: #666666">*</span> dWo
output_bias <span style="color: #666666">-=</span> eta <span style="color: #666666">*</span> dBo
hidden_weights <span style="color: #666666">-=</span> eta <span style="color: #666666">*</span> dWh
hidden_bias <span style="color: #666666">-=</span> eta <span style="color: #666666">*</span> dBh
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;New accuracy on training data: &quot;</span> <span style="color: #666666">+</span> <span style="color: #008000">str</span>(accuracy_score(predict(X_train), Y_train)))
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._NeuralNet-bs043.html">&laquo;</a></li>
<li><a href="._NeuralNet-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs036.html">37</a></li>
<li><a href="._NeuralNet-bs037.html">38</a></li>
<li><a href="._NeuralNet-bs038.html">39</a></li>
<li><a href="._NeuralNet-bs039.html">40</a></li>
<li><a href="._NeuralNet-bs040.html">41</a></li>
<li><a href="._NeuralNet-bs041.html">42</a></li>
<li><a href="._NeuralNet-bs042.html">43</a></li>
<li><a href="._NeuralNet-bs043.html">44</a></li>
<li class="active"><a href="._NeuralNet-bs044.html">45</a></li>
<li><a href="._NeuralNet-bs045.html">46</a></li>
<li><a href="._NeuralNet-bs046.html">47</a></li>
<li><a href="._NeuralNet-bs047.html">48</a></li>
<li><a href="._NeuralNet-bs048.html">49</a></li>
<li><a href="._NeuralNet-bs049.html">50</a></li>
<li><a href="._NeuralNet-bs050.html">51</a></li>
<li><a href="._NeuralNet-bs051.html">52</a></li>
<li><a href="._NeuralNet-bs052.html">53</a></li>
<li><a href="._NeuralNet-bs053.html">54</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs057.html">58</a></li>
<li><a href="._NeuralNet-bs045.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink