lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

upgrading neural nets with cans

Browse Source
This commit is contained in:
mhjensen
2018-10-19 05:04:15 +02:00
parent 98f03271a5
commit 2e478fadab
258 changed files with 60828 additions and 1199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/NeuralNet/html/._NeuralNet-bs070.html
+58 -15
View File
@@ -6,9 +6,9 @@ Automatically generated HTML file from DocOnce source
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning">
<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks">
<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</title>
<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -151,8 +151,30 @@ Automatically generated HTML file from DocOnce source
'___sec67'),
('3D volumes of neurons', 2, None, '___sec68'),
('Layers used to build CNNs', 2, None, '___sec69'),
('CNNs in brief', 2, None, '___sec70'),
('CNNs in more detail', 2, None, '___sec71')]}
('Transforming images', 2, None, '___sec70'),
('CNNs in brief', 2, None, '___sec71'),
('CNNs in more detail, building convolutional neural networks in '
'Tensorflow and Keras',
2,
None,
'___sec72'),
('Setting it up', 2, None, '___sec73'),
('The MNIST dataset again', 2, None, '___sec74'),
('Strong correlations', 2, None, '___sec75'),
('Layers of a CNN', 2, None, '___sec76'),
('Systematic reduction', 2, None, '___sec77'),
('Prerequisites: Collect and pre-process data',
2,
None,
'___sec78'),
('Importing Keras and Tensorflow', 2, None, '___sec79'),
('Using TensorFlow backend', 2, None, '___sec80'),
('Train the model', 2, None, '___sec81'),
('Visualizing the results', 2, None, '___sec82'),
('Running with Keras', 2, None, '___sec83'),
('Final part', 2, None, '___sec84'),
('Final visualization', 2, None, '___sec85'),
('Fun links', 2, None, '___sec86')]}
end of tocinfo -->
<body>
@@ -182,7 +204,7 @@ MathJax.Hub.Config({
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</a>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
@@ -260,8 +282,23 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._NeuralNet-bs068.html#___sec67" style="font-size: 80%;"><b>Regular NNs don’t scale well to full images</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs069.html#___sec68" style="font-size: 80%;"><b>3D volumes of neurons</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec69" style="font-size: 80%;"><b>Layers used to build CNNs</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs072.html#___sec71" style="font-size: 80%;"><b>CNNs in more detail</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>Transforming images</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs072.html#___sec71" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs073.html#___sec72" style="font-size: 80%;"><b>CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs074.html#___sec73" style="font-size: 80%;"><b>Setting it up</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs075.html#___sec74" style="font-size: 80%;"><b>The MNIST dataset again</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs076.html#___sec75" style="font-size: 80%;"><b>Strong correlations</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs077.html#___sec76" style="font-size: 80%;"><b>Layers of a CNN</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs078.html#___sec77" style="font-size: 80%;"><b>Systematic reduction</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs079.html#___sec78" style="font-size: 80%;"><b>Prerequisites: Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs080.html#___sec79" style="font-size: 80%;"><b>Importing Keras and Tensorflow</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs081.html#___sec80" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs082.html#___sec81" style="font-size: 80%;"><b>Train the model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs083.html#___sec82" style="font-size: 80%;"><b>Visualizing the results</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs084.html#___sec83" style="font-size: 80%;"><b>Running with Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs085.html#___sec84" style="font-size: 80%;"><b>Final part</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs086.html#___sec85" style="font-size: 80%;"><b>Final visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs087.html#___sec86" style="font-size: 80%;"><b>Fun links</b></a></li>
</ul>
</li>
@@ -291,16 +328,13 @@ will stack these layers to form a full CNN architecture.
A simple CNN for image classification could have the architecture:
<ul>
<li> INPUT (\( 32\times 32 \times 3 \)) will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B.</li>
<li> CONV layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as \( [32\times 32\times 12] \) if we decided to use 12 filters.</li>
<li> RELU layer will apply an elementwise activation function, such as the \( max(0,x) \) thresholding at zero. This leaves the size of the volume unchanged (\( [32\times 32\times 12] \)).</li>
<li> POOL layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as \( [16\times 16\times 12] \).</li>
<li> FC (i.e. fully-connected) layer will compute the class scores, resulting in volume of size \( [1\times 1\times 10] \), where each of the 10 numbers correspond to a class score, such as among the 10 categories of the MNIST images we considered above . As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume.</li>
<li> <b>INPUT</b> (\( 32\times 32 \times 3 \)) will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B.</li>
<li> <b>CONV</b> (convolutional )layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as \( [32\times 32\times 12] \) if we decided to use 12 filters.</li>
<li> <b>RELU</b> layer will apply an elementwise activation function, such as the \( max(0,x) \) thresholding at zero. This leaves the size of the volume unchanged (\( [32\times 32\times 12] \)).</li>
<li> <b>POOL</b> (pooling) layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as \( [16\times 16\times 12] \).</li>
<li> <b>FC</b> (i.e. fully-connected) layer will compute the class scores, resulting in volume of size \( [1\times 1\times 10] \), where each of the 10 numbers correspond to a class score, such as among the 10 categories of the MNIST images we considered above . As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume.</li>
</ul>
In this way, CNNs transform the original image layer by layer from the original pixel values to the final class scores. Note that some layers contain parameters and other don&#8217;t. In particular, the CONV/FC layers perform transformations that are a function of not only the activations in the input volume, but also of the parameters (the weights and biases of the neurons). On the other hand, the RELU/POOL layers will implement a fixed function. The parameters in the CONV/FC layers will be trained with gradient descent so that the class scores that the CNN computes are consistent with the labels in the training set for each image.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
@@ -318,6 +352,15 @@ In this way, CNNs transform the original image layer by layer from the original
<li class="active"><a href="._NeuralNet-bs070.html">71</a></li>
<li><a href="._NeuralNet-bs071.html">72</a></li>
<li><a href="._NeuralNet-bs072.html">73</a></li>
<li><a href="._NeuralNet-bs073.html">74</a></li>
<li><a href="._NeuralNet-bs074.html">75</a></li>
<li><a href="._NeuralNet-bs075.html">76</a></li>
<li><a href="._NeuralNet-bs076.html">77</a></li>
<li><a href="._NeuralNet-bs077.html">78</a></li>
<li><a href="._NeuralNet-bs078.html">79</a></li>
<li><a href="._NeuralNet-bs079.html">80</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs087.html">88</a></li>
<li><a href="._NeuralNet-bs071.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->