lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

upgrading neural nets with cans

Browse Source
This commit is contained in:
mhjensen
2018-10-19 05:04:15 +02:00
parent 98f03271a5
commit 2e478fadab
258 changed files with 60828 additions and 1199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/NeuralNet/html/._NeuralNet-bs068.html
+58 -11
View File
@@ -6,9 +6,9 @@ Automatically generated HTML file from DocOnce source
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning">
<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks">
<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</title>
<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -151,8 +151,30 @@ Automatically generated HTML file from DocOnce source
'___sec67'),
('3D volumes of neurons', 2, None, '___sec68'),
('Layers used to build CNNs', 2, None, '___sec69'),
('CNNs in brief', 2, None, '___sec70'),
('CNNs in more detail', 2, None, '___sec71')]}
('Transforming images', 2, None, '___sec70'),
('CNNs in brief', 2, None, '___sec71'),
('CNNs in more detail, building convolutional neural networks in '
'Tensorflow and Keras',
2,
None,
'___sec72'),
('Setting it up', 2, None, '___sec73'),
('The MNIST dataset again', 2, None, '___sec74'),
('Strong correlations', 2, None, '___sec75'),
('Layers of a CNN', 2, None, '___sec76'),
('Systematic reduction', 2, None, '___sec77'),
('Prerequisites: Collect and pre-process data',
2,
None,
'___sec78'),
('Importing Keras and Tensorflow', 2, None, '___sec79'),
('Using TensorFlow backend', 2, None, '___sec80'),
('Train the model', 2, None, '___sec81'),
('Visualizing the results', 2, None, '___sec82'),
('Running with Keras', 2, None, '___sec83'),
('Final part', 2, None, '___sec84'),
('Final visualization', 2, None, '___sec85'),
('Fun links', 2, None, '___sec86')]}
end of tocinfo -->
<body>
@@ -182,7 +204,7 @@ MathJax.Hub.Config({
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</a>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
@@ -260,8 +282,23 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec67" style="font-size: 80%;"><b>Regular NNs don’t scale well to full images</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs069.html#___sec68" style="font-size: 80%;"><b>3D volumes of neurons</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs070.html#___sec69" style="font-size: 80%;"><b>Layers used to build CNNs</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs072.html#___sec71" style="font-size: 80%;"><b>CNNs in more detail</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>Transforming images</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs072.html#___sec71" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs073.html#___sec72" style="font-size: 80%;"><b>CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs074.html#___sec73" style="font-size: 80%;"><b>Setting it up</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs075.html#___sec74" style="font-size: 80%;"><b>The MNIST dataset again</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs076.html#___sec75" style="font-size: 80%;"><b>Strong correlations</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs077.html#___sec76" style="font-size: 80%;"><b>Layers of a CNN</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs078.html#___sec77" style="font-size: 80%;"><b>Systematic reduction</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs079.html#___sec78" style="font-size: 80%;"><b>Prerequisites: Collect and pre-process data</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs080.html#___sec79" style="font-size: 80%;"><b>Importing Keras and Tensorflow</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs081.html#___sec80" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs082.html#___sec81" style="font-size: 80%;"><b>Train the model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs083.html#___sec82" style="font-size: 80%;"><b>Visualizing the results</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs084.html#___sec83" style="font-size: 80%;"><b>Running with Keras</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs085.html#___sec84" style="font-size: 80%;"><b>Final part</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs086.html#___sec85" style="font-size: 80%;"><b>Final visualization</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs087.html#___sec86" style="font-size: 80%;"><b>Fun links</b></a></li>
</ul>
</li>
@@ -283,14 +320,17 @@ MathJax.Hub.Config({
As an example, consider
an image of size \( 32\times 32\times 3 \) (32 wide, 32 high, 3 color channels), so a
single fully-connected neuron in a first hidden layer of a regular
Neural Network would have \( 32*32*3 = 3072 \) weights. This amount still
Neural Network would have \( 32\times 32\times 3 = 3072 \) weights. This amount still
seems manageable, but clearly this fully-connected structure does not
scale to larger images. For example, an image of more respectable
size, say \( 200\times 200\times 3 \), would lead to neurons that have
\( 200*200*3 =120,000 \) weights. Moreover, we would almost certainly want to have
several such neurons, so the parameters would add up quickly! Clearly,
\( 200\times 200\times 3 = 120,000 \) weights.
<p>
We could have
several such neurons, and the parameters would add up quickly! Clearly,
this full connectivity is wasteful and the huge number of parameters
would quickly lead to overfitting.
would quickly lead to possible overfitting.
<p>
<center> <!-- FIGURE -->
@@ -319,6 +359,13 @@ would quickly lead to overfitting.
<li><a href="._NeuralNet-bs070.html">71</a></li>
<li><a href="._NeuralNet-bs071.html">72</a></li>
<li><a href="._NeuralNet-bs072.html">73</a></li>
<li><a href="._NeuralNet-bs073.html">74</a></li>
<li><a href="._NeuralNet-bs074.html">75</a></li>
<li><a href="._NeuralNet-bs075.html">76</a></li>
<li><a href="._NeuralNet-bs076.html">77</a></li>
<li><a href="._NeuralNet-bs077.html">78</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs087.html">88</a></li>
<li><a href="._NeuralNet-bs069.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->