upgrading neural nets with cans

2018-10-19 05:04:15 +02:00
parent 98f03271a5
commit 2e478fadab
258 changed files with 60828 additions and 1199 deletions
@@ -6,9 +6,9 @@ Automatically generated HTML file from DocOnce source
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
-<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning">
+<meta name="description" content="Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks">

-<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</title>
+<title>Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</title>

 <!-- Bootstrap style: bootstrap -->
 <link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -151,8 +151,30 @@ Automatically generated HTML file from DocOnce source
               '___sec67'),
              ('3D volumes of neurons', 2, None, '___sec68'),
              ('Layers used to build CNNs', 2, None, '___sec69'),
-              ('CNNs in brief', 2, None, '___sec70'),
-              ('CNNs in more detail', 2, None, '___sec71')]}
+              ('Transforming images', 2, None, '___sec70'),
+              ('CNNs in brief', 2, None, '___sec71'),
+              ('CNNs in more detail, building convolutional neural networks in '
+               'Tensorflow and Keras',
+               2,
+               None,
+               '___sec72'),
+              ('Setting it up', 2, None, '___sec73'),
+              ('The MNIST dataset again', 2, None, '___sec74'),
+              ('Strong correlations', 2, None, '___sec75'),
+              ('Layers of a CNN', 2, None, '___sec76'),
+              ('Systematic reduction', 2, None, '___sec77'),
+              ('Prerequisites: Collect and pre-process data',
+               2,
+               None,
+               '___sec78'),
+              ('Importing Keras and Tensorflow', 2, None, '___sec79'),
+              ('Using TensorFlow backend', 2, None, '___sec80'),
+              ('Train the model', 2, None, '___sec81'),
+              ('Visualizing the results', 2, None, '___sec82'),
+              ('Running with Keras', 2, None, '___sec83'),
+              ('Final part', 2, None, '___sec84'),
+              ('Final visualization', 2, None, '___sec85'),
+              ('Fun links', 2, None, '___sec86')]}
 end of tocinfo -->

 <body>
@@ -182,7 +204,7 @@ MathJax.Hub.Config({
      <span class="icon-bar"></span>
      <span class="icon-bar"></span>
    </button>
-    <a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning</a>
+    <a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks</a>
  </div>

  <div class="navbar-collapse collapse navbar-responsive-collapse">
@@ -260,8 +282,23 @@ MathJax.Hub.Config({
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs068.html#___sec67" style="font-size: 80%;"><b>Regular NNs don’t scale well to full images</b></a></li>
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs069.html#___sec68" style="font-size: 80%;"><b>3D volumes of neurons</b></a></li>
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs070.html#___sec69" style="font-size: 80%;"><b>Layers used to build CNNs</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
-     <!-- navigation toc: --> <li><a href="#___sec71" style="font-size: 80%;"><b>CNNs in more detail</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs071.html#___sec70" style="font-size: 80%;"><b>Transforming images</b></a></li>
+     <!-- navigation toc: --> <li><a href="#___sec71" style="font-size: 80%;"><b>CNNs in brief</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs073.html#___sec72" style="font-size: 80%;"><b>CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs074.html#___sec73" style="font-size: 80%;"><b>Setting it up</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs075.html#___sec74" style="font-size: 80%;"><b>The MNIST dataset again</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs076.html#___sec75" style="font-size: 80%;"><b>Strong correlations</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs077.html#___sec76" style="font-size: 80%;"><b>Layers of a CNN</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs078.html#___sec77" style="font-size: 80%;"><b>Systematic reduction</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs079.html#___sec78" style="font-size: 80%;"><b>Prerequisites: Collect and pre-process data</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs080.html#___sec79" style="font-size: 80%;"><b>Importing Keras and Tensorflow</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs081.html#___sec80" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs082.html#___sec81" style="font-size: 80%;"><b>Train the model</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs083.html#___sec82" style="font-size: 80%;"><b>Visualizing the results</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs084.html#___sec83" style="font-size: 80%;"><b>Running with Keras</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs085.html#___sec84" style="font-size: 80%;"><b>Final part</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs086.html#___sec85" style="font-size: 80%;"><b>Final visualization</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs087.html#___sec86" style="font-size: 80%;"><b>Fun links</b></a></li>

        </ul>
      </li>
@@ -277,11 +314,25 @@ MathJax.Hub.Config({
 <a name="part0072"></a>
 <!-- !split -->

-<h2 id="___sec71" class="anchor">CNNs in more detail </h2>
+<h2 id="___sec71" class="anchor">CNNs in brief </h2>

 <p>
-More material to come with examples.
+In summary:

+<ul>
+<li> A CNN architecture is in the simplest case a list of Layers that transform the image volume into an output volume (e.g. holding the class scores)</li>
+<li> There are a few distinct types of Layers (e.g. CONV/FC/RELU/POOL are by far the most popular)</li>
+<li> Each Layer accepts an input 3D volume and transforms it to an output 3D volume through a differentiable function</li>
+<li> Each Layer may or may not have parameters (e.g. CONV/FC do, RELU/POOL don&#8217;t)</li>
+<li> Each Layer may or may not have additional hyperparameters (e.g. CONV/FC/POOL do, RELU doesn&#8217;t)</li>
+</ul>
+
+For more material on convolutional networks, we strongly recommend
+the course
+<a href="https://www.uio.no/studier/emner/matnat/ifi/IN5400/index-eng.html" target="_self">IN5400 &#8211; Machine Learning for Image Analysis</a>
+and the slides of <a href="http://cs231n.github.io/convolutional-networks/" target="_self">CS231</a> which is taught at Stanford University (consistently ranked as one of the top computer science programs in the world). <a href="http://neuralnetworksanddeeplearning.com/chap6.html" target="_self">Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs</a>.
+
+<p>
 <p>
 <!-- navigation buttons at the bottom of the page -->
 <ul class="pagination">
@@ -297,6 +348,18 @@ More material to come with examples.
  <li><a href="._NeuralNet-bs070.html">71</a></li>
  <li><a href="._NeuralNet-bs071.html">72</a></li>
  <li class="active"><a href="._NeuralNet-bs072.html">73</a></li>
+  <li><a href="._NeuralNet-bs073.html">74</a></li>
+  <li><a href="._NeuralNet-bs074.html">75</a></li>
+  <li><a href="._NeuralNet-bs075.html">76</a></li>
+  <li><a href="._NeuralNet-bs076.html">77</a></li>
+  <li><a href="._NeuralNet-bs077.html">78</a></li>
+  <li><a href="._NeuralNet-bs078.html">79</a></li>
+  <li><a href="._NeuralNet-bs079.html">80</a></li>
+  <li><a href="._NeuralNet-bs080.html">81</a></li>
+  <li><a href="._NeuralNet-bs081.html">82</a></li>
+  <li><a href="">...</a></li>
+  <li><a href="._NeuralNet-bs087.html">88</a></li>
+  <li><a href="._NeuralNet-bs073.html">&raquo;</a></li>
 </ul>
 <!-- ------------------- end of main content --------------- -->