lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2022-10-26 06:25:50 +02:00
parent c9a3c8a972
commit b84e394d58
7 changed files with 316 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week43/html/week43-bs.html
+98 -4
View File
@@ -8,7 +8,8 @@ doconce format html week43.do.txt --html_style=bootstrap --pygments_html_style=d
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title></title>
<meta name="description" content="Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks">
<title>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<!-- doconce format html week43.do.txt --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=week43-bs --no_mako -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -201,10 +202,101 @@ MathJax.Hub.Config({
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week43-bs.html">Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#plans-for-week-43" style="font-size: 80%;">Plans for week 43</a></li>
<!-- navigation toc: --> <li><a href="#reading-recommendations" style="font-size: 80%;">Reading Recommendations</a></li>
<!-- navigation toc: --> <li><a href="#convolutional-neural-networks-recognizing-images" style="font-size: 80%;">Convolutional Neural Networks (recognizing images)</a></li>
<!-- navigation toc: --> <li><a href="#what-is-the-difference" style="font-size: 80%;">What is the Difference</a></li>
<!-- navigation toc: --> <li><a href="#neural-networks-vs-cnns" style="font-size: 80%;">Neural Networks vs CNNs</a></li>
<!-- navigation toc: --> <li><a href="#why-cnns-for-images-sound-files-medical-images-from-ct-scans-etc" style="font-size: 80%;">Why CNNS for images, sound files, medical images from CT scans etc?</a></li>
<!-- navigation toc: --> <li><a href="#regular-nns-don-t-scale-well-to-full-images" style="font-size: 80%;">Regular NNs don’t scale well to full images</a></li>
<!-- navigation toc: --> <li><a href="#3d-volumes-of-neurons" style="font-size: 80%;">3D volumes of neurons</a></li>
<!-- navigation toc: --> <li><a href="#layers-used-to-build-cnns" style="font-size: 80%;">Layers used to build CNNs</a></li>
<!-- navigation toc: --> <li><a href="#transforming-images" style="font-size: 80%;">Transforming images</a></li>
<!-- navigation toc: --> <li><a href="#cnns-in-brief" style="font-size: 80%;">CNNs in brief</a></li>
<!-- navigation toc: --> <li><a href="#key-idea" style="font-size: 80%;">Key Idea</a></li>
<!-- navigation toc: --> <li><a href="#mathematics-of-cnns" style="font-size: 80%;">Mathematics of CNNs</a></li>
<!-- navigation toc: --> <li><a href="#convolution-examples-polynomial-multiplication" style="font-size: 80%;">Convolution Examples: Polynomial multiplication</a></li>
<!-- navigation toc: --> <li><a href="#efficient-polynomial-multiplication" style="font-size: 80%;">Efficient Polynomial Multiplication</a></li>
<!-- navigation toc: --> <li><a href="#a-more-efficient-way-of-coding-the-above-convolution" style="font-size: 80%;">A more efficient way of coding the above Convolution</a></li>
<!-- navigation toc: --> <li><a href="#convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms" style="font-size: 80%;">Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)</a></li>
<!-- navigation toc: --> <li><a href="#principle-of-superposition" style="font-size: 80%;">Principle of Superposition</a></li>
<!-- navigation toc: --> <li><a href="#simple-code-example" style="font-size: 80%;">Simple Code Example</a></li>
<!-- navigation toc: --> <li><a href="#wrapping-up-fourier-transforms" style="font-size: 80%;">Wrapping up Fourier transforms</a></li>
<!-- navigation toc: --> <li><a href="#finding-the-coefficients" style="font-size: 80%;">Finding the Coefficients</a></li>
<!-- navigation toc: --> <li><a href="#final-words-on-fourier-transforms" style="font-size: 80%;">Final words on Fourier Transforms</a></li>
<!-- navigation toc: --> <li><a href="#two-dimensional-objects" style="font-size: 80%;">Two-dimensional Objects</a></li>
<!-- navigation toc: --> <li><a href="#cross-correlation" style="font-size: 80%;">Cross-Correlation</a></li>
<!-- navigation toc: --> <li><a href="#more-on-dimensionalities" style="font-size: 80%;">More on Dimensionalities</a></li>
<!-- navigation toc: --> <li><a href="#further-dimensionality-remarks" style="font-size: 80%;">Further Dimensionality Remarks</a></li>
<!-- navigation toc: --> <li><a href="#cnns-in-more-detail-lecture-from-in5400" style="font-size: 80%;">CNNs in more detail, Lecture from IN5400</a></li>
<!-- navigation toc: --> <li><a href="#cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras" style="font-size: 80%;">CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</a></li>
<!-- navigation toc: --> <li><a href="#setting-it-up" style="font-size: 80%;">Setting it up</a></li>
<!-- navigation toc: --> <li><a href="#the-mnist-dataset-again" style="font-size: 80%;">The MNIST dataset again</a></li>
<!-- navigation toc: --> <li><a href="#strong-correlations" style="font-size: 80%;">Strong correlations</a></li>
<!-- navigation toc: --> <li><a href="#layers-of-a-cnn" style="font-size: 80%;">Layers of a CNN</a></li>
<!-- navigation toc: --> <li><a href="#systematic-reduction" style="font-size: 80%;">Systematic reduction</a></li>
<!-- navigation toc: --> <li><a href="#prerequisites-collect-and-pre-process-data" style="font-size: 80%;">Prerequisites: Collect and pre-process data</a></li>
<!-- navigation toc: --> <li><a href="#importing-keras-and-tensorflow" style="font-size: 80%;">Importing Keras and Tensorflow</a></li>
<!-- navigation toc: --> <li><a href="#running-with-keras" style="font-size: 80%;">Running with Keras</a></li>
<!-- navigation toc: --> <li><a href="#final-part" style="font-size: 80%;">Final part</a></li>
<!-- navigation toc: --> <li><a href="#final-visualization" style="font-size: 80%;">Final visualization</a></li>
<!-- navigation toc: --> <li><a href="#the-cifar01-data-set" style="font-size: 80%;">The CIFAR01 data set</a></li>
<!-- navigation toc: --> <li><a href="#verifying-the-data-set" style="font-size: 80%;">Verifying the data set</a></li>
<!-- navigation toc: --> <li><a href="#set-up-the-model" style="font-size: 80%;">Set up the model</a></li>
<!-- navigation toc: --> <li><a href="#add-dense-layers-on-top" style="font-size: 80%;">Add Dense layers on top</a></li>
<!-- navigation toc: --> <li><a href="#compile-and-train-the-model" style="font-size: 80%;">Compile and train the model</a></li>
<!-- navigation toc: --> <li><a href="#finally-evaluate-the-model" style="font-size: 80%;">Finally, evaluate the model</a></li>
<!-- navigation toc: --> <li><a href="#recurrent-neural-networks-overarching-view" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="#set-up-of-an-rnn" style="font-size: 80%;">Set up of an RNN</a></li>
<!-- navigation toc: --> <li><a href="#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="#an-extrapolation-example" style="font-size: 80%;">An extrapolation example</a></li>
<!-- navigation toc: --> <li><a href="#formatting-the-data" style="font-size: 80%;">Formatting the Data</a></li>
<!-- navigation toc: --> <li><a href="#predicting-new-points-with-a-trained-recurrent-neural-network" style="font-size: 80%;">Predicting New Points With A Trained Recurrent Neural Network</a></li>
<!-- navigation toc: --> <li><a href="#other-things-to-try" style="font-size: 80%;">Other Things to Try</a></li>
<!-- navigation toc: --> <li><a href="#other-types-of-recurrent-neural-networks" style="font-size: 80%;">Other Types of Recurrent Neural Networks</a></li>
<!-- navigation toc: --> <li><a href="#generative-models" style="font-size: 80%;">Generative Models</a></li>
<!-- navigation toc: --> <li><a href="#generative-adversarial-networks" style="font-size: 80%;">Generative Adversarial Networks</a></li>
<!-- navigation toc: --> <li><a href="#discriminator" style="font-size: 80%;">Discriminator</a></li>
<!-- navigation toc: --> <li><a href="#learning-process" style="font-size: 80%;">Learning Process</a></li>
<!-- navigation toc: --> <li><a href="#more-about-the-learning-process" style="font-size: 80%;">More about the Learning Process</a></li>
<!-- navigation toc: --> <li><a href="#additional-references" style="font-size: 80%;">Additional References</a></li>
<!-- navigation toc: --> <li><a href="#writing-our-first-generative-adversarial-network" style="font-size: 80%;">Writing Our First Generative Adversarial Network</a></li>
<!-- navigation toc: --> <li><a href="#mnist-and-gans" style="font-size: 80%;">MNIST and GANs</a></li>
<!-- navigation toc: --> <li><a href="#other-models" style="font-size: 80%;">Other Models</a></li>
<!-- navigation toc: --> <li><a href="#training-step" style="font-size: 80%;">Training Step</a></li>
<!-- navigation toc: --> <li><a href="#checkpoints" style="font-size: 80%;">Checkpoints</a></li>
<!-- navigation toc: --> <li><a href="#exploring-the-latent-space" style="font-size: 80%;">Exploring the Latent Space</a></li>
<!-- navigation toc: --> <li><a href="#getting-results" style="font-size: 80%;">Getting Results</a></li>
<!-- navigation toc: --> <li><a href="#interpolating-between-mnist-digits" style="font-size: 80%;">Interpolating Between MNIST Digits</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<!-- ------------------- main content ---------------------- -->
<p>ATITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
<div class="jumbotron">
<center>
<h1>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
<center>
@@ -222,7 +314,9 @@ MathJax.Hub.Config({
<h4>Oct 26, 2022</h4>
</center> <!-- date -->
<br>
</p>
<!-- potential-jumbotron-button -->
</div> <!-- end jumbotron -->
<!-- !split -->
<h2 id="plans-for-week-43" class="anchor">Plans for week 43 </h2>
@@ -262,7 +356,7 @@ MathJax.Hub.Config({
<!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<ol>
<li> <a href="https://www.deeplearningbook.org/contents/convnets.html" target="_self">Goodfellow, Bengio, Courville, chapter 9</a></li>
<li> <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_self">Lectures from CS231 at Stanford</a></li>
<li> We will follow to a large extent the <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_self">lectures from CS231 at Stanford</a></li>
<li> <a href="http://neuralnetworksanddeeplearning.com/chap6.html" target="_self">Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs</a>.</li>
</ol>
</div>
doc/pub/week43/html/week43-reveal.html
+6 -4
View File
@@ -9,7 +9,8 @@ doconce format html week43-reveal.html week43-reveal reveal --html_slide_theme=b
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title></title>
<meta name="description" content="Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks">
<title>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</title>
<!-- reveal.js: https://lab.hakim.se/reveal-js/ -->
@@ -166,7 +167,9 @@ MathJax.Hub.Config({
<section>
<!-- ------------------- main content ---------------------- -->
<p>ATITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
<center>
<h1 style="text-align: center;">Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
<center>
@@ -184,7 +187,6 @@ MathJax.Hub.Config({
<h4>Oct 26, 2022</h4>
</center> <!-- date -->
<br>
</p>
<center style="font-size:80%">
@@ -229,7 +231,7 @@ MathJax.Hub.Config({
<p>
<ol>
<p><li> <a href="https://www.deeplearningbook.org/contents/convnets.html" target="_blank">Goodfellow, Bengio, Courville, chapter 9</a></li>
<p><li> <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">Lectures from CS231 at Stanford</a></li>
<p><li> We will follow to a large extent the <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">lectures from CS231 at Stanford</a></li>
<p><li> <a href="http://neuralnetworksanddeeplearning.com/chap6.html" target="_blank">Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs</a>.</li>
</ol>
</div>
doc/pub/week43/html/week43-solarized.html
+6 -4
View File
@@ -8,7 +8,8 @@ doconce format html week43.do.txt --pygments_html_style=perldoc --html_style=sol
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title></title>
<meta name="description" content="Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks">
<title>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</title>
<link href="https://cdn.rawgit.com/doconce/doconce/master/bundled/html_styles/style_solarized_box/css/solarized_light_code.css" rel="stylesheet" type="text/css" title="light"/>
<script src="https://cdn.rawgit.com/doconce/doconce/master/bundled/html_styles/style_solarized_box/js/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
@@ -229,7 +230,9 @@ MathJax.Hub.Config({
<!-- ------------------- main content ---------------------- -->
<p>ATITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
<center>
<h1>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
<center>
@@ -247,7 +250,6 @@ MathJax.Hub.Config({
<h4>Oct 26, 2022</h4>
</center> <!-- date -->
<br>
</p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="plans-for-week-43">Plans for week 43 </h2>
@@ -285,7 +287,7 @@ MathJax.Hub.Config({
<p>
<ol>
<li> <a href="https://www.deeplearningbook.org/contents/convnets.html" target="_blank">Goodfellow, Bengio, Courville, chapter 9</a></li>
<li> <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">Lectures from CS231 at Stanford</a></li>
<li> We will follow to a large extent the <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">lectures from CS231 at Stanford</a></li>
<li> <a href="http://neuralnetworksanddeeplearning.com/chap6.html" target="_blank">Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs</a>.</li>
</ol>
</div>
doc/pub/week43/html/week43.html
+6 -4
View File
@@ -8,7 +8,8 @@ doconce format html week43.do.txt --pygments_html_style=default --html_style=blo
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title></title>
<meta name="description" content="Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks">
<title>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</title>
<style type="text/css">
/* bloodish style */
body {
@@ -306,7 +307,9 @@ MathJax.Hub.Config({
<!-- ------------------- main content ---------------------- -->
<p>ATITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
<center>
<h1>Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
<center>
@@ -324,7 +327,6 @@ MathJax.Hub.Config({
<h4>Oct 26, 2022</h4>
</center> <!-- date -->
<br>
</p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="plans-for-week-43">Plans for week 43 </h2>
@@ -362,7 +364,7 @@ MathJax.Hub.Config({
<p>
<ol>
<li> <a href="https://www.deeplearningbook.org/contents/convnets.html" target="_blank">Goodfellow, Bengio, Courville, chapter 9</a></li>
<li> <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">Lectures from CS231 at Stanford</a></li>
<li> We will follow to a large extent the <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf" target="_blank">lectures from CS231 at Stanford</a></li>
<li> <a href="http://neuralnetworksanddeeplearning.com/chap6.html" target="_blank">Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs</a>.</li>
</ol>
</div>
doc/pub/week43/ipynb/ipynb-week43-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week43/ipynb/week43.ipynb
+198 -189
View File
File diff suppressed because it is too large Load Diff
doc/src/week43/week43.do.txt
+2 -2
View File
@@ -1,4 +1,4 @@
ATITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
TITLE: Week 43: Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks
AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
DATE: today
@@ -27,7 +27,7 @@ DATE: today
!bblock CNN readings
o "Goodfellow, Bengio, Courville, chapter 9":"https://www.deeplearningbook.org/contents/convnets.html"
o "Lectures from CS231 at Stanford":"http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf"
o We will follow to a large extent the "lectures from CS231 at Stanford":"http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf"
o "Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs":"http://neuralnetworksanddeeplearning.com/chap6.html".
!eblock