diff --git a/doc/pub/week42/html/._week42-bs000.html b/doc/pub/week42/html/._week42-bs000.html index bdb5f998a..d836f3987 100644 --- a/doc/pub/week42/html/._week42-bs000.html +++ b/doc/pub/week42/html/._week42-bs000.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -392,7 +410,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs001.html b/doc/pub/week42/html/._week42-bs001.html index e61672723..6b5cf1166 100644 --- a/doc/pub/week42/html/._week42-bs001.html +++ b/doc/pub/week42/html/._week42-bs001.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -401,7 +419,7 @@ MathJax.Hub.Config({
  • 10
  • 11
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs002.html b/doc/pub/week42/html/._week42-bs002.html index 4f774fbe8..adb90d7e3 100644 --- a/doc/pub/week42/html/._week42-bs002.html +++ b/doc/pub/week42/html/._week42-bs002.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -373,7 +391,7 @@ we will also study the usage of 11
  • 12
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs003.html b/doc/pub/week42/html/._week42-bs003.html index e8ce13760..736b8e062 100644 --- a/doc/pub/week42/html/._week42-bs003.html +++ b/doc/pub/week42/html/._week42-bs003.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -390,7 +408,7 @@ and output layer to any given precision.
  • 12
  • 13
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs004.html b/doc/pub/week42/html/._week42-bs004.html index d8b08302c..72a59fbf2 100644 --- a/doc/pub/week42/html/._week42-bs004.html +++ b/doc/pub/week42/html/._week42-bs004.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -393,7 +411,7 @@ for the solution to be unique.
  • 13
  • 14
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs005.html b/doc/pub/week42/html/._week42-bs005.html index cd9f39038..849c19ea5 100644 --- a/doc/pub/week42/html/._week42-bs005.html +++ b/doc/pub/week42/html/._week42-bs005.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -398,7 +416,7 @@ As described previously, an optimization method could be used to minimize the pa
  • 14
  • 15
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs006.html b/doc/pub/week42/html/._week42-bs006.html index 01691e7f7..22d2f9d49 100644 --- a/doc/pub/week42/html/._week42-bs006.html +++ b/doc/pub/week42/html/._week42-bs006.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -400,7 +418,7 @@ The neural net should then find the parameters \( P \) that minimizes the cost f
  • 15
  • 16
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs007.html b/doc/pub/week42/html/._week42-bs007.html index 7dcef6a2a..ecdd75585 100644 --- a/doc/pub/week42/html/._week42-bs007.html +++ b/doc/pub/week42/html/._week42-bs007.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -382,7 +400,7 @@ Automatic differentiation is a method of finding the derivatives numerically wit
  • 16
  • 17
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs008.html b/doc/pub/week42/html/._week42-bs008.html index 4146b5adc..1dfcfff2c 100644 --- a/doc/pub/week42/html/._week42-bs008.html +++ b/doc/pub/week42/html/._week42-bs008.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -400,7 +418,7 @@ Having an analytical solution at hand, it is possible to use it to compare how w
  • 17
  • 18
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs009.html b/doc/pub/week42/html/._week42-bs009.html index bf5fd99ed..d61f6ebe0 100644 --- a/doc/pub/week42/html/._week42-bs009.html +++ b/doc/pub/week42/html/._week42-bs009.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -391,7 +409,7 @@ In this example, \( \gamma = 2 \) and \( g_0 = 10 \).
  • 18
  • 19
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs010.html b/doc/pub/week42/html/._week42-bs010.html index 47d5983df..819950858 100644 --- a/doc/pub/week42/html/._week42-bs010.html +++ b/doc/pub/week42/html/._week42-bs010.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -385,7 +403,7 @@ with \( h_1(x) \) ensuring that \( g_t(x) \) satisfies some conditions and \( h_
  • 19
  • 20
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs011.html b/doc/pub/week42/html/._week42-bs011.html index 735600d13..57659d870 100644 --- a/doc/pub/week42/html/._week42-bs011.html +++ b/doc/pub/week42/html/._week42-bs011.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -397,7 +415,7 @@ $$
  • 20
  • 21
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs012.html b/doc/pub/week42/html/._week42-bs012.html index d154967f2..1b1fa9e4c 100644 --- a/doc/pub/week42/html/._week42-bs012.html +++ b/doc/pub/week42/html/._week42-bs012.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -406,7 +424,7 @@ is fulfilled as best as possible.
  • 21
  • 22
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs013.html b/doc/pub/week42/html/._week42-bs013.html index 5c09bfa71..91aa9eec4 100644 --- a/doc/pub/week42/html/._week42-bs013.html +++ b/doc/pub/week42/html/._week42-bs013.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -402,7 +420,7 @@ for an input value \( x \).
  • 22
  • 23
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs014.html b/doc/pub/week42/html/._week42-bs014.html index a87f4f87e..dc8704f7a 100644 --- a/doc/pub/week42/html/._week42-bs014.html +++ b/doc/pub/week42/html/._week42-bs014.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -400,7 +418,7 @@ $$
  • 23
  • 24
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs015.html b/doc/pub/week42/html/._week42-bs015.html index abfd4e253..57a841ecf 100644 --- a/doc/pub/week42/html/._week42-bs015.html +++ b/doc/pub/week42/html/._week42-bs015.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -385,7 +403,7 @@ The input layer will consist of \( N_{\text{input} } \) neurons, passing its ele
  • 24
  • 25
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs016.html b/doc/pub/week42/html/._week42-bs016.html index a6b4c6c17..a8e7a530d 100644 --- a/doc/pub/week42/html/._week42-bs016.html +++ b/doc/pub/week42/html/._week42-bs016.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -394,7 +412,7 @@ $$
  • 25
  • 26
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs017.html b/doc/pub/week42/html/._week42-bs017.html index ab6da065a..1bca0dab2 100644 --- a/doc/pub/week42/html/._week42-bs017.html +++ b/doc/pub/week42/html/._week42-bs017.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -395,7 +413,7 @@ $$
  • 26
  • 27
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs018.html b/doc/pub/week42/html/._week42-bs018.html index 5372b7921..89053614f 100644 --- a/doc/pub/week42/html/._week42-bs018.html +++ b/doc/pub/week42/html/._week42-bs018.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -410,7 +428,7 @@ it is assumes that the number of neurons in the output layer is one.
  • 27
  • 28
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs019.html b/doc/pub/week42/html/._week42-bs019.html index 697a39ca3..68256d1ef 100644 --- a/doc/pub/week42/html/._week42-bs019.html +++ b/doc/pub/week42/html/._week42-bs019.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -393,7 +411,7 @@ $$
  • 28
  • 29
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs020.html b/doc/pub/week42/html/._week42-bs020.html index 3c685e5d1..31cd5cb96 100644 --- a/doc/pub/week42/html/._week42-bs020.html +++ b/doc/pub/week42/html/._week42-bs020.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -394,7 +412,7 @@ In this case we seek a continuous range of values since we are approximating a f
  • 29
  • 30
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs021.html b/doc/pub/week42/html/._week42-bs021.html index 44bcac8a2..8b7e4e320 100644 --- a/doc/pub/week42/html/._week42-bs021.html +++ b/doc/pub/week42/html/._week42-bs021.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -393,7 +411,7 @@ Here, gradient descent with a constant step size has been chosen.
  • 30
  • 31
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs022.html b/doc/pub/week42/html/._week42-bs022.html index 1fc2c7771..38608ef8d 100644 --- a/doc/pub/week42/html/._week42-bs022.html +++ b/doc/pub/week42/html/._week42-bs022.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -415,7 +433,7 @@ $$
  • 31
  • 32
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs023.html b/doc/pub/week42/html/._week42-bs023.html index b2df8c5f1..23a977b9c 100644 --- a/doc/pub/week42/html/._week42-bs023.html +++ b/doc/pub/week42/html/._week42-bs023.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -525,7 +543,7 @@ MathJax.Hub.Config({
  • 32
  • 33
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs024.html b/doc/pub/week42/html/._week42-bs024.html index 0ae0e62cb..aa491419e 100644 --- a/doc/pub/week42/html/._week42-bs024.html +++ b/doc/pub/week42/html/._week42-bs024.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -545,7 +563,7 @@ The number of neurons within each hidden layer are given as a list of integers i
  • 33
  • 34
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs025.html b/doc/pub/week42/html/._week42-bs025.html index f42658e07..0d35b2101 100644 --- a/doc/pub/week42/html/._week42-bs025.html +++ b/doc/pub/week42/html/._week42-bs025.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -397,7 +415,7 @@ Here, we stay with a more simple approach and implement for comparison, the simp
  • 34
  • 35
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs026.html b/doc/pub/week42/html/._week42-bs026.html index 928da4c2d..2f9a16bff 100644 --- a/doc/pub/week42/html/._week42-bs026.html +++ b/doc/pub/week42/html/._week42-bs026.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -393,7 +411,7 @@ In this example, we let \( \alpha = 2 \), \( A = 1 \), and \( g_0 = 1.2 \).
  • 35
  • 36
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs027.html b/doc/pub/week42/html/._week42-bs027.html index 6407b2344..51a25b4db 100644 --- a/doc/pub/week42/html/._week42-bs027.html +++ b/doc/pub/week42/html/._week42-bs027.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -398,7 +416,7 @@ $$
  • 36
  • 37
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs028.html b/doc/pub/week42/html/._week42-bs028.html index ced9a147a..7450e5623 100644 --- a/doc/pub/week42/html/._week42-bs028.html +++ b/doc/pub/week42/html/._week42-bs028.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -547,7 +565,7 @@ The network will be the similar as for the exponential decay example, but with s
  • 37
  • 38
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs029.html b/doc/pub/week42/html/._week42-bs029.html index 619d24e31..cab10f322 100644 --- a/doc/pub/week42/html/._week42-bs029.html +++ b/doc/pub/week42/html/._week42-bs029.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -501,7 +519,7 @@ extending the program that uses the network using Autograd:
  • 38
  • 39
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs030.html b/doc/pub/week42/html/._week42-bs030.html index 9968562a5..bfa68bff7 100644 --- a/doc/pub/week42/html/._week42-bs030.html +++ b/doc/pub/week42/html/._week42-bs030.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -403,7 +421,7 @@ In addition, it could be interesting to see how a typical method for numerically
  • 39
  • 40
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs031.html b/doc/pub/week42/html/._week42-bs031.html index 1d537b9dd..28043a4c8 100644 --- a/doc/pub/week42/html/._week42-bs031.html +++ b/doc/pub/week42/html/._week42-bs031.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -410,7 +428,7 @@ $$
  • 40
  • 41
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs032.html b/doc/pub/week42/html/._week42-bs032.html index 97df5525e..1cfbb4412 100644 --- a/doc/pub/week42/html/._week42-bs032.html +++ b/doc/pub/week42/html/._week42-bs032.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -531,7 +549,7 @@ MathJax.Hub.Config({
  • 41
  • 42
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs033.html b/doc/pub/week42/html/._week42-bs033.html index fb76e1d1b..06fa1b87e 100644 --- a/doc/pub/week42/html/._week42-bs033.html +++ b/doc/pub/week42/html/._week42-bs033.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -470,7 +488,7 @@ which makes it possible to solve for the vector \( \boldsymbol{g} \).
  • 42
  • 43
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs034.html b/doc/pub/week42/html/._week42-bs034.html index 5c9bc5e58..cbb761dc8 100644 --- a/doc/pub/week42/html/._week42-bs034.html +++ b/doc/pub/week42/html/._week42-bs034.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -574,7 +592,7 @@ We can then compare the result from this numerical scheme with the output from o
  • 43
  • 44
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs035.html b/doc/pub/week42/html/._week42-bs035.html index 57dd558e5..507cfa53d 100644 --- a/doc/pub/week42/html/._week42-bs035.html +++ b/doc/pub/week42/html/._week42-bs035.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -395,7 +413,7 @@ where \( f \) is an expression involving all kinds of possible mixed derivatives
  • 44
  • 45
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs036.html b/doc/pub/week42/html/._week42-bs036.html index 1ebc9758c..251e92dd9 100644 --- a/doc/pub/week42/html/._week42-bs036.html +++ b/doc/pub/week42/html/._week42-bs036.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -395,7 +413,7 @@ The role of the function \( h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P)) \), is to ensu
  • 45
  • 46
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs037.html b/doc/pub/week42/html/._week42-bs037.html index 80370e53a..832b00ada 100644 --- a/doc/pub/week42/html/._week42-bs037.html +++ b/doc/pub/week42/html/._week42-bs037.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -395,7 +413,7 @@ $$
  • 46
  • 47
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs038.html b/doc/pub/week42/html/._week42-bs038.html index 5dc91ccc6..4d67376ae 100644 --- a/doc/pub/week42/html/._week42-bs038.html +++ b/doc/pub/week42/html/._week42-bs038.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -391,7 +409,7 @@ $$
  • 47
  • 48
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs039.html b/doc/pub/week42/html/._week42-bs039.html index 01b9a0f48..ef2217815 100644 --- a/doc/pub/week42/html/._week42-bs039.html +++ b/doc/pub/week42/html/._week42-bs039.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -397,7 +415,7 @@ with \( u(x) \) being some given function.
  • 48
  • 49
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs040.html b/doc/pub/week42/html/._week42-bs040.html index 44cf7c4b5..1de5b9aa6 100644 --- a/doc/pub/week42/html/._week42-bs040.html +++ b/doc/pub/week42/html/._week42-bs040.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -404,7 +422,7 @@ First, we will look into how Autograd could be used in a network tailored to sol
  • 49
  • 50
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs041.html b/doc/pub/week42/html/._week42-bs041.html index 107456adf..1c46d4057 100644 --- a/doc/pub/week42/html/._week42-bs041.html +++ b/doc/pub/week42/html/._week42-bs041.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -434,7 +452,7 @@ network at each possible pair \( (x,t) \), given an array for the desired
  • 50
  • 51
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs042.html b/doc/pub/week42/html/._week42-bs042.html index 8744d2672..64db9f808 100644 --- a/doc/pub/week42/html/._week42-bs042.html +++ b/doc/pub/week42/html/._week42-bs042.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -401,7 +419,7 @@ since \( (0) = u(1) = 0 \) and \( u(x) = \sin(\pi x) \).
  • 51
  • 52
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs043.html b/doc/pub/week42/html/._week42-bs043.html index 9815fe948..26ac6cfd9 100644 --- a/doc/pub/week42/html/._week42-bs043.html +++ b/doc/pub/week42/html/._week42-bs043.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -439,7 +457,7 @@ mixed derivatives of \( g(x,t) \).
  • 52
  • 53
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs044.html b/doc/pub/week42/html/._week42-bs044.html index 255c41526..63fe51927 100644 --- a/doc/pub/week42/html/._week42-bs044.html +++ b/doc/pub/week42/html/._week42-bs044.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -623,7 +641,7 @@ Using TensorFlow results in a much better execution time. Try it!
  • 53
  • 54
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs045.html b/doc/pub/week42/html/._week42-bs045.html index 69e2c3381..b0c8b4810 100644 --- a/doc/pub/week42/html/._week42-bs045.html +++ b/doc/pub/week42/html/._week42-bs045.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -401,7 +419,7 @@ where \( \frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} \) means the derivati
  • 54
  • 55
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs046.html b/doc/pub/week42/html/._week42-bs046.html index 56450bb71..7028af6e7 100644 --- a/doc/pub/week42/html/._week42-bs046.html +++ b/doc/pub/week42/html/._week42-bs046.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -402,7 +420,7 @@ In this example, let \( c = 1 \) and \( u(x) = \sin(\pi x) \) and \( v(x) = -\pi
  • 55
  • 56
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs047.html b/doc/pub/week42/html/._week42-bs047.html index 2ee79c61e..c2dc521fc 100644 --- a/doc/pub/week42/html/._week42-bs047.html +++ b/doc/pub/week42/html/._week42-bs047.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -396,7 +414,7 @@ Note that this trial solution satisfies the conditions only if \( u(0) = v(0) =
  • 56
  • 57
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs048.html b/doc/pub/week42/html/._week42-bs048.html index d3b52cf64..df1bdba67 100644 --- a/doc/pub/week42/html/._week42-bs048.html +++ b/doc/pub/week42/html/._week42-bs048.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -384,7 +402,7 @@ $$
  • 57
  • 58
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs049.html b/doc/pub/week42/html/._week42-bs049.html index 3d66dc95c..786b57a9f 100644 --- a/doc/pub/week42/html/._week42-bs049.html +++ b/doc/pub/week42/html/._week42-bs049.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -601,7 +619,7 @@ MathJax.Hub.Config({
  • 58
  • 59
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs050.html b/doc/pub/week42/html/._week42-bs050.html index 860c886a0..71ff8ed29 100644 --- a/doc/pub/week42/html/._week42-bs050.html +++ b/doc/pub/week42/html/._week42-bs050.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -383,7 +401,7 @@ MathJax.Hub.Config({
  • 59
  • 60
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs051.html b/doc/pub/week42/html/._week42-bs051.html index bad554ea4..eaecaa6c0 100644 --- a/doc/pub/week42/html/._week42-bs051.html +++ b/doc/pub/week42/html/._week42-bs051.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -411,7 +429,7 @@ Another good read is the article here 60
  • 61
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs052.html b/doc/pub/week42/html/._week42-bs052.html index b3e8dbea1..bbf7e558a 100644 --- a/doc/pub/week42/html/._week42-bs052.html +++ b/doc/pub/week42/html/._week42-bs052.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -386,7 +404,7 @@ before the transformation.
  • 61
  • 62
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs053.html b/doc/pub/week42/html/._week42-bs053.html index 72cd50ce9..badd1e515 100644 --- a/doc/pub/week42/html/._week42-bs053.html +++ b/doc/pub/week42/html/._week42-bs053.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -400,7 +418,7 @@ in the input).
  • 62
  • 63
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs054.html b/doc/pub/week42/html/._week42-bs054.html index cc257de62..d0c2972f9 100644 --- a/doc/pub/week42/html/._week42-bs054.html +++ b/doc/pub/week42/html/._week42-bs054.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -400,7 +418,7 @@ would quickly lead to possible overfitting.
  • 63
  • 64
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs055.html b/doc/pub/week42/html/._week42-bs055.html index f61725188..37b126799 100644 --- a/doc/pub/week42/html/._week42-bs055.html +++ b/doc/pub/week42/html/._week42-bs055.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -412,7 +430,7 @@ dimension.
  • 64
  • 65
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs056.html b/doc/pub/week42/html/._week42-bs056.html index 9a6a3f3bb..d4b0d36c3 100644 --- a/doc/pub/week42/html/._week42-bs056.html +++ b/doc/pub/week42/html/._week42-bs056.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -395,7 +413,7 @@ A simple CNN for image classification could have the architecture:
  • 65
  • 66
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs057.html b/doc/pub/week42/html/._week42-bs057.html index 20e29e590..ceca9f7f2 100644 --- a/doc/pub/week42/html/._week42-bs057.html +++ b/doc/pub/week42/html/._week42-bs057.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -391,7 +409,7 @@ are consistent with the labels in the training set for each image.
  • 66
  • 67
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs058.html b/doc/pub/week42/html/._week42-bs058.html index 6ad72e677..89b2ce73c 100644 --- a/doc/pub/week42/html/._week42-bs058.html +++ b/doc/pub/week42/html/._week42-bs058.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -393,7 +411,7 @@ and the slides of 67
  • 68
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs059.html b/doc/pub/week42/html/._week42-bs059.html index f6ce5856f..539716f0c 100644 --- a/doc/pub/week42/html/._week42-bs059.html +++ b/doc/pub/week42/html/._week42-bs059.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,18 +367,41 @@ MathJax.Hub.Config({ -

    CNNs in more detail, building convolutional neural networks in Tensorflow and Keras

    +

    Mathematics of CNNs

    -As discussed above, CNNs are neural networks built from the assumption that the inputs -to the network are 2D images. This is important because the number of features or pixels in images -grows very fast with the image size, and an enormous number of weights and biases are needed in order to build an accurate network. +The mathematics of CNNs is based on the mathematical operation of +convolution. In mathematics (in particular in functional analysis), +convolution is represented by matheematical operation (integration, +summation etc) on two function in order to produce a third function +that expresses how the shape of one gets modified by the other. +Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning.

    -As before, we still have our input, a hidden layer and an output. What's novel about convolutional networks -are the convolutional and pooling layers stacked in pairs between the input and the hidden layer. -In addition, the data is no longer represented as a 2D feature matrix, instead each input is a number of 2D -matrices, typically 1 for each color dimension (Red, Green, Blue). +Mathematically, convolution is defined as follows (one-dimensional example): +Let us define a continuous function \( y(t) \) given by +$$ +y(t) = \int x(a) w(t-a) da, +$$ + +where \( x(a) \) represents a so-called input and \( w(t-a) \) is normally called the weight function or kernel. + +

    +The above integral is written in a more compact form as +$$ +y(t) = \left(x * w\right)(t). +$$ + +

    +The discretized version reads +$$ +y(t) = \sum_{a=-\infty}^{a=\infty}x(a)w(t-a). +$$ + +Computing the inverse of the above convolution operations is known as deconvolution. + +

    +How can we use this? And what does it mean? Let us study some familiar examples first.

    @@ -388,7 +429,7 @@ matrices, typically 1 for each color dimension (Red, Green, Blue).

  • 68
  • 69
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs060.html b/doc/pub/week42/html/._week42-bs060.html index 2cfb751fa..a05956e0b 100644 --- a/doc/pub/week42/html/._week42-bs060.html +++ b/doc/pub/week42/html/._week42-bs060.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,14 +367,11 @@ MathJax.Hub.Config({ -

    Setting it up

    +

    Convolution Examples: Polynomial multiplication

    -It means that to represent the entire -dataset of images, we require a 4D matrix or tensor. This tensor has the dimensions: -$$ -(n_{inputs},\, n_{pixels, width},\, n_{pixels, height},\, depth) . -$$ +We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function. +Let us remind of this and recast it in terms of the mathematical operation of convolution.

    @@ -384,7 +399,7 @@ $$

  • 69
  • 70
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs061.html b/doc/pub/week42/html/._week42-bs061.html index 136743ddb..a3b57b303 100644 --- a/doc/pub/week42/html/._week42-bs061.html +++ b/doc/pub/week42/html/._week42-bs061.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,20 +367,7 @@ MathJax.Hub.Config({ -

    The MNIST dataset again

    - -

    -The MNIST dataset consists of grayscale images with a pixel size of -\( 28\times 28 \), meaning we require \( 28 \times 28 = 724 \) weights to each -neuron in the first hidden layer. - -

    -If we were to analyze images of size \( 128\times 128 \) we would require -\( 128 \times 128 = 16384 \) weights to each neuron. Even worse if we were -dealing with color images, as most images are, we have an image matrix -of size \( 128\times 128 \) for each color dimension (Red, Green, Blue), -meaning 3 times the number of weights \( = 49152 \) are required for every -single neuron in the first hidden layer. +

    Convolution Examples: Probability Theory

    @@ -390,7 +395,7 @@ single neuron in the first hidden layer.

  • 70
  • 71
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs062.html b/doc/pub/week42/html/._week42-bs062.html index c4f97e513..f9461869d 100644 --- a/doc/pub/week42/html/._week42-bs062.html +++ b/doc/pub/week42/html/._week42-bs062.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,20 +367,176 @@ MathJax.Hub.Config({ -

    Strong correlations

    +

    Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)

    -Images typically have strong local correlations, meaning that a small -part of the image varies little from its neighboring regions. If for -example we have an image of a blue car, we can roughly assume that a -small blue part of the image is surrounded by other blue regions. +If one has several driving forces, \( F(t)=\sum_n F_n(t) \), one can find +the particular solution to each \( F_n \), \( x_{pn}(t) \), and the particular +solution for the entire driving force is + +$$ +\begin{equation} +x_p(t)=\sum_nx_{pn}(t). +\tag{21} +\end{equation} +$$

    -Therefore, instead of connecting every single pixel to a neuron in the -first hidden layer, as we have previously done with deep neural -networks, we can instead connect each neuron to a small part of the -image (in all 3 RGB depth dimensions). The size of each small area is -fixed, and known as a receptive. +This is known as the principal of superposition. It only applies when +the homogenous equation is linear. If there were an anharmonic term +such as \( x^3 \) in the homogenous equation, then when one summed various +solutions, \( x=(\sum_n x_n)^2 \), one would get cross +terms. Superposition is especially useful when \( F(t) \) can be written +as a sum of sinusoidal terms, because the solutions for each +sinusoidal (sine or cosine) term is analytic, as we saw above. + +

    +Driving forces are often periodic, even when they are not +sinusoidal. Periodicity implies that for some time \( \tau \) + +$$ +\begin{eqnarray} +F(t+\tau)=F(t). +\end{eqnarray} +$$ + +

    +One example of a non-sinusoidal periodic force is a square wave. Many +components in electric circuits are non-linear, e.g. diodes, which +makes many wave forms non-sinusoidal even when the circuits are being +driven by purely sinusoidal sources. + +

    +The code here shows a typical example of such a square wave generated using the functionality included in the scipy Python package. We have used a period of \( \tau=0.2 \). + +

    + + +

    import numpy as np
    +import math
    +from scipy import signal
    +import matplotlib.pyplot as plt
    +
    +# number of points                                                                                       
    +n = 500
    +# start and final times                                                                                  
    +t0 = 0.0
    +tn = 1.0
    +# Period                                                                                                 
    +t = np.linspace(t0, tn, n, endpoint=False)
    +SqrSignal = np.zeros(n)
    +SqrSignal = 1.0+signal.square(2*np.pi*5*t)
    +plt.plot(t, SqrSignal)
    +plt.ylim(-0.5, 2.5)
    +plt.show()
    +
    +

    +For the sinusoidal example the +period is \( \tau=2\pi/\omega \). However, higher harmonics can also +satisfy the periodicity requirement. In general, any force that +satisfies the periodicity requirement can be expressed as a sum over +harmonics, + +$$ +\begin{equation} +F(t)=\frac{f_0}{2}+\sum_{n>0} f_n\cos(2n\pi t/\tau)+g_n\sin(2n\pi t/\tau). +\tag{22} +\end{equation} +$$ + +

    +We can write down the answer for +\( x_{pn}(t) \), by substituting \( f_n/m \) or \( g_n/m \) for \( F_0/m \). By +writing each factor \( 2n\pi t/\tau \) as \( n\omega t \), with \( \omega\equiv +2\pi/\tau \), + +$$ +\begin{equation} +\tag{23} +F(t)=\frac{f_0}{2}+\sum_{n>0}f_n\cos(n\omega t)+g_n\sin(n\omega t). +\end{equation} +$$ + +

    +The solutions for \( x(t) \) then come from replacing \( \omega \) with +\( n\omega \) for each term in the particular solution, + +$$ +\begin{eqnarray} +x_p(t)&=&\frac{f_0}{2k}+\sum_{n>0} \alpha_n\cos(n\omega t-\delta_n)+\beta_n\sin(n\omega t-\delta_n),\\ +\nonumber +\alpha_n&=&\frac{f_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\beta_n&=&\frac{g_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\delta_n&=&\tan^{-1}\left(\frac{2\beta n\omega}{\omega_0^2-n^2\omega^2}\right). +\end{eqnarray} +$$ + +

    +Because the forces have been applied for a long time, any non-zero +damping eliminates the homogenous parts of the solution, so one need +only consider the particular solution for each \( n \). + +

    +The problem will considered solved if one can find expressions for the +coefficients \( f_n \) and \( g_n \), even though the solutions are expressed +as an infinite sum. The coefficients can be extracted from the +function \( F(t) \) by + +$$ +\begin{eqnarray} +\tag{24} +f_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\cos(2n\pi t/\tau),\\ +\nonumber +g_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\sin(2n\pi t/\tau). +\end{eqnarray} +$$ + +

    +To check the consistency of these expressions and to verify +Eq. (24), one can insert the expansion of \( F(t) \) in +Eq. (23) into the expression for the coefficients in +Eq. (24) and see whether + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~\left\{ +\frac{f_0}{2}+\sum_{m>0}f_m\cos(m\omega t)+g_m\sin(m\omega t) +\right\}\cos(n\omega t). +\end{eqnarray} +$$ + +

    +Immediately, one can throw away all the terms with \( g_m \) because they +convolute an even and an odd function. The term with \( f_0/2 \) +disappears because \( \cos(n\omega t) \) is equally positive and negative +over the interval and will integrate to zero. For all the terms +\( f_m\cos(m\omega t) \) appearing in the sum, one can use angle addition +formulas to see that \( \cos(m\omega t)\cos(n\omega +t)=(1/2)(\cos[(m+n)\omega t]+\cos[(m-n)\omega t] \). This will integrate +to zero unless \( m=n \). In that case the \( m=n \) term gives + +$$ +\begin{equation} +\int_{-\tau/2}^{\tau/2}dt~\cos^2(m\omega t)=\frac{\tau}{2}, +\tag{25} +\end{equation} +$$ + +

    +and + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~f_n/2\\ +\nonumber +&=&f_n~\checkmark. +\end{eqnarray} +$$ + +

    +The same method can be used to check for the consistency of \( g_n \).

    @@ -390,7 +564,7 @@ fixed, and known as a 71

  • 72
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs063.html b/doc/pub/week42/html/._week42-bs063.html index 8758ce04d..332aeb540 100644 --- a/doc/pub/week42/html/._week42-bs063.html +++ b/doc/pub/week42/html/._week42-bs063.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -347,26 +365,20 @@ MathJax.Hub.Config({

     

     

     

    - + -

    Layers of a CNN

    -The layers of a convolutional neural network arrange neurons in 3D: width, height and depth. -The input image is typically a square matrix of depth 3. +

    CNNs in more detail, building convolutional neural networks in Tensorflow and Keras

    -A convolution is performed on the image which outputs -a 3D volume of neurons. The weights to the input are arranged in a number of 2D matrices, known as filters. +As discussed above, CNNs are neural networks built from the assumption that the inputs +to the network are 2D images. This is important because the number of features or pixels in images +grows very fast with the image size, and an enormous number of weights and biases are needed in order to build an accurate network.

    -Each filter slides along the input image, taking the dot product -between each small part of the image and the filter, in all depth -dimensions. This is then passed through a non-linear function, -typically the Rectified Linear (ReLu) function, which serves as the -activation of the neurons in the first convolutional layer. This is -further passed through a pooling layer, which reduces the size of the -convolutional layer, e.g. by taking the maximum or average across some -small regions, and this serves as input to the next convolutional -layer. +As before, we still have our input, a hidden layer and an output. What's novel about convolutional networks +are the convolutional and pooling layers stacked in pairs between the input and the hidden layer. +In addition, the data is no longer represented as a 2D feature matrix, instead each input is a number of 2D +matrices, typically 1 for each color dimension (Red, Green, Blue).

    @@ -394,7 +406,7 @@ layer.

  • 72
  • 73
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs064.html b/doc/pub/week42/html/._week42-bs064.html index 5dadf870b..1b3079d7f 100644 --- a/doc/pub/week42/html/._week42-bs064.html +++ b/doc/pub/week42/html/._week42-bs064.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,17 +367,14 @@ MathJax.Hub.Config({ -

    Systematic reduction

    +

    Setting it up

    -By systematically reducing the size of the input volume, through -convolution and pooling, the network should create representations of -small parts of the input, and then from them assemble representations -of larger areas. The final pooling layer is flattened to serve as -input to a hidden layer, such that each neuron in the final pooling -layer is connected to every single neuron in the hidden layer. This -then serves as input to the output layer, e.g. a softmax output for -classification. +It means that to represent the entire +dataset of images, we require a 4D matrix or tensor. This tensor has the dimensions: +$$ +(n_{inputs},\, n_{pixels, width},\, n_{pixels, height},\, depth) . +$$

    @@ -387,7 +402,7 @@ classification.

  • 73
  • 74
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs065.html b/doc/pub/week42/html/._week42-bs065.html index c7c7dec6a..8b920ef6f 100644 --- a/doc/pub/week42/html/._week42-bs065.html +++ b/doc/pub/week42/html/._week42-bs065.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,51 +367,21 @@ MathJax.Hub.Config({ -

    Prerequisites: Collect and pre-process data

    +

    The MNIST dataset again

    +

    +The MNIST dataset consists of grayscale images with a pixel size of +\( 28\times 28 \), meaning we require \( 28 \times 28 = 724 \) weights to each +neuron in the first hidden layer. - -

    # import necessary packages
    -import numpy as np
    -import matplotlib.pyplot as plt
    -from sklearn import datasets
    +

    +If we were to analyze images of size \( 128\times 128 \) we would require +\( 128 \times 128 = 16384 \) weights to each neuron. Even worse if we were +dealing with color images, as most images are, we have an image matrix +of size \( 128\times 128 \) for each color dimension (Red, Green, Blue), +meaning 3 times the number of weights \( = 49152 \) are required for every +single neuron in the first hidden layer. - -# ensure the same random numbers appear every time -np.random.seed(0) - -# display images in notebook -%matplotlib inline -plt.rcParams['figure.figsize'] = (12,12) - - -# download MNIST dataset -digits = datasets.load_digits() - -# define inputs and labels -inputs = digits.images -labels = digits.target - -# RGB images have a depth of 3 -# our images are grayscale so they should have a depth of 1 -inputs = inputs[:,:,:,np.newaxis] - -print("inputs = (n_inputs, pixel_width, pixel_height, depth) = " + str(inputs.shape)) -print("labels = (n_inputs) = " + str(labels.shape)) - - -# choose some random images to display -n_inputs = len(inputs) -indices = np.arange(n_inputs) -random_indices = np.random.choice(indices, size=5) - -for i, image in enumerate(digits.images[random_indices]): - plt.subplot(1, 5, i+1) - plt.axis('off') - plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest') - plt.title("Label: %d" % digits.target[random_indices[i]]) -plt.show() -

    @@ -420,7 +408,7 @@ plt.show()

  • 74
  • 75
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs066.html b/doc/pub/week42/html/._week42-bs066.html index a460c99ee..e1c0abda7 100644 --- a/doc/pub/week42/html/._week42-bs066.html +++ b/doc/pub/week42/html/._week42-bs066.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,33 +367,21 @@ MathJax.Hub.Config({ -

    Importing Keras and Tensorflow

    +

    Strong correlations

    +

    +Images typically have strong local correlations, meaning that a small +part of the image varies little from its neighboring regions. If for +example we have an image of a blue car, we can roughly assume that a +small blue part of the image is surrounded by other blue regions. - -

    from tensorflow.keras import datasets, layers, models
    -from tensorflow.keras.layers import Input
    -from tensorflow.keras.models import Sequential      #This allows appending layers to existing models
    -from tensorflow.keras.layers import Dense           #This allows defining the characteristics of a particular layer
    -from tensorflow.keras import optimizers             #This allows using whichever optimiser we want (sgd,adam,RMSprop)
    -from tensorflow.keras import regularizers           #This allows using whichever regularizer we want (l1,l2,l1_l2)
    -from tensorflow.keras.utils import to_categorical   #This allows using categorical cross entropy as the cost function
    -#from tensorflow.keras import Conv2D
    -#from tensorflow.keras import MaxPooling2D
    -#from tensorflow.keras import Flatten
    +

    +Therefore, instead of connecting every single pixel to a neuron in the +first hidden layer, as we have previously done with deep neural +networks, we can instead connect each neuron to a small part of the +image (in all 3 RGB depth dimensions). The size of each small area is +fixed, and known as a receptive. -from sklearn.model_selection import train_test_split - -# representation of labels -labels = to_categorical(labels) - -# split into train and test data -# one-liner from scikit-learn library -train_size = 0.8 -test_size = 1 - train_size -X_train, X_test, Y_train, Y_test = train_test_split(inputs, labels, train_size=train_size, - test_size=test_size) -

    @@ -401,6 +407,8 @@ X_train, X_test, Y_train, Y_test = train_tes

  • 74
  • 75
  • 76
  • +
  • ...
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs067.html b/doc/pub/week42/html/._week42-bs067.html index b17dcb4a2..4c18556fe 100644 --- a/doc/pub/week42/html/._week42-bs067.html +++ b/doc/pub/week42/html/._week42-bs067.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,38 +367,25 @@ MathJax.Hub.Config({ -

    Running with Keras

    +

    Layers of a CNN

    +The layers of a convolutional neural network arrange neurons in 3D: width, height and depth. +The input image is typically a square matrix of depth 3.

    +A convolution is performed on the image which outputs +a 3D volume of neurons. The weights to the input are arranged in a number of 2D matrices, known as filters. - -

    def create_convolutional_neural_network_keras(input_shape, receptive_field,
    -                                              n_filters, n_neurons_connected, n_categories,
    -                                              eta, lmbd):
    -    model = Sequential()
    -    model.add(layers.Conv2D(n_filters, (receptive_field, receptive_field), input_shape=input_shape, padding='same',
    -              activation='relu', kernel_regularizer=regularizers.l2(lmbd)))
    -    model.add(layers.MaxPooling2D(pool_size=(2, 2)))
    -    model.add(layers.Flatten())
    -    model.add(layers.Dense(n_neurons_connected, activation='relu', kernel_regularizer=regularizers.l2(lmbd)))
    -    model.add(layers.Dense(n_categories, activation='softmax', kernel_regularizer=regularizers.l2(lmbd)))
    -    
    -    sgd = optimizers.SGD(lr=eta)
    -    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
    -    
    -    return model
    +

    +Each filter slides along the input image, taking the dot product +between each small part of the image and the filter, in all depth +dimensions. This is then passed through a non-linear function, +typically the Rectified Linear (ReLu) function, which serves as the +activation of the neurons in the first convolutional layer. This is +further passed through a pooling layer, which reduces the size of the +convolutional layer, e.g. by taking the maximum or average across some +small regions, and this serves as input to the next convolutional +layer. -epochs = 100 -batch_size = 100 -input_shape = X_train.shape[1:4] -receptive_field = 3 -n_filters = 10 -n_neurons_connected = 50 -n_categories = 10 - -eta_vals = np.logspace(-5, 1, 7) -lmbd_vals = np.logspace(-5, 1, 7) -

    @@ -405,6 +410,9 @@ lmbd_vals = np.

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • ...
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs068.html b/doc/pub/week42/html/._week42-bs068.html index 8b6d68051..4af7f5239 100644 --- a/doc/pub/week42/html/._week42-bs068.html +++ b/doc/pub/week42/html/._week42-bs068.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,28 +367,18 @@ MathJax.Hub.Config({ -

    Final part

    +

    Systematic reduction

    +By systematically reducing the size of the input volume, through +convolution and pooling, the network should create representations of +small parts of the input, and then from them assemble representations +of larger areas. The final pooling layer is flattened to serve as +input to a hidden layer, such that each neuron in the final pooling +layer is connected to every single neuron in the hidden layer. This +then serves as input to the output layer, e.g. a softmax output for +classification. - -

    CNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)
    -        
    -for i, eta in enumerate(eta_vals):
    -    for j, lmbd in enumerate(lmbd_vals):
    -        CNN = create_convolutional_neural_network_keras(input_shape, receptive_field,
    -                                              n_filters, n_neurons_connected, n_categories,
    -                                              eta, lmbd)
    -        CNN.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, verbose=0)
    -        scores = CNN.evaluate(X_test, Y_test)
    -        
    -        CNN_keras[i][j] = CNN
    -        
    -        print("Learning rate = ", eta)
    -        print("Lambda = ", lmbd)
    -        print("Test accuracy: %.3f" % scores[1])
    -        print()
    -

    @@ -394,6 +402,10 @@ MathJax.Hub.Config({

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • ...
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs069.html b/doc/pub/week42/html/._week42-bs069.html index 2a2669f5f..a6d31362f 100644 --- a/doc/pub/week42/html/._week42-bs069.html +++ b/doc/pub/week42/html/._week42-bs069.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,40 +367,49 @@ MathJax.Hub.Config({ -

    Final visualization

    - +

    Prerequisites: Collect and pre-process data

    -

    # visual representation of grid search
    -# uses seaborn heatmap, could probably do this in matplotlib
    -import seaborn as sns
    +
    # import necessary packages
    +import numpy as np
    +import matplotlib.pyplot as plt
    +from sklearn import datasets
     
    -sns.set()
     
    -train_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
    -test_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
    +# ensure the same random numbers appear every time
    +np.random.seed(0)
     
    -for i in range(len(eta_vals)):
    -    for j in range(len(lmbd_vals)):
    -        CNN = CNN_keras[i][j]
    +# display images in notebook
    +%matplotlib inline
    +plt.rcParams['figure.figsize'] = (12,12)
     
    -        train_accuracy[i][j] = CNN.evaluate(X_train, Y_train)[1]
    -        test_accuracy[i][j] = CNN.evaluate(X_test, Y_test)[1]
     
    -        
    -fig, ax = plt.subplots(figsize = (10, 10))
    -sns.heatmap(train_accuracy, annot=True, ax=ax, cmap="viridis")
    -ax.set_title("Training Accuracy")
    -ax.set_ylabel("$\eta$")
    -ax.set_xlabel("$\lambda$")
    -plt.show()
    +# download MNIST dataset
    +digits = datasets.load_digits()
     
    -fig, ax = plt.subplots(figsize = (10, 10))
    -sns.heatmap(test_accuracy, annot=True, ax=ax, cmap="viridis")
    -ax.set_title("Test Accuracy")
    -ax.set_ylabel("$\eta$")
    -ax.set_xlabel("$\lambda$")
    +# define inputs and labels
    +inputs = digits.images
    +labels = digits.target
    +
    +# RGB images have a depth of 3
    +# our images are grayscale so they should have a depth of 1
    +inputs = inputs[:,:,:,np.newaxis]
    +
    +print("inputs = (n_inputs, pixel_width, pixel_height, depth) = " + str(inputs.shape))
    +print("labels = (n_inputs) = " + str(labels.shape))
    +
    +
    +# choose some random images to display
    +n_inputs = len(inputs)
    +indices = np.arange(n_inputs)
    +random_indices = np.random.choice(indices, size=5)
    +
    +for i, image in enumerate(digits.images[random_indices]):
    +    plt.subplot(1, 5, i+1)
    +    plt.axis('off')
    +    plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
    +    plt.title("Label: %d" % digits.target[random_indices[i]])
     plt.show()
     

    @@ -407,6 +434,11 @@ plt.show()

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • ...
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs070.html b/doc/pub/week42/html/._week42-bs070.html index c8a6e82e8..be0718ece 100644 --- a/doc/pub/week42/html/._week42-bs070.html +++ b/doc/pub/week42/html/._week42-bs070.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,27 +367,32 @@ MathJax.Hub.Config({ -

    The CIFAR01 data set

    - -

    -The CIFAR10 dataset contains 60,000 color images in 10 classes, with -6,000 images in each class. The dataset is divided into 50,000 -training images and 10,000 testing images. The classes are mutually -exclusive and there is no overlap between them. - +

    Importing Keras and Tensorflow

    -

    import tensorflow as tf
    +
    from tensorflow.keras import datasets, layers, models
    +from tensorflow.keras.layers import Input
    +from tensorflow.keras.models import Sequential      #This allows appending layers to existing models
    +from tensorflow.keras.layers import Dense           #This allows defining the characteristics of a particular layer
    +from tensorflow.keras import optimizers             #This allows using whichever optimiser we want (sgd,adam,RMSprop)
    +from tensorflow.keras import regularizers           #This allows using whichever regularizer we want (l1,l2,l1_l2)
    +from tensorflow.keras.utils import to_categorical   #This allows using categorical cross entropy as the cost function
    +#from tensorflow.keras import Conv2D
    +#from tensorflow.keras import MaxPooling2D
    +#from tensorflow.keras import Flatten
     
    -from tensorflow.keras import datasets, layers, models
    -import matplotlib.pyplot as plt
    +from sklearn.model_selection import train_test_split
     
    -# We import the data set
    -(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()
    +# representation of labels
    +labels = to_categorical(labels)
     
    -# Normalize pixel values to be between 0 and 1 by dividing by 255. 
    -train_images, test_images = train_images / 255.0, test_images / 255.0
    +# split into train and test data
    +# one-liner from scikit-learn library
    +train_size = 0.8
    +test_size = 1 - train_size
    +X_train, X_test, Y_train, Y_test = train_test_split(inputs, labels, train_size=train_size,
    +                                                    test_size=test_size)
     

    @@ -392,6 +415,10 @@ train_images, test_images = train_images 74

  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs071.html b/doc/pub/week42/html/._week42-bs071.html index f10d9e902..9d354436a 100644 --- a/doc/pub/week42/html/._week42-bs071.html +++ b/doc/pub/week42/html/._week42-bs071.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -347,30 +365,39 @@ MathJax.Hub.Config({

     

     

     

    - + -

    Verifying the data set

    - -

    -To verify that the dataset looks correct, let's plot the first 25 images from the training set and display the class name below each image. +

    Running with Keras

    -

    class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
    -               'dog', 'frog', 'horse', 'ship', 'truck']
    -​
    -plt.figure(figsize=(10,10))
    -for i in range(25):
    -    plt.subplot(5,5,i+1)
    -    plt.xticks([])
    -    plt.yticks([])
    -    plt.grid(False)
    -    plt.imshow(train_images[i], cmap=plt.cm.binary)
    -    # The CIFAR labels happen to be arrays, 
    -    # which is why you need the extra index
    -    plt.xlabel(class_names[train_labels[i][0]])
    -plt.show()
    +
    def create_convolutional_neural_network_keras(input_shape, receptive_field,
    +                                              n_filters, n_neurons_connected, n_categories,
    +                                              eta, lmbd):
    +    model = Sequential()
    +    model.add(layers.Conv2D(n_filters, (receptive_field, receptive_field), input_shape=input_shape, padding='same',
    +              activation='relu', kernel_regularizer=regularizers.l2(lmbd)))
    +    model.add(layers.MaxPooling2D(pool_size=(2, 2)))
    +    model.add(layers.Flatten())
    +    model.add(layers.Dense(n_neurons_connected, activation='relu', kernel_regularizer=regularizers.l2(lmbd)))
    +    model.add(layers.Dense(n_categories, activation='softmax', kernel_regularizer=regularizers.l2(lmbd)))
    +    
    +    sgd = optimizers.SGD(lr=eta)
    +    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
    +    
    +    return model
    +
    +epochs = 100
    +batch_size = 100
    +input_shape = X_train.shape[1:4]
    +receptive_field = 3
    +n_filters = 10
    +n_neurons_connected = 50
    +n_categories = 10
    +
    +eta_vals = np.logspace(-5, 1, 7)
    +lmbd_vals = np.logspace(-5, 1, 7)
     

    @@ -392,6 +419,10 @@ plt.show()

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs072.html b/doc/pub/week42/html/._week42-bs072.html index 076185341..fc372eb30 100644 --- a/doc/pub/week42/html/._week42-bs072.html +++ b/doc/pub/week42/html/._week42-bs072.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,31 +367,28 @@ MathJax.Hub.Config({ -

    Set up the model

    - -

    -The 6 lines of code below define the convolutional base using a common pattern: a stack of Conv2D and MaxPooling2D layers. - -

    -As input, a CNN takes tensors of shape (image_height, image_width, color_channels), ignoring the batch size. If you are new to these dimensions, color_channels refers to (R,G,B). In this example, you will configure our CNN to process inputs of shape (32, 32, 3), which is the format of CIFAR images. You can do this by passing the argument input_shape to our first layer. +

    Final part

    -

    model = models.Sequential()
    -model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
    -model.add(layers.MaxPooling2D((2, 2)))
    -model.add(layers.Conv2D(64, (3, 3), activation='relu'))
    -model.add(layers.MaxPooling2D((2, 2)))
    -model.add(layers.Conv2D(64, (3, 3), activation='relu'))
    -
    -# Let's display the architecture of our model so far.
    -
    -model.summary()
    +
    CNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)
    +        
    +for i, eta in enumerate(eta_vals):
    +    for j, lmbd in enumerate(lmbd_vals):
    +        CNN = create_convolutional_neural_network_keras(input_shape, receptive_field,
    +                                              n_filters, n_neurons_connected, n_categories,
    +                                              eta, lmbd)
    +        CNN.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, verbose=0)
    +        scores = CNN.evaluate(X_test, Y_test)
    +        
    +        CNN_keras[i][j] = CNN
    +        
    +        print("Learning rate = ", eta)
    +        print("Lambda = ", lmbd)
    +        print("Test accuracy: %.3f" % scores[1])
    +        print()
     
    -

    -You can see that the output of every Conv2D and MaxPooling2D layer is a 3D tensor of shape (height, width, channels). The width and height dimensions tend to shrink as you go deeper in the network. The number of output channels for each Conv2D layer is controlled by the first argument (e.g., 32 or 64). Typically, as the width and height shrink, you can afford (computationally) to add more output channels in each Conv2D layer. -

    @@ -393,6 +408,10 @@ You can see that the output of every Conv2D and MaxPooling2D layer is a 3D tenso

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs073.html b/doc/pub/week42/html/._week42-bs073.html index 58e9bdef1..fe0670070 100644 --- a/doc/pub/week42/html/._week42-bs073.html +++ b/doc/pub/week42/html/._week42-bs073.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,30 +367,42 @@ MathJax.Hub.Config({ -

    Add Dense layers on top

    - -

    -To complete our model, you will feed the last output tensor from the -convolutional base (of shape (4, 4, 64)) into one or more Dense layers -to perform classification. Dense layers take vectors as input (which -are 1D), while the current output is a 3D tensor. First, you will -flatten (or unroll) the 3D output to 1D, then add one or more Dense -layers on top. CIFAR has 10 output classes, so you use a final Dense -layer with 10 outputs and a softmax activation. +

    Final visualization

    -

    model.add(layers.Flatten())
    -model.add(layers.Dense(64, activation='relu'))
    -model.add(layers.Dense(10))
    -Here's the complete architecture of our model.
    +
    # visual representation of grid search
    +# uses seaborn heatmap, could probably do this in matplotlib
    +import seaborn as sns
     
    -model.summary()
    +sns.set()
    +
    +train_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
    +test_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
    +
    +for i in range(len(eta_vals)):
    +    for j in range(len(lmbd_vals)):
    +        CNN = CNN_keras[i][j]
    +
    +        train_accuracy[i][j] = CNN.evaluate(X_train, Y_train)[1]
    +        test_accuracy[i][j] = CNN.evaluate(X_test, Y_test)[1]
    +
    +        
    +fig, ax = plt.subplots(figsize = (10, 10))
    +sns.heatmap(train_accuracy, annot=True, ax=ax, cmap="viridis")
    +ax.set_title("Training Accuracy")
    +ax.set_ylabel("$\eta$")
    +ax.set_xlabel("$\lambda$")
    +plt.show()
    +
    +fig, ax = plt.subplots(figsize = (10, 10))
    +sns.heatmap(test_accuracy, annot=True, ax=ax, cmap="viridis")
    +ax.set_title("Test Accuracy")
    +ax.set_ylabel("$\eta$")
    +ax.set_xlabel("$\lambda$")
    +plt.show()
     
    -

    -As you can see, our (4, 4, 64) outputs were flattened into vectors of shape (1024) before going through two Dense layers. -

    @@ -391,6 +421,10 @@ As you can see, our (4, 4, 64) outputs were flattened into vectors of shape (102

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs074.html b/doc/pub/week42/html/._week42-bs074.html index 1162ebf7b..a9d7403fe 100644 --- a/doc/pub/week42/html/._week42-bs074.html +++ b/doc/pub/week42/html/._week42-bs074.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,17 +367,27 @@ MathJax.Hub.Config({ -

    Compile and train the model

    +

    The CIFAR01 data set

    + +

    +The CIFAR10 dataset contains 60,000 color images in 10 classes, with +6,000 images in each class. The dataset is divided into 50,000 +training images and 10,000 testing images. The classes are mutually +exclusive and there is no overlap between them.

    -

    model.compile(optimizer='adam',
    -              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    -              metrics=['accuracy'])
    -​
    -history = model.fit(train_images, train_labels, epochs=10, 
    -                    validation_data=(test_images, test_labels))
    +
    import tensorflow as tf
    +
    +from tensorflow.keras import datasets, layers, models
    +import matplotlib.pyplot as plt
    +
    +# We import the data set
    +(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()
    +
    +# Normalize pixel values to be between 0 and 1 by dividing by 255. 
    +train_images, test_images = train_images / 255.0, test_images / 255.0
     

    @@ -378,6 +406,10 @@ history = model

  • 74
  • 75
  • 76
  • +
  • 77
  • +
  • 78
  • +
  • 79
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/._week42-bs075.html b/doc/pub/week42/html/._week42-bs075.html index 329da77c6..9f13cf2fb 100644 --- a/doc/pub/week42/html/._week42-bs075.html +++ b/doc/pub/week42/html/._week42-bs075.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -349,24 +367,30 @@ MathJax.Hub.Config({ -

    Finally, evaluate the model

    +

    Verifying the data set

    + +

    +To verify that the dataset looks correct, let's plot the first 25 images from the training set and display the class name below each image.

    -

    plt.plot(history.history['accuracy'], label='accuracy')
    -plt.plot(history.history['val_accuracy'], label = 'val_accuracy')
    -plt.xlabel('Epoch')
    -plt.ylabel('Accuracy')
    -plt.ylim([0.5, 1])
    -plt.legend(loc='lower right')
    -
    -test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)
    -
    -print(test_acc)
    +
    class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
    +               'dog', 'frog', 'horse', 'ship', 'truck']
    +​
    +plt.figure(figsize=(10,10))
    +for i in range(25):
    +    plt.subplot(5,5,i+1)
    +    plt.xticks([])
    +    plt.yticks([])
    +    plt.grid(False)
    +    plt.imshow(train_images[i], cmap=plt.cm.binary)
    +    # The CIFAR labels happen to be arrays, 
    +    # which is why you need the extra index
    +    plt.xlabel(class_names[train_labels[i][0]])
    +plt.show()
     

    -

      @@ -382,6 +406,11 @@ test_loss, test_acc = model74
    • 75
    • 76
    • +
    • 77
    • +
    • 78
    • +
    • 79
    • +
    • 80
    • +
    • »
    diff --git a/doc/pub/week42/html/week42-bs.html b/doc/pub/week42/html/week42-bs.html index bdb5f998a..d836f3987 100644 --- a/doc/pub/week42/html/week42-bs.html +++ b/doc/pub/week42/html/week42-bs.html @@ -189,6 +189,20 @@ Automatically generated HTML file from DocOnce source 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -317,23 +331,27 @@ MathJax.Hub.Config({
  • Layers used to build CNNs
  • Transforming images
  • CNNs in brief
  • -
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • -
  • Setting it up
  • -
  • The MNIST dataset again
  • -
  • Strong correlations
  • -
  • Layers of a CNN
  • -
  • Systematic reduction
  • -
  • Prerequisites: Collect and pre-process data
  • -
  • Importing Keras and Tensorflow
  • -
  • Running with Keras
  • -
  • Final part
  • -
  • Final visualization
  • -
  • The CIFAR01 data set
  • -
  • Verifying the data set
  • -
  • Set up the model
  • -
  • Add Dense layers on top
  • -
  • Compile and train the model
  • -
  • Finally, evaluate the model
  • +
  • Mathematics of CNNs
  • +
  • Convolution Examples: Polynomial multiplication
  • +
  • Convolution Examples: Probability Theory
  • +
  • Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)
  • +
  • CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
  • +
  • Setting it up
  • +
  • The MNIST dataset again
  • +
  • Strong correlations
  • +
  • Layers of a CNN
  • +
  • Systematic reduction
  • +
  • Prerequisites: Collect and pre-process data
  • +
  • Importing Keras and Tensorflow
  • +
  • Running with Keras
  • +
  • Final part
  • +
  • Final visualization
  • +
  • The CIFAR01 data set
  • +
  • Verifying the data set
  • +
  • Set up the model
  • +
  • Add Dense layers on top
  • +
  • Compile and train the model
  • +
  • Finally, evaluate the model
  • @@ -392,7 +410,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
  • -
  • 76
  • +
  • 80
  • »
  • diff --git a/doc/pub/week42/html/week42-reveal.html b/doc/pub/week42/html/week42-reveal.html index fdfb2efaf..7dd4ffbf7 100644 --- a/doc/pub/week42/html/week42-reveal.html +++ b/doc/pub/week42/html/week42-reveal.html @@ -3177,6 +3177,257 @@ and the slides of +
    +

    Mathematics of CNNs

    + +

    +The mathematics of CNNs is based on the mathematical operation of +convolution. In mathematics (in particular in functional analysis), +convolution is represented by matheematical operation (integration, +summation etc) on two function in order to produce a third function +that expresses how the shape of one gets modified by the other. +Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning. + +

    +Mathematically, convolution is defined as follows (one-dimensional example): +Let us define a continuous function \( y(t) \) given by +

     
    +$$ +y(t) = \int x(a) w(t-a) da, +$$ +

     
    + +where \( x(a) \) represents a so-called input and \( w(t-a) \) is normally called the weight function or kernel. + +

    +The above integral is written in a more compact form as +

     
    +$$ +y(t) = \left(x * w\right)(t). +$$ +

     
    + +

    +The discretized version reads +

     
    +$$ +y(t) = \sum_{a=-\infty}^{a=\infty}x(a)w(t-a). +$$ +

     
    + +Computing the inverse of the above convolution operations is known as deconvolution. + +

    +How can we use this? And what does it mean? Let us study some familiar examples first. +

    + + +
    +

    Convolution Examples: Polynomial multiplication

    + +

    +We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function. +Let us remind of this and recast it in terms of the mathematical operation of convolution. +

    + + +
    +

    Convolution Examples: Probability Theory

    +
    + + +
    +

    Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)

    + +

    +If one has several driving forces, \( F(t)=\sum_n F_n(t) \), one can find +the particular solution to each \( F_n \), \( x_{pn}(t) \), and the particular +solution for the entire driving force is + +

     
    +$$ +\begin{equation} +x_p(t)=\sum_nx_{pn}(t). +\tag{21} +\end{equation} +$$ +

     
    + +

    +This is known as the principal of superposition. It only applies when +the homogenous equation is linear. If there were an anharmonic term +such as \( x^3 \) in the homogenous equation, then when one summed various +solutions, \( x=(\sum_n x_n)^2 \), one would get cross +terms. Superposition is especially useful when \( F(t) \) can be written +as a sum of sinusoidal terms, because the solutions for each +sinusoidal (sine or cosine) term is analytic, as we saw above. + +

    +Driving forces are often periodic, even when they are not +sinusoidal. Periodicity implies that for some time \( \tau \) + +

     
    +$$ +\begin{eqnarray} +F(t+\tau)=F(t). +\end{eqnarray} +$$ +

     
    + +

    +One example of a non-sinusoidal periodic force is a square wave. Many +components in electric circuits are non-linear, e.g. diodes, which +makes many wave forms non-sinusoidal even when the circuits are being +driven by purely sinusoidal sources. + +

    +The code here shows a typical example of such a square wave generated using the functionality included in the scipy Python package. We have used a period of \( \tau=0.2 \). + +

    + + +

    import numpy as np
    +import math
    +from scipy import signal
    +import matplotlib.pyplot as plt
    +
    +# number of points                                                                                       
    +n = 500
    +# start and final times                                                                                  
    +t0 = 0.0
    +tn = 1.0
    +# Period                                                                                                 
    +t = np.linspace(t0, tn, n, endpoint=False)
    +SqrSignal = np.zeros(n)
    +SqrSignal = 1.0+signal.square(2*np.pi*5*t)
    +plt.plot(t, SqrSignal)
    +plt.ylim(-0.5, 2.5)
    +plt.show()
    +
    +

    +For the sinusoidal example the +period is \( \tau=2\pi/\omega \). However, higher harmonics can also +satisfy the periodicity requirement. In general, any force that +satisfies the periodicity requirement can be expressed as a sum over +harmonics, + +

     
    +$$ +\begin{equation} +F(t)=\frac{f_0}{2}+\sum_{n>0} f_n\cos(2n\pi t/\tau)+g_n\sin(2n\pi t/\tau). +\tag{22} +\end{equation} +$$ +

     
    + +

    +We can write down the answer for +\( x_{pn}(t) \), by substituting \( f_n/m \) or \( g_n/m \) for \( F_0/m \). By +writing each factor \( 2n\pi t/\tau \) as \( n\omega t \), with \( \omega\equiv +2\pi/\tau \), + +

     
    +$$ +\begin{equation} +\tag{23} +F(t)=\frac{f_0}{2}+\sum_{n>0}f_n\cos(n\omega t)+g_n\sin(n\omega t). +\end{equation} +$$ +

     
    + +

    +The solutions for \( x(t) \) then come from replacing \( \omega \) with +\( n\omega \) for each term in the particular solution, + +

     
    +$$ +\begin{eqnarray} +x_p(t)&=&\frac{f_0}{2k}+\sum_{n>0} \alpha_n\cos(n\omega t-\delta_n)+\beta_n\sin(n\omega t-\delta_n),\\ +\nonumber +\alpha_n&=&\frac{f_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\beta_n&=&\frac{g_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\delta_n&=&\tan^{-1}\left(\frac{2\beta n\omega}{\omega_0^2-n^2\omega^2}\right). +\end{eqnarray} +$$ +

     
    + +

    +Because the forces have been applied for a long time, any non-zero +damping eliminates the homogenous parts of the solution, so one need +only consider the particular solution for each \( n \). + +

    +The problem will considered solved if one can find expressions for the +coefficients \( f_n \) and \( g_n \), even though the solutions are expressed +as an infinite sum. The coefficients can be extracted from the +function \( F(t) \) by + +

     
    +$$ +\begin{eqnarray} +\tag{24} +f_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\cos(2n\pi t/\tau),\\ +\nonumber +g_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\sin(2n\pi t/\tau). +\end{eqnarray} +$$ +

     
    + +

    +To check the consistency of these expressions and to verify +Eq. (24), one can insert the expansion of \( F(t) \) in +Eq. (23) into the expression for the coefficients in +Eq. (24) and see whether + +

     
    +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~\left\{ +\frac{f_0}{2}+\sum_{m>0}f_m\cos(m\omega t)+g_m\sin(m\omega t) +\right\}\cos(n\omega t). +\end{eqnarray} +$$ +

     
    + +

    +Immediately, one can throw away all the terms with \( g_m \) because they +convolute an even and an odd function. The term with \( f_0/2 \) +disappears because \( \cos(n\omega t) \) is equally positive and negative +over the interval and will integrate to zero. For all the terms +\( f_m\cos(m\omega t) \) appearing in the sum, one can use angle addition +formulas to see that \( \cos(m\omega t)\cos(n\omega +t)=(1/2)(\cos[(m+n)\omega t]+\cos[(m-n)\omega t] \). This will integrate +to zero unless \( m=n \). In that case the \( m=n \) term gives + +

     
    +$$ +\begin{equation} +\int_{-\tau/2}^{\tau/2}dt~\cos^2(m\omega t)=\frac{\tau}{2}, +\tag{25} +\end{equation} +$$ +

     
    + +

    +and + +

     
    +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~f_n/2\\ +\nonumber +&=&f_n~\checkmark. +\end{eqnarray} +$$ +

     
    + +

    +The same method can be used to check for the consistency of \( g_n \). +

    + +

    CNNs in more detail, building convolutional neural networks in Tensorflow and Keras

    diff --git a/doc/pub/week42/html/week42-solarized.html b/doc/pub/week42/html/week42-solarized.html index 3420b64c4..1c707d3b4 100644 --- a/doc/pub/week42/html/week42-solarized.html +++ b/doc/pub/week42/html/week42-solarized.html @@ -209,6 +209,20 @@ div { text-align: justify; text-justify: inter-word; } 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -3173,6 +3187,233 @@ and the slides of









    +

    Mathematics of CNNs

    + +

    +The mathematics of CNNs is based on the mathematical operation of +convolution. In mathematics (in particular in functional analysis), +convolution is represented by matheematical operation (integration, +summation etc) on two function in order to produce a third function +that expresses how the shape of one gets modified by the other. +Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning. + +

    +Mathematically, convolution is defined as follows (one-dimensional example): +Let us define a continuous function \( y(t) \) given by +$$ +y(t) = \int x(a) w(t-a) da, +$$ + +where \( x(a) \) represents a so-called input and \( w(t-a) \) is normally called the weight function or kernel. + +

    +The above integral is written in a more compact form as +$$ +y(t) = \left(x * w\right)(t). +$$ + +

    +The discretized version reads +$$ +y(t) = \sum_{a=-\infty}^{a=\infty}x(a)w(t-a). +$$ + +Computing the inverse of the above convolution operations is known as deconvolution. + +

    +How can we use this? And what does it mean? Let us study some familiar examples first. + +

    +









    + +

    Convolution Examples: Polynomial multiplication

    + +

    +We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function. +Let us remind of this and recast it in terms of the mathematical operation of convolution. + +

    +









    + +

    Convolution Examples: Probability Theory

    + +

    +









    + +

    Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)

    + +

    +If one has several driving forces, \( F(t)=\sum_n F_n(t) \), one can find +the particular solution to each \( F_n \), \( x_{pn}(t) \), and the particular +solution for the entire driving force is + +$$ +\begin{equation} +x_p(t)=\sum_nx_{pn}(t). +\label{_auto3} +\end{equation} +$$ + +

    +This is known as the principal of superposition. It only applies when +the homogenous equation is linear. If there were an anharmonic term +such as \( x^3 \) in the homogenous equation, then when one summed various +solutions, \( x=(\sum_n x_n)^2 \), one would get cross +terms. Superposition is especially useful when \( F(t) \) can be written +as a sum of sinusoidal terms, because the solutions for each +sinusoidal (sine or cosine) term is analytic, as we saw above. + +

    +Driving forces are often periodic, even when they are not +sinusoidal. Periodicity implies that for some time \( \tau \) + +$$ +\begin{eqnarray} +F(t+\tau)=F(t). +\end{eqnarray} +$$ + +

    +One example of a non-sinusoidal periodic force is a square wave. Many +components in electric circuits are non-linear, e.g. diodes, which +makes many wave forms non-sinusoidal even when the circuits are being +driven by purely sinusoidal sources. + +

    +The code here shows a typical example of such a square wave generated using the functionality included in the scipy Python package. We have used a period of \( \tau=0.2 \). + +

    + + +

    import numpy as np
    +import math
    +from scipy import signal
    +import matplotlib.pyplot as plt
    +
    +# number of points                                                                                       
    +n = 500
    +# start and final times                                                                                  
    +t0 = 0.0
    +tn = 1.0
    +# Period                                                                                                 
    +t = np.linspace(t0, tn, n, endpoint=False)
    +SqrSignal = np.zeros(n)
    +SqrSignal = 1.0+signal.square(2*np.pi*5*t)
    +plt.plot(t, SqrSignal)
    +plt.ylim(-0.5, 2.5)
    +plt.show()
    +
    +

    +For the sinusoidal example the +period is \( \tau=2\pi/\omega \). However, higher harmonics can also +satisfy the periodicity requirement. In general, any force that +satisfies the periodicity requirement can be expressed as a sum over +harmonics, + +$$ +\begin{equation} +F(t)=\frac{f_0}{2}+\sum_{n>0} f_n\cos(2n\pi t/\tau)+g_n\sin(2n\pi t/\tau). +\label{_auto4} +\end{equation} +$$ + +

    +We can write down the answer for +\( x_{pn}(t) \), by substituting \( f_n/m \) or \( g_n/m \) for \( F_0/m \). By +writing each factor \( 2n\pi t/\tau \) as \( n\omega t \), with \( \omega\equiv +2\pi/\tau \), + +$$ +\begin{equation} +\label{eq:fourierdef1} +F(t)=\frac{f_0}{2}+\sum_{n>0}f_n\cos(n\omega t)+g_n\sin(n\omega t). +\end{equation} +$$ + +

    +The solutions for \( x(t) \) then come from replacing \( \omega \) with +\( n\omega \) for each term in the particular solution, + +$$ +\begin{eqnarray} +x_p(t)&=&\frac{f_0}{2k}+\sum_{n>0} \alpha_n\cos(n\omega t-\delta_n)+\beta_n\sin(n\omega t-\delta_n),\\ +\nonumber +\alpha_n&=&\frac{f_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\beta_n&=&\frac{g_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\delta_n&=&\tan^{-1}\left(\frac{2\beta n\omega}{\omega_0^2-n^2\omega^2}\right). +\end{eqnarray} +$$ + +

    +Because the forces have been applied for a long time, any non-zero +damping eliminates the homogenous parts of the solution, so one need +only consider the particular solution for each \( n \). + +

    +The problem will considered solved if one can find expressions for the +coefficients \( f_n \) and \( g_n \), even though the solutions are expressed +as an infinite sum. The coefficients can be extracted from the +function \( F(t) \) by + +$$ +\begin{eqnarray} +\label{eq:fourierdef2} +f_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\cos(2n\pi t/\tau),\\ +\nonumber +g_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\sin(2n\pi t/\tau). +\end{eqnarray} +$$ + +

    +To check the consistency of these expressions and to verify +Eq. \eqref{eq:fourierdef2}, one can insert the expansion of \( F(t) \) in +Eq. \eqref{eq:fourierdef1} into the expression for the coefficients in +Eq. \eqref{eq:fourierdef2} and see whether + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~\left\{ +\frac{f_0}{2}+\sum_{m>0}f_m\cos(m\omega t)+g_m\sin(m\omega t) +\right\}\cos(n\omega t). +\end{eqnarray} +$$ + +

    +Immediately, one can throw away all the terms with \( g_m \) because they +convolute an even and an odd function. The term with \( f_0/2 \) +disappears because \( \cos(n\omega t) \) is equally positive and negative +over the interval and will integrate to zero. For all the terms +\( f_m\cos(m\omega t) \) appearing in the sum, one can use angle addition +formulas to see that \( \cos(m\omega t)\cos(n\omega +t)=(1/2)(\cos[(m+n)\omega t]+\cos[(m-n)\omega t] \). This will integrate +to zero unless \( m=n \). In that case the \( m=n \) term gives + +$$ +\begin{equation} +\int_{-\tau/2}^{\tau/2}dt~\cos^2(m\omega t)=\frac{\tau}{2}, +\label{_auto5} +\end{equation} +$$ + +

    +and + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~f_n/2\\ +\nonumber +&=&f_n~\checkmark. +\end{eqnarray} +$$ + +

    +The same method can be used to check for the consistency of \( g_n \). + +

    +









    +

    CNNs in more detail, building convolutional neural networks in Tensorflow and Keras

    diff --git a/doc/pub/week42/html/week42.html b/doc/pub/week42/html/week42.html index 93500401d..38205a2d2 100644 --- a/doc/pub/week42/html/week42.html +++ b/doc/pub/week42/html/week42.html @@ -214,6 +214,20 @@ div { text-align: justify; text-justify: inter-word; } 'layers-used-to-build-cnns'), ('Transforming images', 2, None, 'transforming-images'), ('CNNs in brief', 2, None, 'cnns-in-brief'), + ('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'), + ('Convolution Examples: Polynomial multiplication', + 2, + None, + 'convolution-examples-polynomial-multiplication'), + ('Convolution Examples: Probability Theory', + 2, + None, + 'convolution-examples-probability-theory'), + ('Convolution Examples: Principle of Superposition and Periodic ' + 'Forces (Fourier Transforms)', + 2, + None, + 'convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms'), ('CNNs in more detail, building convolutional neural networks in ' 'Tensorflow and Keras', 2, @@ -3178,6 +3192,233 @@ and the slides of









    +

    Mathematics of CNNs

    + +

    +The mathematics of CNNs is based on the mathematical operation of +convolution. In mathematics (in particular in functional analysis), +convolution is represented by matheematical operation (integration, +summation etc) on two function in order to produce a third function +that expresses how the shape of one gets modified by the other. +Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning. + +

    +Mathematically, convolution is defined as follows (one-dimensional example): +Let us define a continuous function \( y(t) \) given by +$$ +y(t) = \int x(a) w(t-a) da, +$$ + +where \( x(a) \) represents a so-called input and \( w(t-a) \) is normally called the weight function or kernel. + +

    +The above integral is written in a more compact form as +$$ +y(t) = \left(x * w\right)(t). +$$ + +

    +The discretized version reads +$$ +y(t) = \sum_{a=-\infty}^{a=\infty}x(a)w(t-a). +$$ + +Computing the inverse of the above convolution operations is known as deconvolution. + +

    +How can we use this? And what does it mean? Let us study some familiar examples first. + +

    +









    + +

    Convolution Examples: Polynomial multiplication

    + +

    +We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function. +Let us remind of this and recast it in terms of the mathematical operation of convolution. + +

    +









    + +

    Convolution Examples: Probability Theory

    + +

    +









    + +

    Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)

    + +

    +If one has several driving forces, \( F(t)=\sum_n F_n(t) \), one can find +the particular solution to each \( F_n \), \( x_{pn}(t) \), and the particular +solution for the entire driving force is + +$$ +\begin{equation} +x_p(t)=\sum_nx_{pn}(t). +\label{_auto3} +\end{equation} +$$ + +

    +This is known as the principal of superposition. It only applies when +the homogenous equation is linear. If there were an anharmonic term +such as \( x^3 \) in the homogenous equation, then when one summed various +solutions, \( x=(\sum_n x_n)^2 \), one would get cross +terms. Superposition is especially useful when \( F(t) \) can be written +as a sum of sinusoidal terms, because the solutions for each +sinusoidal (sine or cosine) term is analytic, as we saw above. + +

    +Driving forces are often periodic, even when they are not +sinusoidal. Periodicity implies that for some time \( \tau \) + +$$ +\begin{eqnarray} +F(t+\tau)=F(t). +\end{eqnarray} +$$ + +

    +One example of a non-sinusoidal periodic force is a square wave. Many +components in electric circuits are non-linear, e.g. diodes, which +makes many wave forms non-sinusoidal even when the circuits are being +driven by purely sinusoidal sources. + +

    +The code here shows a typical example of such a square wave generated using the functionality included in the scipy Python package. We have used a period of \( \tau=0.2 \). + +

    + + +

    import numpy as np
    +import math
    +from scipy import signal
    +import matplotlib.pyplot as plt
    +
    +# number of points                                                                                       
    +n = 500
    +# start and final times                                                                                  
    +t0 = 0.0
    +tn = 1.0
    +# Period                                                                                                 
    +t = np.linspace(t0, tn, n, endpoint=False)
    +SqrSignal = np.zeros(n)
    +SqrSignal = 1.0+signal.square(2*np.pi*5*t)
    +plt.plot(t, SqrSignal)
    +plt.ylim(-0.5, 2.5)
    +plt.show()
    +
    +

    +For the sinusoidal example the +period is \( \tau=2\pi/\omega \). However, higher harmonics can also +satisfy the periodicity requirement. In general, any force that +satisfies the periodicity requirement can be expressed as a sum over +harmonics, + +$$ +\begin{equation} +F(t)=\frac{f_0}{2}+\sum_{n>0} f_n\cos(2n\pi t/\tau)+g_n\sin(2n\pi t/\tau). +\label{_auto4} +\end{equation} +$$ + +

    +We can write down the answer for +\( x_{pn}(t) \), by substituting \( f_n/m \) or \( g_n/m \) for \( F_0/m \). By +writing each factor \( 2n\pi t/\tau \) as \( n\omega t \), with \( \omega\equiv +2\pi/\tau \), + +$$ +\begin{equation} +\label{eq:fourierdef1} +F(t)=\frac{f_0}{2}+\sum_{n>0}f_n\cos(n\omega t)+g_n\sin(n\omega t). +\end{equation} +$$ + +

    +The solutions for \( x(t) \) then come from replacing \( \omega \) with +\( n\omega \) for each term in the particular solution, + +$$ +\begin{eqnarray} +x_p(t)&=&\frac{f_0}{2k}+\sum_{n>0} \alpha_n\cos(n\omega t-\delta_n)+\beta_n\sin(n\omega t-\delta_n),\\ +\nonumber +\alpha_n&=&\frac{f_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\beta_n&=&\frac{g_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\delta_n&=&\tan^{-1}\left(\frac{2\beta n\omega}{\omega_0^2-n^2\omega^2}\right). +\end{eqnarray} +$$ + +

    +Because the forces have been applied for a long time, any non-zero +damping eliminates the homogenous parts of the solution, so one need +only consider the particular solution for each \( n \). + +

    +The problem will considered solved if one can find expressions for the +coefficients \( f_n \) and \( g_n \), even though the solutions are expressed +as an infinite sum. The coefficients can be extracted from the +function \( F(t) \) by + +$$ +\begin{eqnarray} +\label{eq:fourierdef2} +f_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\cos(2n\pi t/\tau),\\ +\nonumber +g_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\sin(2n\pi t/\tau). +\end{eqnarray} +$$ + +

    +To check the consistency of these expressions and to verify +Eq. \eqref{eq:fourierdef2}, one can insert the expansion of \( F(t) \) in +Eq. \eqref{eq:fourierdef1} into the expression for the coefficients in +Eq. \eqref{eq:fourierdef2} and see whether + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~\left\{ +\frac{f_0}{2}+\sum_{m>0}f_m\cos(m\omega t)+g_m\sin(m\omega t) +\right\}\cos(n\omega t). +\end{eqnarray} +$$ + +

    +Immediately, one can throw away all the terms with \( g_m \) because they +convolute an even and an odd function. The term with \( f_0/2 \) +disappears because \( \cos(n\omega t) \) is equally positive and negative +over the interval and will integrate to zero. For all the terms +\( f_m\cos(m\omega t) \) appearing in the sum, one can use angle addition +formulas to see that \( \cos(m\omega t)\cos(n\omega +t)=(1/2)(\cos[(m+n)\omega t]+\cos[(m-n)\omega t] \). This will integrate +to zero unless \( m=n \). In that case the \( m=n \) term gives + +$$ +\begin{equation} +\int_{-\tau/2}^{\tau/2}dt~\cos^2(m\omega t)=\frac{\tau}{2}, +\label{_auto5} +\end{equation} +$$ + +

    +and + +$$ +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~f_n/2\\ +\nonumber +&=&f_n~\checkmark. +\end{eqnarray} +$$ + +

    +The same method can be used to check for the consistency of \( g_n \). + +

    +









    +

    CNNs in more detail, building convolutional neural networks in Tensorflow and Keras

    diff --git a/doc/pub/week42/ipynb/ipynb-week42-src.tar.gz b/doc/pub/week42/ipynb/ipynb-week42-src.tar.gz index a5503eb12..1bce3c295 100644 Binary files a/doc/pub/week42/ipynb/ipynb-week42-src.tar.gz and b/doc/pub/week42/ipynb/ipynb-week42-src.tar.gz differ diff --git a/doc/pub/week42/ipynb/week42.ipynb b/doc/pub/week42/ipynb/week42.ipynb index 7676499c9..b1c81a271 100644 --- a/doc/pub/week42/ipynb/week42.ipynb +++ b/doc/pub/week42/ipynb/week42.ipynb @@ -3283,6 +3283,359 @@ "and the slides of [CS231](http://cs231n.github.io/convolutional-networks/) which is taught at Stanford University (consistently ranked as one of the top computer science programs in the world). [Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs](http://neuralnetworksanddeeplearning.com/chap6.html).\n", "\n", "\n", + "## Mathematics of CNNs\n", + "\n", + "The mathematics of CNNs is based on the mathematical operation of\n", + "**convolution**. In mathematics (in particular in functional analysis),\n", + "convolution is represented by matheematical operation (integration,\n", + "summation etc) on two function in order to produce a third function\n", + "that expresses how the shape of one gets modified by the other.\n", + "Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning.\n", + "\n", + "Mathematically, convolution is defined as follows (one-dimensional example):\n", + "Let us define a continuous function $y(t)$ given by" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "y(t) = \\int x(a) w(t-a) da,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $x(a)$ represents a so-called input and $w(t-a)$ is normally called the weight function or kernel.\n", + "\n", + "The above integral is written in a more compact form as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "y(t) = \\left(x * w\\right)(t).\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The discretized version reads" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "y(t) = \\sum_{a=-\\infty}^{a=\\infty}x(a)w(t-a).\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Computing the inverse of the above convolution operations is known as deconvolution.\n", + "\n", + "How can we use this? And what does it mean? Let us study some familiar examples first.\n", + "\n", + "\n", + "## Convolution Examples: Polynomial multiplication\n", + "\n", + "We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function.\n", + "Let us remind of this and recast it in terms of the mathematical operation of convolution.\n", + "\n", + "## Convolution Examples: Probability Theory\n", + "\n", + "\n", + "## Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)\n", + "\n", + "If one has several driving forces, $F(t)=\\sum_n F_n(t)$, one can find\n", + "the particular solution to each $F_n$, $x_{pn}(t)$, and the particular\n", + "solution for the entire driving force is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "

    \n", + "\n", + "$$\n", + "\\begin{equation}\n", + "x_p(t)=\\sum_nx_{pn}(t).\n", + "\\label{_auto3} \\tag{21}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is known as the principal of superposition. It only applies when\n", + "the homogenous equation is linear. If there were an anharmonic term\n", + "such as $x^3$ in the homogenous equation, then when one summed various\n", + "solutions, $x=(\\sum_n x_n)^2$, one would get cross\n", + "terms. Superposition is especially useful when $F(t)$ can be written\n", + "as a sum of sinusoidal terms, because the solutions for each\n", + "sinusoidal (sine or cosine) term is analytic, as we saw above.\n", + "\n", + "Driving forces are often periodic, even when they are not\n", + "sinusoidal. Periodicity implies that for some time $\\tau$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{eqnarray}\n", + "F(t+\\tau)=F(t). \n", + "\\end{eqnarray}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "One example of a non-sinusoidal periodic force is a square wave. Many\n", + "components in electric circuits are non-linear, e.g. diodes, which\n", + "makes many wave forms non-sinusoidal even when the circuits are being\n", + "driven by purely sinusoidal sources.\n", + "\n", + "The code here shows a typical example of such a square wave generated using the functionality included in the **scipy** Python package. We have used a period of $\\tau=0.2$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import math\n", + "from scipy import signal\n", + "import matplotlib.pyplot as plt\n", + "\n", + "# number of points \n", + "n = 500\n", + "# start and final times \n", + "t0 = 0.0\n", + "tn = 1.0\n", + "# Period \n", + "t = np.linspace(t0, tn, n, endpoint=False)\n", + "SqrSignal = np.zeros(n)\n", + "SqrSignal = 1.0+signal.square(2*np.pi*5*t)\n", + "plt.plot(t, SqrSignal)\n", + "plt.ylim(-0.5, 2.5)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the sinusoidal example the\n", + "period is $\\tau=2\\pi/\\omega$. However, higher harmonics can also\n", + "satisfy the periodicity requirement. In general, any force that\n", + "satisfies the periodicity requirement can be expressed as a sum over\n", + "harmonics," + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
    \n", + "\n", + "$$\n", + "\\begin{equation}\n", + "F(t)=\\frac{f_0}{2}+\\sum_{n>0} f_n\\cos(2n\\pi t/\\tau)+g_n\\sin(2n\\pi t/\\tau).\n", + "\\label{_auto4} \\tag{22}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can write down the answer for\n", + "$x_{pn}(t)$, by substituting $f_n/m$ or $g_n/m$ for $F_0/m$. By\n", + "writing each factor $2n\\pi t/\\tau$ as $n\\omega t$, with $\\omega\\equiv\n", + "2\\pi/\\tau$," + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
    \n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\label{eq:fourierdef1} \\tag{23}\n", + "F(t)=\\frac{f_0}{2}+\\sum_{n>0}f_n\\cos(n\\omega t)+g_n\\sin(n\\omega t).\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The solutions for $x(t)$ then come from replacing $\\omega$ with\n", + "$n\\omega$ for each term in the particular solution," + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{eqnarray}\n", + "x_p(t)&=&\\frac{f_0}{2k}+\\sum_{n>0} \\alpha_n\\cos(n\\omega t-\\delta_n)+\\beta_n\\sin(n\\omega t-\\delta_n),\\\\\n", + "\\nonumber\n", + "\\alpha_n&=&\\frac{f_n/m}{\\sqrt{((n\\omega)^2-\\omega_0^2)+4\\beta^2n^2\\omega^2}},\\\\\n", + "\\nonumber\n", + "\\beta_n&=&\\frac{g_n/m}{\\sqrt{((n\\omega)^2-\\omega_0^2)+4\\beta^2n^2\\omega^2}},\\\\\n", + "\\nonumber\n", + "\\delta_n&=&\\tan^{-1}\\left(\\frac{2\\beta n\\omega}{\\omega_0^2-n^2\\omega^2}\\right).\n", + "\\end{eqnarray}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Because the forces have been applied for a long time, any non-zero\n", + "damping eliminates the homogenous parts of the solution, so one need\n", + "only consider the particular solution for each $n$.\n", + "\n", + "The problem will considered solved if one can find expressions for the\n", + "coefficients $f_n$ and $g_n$, even though the solutions are expressed\n", + "as an infinite sum. The coefficients can be extracted from the\n", + "function $F(t)$ by" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
    \n", + "\n", + "$$\n", + "\\begin{eqnarray}\n", + "\\label{eq:fourierdef2} \\tag{24}\n", + "f_n&=&\\frac{2}{\\tau}\\int_{-\\tau/2}^{\\tau/2} dt~F(t)\\cos(2n\\pi t/\\tau),\\\\\n", + "\\nonumber\n", + "g_n&=&\\frac{2}{\\tau}\\int_{-\\tau/2}^{\\tau/2} dt~F(t)\\sin(2n\\pi t/\\tau).\n", + "\\end{eqnarray}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To check the consistency of these expressions and to verify\n", + "Eq. ([24](#eq:fourierdef2)), one can insert the expansion of $F(t)$ in\n", + "Eq. ([23](#eq:fourierdef1)) into the expression for the coefficients in\n", + "Eq. ([24](#eq:fourierdef2)) and see whether" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{eqnarray}\n", + "f_n&=?&\\frac{2}{\\tau}\\int_{-\\tau/2}^{\\tau/2} dt~\\left\\{\n", + "\\frac{f_0}{2}+\\sum_{m>0}f_m\\cos(m\\omega t)+g_m\\sin(m\\omega t)\n", + "\\right\\}\\cos(n\\omega t).\n", + "\\end{eqnarray}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Immediately, one can throw away all the terms with $g_m$ because they\n", + "convolute an even and an odd function. The term with $f_0/2$\n", + "disappears because $\\cos(n\\omega t)$ is equally positive and negative\n", + "over the interval and will integrate to zero. For all the terms\n", + "$f_m\\cos(m\\omega t)$ appearing in the sum, one can use angle addition\n", + "formulas to see that $\\cos(m\\omega t)\\cos(n\\omega\n", + "t)=(1/2)(\\cos[(m+n)\\omega t]+\\cos[(m-n)\\omega t]$. This will integrate\n", + "to zero unless $m=n$. In that case the $m=n$ term gives" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
    \n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\int_{-\\tau/2}^{\\tau/2}dt~\\cos^2(m\\omega t)=\\frac{\\tau}{2},\n", + "\\label{_auto5} \\tag{25}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "and" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{eqnarray}\n", + "f_n&=?&\\frac{2}{\\tau}\\int_{-\\tau/2}^{\\tau/2} dt~f_n/2\\\\\n", + "\\nonumber\n", + "&=&f_n~\\checkmark.\n", + "\\end{eqnarray}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The same method can be used to check for the consistency of $g_n$.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", "\n", "## CNNs in more detail, building convolutional neural networks in Tensorflow and Keras\n", "\n", diff --git a/doc/src/week42/week42.do.txt b/doc/src/week42/week42.do.txt index 66c31fe7b..fff0e917f 100644 --- a/doc/src/week42/week42.do.txt +++ b/doc/src/week42/week42.do.txt @@ -2597,6 +2597,214 @@ the course and the slides of "CS231":"http://cs231n.github.io/convolutional-networks/" which is taught at Stanford University (consistently ranked as one of the top computer science programs in the world). "Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs":"http://neuralnetworksanddeeplearning.com/chap6.html". +!split +===== Mathematics of CNNs ===== + +The mathematics of CNNs is based on the mathematical operation of +_convolution_. In mathematics (in particular in functional analysis), +convolution is represented by matheematical operation (integration, +summation etc) on two function in order to produce a third function +that expresses how the shape of one gets modified by the other. +Convolution has a plethora of applications in a variety of disciplines, spanning from statistics to signal processing, computer vision, solutions of differential equations,linear algebra, engineering, and yes, machine learning. + +Mathematically, convolution is defined as follows (one-dimensional example): +Let us define a continuous function $y(t)$ given by +!bt +\[ +y(t) = \int x(a) w(t-a) da, +\] +!et +where $x(a)$ represents a so-called input and $w(t-a)$ is normally called the weight function or kernel. + +The above integral is written in a more compact form as +!bt +\[ +y(t) = \left(x * w\right)(t). +\] +!et + +The discretized version reads +!bt +\[ +y(t) = \sum_{a=-\infty}^{a=\infty}x(a)w(t-a). +\] +!et +Computing the inverse of the above convolution operations is known as deconvolution. + +How can we use this? And what does it mean? Let us study some familiar examples first. + + +!split +===== Convolution Examples: Polynomial multiplication ===== + +We have already met such an example in project 1 when we tried to set up the design matrix for a two-dimensional function. +Let us remind of this and recast it in terms of the mathematical operation of convolution. + +!split +===== Convolution Examples: Probability Theory ===== + + +!split +===== Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms) ===== + +If one has several driving forces, $F(t)=\sum_n F_n(t)$, one can find +the particular solution to each $F_n$, $x_{pn}(t)$, and the particular +solution for the entire driving force is + +!bt +\begin{equation} +x_p(t)=\sum_nx_{pn}(t). +\end{equation} +!et + +This is known as the principal of superposition. It only applies when +the homogenous equation is linear. If there were an anharmonic term +such as $x^3$ in the homogenous equation, then when one summed various +solutions, $x=(\sum_n x_n)^2$, one would get cross +terms. Superposition is especially useful when $F(t)$ can be written +as a sum of sinusoidal terms, because the solutions for each +sinusoidal (sine or cosine) term is analytic, as we saw above. + +Driving forces are often periodic, even when they are not +sinusoidal. Periodicity implies that for some time $\tau$ + +!bt +\begin{eqnarray} +F(t+\tau)=F(t). +\end{eqnarray} +!et + +One example of a non-sinusoidal periodic force is a square wave. Many +components in electric circuits are non-linear, e.g. diodes, which +makes many wave forms non-sinusoidal even when the circuits are being +driven by purely sinusoidal sources. + +The code here shows a typical example of such a square wave generated using the functionality included in the _scipy_ Python package. We have used a period of $\tau=0.2$. + +!bc pycod +import numpy as np +import math +from scipy import signal +import matplotlib.pyplot as plt + +# number of points +n = 500 +# start and final times +t0 = 0.0 +tn = 1.0 +# Period +t = np.linspace(t0, tn, n, endpoint=False) +SqrSignal = np.zeros(n) +SqrSignal = 1.0+signal.square(2*np.pi*5*t) +plt.plot(t, SqrSignal) +plt.ylim(-0.5, 2.5) +plt.show() +!ec + + +For the sinusoidal example the +period is $\tau=2\pi/\omega$. However, higher harmonics can also +satisfy the periodicity requirement. In general, any force that +satisfies the periodicity requirement can be expressed as a sum over +harmonics, + +!bt +\begin{equation} +F(t)=\frac{f_0}{2}+\sum_{n>0} f_n\cos(2n\pi t/\tau)+g_n\sin(2n\pi t/\tau). +\end{equation} +!et + +We can write down the answer for +$x_{pn}(t)$, by substituting $f_n/m$ or $g_n/m$ for $F_0/m$. By +writing each factor $2n\pi t/\tau$ as $n\omega t$, with $\omega\equiv +2\pi/\tau$, + +!bt +\begin{equation} +label{eq:fourierdef1} +F(t)=\frac{f_0}{2}+\sum_{n>0}f_n\cos(n\omega t)+g_n\sin(n\omega t). +\end{equation} +!et + +The solutions for $x(t)$ then come from replacing $\omega$ with +$n\omega$ for each term in the particular solution, + +!bt +\begin{eqnarray} +x_p(t)&=&\frac{f_0}{2k}+\sum_{n>0} \alpha_n\cos(n\omega t-\delta_n)+\beta_n\sin(n\omega t-\delta_n),\\ +\nonumber +\alpha_n&=&\frac{f_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\beta_n&=&\frac{g_n/m}{\sqrt{((n\omega)^2-\omega_0^2)+4\beta^2n^2\omega^2}},\\ +\nonumber +\delta_n&=&\tan^{-1}\left(\frac{2\beta n\omega}{\omega_0^2-n^2\omega^2}\right). +\end{eqnarray} +!et + +Because the forces have been applied for a long time, any non-zero +damping eliminates the homogenous parts of the solution, so one need +only consider the particular solution for each $n$. + +The problem will considered solved if one can find expressions for the +coefficients $f_n$ and $g_n$, even though the solutions are expressed +as an infinite sum. The coefficients can be extracted from the +function $F(t)$ by + +!bt +\begin{eqnarray} +label{eq:fourierdef2} +f_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\cos(2n\pi t/\tau),\\ +\nonumber +g_n&=&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~F(t)\sin(2n\pi t/\tau). +\end{eqnarray} +!et + +To check the consistency of these expressions and to verify +Eq. (ref{eq:fourierdef2}), one can insert the expansion of $F(t)$ in +Eq. (ref{eq:fourierdef1}) into the expression for the coefficients in +Eq. (ref{eq:fourierdef2}) and see whether + +!bt +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~\left\{ +\frac{f_0}{2}+\sum_{m>0}f_m\cos(m\omega t)+g_m\sin(m\omega t) +\right\}\cos(n\omega t). +\end{eqnarray} +!et + +Immediately, one can throw away all the terms with $g_m$ because they +convolute an even and an odd function. The term with $f_0/2$ +disappears because $\cos(n\omega t)$ is equally positive and negative +over the interval and will integrate to zero. For all the terms +$f_m\cos(m\omega t)$ appearing in the sum, one can use angle addition +formulas to see that $\cos(m\omega t)\cos(n\omega +t)=(1/2)(\cos[(m+n)\omega t]+\cos[(m-n)\omega t]$. This will integrate +to zero unless $m=n$. In that case the $m=n$ term gives + +!bt +\begin{equation} +\int_{-\tau/2}^{\tau/2}dt~\cos^2(m\omega t)=\frac{\tau}{2}, +\end{equation} +!et + +and + +!bt +\begin{eqnarray} +f_n&=?&\frac{2}{\tau}\int_{-\tau/2}^{\tau/2} dt~f_n/2\\ +\nonumber +&=&f_n~\checkmark. +\end{eqnarray} +!et + +The same method can be used to check for the consistency of $g_n$. + + + + + + + !split ===== CNNs in more detail, building convolutional neural networks in Tensorflow and Keras ===== @@ -2977,3 +3185,6 @@ print(test_acc) + + +