lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added more text

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-10-22 07:55:23 +02:00
parent af905dd693
commit 4618a2a7ee
98 changed files with 3809 additions and 3536 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week42/html/._week42-bs064.html
+65 -47
View File
@@ -190,6 +190,7 @@ Automatically generated HTML file from DocOnce source
'layers-used-to-build-cnns'),
('Transforming images', 2, None, 'transforming-images'),
('CNNs in brief', 2, None, 'cnns-in-brief'),
('Key Idea', 2, None, 'key-idea'),
('Mathematics of CNNs', 2, None, 'mathematics-of-cnns'),
('Convolution Examples: Polynomial multiplication',
2,
@@ -358,37 +359,38 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week42-bs057.html#layers-used-to-build-cnns" style="font-size: 80%;">Layers used to build CNNs</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs058.html#transforming-images" style="font-size: 80%;">Transforming images</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs059.html#cnns-in-brief" style="font-size: 80%;">CNNs in brief</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs060.html#mathematics-of-cnns" style="font-size: 80%;">Mathematics of CNNs</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs061.html#convolution-examples-polynomial-multiplication" style="font-size: 80%;">Convolution Examples: Polynomial multiplication</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs062.html#efficient-polynomial-multiplication" style="font-size: 80%;">Efficient Polynomial Multiplication</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs063.html#a-more-efficient-way-of-coding-the-above-convolution" style="font-size: 80%;">A more efficient way of coding the above Convolution</a></li>
<!-- navigation toc: --> <li><a href="#convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms" style="font-size: 80%;">Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs065.html#principle-of-superposition" style="font-size: 80%;">Principle of Superposition</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs066.html#simple-code-example" style="font-size: 80%;">Simple Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs067.html#wrapping-up-fourier-transforms" style="font-size: 80%;">Wrapping up Fourier transforms</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs068.html#finding-the-coefficients" style="font-size: 80%;">Finding the Coefficients</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs069.html#final-words-on-fourier-transforms" style="font-size: 80%;">Final words on Fourier Transforms</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs070.html#two-dimensional-objects" style="font-size: 80%;">Two-dimensional Objects</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs071.html#cross-correlation" style="font-size: 80%;">Cross-Correlation</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs072.html#more-on-dimensionalities" style="font-size: 80%;">More on Dimensionalities</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs073.html#further-dimensionality-remarks" style="font-size: 80%;">Further Dimensionality Remarks</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs074.html#cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras" style="font-size: 80%;">CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs075.html#setting-it-up" style="font-size: 80%;">Setting it up</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs076.html#the-mnist-dataset-again" style="font-size: 80%;">The MNIST dataset again</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs077.html#strong-correlations" style="font-size: 80%;">Strong correlations</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs078.html#layers-of-a-cnn" style="font-size: 80%;">Layers of a CNN</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs079.html#systematic-reduction" style="font-size: 80%;">Systematic reduction</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs080.html#prerequisites-collect-and-pre-process-data" style="font-size: 80%;">Prerequisites: Collect and pre-process data</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs081.html#importing-keras-and-tensorflow" style="font-size: 80%;">Importing Keras and Tensorflow</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs082.html#running-with-keras" style="font-size: 80%;">Running with Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs083.html#final-part" style="font-size: 80%;">Final part</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs084.html#final-visualization" style="font-size: 80%;">Final visualization</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs085.html#the-cifar01-data-set" style="font-size: 80%;">The CIFAR01 data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs086.html#verifying-the-data-set" style="font-size: 80%;">Verifying the data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs087.html#set-up-the-model" style="font-size: 80%;">Set up the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs088.html#add-dense-layers-on-top" style="font-size: 80%;">Add Dense layers on top</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs089.html#compile-and-train-the-model" style="font-size: 80%;">Compile and train the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs090.html#finally-evaluate-the-model" style="font-size: 80%;">Finally, evaluate the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs060.html#key-idea" style="font-size: 80%;">Key Idea</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs061.html#mathematics-of-cnns" style="font-size: 80%;">Mathematics of CNNs</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs062.html#convolution-examples-polynomial-multiplication" style="font-size: 80%;">Convolution Examples: Polynomial multiplication</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs063.html#efficient-polynomial-multiplication" style="font-size: 80%;">Efficient Polynomial Multiplication</a></li>
<!-- navigation toc: --> <li><a href="#a-more-efficient-way-of-coding-the-above-convolution" style="font-size: 80%;">A more efficient way of coding the above Convolution</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs065.html#convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms" style="font-size: 80%;">Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs066.html#principle-of-superposition" style="font-size: 80%;">Principle of Superposition</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs067.html#simple-code-example" style="font-size: 80%;">Simple Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs068.html#wrapping-up-fourier-transforms" style="font-size: 80%;">Wrapping up Fourier transforms</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs069.html#finding-the-coefficients" style="font-size: 80%;">Finding the Coefficients</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs070.html#final-words-on-fourier-transforms" style="font-size: 80%;">Final words on Fourier Transforms</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs071.html#two-dimensional-objects" style="font-size: 80%;">Two-dimensional Objects</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs072.html#cross-correlation" style="font-size: 80%;">Cross-Correlation</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs073.html#more-on-dimensionalities" style="font-size: 80%;">More on Dimensionalities</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs074.html#further-dimensionality-remarks" style="font-size: 80%;">Further Dimensionality Remarks</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs075.html#cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras" style="font-size: 80%;">CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs076.html#setting-it-up" style="font-size: 80%;">Setting it up</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs077.html#the-mnist-dataset-again" style="font-size: 80%;">The MNIST dataset again</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs078.html#strong-correlations" style="font-size: 80%;">Strong correlations</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs079.html#layers-of-a-cnn" style="font-size: 80%;">Layers of a CNN</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs080.html#systematic-reduction" style="font-size: 80%;">Systematic reduction</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs081.html#prerequisites-collect-and-pre-process-data" style="font-size: 80%;">Prerequisites: Collect and pre-process data</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs082.html#importing-keras-and-tensorflow" style="font-size: 80%;">Importing Keras and Tensorflow</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs083.html#running-with-keras" style="font-size: 80%;">Running with Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs084.html#final-part" style="font-size: 80%;">Final part</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs085.html#final-visualization" style="font-size: 80%;">Final visualization</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs086.html#the-cifar01-data-set" style="font-size: 80%;">The CIFAR01 data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs087.html#verifying-the-data-set" style="font-size: 80%;">Verifying the data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs088.html#set-up-the-model" style="font-size: 80%;">Set up the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs089.html#add-dense-layers-on-top" style="font-size: 80%;">Add Dense layers on top</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs090.html#compile-and-train-the-model" style="font-size: 80%;">Compile and train the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs091.html#finally-evaluate-the-model" style="font-size: 80%;">Finally, evaluate the model</a></li>
</ul>
</li>
@@ -404,27 +406,43 @@ MathJax.Hub.Config({
<a name="part0064"></a>
<!-- !split -->
<h2 id="convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms" class="anchor">Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms) </h2>
<h2 id="a-more-efficient-way-of-coding-the-above-convolution" class="anchor">A more efficient way of coding the above Convolution </h2>
<p>
For problems with so-called harmonic oscillations, given by for example the following differential equation
$$
m\frac{d^2x}{dt^2}+\eta\frac{dx}{dt}+x(t)=F(t),
$$
Since we only have a finite number of \( \alpha \) and \( \beta \) values
which are non-zero, we can rewrite the above convolution expressions
as a matrix-vector multiplication
where \( F(t) \) is an applied external force acting on the system (often called a driving force), one can use the theory of Fourier transformations to find the solutions of this type of equations.
$$
\boldsymbol{\delta}=\begin{bmatrix}\alpha_0 & 0 & 0 & 0 \\
\alpha_1 & \alpha_0 & 0 & 0 \\
\alpha_2 & \alpha_1 & \alpha_0 & 0 \\
0 & \alpha_2 & \alpha_1 & \alpha_0 \\
0 & 0 & \alpha_2 & \alpha_1 \\
0 & 0 & 0 & \alpha_2
\end{bmatrix}\begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \\ \beta_3\end{bmatrix}.
$$
<p>
If one has several driving forces, \( F(t)=\sum_n F_n(t) \), one can find
the particular solution to each \( F_n \), \( x_{pn}(t) \), and the particular
solution for the entire driving force is then given by a series like
The process is commutative and we can easily see that we can rewrite the multiplication in terms of a matrix holding \( \beta \) and a vector holding \( \alpha \).
In this case we have
$$
\boldsymbol{\delta}=\begin{bmatrix}\beta_0 & 0 & 0 \\
\beta_1 & \beta_0 & 0 \\
\beta_2 & \beta_1 & \beta_0 \\
\beta_3 & \beta_2 & \beta_1 \\
0 & \beta_3 & \beta_2 \\
0 & 0 & \beta_3
\end{bmatrix}\begin{bmatrix} \alpha_0 \\ \alpha_1 \\ \alpha_2\end{bmatrix}.
$$
$$
\begin{equation}
x_p(t)=\sum_nx_{pn}(t).
\tag{21}
\end{equation}
$$
<p>
Note that the use of these matrices is for mathematical purposes only and not implementation purposes.
When implementing the above equation we do not encode (and allocate memory) the matrices explicitely.
We rather code the convolutions in the minimal memory footprint that they require.
<p>
Does the number of floating point operations change here when we use the commutative property?
<p>
<p>
@@ -452,7 +470,7 @@ $$
<li><a href="._week42-bs072.html">73</a></li>
<li><a href="._week42-bs073.html">74</a></li>
<li><a href="">...</a></li>
<li><a href="._week42-bs090.html">91</a></li>
<li><a href="._week42-bs091.html">92</a></li>
<li><a href="._week42-bs065.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->