updating lectures

doc/src/week42/week42.do.txt

@@ -20,6 +20,14 @@ DATE: today
 
 !bblock  Excellent lectures on CNNs 
 * "Video  on Convolutional Neural Networks from MIT":"https://www.youtube.com/watch?v=iaSUYvmCekI&ab_channel=AlexanderAmini"
+* "Video on CNNs from Stanford":"https://www.youtube.com/watch?v=bNb2fEVKeEo&list=PLC1qU-LWwrF64f4QKQT-Vg5Wr4qEE1Zxk&index=6&ab_channel=StanfordUniversitySchoolofEngineering"
+!eblock
+
+
+!bblock  And Lecture material on CNNs
+* "Lectures from IN5400 spring 2019":"https://www.uio.no/studier/emner/matnat/ifi/IN5400/v19/material/week5/in5400_2019_week5_convolutional_nerual_networks.pdf"
+* "Lectures from IN5400 spring 2021":"https://www.uio.no/studier/emner/matnat/ifi/IN5400/v21/lecture-slides/in5400_2021_w5_lecture_convolutions.pdf"
+* "See also Michael Nielsen's Lectures":"http://neuralnetworksanddeeplearning.com/chap6.html"
 !eblock
 
 
@@ -2444,7 +2452,10 @@ loss function (for example Softmax) on the last (fully-connected) layer
 and all the tips/tricks we developed for learning regular Neural
 Networks still apply (back propagation, gradient descent etc etc).
 
-What is the difference? _CNN architectures make the explicit assumption that
+!split
+===== What is the Difference =====
+
+_CNN architectures make the explicit assumption that
 the inputs are images, which allows us to encode certain properties
 into the architecture. These then make the forward function more
 efficient to implement and vastly reduce the amount of parameters in
@@ -2957,9 +2968,38 @@ plt.show()
 
 
 !split
-===== Convolution Examples: Probability Theory =====
+=====  Two-dimensional Objects =====
 
-More text will be added here
+We often use convolutions over more than one dimension at a time. If
+we have a two-dimensional image $I$ as input, we can have a _filter_
+defined by a two-dimensional _kernel_ $K$. This leads to an output $S$
+
+!bt
+\[
+S_(i,j)=(I * K)(i,j) = \sum_m\sum_n I(m,n)K(i-m,j-n).
+\]
+!et
+
+Convolution is a commutatitave process, which means we can rewrite this equation as
+!bt
+\[
+S_(i,j)=(I * K)(i,j) = \sum_m\sum_n I(i-m,j-n)K(m,n).
+\]
+!et
+
+Normally the latter is more straightforward to implement in  a machine elarning library since there is less variation in the range of values of $m$ and $n$.
+
+!split
+===== Cross-Correlation =====
+
+
+
+Many deep learning libraries implement cross-correlation instead of convolution
+!bt
+\[
+S_(i,j)=(I * K)(i,j) = \sum_m\sum_n I(i+m,j-+)K(m,n).
+\]
+!et
 
 
 !split