more typos

doc/src/week42/week42.do.txt

@@ -2663,16 +2663,16 @@ z(t) = \delta_0+\delta_1 t+\delta_2 t^2+\delta_3 t^3+\delta_4 t^4+\delta_5 t^5.
 !split
 ===== Efficient Polynomial Multiplication =====
 
-Computing polynomial products can be implemented efficiently if we rewrite the the more brute force multiplications using convolution.
+Computing polynomial products can be implemented efficiently if we rewrite the more brute force multiplications using convolution.
 We note first that the new coefficients are given as
 
 !bt
 \begin{split}
 \delta_0=&\alpha_0\beta_0\\
-\delta_1=&\alpha_1\beta_0+\beta_0\alpha_1\\
-\delta_2=&\alpha_0\beta_2+\beta_1\alpha_1+\alpha_2\beta_0\\
-\delta_3=&\alpha_1\beta_2+\beta_1\alpha_2+\alpha_0\beta_3\\
-\delta_4=&\alpha_2\beta_2+\beta_3\alpha_1\\
+\delta_1=&\alpha_1\beta_0+\alpha_1\beta_0\\
+\delta_2=&\alpha_0\beta_2+\alpha_1\beta_1+\alpha_2\beta_0\\
+\delta_3=&\alpha_1\beta_2+\alpha_2\beta_1+\alpha_0\beta_3\\
+\delta_4=&\alpha_2\beta_2+\alpha_1\beta_3\\
 \delta_5=&\alpha_2\beta_3.\\
 \end{split}
 !et
@@ -2710,12 +2710,27 @@ as a matrix-vector multiplication
 			    0 & \alpha_2 & \alpha_1 & \alpha_0 \\
 			    0 & 0 & \alpha_2 & \alpha_1 \\
 			    0 & 0 & 0 & \alpha_2
-			    \end{bmatrix}\begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \\ \beta_3\end{bmatrix}
+			    \end{bmatrix}\begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \\ \beta_3\end{bmatrix}.
 \]
 !et
 
-The process is commutative and we can easily see that we can rewrite the multiplication in terms of  a martrix holding $\beta$ and a vector holding $\alpha$.
+The process is commutative and we can easily see that we can rewrite the multiplication in terms of  a matrix holding $\beta$ and a vector holding $\alpha$.
+In this case we have
+!bt
+\[
+\bm{\delta}=\begin{bmatrix}\beta_0 & 0 & 0  \\
+                            \beta_1 & \beta_0 & 0  \\
+			    \beta_2 & \beta_1 & \beta_0  \\
+			    \beta_3 & \beta_2 & \beta_1 \\
+			    0 & \beta_3 & \beta_2 \\
+			    0 & 0 & \beta_3
+			    \end{bmatrix}\begin{bmatrix} \alpha_0 \\ \alpha_1 \\ \alpha_2\end{bmatrix}.
+\]
+!et
 
+Note that the use of these  matrices is for mathematical purposes only  and not implementation purposes.
+When implementing the above equation we do not encode (and allocate memory) the matrices explicitely.
+We rather code the convolutions in the minimal memory footprint that they require.
 
 
 
@@ -2946,6 +2961,55 @@ plt.show()
 More text will be added here
 
 
+!split
+===== More on Dimensionalities =====
+
+In feilds like signal processing (and imaging as well), one designs
+so-called filters. These filters are defined by the convolutions and
+are often hand-crafted. One may specify filters for smoothing, edge
+detection, frequency reshaping, and similar operations. However with
+neural networks the idea is to automatically learn the filters and use
+many of them in conjunction with non-linear operations (activation
+functions).
+
+As an example consider a neural network operating on sound sequence
+data.  Assume that we an input vector $\bm{x}$ of length $d=10^6$.  We
+construct then a neural network with onle hidden layer only with
+$10^4$ nodes. This means that we will have a weight matrix with
+$10^4\times 10^6=10^{10}$ weights to be determined, together with $10^4$ biases.
+
+Assume furthermore that we have an output layer which is meant to train whether the sound sequence represents a human voice (true) or something else (false).
+It means that we have only one output node. But since this output node connects to $10^4$ nodes in the hidden layer, there are in total $10^4$ weights to be determined for the output layer, plus one bias. In total we have
+
+!bt
+\[
+\mathrm{NumberParameters}=10^{10}+10^4+10^4+1 \approx 10^{10},
+\]
+!et
+that is ten billion parameters to determine. 
+
+
+!split
+===== Further Dimensionality Remarks =====
+
+In today’s architecture one can train such neural networks, however
+this is a huge number of parameters for the task at hand. In general,
+it is a very wasteful and inefficient use of dense matrices as
+parameters. Just as importantly, such trained network parameters are
+very specific for the type of input data on which they were trained
+and the network is not likely to generalize easily to variations in
+the input.
+
+
+The main principles that justify convolutions is locality of
+information and repetion of patterns within the signal. Sound samples
+of the input in adjacent spots are much more likely to affect each
+other than those that are very far away. Similarly, sounds are
+repeated in multiple times in the signal. While slightly simplistic,
+reasoning about such a sound example demonstrates this. The same
+principles then apply to images and other similar data.
+
+
 
 !split
 ===== CNNs in more detail, building convolutional neural networks in Tensorflow and Keras =====