update on cnn

doc/src/NeuralNet/NeuralNet.do.txt

@@ -1054,7 +1054,7 @@ The MNIST dataset consists of 70 000 images of size $28\times 28$ pixels, each l
 The scikit-learn dataset we will use consists of a selection of 1797 images of size $8\times 8$ collected and processed from this database.  
   
 To feed data into a feed-forward neural network we need to represent
-the inputs as a feature matrix $X = (n_{inputs}, n_{features})$.  Each
+the inputs as a design/feature matrix $X = (n_{inputs}, n_{features})$.  Each
 row represents an *input*, in this case a handwritten digit, and
 each column represents a *feature*, in this case a pixel.  The
 correct answers, also known as *labels* or *targets* are
@@ -1063,7 +1063,7 @@ $Y = (n_{inputs}) = (5, 3, 1, 8,...)$.
   
 As an example, say we want to build a neural network using supervised learning to predict Body-Mass Index (BMI) from
 measurements of height (in m)  
-and weight (in kg). If we have measurements of 5 people the feature matrix could be for example:  
+and weight (in kg). If we have measurements of 5 people the design/feature matrix could be for example:  
   
 $$ X = \begin{bmatrix}
 1.85 & 81\\
@@ -1079,7 +1079,7 @@ $$ Y = (23.7, 22.2, 27.1, 17.5, 21.1) $$
   
 Since each input image is a 2D matrix, we need to flatten the image
 (i.e. "unravel" the 2D matrix into a 1D array) to turn the data into a
-feature matrix. This means we lose all spatial information in the
+design/feature matrix. This means we lose all spatial information in the
 image, such as locality and translational invariance. More complicated
 architectures such as Convolutional Neural Networks can take advantage
 of such information, and are most commonly applied when analyzing
@@ -1430,9 +1430,9 @@ reduces *overfitting*.
   
 We will measure the size of the weights using the so called *L2-norm*, meaning our cost function becomes:  
   
-$$ \nabla \mathcal{C}(\theta) = \frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}_i(\theta) \quad \rightarrow \quad
-\frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}_i(\theta) + \lambda \lvert \lvert \hat{w} \rvert \rvert_2^2 
-= \frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}(\theta) + \lambda \sum_{ij} w_{ij}^2,$$  
+$$  \mathcal{C}(\theta) = \frac{1}{N} \sum_{i=1}^N \mathcal{L}_i(\theta) \quad \rightarrow \quad
+\frac{1}{N} \sum_{i=1}^N  \mathcal{L}_i(\theta) + \lambda \lvert \lvert \hat{w} \rvert \rvert_2^2 
+= \frac{1}{N} \sum_{i=1}^N  \mathcal{L}(\theta) + \lambda \sum_{ij} w_{ij}^2,$$  
   
 i.e. we sum up all the weights squared. The factor $\lambda$ is known as a regularization parameter.