updated typos

2018-10-04 05:35:51 +02:00
parent ff08e04b6e
commit 0c44eedada
64 changed files with 1948 additions and 2013 deletions
@@ -110,20 +110,19 @@ Automatically generated HTML file from DocOnce source
              ('Improving performance', 2, None, '___sec44'),
              ('Full object-oriented implementation', 2, None, '___sec45'),
              ('Evaluate model performance on test data', 2, None, '___sec46'),
-              ('Adjust hyperparameters (if necessary, network architecture',
-               2,
-               None,
-               '___sec47'),
-              ('scikit-learn implementation', 2, None, '___sec48'),
+              ('Adjust hyperparameters', 2, None, '___sec47'),
+              ('Visualization', 2, None, '___sec48'),
+              ('scikit-learn implementation', 2, None, '___sec49'),
+              ('Visualization', 2, None, '___sec50'),
              ('Building neural networks in Tensorflow and Keras',
               2,
               None,
-               '___sec49'),
-              ('Tensorflow', 2, None, '___sec50'),
-              ('Collect and pre-process data', 2, None, '___sec51'),
-              ('Using TensorFlow backend', 2, None, '___sec52'),
-              ('Optimizing and using gradient descent', 2, None, '___sec53'),
-              ('Using Keras', 2, None, '___sec54')]}
+               '___sec51'),
+              ('Tensorflow', 2, None, '___sec52'),
+              ('Collect and pre-process data', 2, None, '___sec53'),
+              ('Using TensorFlow backend', 2, None, '___sec54'),
+              ('Optimizing and using gradient descent', 2, None, '___sec55'),
+              ('Using Keras', 2, None, '___sec56')]}
 end of tocinfo -->

 <body>
@@ -208,14 +207,16 @@ MathJax.Hub.Config({
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs045.html#___sec44" style="font-size: 80%;"><b>Improving performance</b></a></li>
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs046.html#___sec45" style="font-size: 80%;"><b>Full object-oriented implementation</b></a></li>
     <!-- navigation toc: --> <li><a href="._NeuralNet-bs047.html#___sec46" style="font-size: 80%;"><b>Evaluate model performance on test data</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs048.html#___sec47" style="font-size: 80%;"><b>Adjust hyperparameters (if necessary, network architecture</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs049.html#___sec48" style="font-size: 80%;"><b>scikit-learn implementation</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs050.html#___sec49" style="font-size: 80%;"><b>Building neural networks in Tensorflow and Keras</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs051.html#___sec50" style="font-size: 80%;"><b>Tensorflow</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs052.html#___sec51" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs053.html#___sec52" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs054.html#___sec53" style="font-size: 80%;"><b>Optimizing and using gradient descent</b></a></li>
-     <!-- navigation toc: --> <li><a href="._NeuralNet-bs055.html#___sec54" style="font-size: 80%;"><b>Using Keras</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs048.html#___sec47" style="font-size: 80%;"><b>Adjust hyperparameters</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs049.html#___sec48" style="font-size: 80%;"><b>Visualization</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs050.html#___sec49" style="font-size: 80%;"><b>scikit-learn implementation</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs051.html#___sec50" style="font-size: 80%;"><b>Visualization</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs052.html#___sec51" style="font-size: 80%;"><b>Building neural networks in Tensorflow and Keras</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs053.html#___sec52" style="font-size: 80%;"><b>Tensorflow</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs054.html#___sec53" style="font-size: 80%;"><b>Collect and pre-process data</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs055.html#___sec54" style="font-size: 80%;"><b>Using TensorFlow backend</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs056.html#___sec55" style="font-size: 80%;"><b>Optimizing and using gradient descent</b></a></li>
+     <!-- navigation toc: --> <li><a href="._NeuralNet-bs057.html#___sec56" style="font-size: 80%;"><b>Using Keras</b></a></li>

        </ul>
      </li>
@@ -237,38 +238,38 @@ MathJax.Hub.Config({
 To more efficently train our network these equations are implemented using matrix operations.  
 The error in the output layer is calculated simply as  

-$$ \Delta_o = \hat{y} - y = (n_{inputs}, n_{categories}) .$$
+$$ \delta_L = \hat{y} - y = (n_{inputs}, n_{categories}) .$$

 <p>
 The gradient for the output weights is calculated as  

-$$ \nabla W_{o} = A^T \Delta_o = (n_{hidden}, n_{categories}) ,$$
+$$ \nabla W_{L} = \hat{a}^T \delta_L   = (n_{hidden}, n_{categories}) ,$$

 <p>
-where \( A = (n_{inputs}, n_{hidden}) \). This simply means that we are summing up the gradients for each input.  
+where \( \hat{a} = (n_{inputs}, n_{hidden}) \). This simply means that we are summing up the gradients for each input.  
 Since we are going backwards we have to transpose the activation matrix.

 <p>
 The gradient with respect to the output bias is then  

-$$ \nabla B_{o} = \sum_{i=1}^{n_{inputs}} \Delta_o = (n_{categories}) .$$
+$$ \nabla \hat{b}_{L} = \sum_{i=1}^{n_{inputs}} \delta_L = (n_{categories}) .$$

 <p>
 The error in the hidden layer is  

-$$ \Delta_h = \Delta_o W_{o}^T \circ f'(Z_{h}) = \Delta_o W_{o}^T \circ A_{h} \circ (1 - A_{h}) = (n_{inputs}, n_{hidden}) ,$$
+$$ \Delta_h = \delta_L W_{L}^T \circ f'(z_{h}) = \delta_L W_{L}^T \circ a_{h} \circ (1 - a_{h}) = (n_{inputs}, n_{hidden}) ,$$

 <p>
-where \( f'(A_{h}) \) is the derivative of the activation in the hidden layer. The matrix products mean
+where \( f'(a_{h}) \) is the derivative of the activation in the hidden layer. The matrix products mean
 that we are summing up the products for each neuron in the output layer. The symbol \( \circ \) denotes
 the <em>Hadamard product</em>, meaning element-wise multiplication.

 <p>
 This again gives us the gradients in the hidden layer:  

-$$ \nabla W_{h} = X^T \Delta_h = (n_{features}, n_{hidden}) ,$$  
+$$ \nabla W_{h} = X^T \delta_h = (n_{features}, n_{hidden}) ,$$  

-$$ \nabla B_{h} = \sum_{i=1}^{n_{inputs}} \Delta_h = (n_{hidden}) .$$
+$$ \nabla b_{h} = \sum_{i=1}^{n_{inputs}} \delta_h = (n_{hidden}) .$$

 <p>

@@ -371,7 +372,7 @@ lmbd <span style="color: #666666">=</span> <span style="color: #666666">0.01</sp
  <li><a href="._NeuralNet-bs052.html">53</a></li>
  <li><a href="._NeuralNet-bs053.html">54</a></li>
  <li><a href="">...</a></li>
-  <li><a href="._NeuralNet-bs055.html">56</a></li>
+  <li><a href="._NeuralNet-bs057.html">58</a></li>
  <li><a href="._NeuralNet-bs045.html">&raquo;</a></li>
 </ul>
 <!-- ------------------- end of main content --------------- -->