lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update on cnn

Browse Source
This commit is contained in:
mhjensen
2019-10-04 05:50:18 +02:00
parent 812fbe454e
commit c2be6efa93
162 changed files with 5235 additions and 2420 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/NeuralNet/html/NeuralNet.html
+7 -7
View File
@@ -208,7 +208,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Oct 3, 2019</h4></center> <!-- date -->
<center><h4>Oct 4, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1373,7 +1373,7 @@ The scikit-learn dataset we will use consists of a selection of 1797 images of s
<p>
To feed data into a feed-forward neural network we need to represent
the inputs as a feature matrix \( X = (n_{inputs}, n_{features}) \). Each
the inputs as a design/feature matrix \( X = (n_{inputs}, n_{features}) \). Each
row represents an <em>input</em>, in this case a handwritten digit, and
each column represents a <em>feature</em>, in this case a pixel. The
correct answers, also known as <em>labels</em> or <em>targets</em> are
@@ -1383,7 +1383,7 @@ represented as a 1D array of integers
<p>
As an example, say we want to build a neural network using supervised learning to predict Body-Mass Index (BMI) from
measurements of height (in m)
and weight (in kg). If we have measurements of 5 people the feature matrix could be for example:
and weight (in kg). If we have measurements of 5 people the design/feature matrix could be for example:
$$ X = \begin{bmatrix}
1.85 &amp; 81\\
@@ -1401,7 +1401,7 @@ $$ Y = (23.7, 22.2, 27.1, 17.5, 21.1) $$
<p>
Since each input image is a 2D matrix, we need to flatten the image
(i.e. "unravel" the 2D matrix into a 1D array) to turn the data into a
feature matrix. This means we lose all spatial information in the
design/feature matrix. This means we lose all spatial information in the
image, such as locality and translational invariance. More complicated
architectures such as Convolutional Neural Networks can take advantage
of such information, and are most commonly applied when analyzing
@@ -1818,9 +1818,9 @@ reduces <em>overfitting</em>.
<p>
We will measure the size of the weights using the so called <em>L2-norm</em>, meaning our cost function becomes:
$$ \nabla \mathcal{C}(\theta) = \frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}_i(\theta) \quad \rightarrow \quad
\frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}_i(\theta) + \lambda \lvert \lvert \hat{w} \rvert \rvert_2^2
= \frac{1}{N} \sum_{i=1}^N \nabla \mathcal{L}(\theta) + \lambda \sum_{ij} w_{ij}^2,$$
$$ \mathcal{C}(\theta) = \frac{1}{N} \sum_{i=1}^N \mathcal{L}_i(\theta) \quad \rightarrow \quad
\frac{1}{N} \sum_{i=1}^N \mathcal{L}_i(\theta) + \lambda \lvert \lvert \hat{w} \rvert \rvert_2^2
= \frac{1}{N} \sum_{i=1}^N \mathcal{L}(\theta) + \lambda \sum_{ij} w_{ij}^2,$$
<p>
i.e. we sum up all the weights squared. The factor \( \lambda \) is known as a regularization parameter.