lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1b2e055ffa91e81d94d08f9b2265527ea3c4f80d
FYS-STK4155/doc/pub/NeuralNet/html/._NeuralNet-bs010.html
T
mhjensen c56b1b1611 chasing slides type
2018-09-28 12:11:44 +02:00

232 lines
9.4 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of machine learning">
<title>Data Analysis and Machine Learning: Elements of machine learning</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Neural networks', 2, None, '___sec0'),
('Artificial neurons', 2, None, '___sec1'),
('Neural network types', 2, None, '___sec2'),
('Feed-forward neural networks', 2, None, '___sec3'),
('Recurrent neural networks', 2, None, '___sec4'),
('Other types of networks', 2, None, '___sec5'),
('Multilayer perceptrons', 2, None, '___sec6'),
('Why multilayer perceptrons?', 2, None, '___sec7'),
('Mathematical model', 2, None, '___sec8'),
('Mathematical model', 2, None, '___sec9'),
('Mathematical model', 2, None, '___sec10'),
('Mathematical model', 2, None, '___sec11'),
('Mathematical model', 2, None, '___sec12'),
('Matrix-vector notation', 3, None, '___sec13'),
('Matrix-vector notation and activation', 3, None, '___sec14'),
('Activation functions', 3, None, '___sec15'),
('Activation functions, Logistic and Hyperbolic ones',
3,
None,
'___sec16'),
('Relevance', 3, None, '___sec17'),
('Setting up a Multi-layer perceptron model',
2,
None,
'___sec18'),
('Two-layer Neural Network', 2, None, '___sec19')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="NeuralNet-bs.html">Data Analysis and Machine Learning: Elements of machine learning</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._NeuralNet-bs001.html#___sec0" style="font-size: 80%;"><b>Neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs002.html#___sec1" style="font-size: 80%;"><b>Artificial neurons</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs003.html#___sec2" style="font-size: 80%;"><b>Neural network types</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs004.html#___sec3" style="font-size: 80%;"><b>Feed-forward neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs005.html#___sec4" style="font-size: 80%;"><b>Recurrent neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs006.html#___sec5" style="font-size: 80%;"><b>Other types of networks</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs007.html#___sec6" style="font-size: 80%;"><b>Multilayer perceptrons</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs008.html#___sec7" style="font-size: 80%;"><b>Why multilayer perceptrons?</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs009.html#___sec8" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs011.html#___sec10" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs012.html#___sec11" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs013.html#___sec12" style="font-size: 80%;"><b>Mathematical model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs014.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Matrix-vector notation</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs015.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Matrix-vector notation and activation</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs016.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Activation functions</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs017.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Activation functions, Logistic and Hyperbolic ones</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs018.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Relevance</a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs019.html#___sec18" style="font-size: 80%;"><b>Setting up a Multi-layer perceptron model</b></a></li>
<!-- navigation toc: --> <li><a href="._NeuralNet-bs020.html#___sec19" style="font-size: 80%;"><b>Two-layer Neural Network</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0010"></a>
<!-- !split -->
<h2 id="___sec9" class="anchor">Mathematical model </h2>
<p>
First, for each node \( i \) in the first hidden layer, we calculate a weighted sum \( u_i^1 \) of the input coordinates \( x_j \),
$$
\begin{equation}
u_i^1 = \sum_{j=1}^2 w_{ij}^1 x_j + b_i^1
\tag{3}
\end{equation}
$$
This value is the argument to the activation function \( f_1 \) of each neuron \( i \),
producing the output \( y_i^1 \) of all neurons in layer 1,
$$
\begin{equation}
y_i^1 = f_1(u_i^1) = f_1\left(\sum_{j=1}^2 w_{ij}^1 x_j + b_i^1\right)
\tag{4}
\end{equation}
$$
<p>
where we assume that all nodes in the same layer have identical
activation functions, hence the notation \( f_l \)
$$
\begin{equation}
y_i^l = f_l(u_i^l) = f_l\left(\sum_{j=1}^{N_{l-1}} w_{ij}^l y_j^{l-1} + b_i^l\right)
\tag{5}
\end{equation}
$$
<p>
where \( N_l \) is the number of nodes in layer \( l \). When the output of
all the nodes in the first hidden layer are computed, the values of
the subsequent layer can be calculated and so forth until the output
is obtained.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._NeuralNet-bs009.html">&laquo;</a></li>
<li><a href="._NeuralNet-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs002.html">3</a></li>
<li><a href="._NeuralNet-bs003.html">4</a></li>
<li><a href="._NeuralNet-bs004.html">5</a></li>
<li><a href="._NeuralNet-bs005.html">6</a></li>
<li><a href="._NeuralNet-bs006.html">7</a></li>
<li><a href="._NeuralNet-bs007.html">8</a></li>
<li><a href="._NeuralNet-bs008.html">9</a></li>
<li><a href="._NeuralNet-bs009.html">10</a></li>
<li class="active"><a href="._NeuralNet-bs010.html">11</a></li>
<li><a href="._NeuralNet-bs011.html">12</a></li>
<li><a href="._NeuralNet-bs012.html">13</a></li>
<li><a href="._NeuralNet-bs013.html">14</a></li>
<li><a href="._NeuralNet-bs014.html">15</a></li>
<li><a href="._NeuralNet-bs015.html">16</a></li>
<li><a href="._NeuralNet-bs016.html">17</a></li>
<li><a href="._NeuralNet-bs017.html">18</a></li>
<li><a href="._NeuralNet-bs018.html">19</a></li>
<li><a href="._NeuralNet-bs019.html">20</a></li>
<li><a href="">...</a></li>
<li><a href="._NeuralNet-bs020.html">21</a></li>
<li><a href="._NeuralNet-bs011.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink