lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5c6c05efbe058f033f3f35def695f64a7c70b9ce
FYS-STK4155/doc/pub/week42/html/._week42-bs032.html
T
mhjensen faf9db223a update week 42
2020-10-16 11:37:12 +02:00

330 lines
22 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks">
<title>Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Plan for week 42', 2, None, '___sec0'),
('Convolutional Neural Networks (recognizing images)',
2,
None,
'___sec1'),
('Neural Networks vs CNNs', 2, None, '___sec2'),
('Why CNNS for images, sound files, medical images from CT scans '
'etc?',
2,
None,
'___sec3'),
('Regular NNs don’t scale well to full images',
2,
None,
'___sec4'),
('3D volumes of neurons', 2, None, '___sec5'),
('Layers used to build CNNs', 2, None, '___sec6'),
('Transforming images', 2, None, '___sec7'),
('CNNs in brief', 2, None, '___sec8'),
('CNNs in more detail, building convolutional neural networks in '
'Tensorflow and Keras',
2,
None,
'___sec9'),
('Setting it up', 2, None, '___sec10'),
('The MNIST dataset again', 2, None, '___sec11'),
('Strong correlations', 2, None, '___sec12'),
('Layers of a CNN', 2, None, '___sec13'),
('Systematic reduction', 2, None, '___sec14'),
('Prerequisites: Collect and pre-process data',
2,
None,
'___sec15'),
('Importing Keras and Tensorflow', 2, None, '___sec16'),
('Running with Keras', 2, None, '___sec17'),
('Final part', 2, None, '___sec18'),
('Final visualization', 2, None, '___sec19'),
('The CIFAR01 data set', 2, None, '___sec20'),
('Verifying the data set', 2, None, '___sec21'),
('Set up the model', 2, None, '___sec22'),
('Add Dense layers on top', 2, None, '___sec23'),
('Compile and train the model', 2, None, '___sec24'),
('Finally, evaluate the model', 2, None, '___sec25'),
('Recurrent neural networks: Overarching view',
2,
None,
'___sec26'),
('Set up of an RNN', 2, None, '___sec27'),
('A simple example', 2, None, '___sec28'),
('An extrapolation example', 2, None, '___sec29'),
('Formatting the Data', 2, None, '___sec30'),
('Predicting New Points With A Trained Recurrent Neural Network',
2,
None,
'___sec31'),
('Other Things to Try', 2, None, '___sec32'),
('Other Types of Recurrent Neural Networks',
2,
None,
'___sec33')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week42-bs.html">Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week42-bs001.html#___sec0" style="font-size: 80%;">Plan for week 42</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs002.html#___sec1" style="font-size: 80%;">Convolutional Neural Networks (recognizing images)</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs003.html#___sec2" style="font-size: 80%;">Neural Networks vs CNNs</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs004.html#___sec3" style="font-size: 80%;">Why CNNS for images, sound files, medical images from CT scans etc?</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs005.html#___sec4" style="font-size: 80%;">Regular NNs don’t scale well to full images</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs006.html#___sec5" style="font-size: 80%;">3D volumes of neurons</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs007.html#___sec6" style="font-size: 80%;">Layers used to build CNNs</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs008.html#___sec7" style="font-size: 80%;">Transforming images</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs009.html#___sec8" style="font-size: 80%;">CNNs in brief</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs010.html#___sec9" style="font-size: 80%;">CNNs in more detail, building convolutional neural networks in Tensorflow and Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs011.html#___sec10" style="font-size: 80%;">Setting it up</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs012.html#___sec11" style="font-size: 80%;">The MNIST dataset again</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs013.html#___sec12" style="font-size: 80%;">Strong correlations</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs014.html#___sec13" style="font-size: 80%;">Layers of a CNN</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs015.html#___sec14" style="font-size: 80%;">Systematic reduction</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs016.html#___sec15" style="font-size: 80%;">Prerequisites: Collect and pre-process data</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs017.html#___sec16" style="font-size: 80%;">Importing Keras and Tensorflow</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs018.html#___sec17" style="font-size: 80%;">Running with Keras</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs019.html#___sec18" style="font-size: 80%;">Final part</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs020.html#___sec19" style="font-size: 80%;">Final visualization</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs021.html#___sec20" style="font-size: 80%;">The CIFAR01 data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs022.html#___sec21" style="font-size: 80%;">Verifying the data set</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs023.html#___sec22" style="font-size: 80%;">Set up the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs024.html#___sec23" style="font-size: 80%;">Add Dense layers on top</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs025.html#___sec24" style="font-size: 80%;">Compile and train the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs026.html#___sec25" style="font-size: 80%;">Finally, evaluate the model</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs027.html#___sec26" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs028.html#___sec27" style="font-size: 80%;">Set up of an RNN</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs029.html#___sec28" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs030.html#___sec29" style="font-size: 80%;">An extrapolation example</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs031.html#___sec30" style="font-size: 80%;">Formatting the Data</a></li>
<!-- navigation toc: --> <li><a href="#___sec31" style="font-size: 80%;">Predicting New Points With A Trained Recurrent Neural Network</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs033.html#___sec32" style="font-size: 80%;">Other Things to Try</a></li>
<!-- navigation toc: --> <li><a href="._week42-bs034.html#___sec33" style="font-size: 80%;">Other Types of Recurrent Neural Networks</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0032"></a>
<!-- !split -->
<h2 id="___sec31" class="anchor">Predicting New Points With A Trained Recurrent Neural Network </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">test_rnn</span> (x1, y_test, plot_min, plot_max):
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot;</span>
<span style="color: #BA2121; font-style: italic"> Inputs:</span>
<span style="color: #BA2121; font-style: italic"> x1 (a list or numpy array): The complete x component of the data set</span>
<span style="color: #BA2121; font-style: italic"> y_test (a list or numpy array): The complete y component of the data set</span>
<span style="color: #BA2121; font-style: italic"> plot_min (an int or float): the smallest x value used in the training data</span>
<span style="color: #BA2121; font-style: italic"> plot_max (an int or float): the largest x valye used in the training data</span>
<span style="color: #BA2121; font-style: italic"> Returns:</span>
<span style="color: #BA2121; font-style: italic"> None.</span>
<span style="color: #BA2121; font-style: italic"> Uses a trained recurrent neural network model to predict future points in the </span>
<span style="color: #BA2121; font-style: italic"> series. Computes the MSE of the predicted data set from the true data set, saves</span>
<span style="color: #BA2121; font-style: italic"> the predicted data set to a csv file, and plots the predicted and true data sets w</span>
<span style="color: #BA2121; font-style: italic"> while also displaying the data range used for training.</span>
<span style="color: #BA2121; font-style: italic"> &quot;&quot;&quot;</span>
<span style="color: #408080; font-style: italic"># Add the training data as the first dim points in the predicted data array as these</span>
<span style="color: #408080; font-style: italic"># are known values.</span>
y_pred <span style="color: #666666">=</span> y_test[:dim]<span style="color: #666666">.</span>tolist()
<span style="color: #408080; font-style: italic"># Generate the first input to the trained recurrent neural network using the last two </span>
<span style="color: #408080; font-style: italic"># points of the training data. Based on how the network was trained this means that it</span>
<span style="color: #408080; font-style: italic"># will predict the first point in the data set after the training data. All of the </span>
<span style="color: #408080; font-style: italic"># brackets are necessary for Tensorflow.</span>
next_input <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array([[[y_test[dim<span style="color: #666666">-2</span>]], [y_test[dim<span style="color: #666666">-1</span>]]]])
<span style="color: #408080; font-style: italic"># Save the very last point in the training data set. This will be used later.</span>
last <span style="color: #666666">=</span> [y_test[dim<span style="color: #666666">-1</span>]]
<span style="color: #408080; font-style: italic"># Iterate until the complete data set is created.</span>
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span> (dim, <span style="color: #008000">len</span>(y_test)):
<span style="color: #408080; font-style: italic"># Predict the next point in the data set using the previous two points.</span>
<span style="color: #008000">next</span> <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(next_input)
<span style="color: #408080; font-style: italic"># Append just the number of the predicted data set</span>
y_pred<span style="color: #666666">.</span>append(<span style="color: #008000">next</span>[<span style="color: #666666">0</span>][<span style="color: #666666">0</span>])
<span style="color: #408080; font-style: italic"># Create the input that will be used to predict the next data point in the data set.</span>
next_input <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array([[last, <span style="color: #008000">next</span>[<span style="color: #666666">0</span>]]], dtype<span style="color: #666666">=</span>np<span style="color: #666666">.</span>float64)
last <span style="color: #666666">=</span> <span style="color: #008000">next</span>
<span style="color: #408080; font-style: italic"># Print the mean squared error between the known data set and the predicted data set.</span>
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;MSE: &#39;</span>, np<span style="color: #666666">.</span>square(np<span style="color: #666666">.</span>subtract(y_test, y_pred))<span style="color: #666666">.</span>mean())
<span style="color: #408080; font-style: italic"># Save the predicted data set as a csv file for later use</span>
name <span style="color: #666666">=</span> datatype <span style="color: #666666">+</span> <span style="color: #BA2121">&#39;Predicted&#39;</span><span style="color: #666666">+</span><span style="color: #008000">str</span>(dim)<span style="color: #666666">+</span><span style="color: #BA2121">&#39;.csv&#39;</span>
np<span style="color: #666666">.</span>savetxt(name, y_pred, delimiter<span style="color: #666666">=</span><span style="color: #BA2121">&#39;,&#39;</span>)
<span style="color: #408080; font-style: italic"># Plot the known data set and the predicted data set. The red box represents the region that was used</span>
<span style="color: #408080; font-style: italic"># for the training data.</span>
fig, ax <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>subplots()
ax<span style="color: #666666">.</span>plot(x1, y_test, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;true&quot;</span>, linewidth<span style="color: #666666">=3</span>)
ax<span style="color: #666666">.</span>plot(x1, y_pred, <span style="color: #BA2121">&#39;g-.&#39;</span>,label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;predicted&quot;</span>, linewidth<span style="color: #666666">=4</span>)
ax<span style="color: #666666">.</span>legend()
<span style="color: #408080; font-style: italic"># Created a red region to represent the points used in the training data.</span>
ax<span style="color: #666666">.</span>axvspan(plot_min, plot_max, alpha<span style="color: #666666">=0.25</span>, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;red&#39;</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic"># Check to make sure the data set is complete</span>
<span style="color: #008000; font-weight: bold">assert</span> <span style="color: #008000">len</span>(X_tot) <span style="color: #666666">==</span> <span style="color: #008000">len</span>(y_tot)
<span style="color: #408080; font-style: italic"># This is the number of points that will be used in as the training data</span>
dim<span style="color: #666666">=12</span>
<span style="color: #408080; font-style: italic"># Separate the training data from the whole data set</span>
X_train <span style="color: #666666">=</span> X_tot[:dim]
y_train <span style="color: #666666">=</span> y_tot[:dim]
<span style="color: #408080; font-style: italic"># Generate the training data for the RNN, using a sequence of 2</span>
rnn_input, rnn_training <span style="color: #666666">=</span> format_data(y_train, <span style="color: #666666">2</span>)
<span style="color: #408080; font-style: italic"># Create a recurrent neural network in Keras and produce a summary of the </span>
<span style="color: #408080; font-style: italic"># machine learning model</span>
model <span style="color: #666666">=</span> rnn(length_of_sequences <span style="color: #666666">=</span> rnn_input<span style="color: #666666">.</span>shape[<span style="color: #666666">1</span>])
model<span style="color: #666666">.</span>summary()
<span style="color: #408080; font-style: italic"># Start the timer. Want to time training+testing</span>
start <span style="color: #666666">=</span> timer()
<span style="color: #408080; font-style: italic"># Fit the model using the training data genenerated above using 150 training iterations and a 5%</span>
<span style="color: #408080; font-style: italic"># validation split. Setting verbose to True prints information about each training iteration.</span>
hist <span style="color: #666666">=</span> model<span style="color: #666666">.</span>fit(rnn_input, rnn_training, batch_size<span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">None</span>, epochs<span style="color: #666666">=150</span>,
verbose<span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">True</span>,validation_split<span style="color: #666666">=0.05</span>)
<span style="color: #008000; font-weight: bold">for</span> label <span style="color: #AA22FF; font-weight: bold">in</span> [<span style="color: #BA2121">&quot;loss&quot;</span>,<span style="color: #BA2121">&quot;val_loss&quot;</span>]:
plt<span style="color: #666666">.</span>plot(hist<span style="color: #666666">.</span>history[label],label<span style="color: #666666">=</span>label)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;loss&quot;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;epoch&quot;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;The final validation loss: </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">.</span>format(hist<span style="color: #666666">.</span>history[<span style="color: #BA2121">&quot;val_loss&quot;</span>][<span style="color: #666666">-1</span>]))
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic"># Use the trained neural network to predict more points of the data set</span>
test_rnn(X_tot, y_tot, X_tot[<span style="color: #666666">0</span>], X_tot[dim<span style="color: #666666">-1</span>])
<span style="color: #408080; font-style: italic"># Stop the timer and calculate the total time needed.</span>
end <span style="color: #666666">=</span> timer()
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Time: &#39;</span>, end<span style="color: #666666">-</span>start)
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._week42-bs031.html">&laquo;</a></li>
<li><a href="._week42-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._week42-bs024.html">25</a></li>
<li><a href="._week42-bs025.html">26</a></li>
<li><a href="._week42-bs026.html">27</a></li>
<li><a href="._week42-bs027.html">28</a></li>
<li><a href="._week42-bs028.html">29</a></li>
<li><a href="._week42-bs029.html">30</a></li>
<li><a href="._week42-bs030.html">31</a></li>
<li><a href="._week42-bs031.html">32</a></li>
<li class="active"><a href="._week42-bs032.html">33</a></li>
<li><a href="._week42-bs033.html">34</a></li>
<li><a href="._week42-bs034.html">35</a></li>
<li><a href="._week42-bs033.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink