lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update on getting started

Browse Source
This commit is contained in:
mhjensen
2019-08-14 17:13:58 +02:00
parent 73f987a013
commit 68e18944b6
8 changed files with 142 additions and 224 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/How2ReadData/html/How2ReadData-bs.html
+43 -65
View File
@@ -55,27 +55,25 @@ Automatically generated HTML file from DocOnce source
None,
'___sec8'),
('Basic Matrix Features', 2, None, '___sec9'),
('Basic Matrix Features', 2, None, '___sec10'),
('Basic Matrix Features', 2, None, '___sec11'),
('Some famous Matrices', 2, None, '___sec12'),
('Basic Matrix Features', 2, None, '___sec13'),
('Numpy and arrays', 2, None, '___sec14'),
('Matrices in Python', 2, None, '___sec15'),
('Meet the Pandas', 2, None, '___sec16'),
('Reading Data and fitting', 2, None, '___sec17'),
('Some famous Matrices', 3, None, '___sec10'),
('More Basic Matrix Features', 3, None, '___sec11'),
('Numpy and arrays', 2, None, '___sec12'),
('Matrices in Python', 2, None, '___sec13'),
('Meet the Pandas', 2, None, '___sec14'),
('Reading Data and fitting', 2, None, '___sec15'),
('Simple linear regression model using _scikit-learn_',
3,
None,
'___sec18'),
'___sec16'),
('To our real data: nuclear binding energies. Brief reminder on '
'masses and binding energies',
3,
None,
'___sec19'),
('Organizing our data', 3, None, '___sec20'),
('Seeing the wood for the trees', 3, None, '___sec21'),
('And what about using neural networks?', 3, None, '___sec22'),
('A first summary', 2, None, '___sec23')]}
'___sec17'),
('Organizing our data', 3, None, '___sec18'),
('Seeing the wood for the trees', 3, None, '___sec19'),
('And what about using neural networks?', 3, None, '___sec20'),
('A first summary', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -123,20 +121,18 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec7" style="font-size: 80%;"><b>Installing R, C++, cython, Numba etc</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec8" style="font-size: 80%;"><b>Numpy examples and Important Matrix and vector handling packages</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Basic Matrix Features</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;"><b>Basic Matrix Features</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec11" style="font-size: 80%;"><b>Basic Matrix Features</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec12" style="font-size: 80%;"><b>Some famous Matrices</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;"><b>Basic Matrix Features</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;"><b>Numpy and arrays</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;"><b>Matrices in Python</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;"><b>Meet the Pandas</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;"><b>Reading Data and fitting</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple linear regression model using <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;To our real data: nuclear binding energies. Brief reminder on masses and binding energies</a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Organizing our data</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Seeing the wood for the trees</a></li>
<!-- navigation toc: --> <li><a href="#___sec22" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;And what about using neural networks?</a></li>
<!-- navigation toc: --> <li><a href="#___sec23" style="font-size: 80%;"><b>A first summary</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Some famous Matrices</a></li>
<!-- navigation toc: --> <li><a href="#___sec11" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;More Basic Matrix Features</a></li>
<!-- navigation toc: --> <li><a href="#___sec12" style="font-size: 80%;"><b>Numpy and arrays</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;"><b>Matrices in Python</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;"><b>Meet the Pandas</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;"><b>Reading Data and fitting</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple linear regression model using <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;To our real data: nuclear binding energies. Brief reminder on masses and binding energies</a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Organizing our data</a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Seeing the wood for the trees</a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;And what about using neural networks?</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;"><b>A first summary</b></a></li>
</ul>
</li>
@@ -170,7 +166,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Aug 13, 2019</h4></center> <!-- date -->
<center><h4>Aug 14, 2019</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
@@ -194,7 +190,7 @@ polynomials with random noise added. We will use the Python
software package <a href="http://scikit-learn.org/stable/" target="_self">Scikit-Learn</a> and
introduce various machine learning algorithms to make fits of
the data and predictions. We move thereafter to more interesting
cases such as nuclear binding energies.
cases such as data from say experiments (below we will look at experimental nuclear binding energies as an example).
These are examples where we can easily set up the data and
then use machine learning algorithms included in for example
<b>Scikit-Learn</b>.
@@ -204,7 +200,7 @@ These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python
packages for machine learning and statistical data analysis.
libraries for machine learning and statistical data analysis.
<p>
Here, we will mainly focus on two
@@ -411,14 +407,14 @@ Here we list several useful Python libraries we strongly recommend (if you use a
<p>
You will also find it convenient to utilize <b>R</b>. We will mainly
use Python during lectures and in various projects and exercises.
use Python during our lectures and in various projects and exercises.
Those of you
already familiar with <b>R</b> should feel free to continue using <b>R</b>, keeping
however an eye on the parallel Python set ups. Similarly, if you are a
Python afecionado, feel free to explore <b>R</b> as well. Jupyter/Ipython
notebook allows you to run <b>R</b> codes interactively in your
browser. The software library <b>R</b> is tuned to statistically analysis
and allows for an easy usage of the tools we will discuss in these
browser. The software library <b>R</b> is really tailored for statistical data analysis
and allows for an easy usage of the tools and algorithms we will discuss in these
lectures.
<p>
@@ -462,7 +458,7 @@ formats, ipython notebooks, latex files, pdf files etc with minimal edits. These
<h2 id="___sec8" class="anchor">Numpy examples and Important Matrix and vector handling packages </h2>
<p>
There are several central software packages for linear algebra and eigenvalue problems. Several of the more
There are several central software libraries for linear algebra and eigenvalue problems. Several of the more
popular ones have been wrapped into ofter software packages like those from the widely used text <b>Numerical Recipes</b>. The original source codes in many of the available packages are often taken from the widely used
software package LAPACK, which follows two other popular packages
developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly here.
@@ -493,14 +489,6 @@ $$
0 & 0 & 0 & 1
\end{bmatrix}
$$
</div>
</div>
<h2 id="___sec10" class="anchor">Basic Matrix Features </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
The inverse of a matrix is defined by
@@ -508,16 +496,6 @@ The inverse of a matrix is defined by
$$
\mathbf{A}^{-1} \cdot \mathbf{A} = I
$$
</div>
</div>
<h2 id="___sec11" class="anchor">Basic Matrix Features </h2>
<p>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
@@ -542,7 +520,7 @@ $$
</div>
<h2 id="___sec12" class="anchor">Some famous Matrices </h2>
<h3 id="___sec10" class="anchor">Some famous Matrices </h3>
<ul>
<li> Diagonal if \( a_{ij}=0 \) for \( i\ne j \)</li>
@@ -556,7 +534,7 @@ $$
<li> Banded, block upper triangular, block lower triangular....</li>
</ul>
<h2 id="___sec13" class="anchor">Basic Matrix Features </h2>
<h3 id="___sec11" class="anchor">More Basic Matrix Features </h3>
<p>
<div class="panel panel-default">
@@ -576,7 +554,7 @@ For an \( N\times N \) matrix \( \mathbf{A} \) the following properties are all
</div>
<h2 id="___sec14" class="anchor">Numpy and arrays </h2>
<h2 id="___sec12" class="anchor">Numpy and arrays </h2>
<a href="http://www.numpy.org/" target="_self">Numpy</a> provides an easy way to handle arrays in Python. The standard way to import this library is as
<p>
@@ -661,7 +639,7 @@ x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>l
<span style="color: #008000; font-weight: bold">print</span>(x<span style="color: #666666">.</span>itemsize)
</pre></div>
<h2 id="___sec15" class="anchor">Matrices in Python </h2>
<h2 id="___sec13" class="anchor">Matrices in Python </h2>
<p>
Having defined vectors, we are now ready to try out matrices. We can
@@ -798,7 +776,7 @@ plt<span style="color: #666666">.</span>plot(x,y,marker<span style="color: #6666
plt<span style="color: #666666">.</span>show()
</pre></div>
<h2 id="___sec16" class="anchor">Meet the Pandas </h2>
<h2 id="___sec14" class="anchor">Meet the Pandas </h2>
<p>
<br /><br /><center><p><img src="fig/pandas.jpg" align="bottom" width=600></p></center><br /><br />
@@ -923,7 +901,7 @@ most operations are vectorized, achieving thereby a high performance when dealin
As we will see below it leads also to a very concice code close to the mathematical operations we may be interested in.
For multidimensional arrays, we recommend strongly <a href="http://xarray.pydata.org/en/stable/" target="_self">xarray</a>. <b>xarray</b> has much of the same flexibility as <b>pandas</b>, but allows for the extension to higher dimensions than two. We will see examples later of the usage of both <b>pandas</b> and <b>xarray</b>.
<h2 id="___sec17" class="anchor">Reading Data and fitting </h2>
<h2 id="___sec15" class="anchor">Reading Data and fitting </h2>
<p>
In order to study various Machine Learning algorithms, we need to
@@ -959,7 +937,7 @@ you can easily implement other algorithms like decision trees and random forests
But before we really start with nuclear physics data, let's just look at some simpler polynomial fitting cases, such as,
(don't be offended) fitting straight lines!
<h3 id="___sec18" class="anchor">Simple linear regression model using <b>scikit-learn</b> </h3>
<h3 id="___sec16" class="anchor">Simple linear regression model using <b>scikit-learn</b> </h3>
<p>
We start with perhaps our simplest possible example, using <b>Scikit-Learn</b> to perform linear regression analysis on a data set produced by us.
@@ -1253,7 +1231,7 @@ plt<span style="color: #666666">.</span>show()
<span style="color: #008000; font-weight: bold">print</span> (error(y))
</pre></div>
<h3 id="___sec19" class="anchor">To our real data: nuclear binding energies. Brief reminder on masses and binding energies </h3>
<h3 id="___sec17" class="anchor">To our real data: nuclear binding energies. Brief reminder on masses and binding energies </h3>
<p>
Let us now dive into nuclear physics and remind ourselves briefly about some basic features about binding
@@ -1331,7 +1309,7 @@ We could also add a so-called pairing term, which is a correction term that
arises from the tendency of proton pairs and neutron pairs to
occur. An even number of particles is more stable than an odd number.
<h3 id="___sec20" class="anchor">Organizing our data </h3>
<h3 id="___sec18" class="anchor">Organizing our data </h3>
<p>
Let us start with reading and organizing our data.
@@ -1517,7 +1495,7 @@ save_fig(<span style="color: #BA2121">&quot;Masses2016&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<h3 id="___sec21" class="anchor">Seeing the wood for the trees </h3>
<h3 id="___sec19" class="anchor">Seeing the wood for the trees </h3>
<p>
As a teaser, let us now see how we can do this with decision trees using <b>scikit-learn</b>. Later we will switch to so-called <b>random forests</b>!
@@ -1556,7 +1534,7 @@ plt<span style="color: #666666">.</span>show()
<span style="color: #008000; font-weight: bold">print</span>(np<span style="color: #666666">.</span>mean( (Energies<span style="color: #666666">-</span>y_1)<span style="color: #666666">**2</span>))
</pre></div>
<h3 id="___sec22" class="anchor">And what about using neural networks? </h3>
<h3 id="___sec20" class="anchor">And what about using neural networks? </h3>
The <b>seaborn</b> package allows us to visualize data in an efficient way. Note that we use <b>scikit-learn</b>'s multi-layer perceptron (or feed forward neural network)
functionality.
@@ -1594,7 +1572,7 @@ ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&
plt<span style="color: #666666">.</span>show()
</pre></div>
<h2 id="___sec23" class="anchor">A first summary </h2>
<h2 id="___sec21" class="anchor">A first summary </h2>
<p>
The aim behind these introductory words was to present to you various