lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

adding more code examples

Browse Source
This commit is contained in:
mhjensen
2018-05-29 08:18:42 -04:00
parent 998e88b58a
commit 0d988ad4fc
12 changed files with 705 additions and 475 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/How2ReadData/html/How2ReadData-bs.html
+145 -97
View File
@@ -49,33 +49,33 @@ Automatically generated HTML file from DocOnce source
2,
None,
'___sec5'),
('Predator-Prey model from ecology', 2, None, '___sec6'),
('Case study from Hudson bay', 2, None, '___sec7'),
('Hudson bay data', 2, None, '___sec8'),
('Plotting the data', 2, None, '___sec9'),
('Non-Linear Least squares in R', 2, None, '___sec6'),
('Predator-Prey model from ecology', 2, None, '___sec7'),
('Case study from Hudson bay', 2, None, '___sec8'),
('Hudson bay data', 2, None, '___sec9'),
('Plotting the data', 2, None, '___sec10'),
('Hares and lynx in Hudson bay from 1900 to 1920',
2,
None,
'___sec10'),
'___sec11'),
('Why now create a computer model for the hare and lynx '
'populations?',
2,
None,
'___sec11'),
('The traditional (top-down) approach', 2, None, '___sec12'),
('Basic mathematics notation', 2, None, '___sec13'),
'___sec12'),
('The traditional (top-down) approach', 2, None, '___sec13'),
('Basic mathematics notation', 2, None, '___sec14'),
('Basic dynamics of the population of hares',
2,
None,
'___sec14'),
('Basic dynamics of the population of lynx', 2, None, '___sec15'),
('Evolution equations', 2, None, '___sec16'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec17'),
('The program', 2, None, '___sec18'),
('The plot', 2, None, '___sec19'),
('Linear regression in Python', 2, None, '___sec20'),
('Linear Least squares in R', 2, None, '___sec21'),
('Non-Linear Least squares in R', 2, None, '___sec22'),
'___sec15'),
('Basic dynamics of the population of lynx', 2, None, '___sec16'),
('Evolution equations', 2, None, '___sec17'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec18'),
('The program', 2, None, '___sec19'),
('The plot', 2, None, '___sec20'),
('Linear regression in Python', 2, None, '___sec21'),
('Linear Least squares in R', 2, None, '___sec22'),
('Example: ecoli lab experiment', 2, None, '___sec23'),
('The program', 2, None, '___sec24'),
('The output', 2, None, '___sec25'),
@@ -133,23 +133,23 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec3" style="font-size: 80%;"><b>Installing R, C++, cython or Julia</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec4" style="font-size: 80%;"><b>Installing R, C++, cython, Numba etc</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec5" style="font-size: 80%;"><b>Simple linear regression model using <b>scikit-learn</b></b></a></li>
<!-- navigation toc: --> <li><a href="#___sec6" style="font-size: 80%;"><b>Predator-Prey model from ecology</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec7" style="font-size: 80%;"><b>Case study from Hudson bay</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec8" style="font-size: 80%;"><b>Hudson bay data</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Plotting the data</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;"><b>Hares and lynx in Hudson bay from 1900 to 1920</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec11" style="font-size: 80%;"><b>Why now create a computer model for the hare and lynx populations?</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec12" style="font-size: 80%;"><b>The traditional (top-down) approach</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;"><b>Basic mathematics notation</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;"><b>Basic dynamics of the population of hares</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;"><b>Basic dynamics of the population of lynx</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;"><b>Evolution equations</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;"><b>Adapt the model to the Hudson Bay case</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;"><b>The program</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;"><b>The plot</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;"><b>Linear regression in Python</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;"><b>Linear Least squares in R</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec22" style="font-size: 80%;"><b>Non-Linear Least squares in R</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec6" style="font-size: 80%;"><b>Non-Linear Least squares in R</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec7" style="font-size: 80%;"><b>Predator-Prey model from ecology</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec8" style="font-size: 80%;"><b>Case study from Hudson bay</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Hudson bay data</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;"><b>Plotting the data</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec11" style="font-size: 80%;"><b>Hares and lynx in Hudson bay from 1900 to 1920</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec12" style="font-size: 80%;"><b>Why now create a computer model for the hare and lynx populations?</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;"><b>The traditional (top-down) approach</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;"><b>Basic mathematics notation</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;"><b>Basic dynamics of the population of hares</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;"><b>Basic dynamics of the population of lynx</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;"><b>Evolution equations</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;"><b>Adapt the model to the Hudson Bay case</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;"><b>The program</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;"><b>The plot</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;"><b>Linear regression in Python</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec22" style="font-size: 80%;"><b>Linear Least squares in R</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec23" style="font-size: 80%;"><b>Example: ecoli lab experiment</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec24" style="font-size: 80%;"><b>The program</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec25" style="font-size: 80%;"><b>The output</b></a></li>
@@ -191,7 +191,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>May 28, 2018</h4></center> <!-- date -->
<center><h4>May 29, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- potential-jumbotron-button -->
@@ -224,16 +224,20 @@ then use machine learning algorithms included in for example
<b>scikit-learn</b>.
<p>
These examples will serve us the purpose of getting started. Furthermore, they
allow us to catch more than two birds with a stone. They will allow us
to bring in some programming specific topics and tools as well as
showing the power of various Python (and R) packages for machine
learning and statistical data analysis. In the lectures on linear
algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which
are relevant for the various algorithms we will discuss. Here, we will
mainly focus on two specific Python packages for Machine Learning,
scikit-learn and tensorflow (see below for links etc).
Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming.
These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python (and
R) packages for machine learning and statistical data analysis. In the
lectures on linear algebra we cover in more detail various programming
features of languages like Python and C++ (and other), we will also
look into more specific linear functions which are relevant for the
various algorithms we will discuss. Here, we will mainly focus on two
specific Python packages for Machine Learning, scikit-learn and
tensorflow (see below for links etc). Moreover, the examples we
introduce will serve as inputs to many of our discussions later, as
well as allowing you to set up models and produce your own data and
get started with programming.
<p>
<!-- !split -->
@@ -618,7 +622,86 @@ years etc.
<p>
We will discuss in more
detail these and more function in the various lectures.
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> Ridge
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> make_pipeline
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">f</span>(x):
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot; function to approximate by polynomial interpolation&quot;&quot;&quot;</span>
<span style="color: #008000; font-weight: bold">return</span> x<span style="color: #666666">*</span>x<span style="color: #666666">*</span>x
<span style="color: #408080; font-style: italic"># generate points used to plot </span>
x_plot <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
<span style="color: #408080; font-style: italic"># generate points and keep a subset of them </span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
rng <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>RandomState(<span style="color: #666666">0</span>)
rng<span style="color: #666666">.</span>shuffle(x)
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sort(x[:<span style="color: #666666">20</span>])
y <span style="color: #666666">=</span> f(x)
<span style="color: #408080; font-style: italic"># create matrix versions of these arrays </span>
X <span style="color: #666666">=</span> x[:, np<span style="color: #666666">.</span>newaxis]
X_plot <span style="color: #666666">=</span> x_plot[:, np<span style="color: #666666">.</span>newaxis]
colors <span style="color: #666666">=</span> [<span style="color: #BA2121">&#39;teal&#39;</span>, <span style="color: #BA2121">&#39;yellowgreen&#39;</span>, <span style="color: #BA2121">&#39;gold&#39;</span>]
lw <span style="color: #666666">=</span> <span style="color: #666666">2</span>
plt<span style="color: #666666">.</span>plot(x_plot, f(x_plot), color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;cornflowerblue&#39;</span>, linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;ground truth&quot;</span>)
plt<span style="color: #666666">.</span>scatter(x, y, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;navy&#39;</span>, s<span style="color: #666666">=30</span>, marker<span style="color: #666666">=</span><span style="color: #BA2121">&#39;o&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;training points&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> count, degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>([<span style="color: #666666">3</span>, <span style="color: #666666">4</span>, <span style="color: #666666">5</span>]):
model <span style="color: #666666">=</span> make_pipeline(PolynomialFeatures(degree), Ridge())
model<span style="color: #666666">.</span>fit(X, y)
y_plot <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_plot)
plt<span style="color: #666666">.</span>plot(x_plot, y_plot, color<span style="color: #666666">=</span>colors[count], linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;degree </span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121">&quot;</span> <span style="color: #666666">%</span> degree)
plt<span style="color: #666666">.</span>legend(loc<span style="color: #666666">=</span><span style="color: #BA2121">&#39;lower left&#39;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<!-- !split -->
<h2 id="___sec6" class="anchor">Non-Linear Least squares in R </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
<!-- code=r (!bc r) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000">set.seed</span>(<span style="color: #666666">1485</span>)
len <span style="color: #666666">=</span> <span style="color: #666666">24</span>
x <span style="color: #666666">=</span> runif(len)
y <span style="color: #666666">=</span> x<span style="color: #666666">^3+</span>rnorm(len, <span style="color: #666666">0</span>,<span style="color: #666666">0.06</span>)
ds <span style="color: #666666">=</span> <span style="color: #B00040">data.frame</span>(x <span style="color: #666666">=</span> x, y <span style="color: #666666">=</span> y)
str(ds)
plot( y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span><span style="color: #BA2121">&quot;Known cubic with noise&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,length <span style="color: #666666">=100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=2</span>, col <span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>)
m <span style="color: #666666">=</span> nls(y <span style="color: #666666">~</span> <span style="color: #008000">I</span>(x<span style="color: #666666">^</span>power), data <span style="color: #666666">=</span> ds, start <span style="color: #666666">=</span> <span style="color: #B00040">list</span>(power<span style="color: #666666">=1</span>), trace <span style="color: #666666">=</span> <span style="color: #008000">T</span>)
<span style="color: #008000">class</span>(m)
<span style="color: #008000">summary</span>(m)
power <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">1</span>], <span style="color: #666666">3</span>)
power.se <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">2</span>], <span style="color: #666666">3</span>)
plot(y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Fitted power model&quot;</span>, sub <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Blue: fit; green: known&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>, <span style="color: #666666">1</span>, length <span style="color: #666666">=</span> <span style="color: #666666">100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=</span> <span style="color: #666666">2</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #B00040">list</span>(x <span style="color: #666666">=</span> s)), lty <span style="color: #666666">=</span> <span style="color: #666666">1</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;blue&quot;</span>)
text(<span style="color: #666666">0</span>, <span style="color: #666666">0.5</span>, <span style="color: #008000">paste</span>(<span style="color: #BA2121">&quot;y =x^ (&quot;</span>, power, <span style="color: #BA2121">&quot; +/- &quot;</span>, power.se, <span style="color: #BA2121">&quot;)&quot;</span>, sep <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;&quot;</span>), pos <span style="color: #666666">=</span> <span style="color: #666666">4</span>)
</pre></div>
<p>
</div>
</div>
<p>
Another useful Python package is
@@ -638,7 +721,7 @@ display(data_pandas)
<p>
<!-- !split -->
<h2 id="___sec6" class="anchor">Predator-Prey model from ecology </h2>
<h2 id="___sec7" class="anchor">Predator-Prey model from ecology </h2>
<p>
<div class="panel panel-default">
@@ -663,7 +746,7 @@ scientific method:
<p>
<!-- !split -->
<h2 id="___sec7" class="anchor">Case study from Hudson bay </h2>
<h2 id="___sec8" class="anchor">Case study from Hudson bay </h2>
<p>
<div class="panel panel-default">
@@ -685,7 +768,7 @@ Here we start by
<p>
<!-- !split -->
<h2 id="___sec8" class="anchor">Hudson bay data </h2>
<h2 id="___sec9" class="anchor">Hudson bay data </h2>
<p>
<div class="panel panel-default">
@@ -740,7 +823,7 @@ One reason that this particular system has been so extensively studied is that t
<p>
<!-- !split -->
<h2 id="___sec9" class="anchor">Plotting the data </h2>
<h2 id="___sec10" class="anchor">Plotting the data </h2>
<p>
<div class="panel panel-default">
@@ -777,7 +860,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split -->
<h2 id="___sec10" class="anchor">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<h2 id="___sec11" class="anchor">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_data.png" align="bottom" width=700></p></center><br /><br />
@@ -785,7 +868,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split -->
<h2 id="___sec11" class="anchor">Why now create a computer model for the hare and lynx populations? </h2>
<h2 id="___sec12" class="anchor">Why now create a computer model for the hare and lynx populations? </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
@@ -819,7 +902,7 @@ climate and other complicating factors. How significant are these?
<p>
<!-- !split -->
<h2 id="___sec12" class="anchor">The traditional (top-down) approach </h2>
<h2 id="___sec13" class="anchor">The traditional (top-down) approach </h2>
<p>
<div class="panel panel-default">
@@ -853,7 +936,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split -->
<h2 id="___sec13" class="anchor">Basic mathematics notation </h2>
<h2 id="___sec14" class="anchor">Basic mathematics notation </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
@@ -874,7 +957,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split -->
<h2 id="___sec14" class="anchor">Basic dynamics of the population of hares </h2>
<h2 id="___sec15" class="anchor">Basic dynamics of the population of hares </h2>
<p>
<div class="panel panel-default">
@@ -903,7 +986,7 @@ $$ \Delta H = a\Delta t H^n - b \Delta t H^nL^n$$
<p>
<!-- !split -->
<h2 id="___sec15" class="anchor">Basic dynamics of the population of lynx </h2>
<h2 id="___sec16" class="anchor">Basic dynamics of the population of lynx </h2>
<p>
<div class="panel panel-default">
@@ -934,7 +1017,7 @@ $$ \Delta L = d\Delta t H^nL^n - c\Delta t L^n$$
<p>
<!-- !split -->
<h2 id="___sec16" class="anchor">Evolution equations </h2>
<h2 id="___sec17" class="anchor">Evolution equations </h2>
<p>
<div class="panel panel-default">
@@ -964,7 +1047,7 @@ Note:
<p>
<!-- !split -->
<h2 id="___sec17" class="anchor">Adapt the model to the Hudson Bay case </h2>
<h2 id="___sec18" class="anchor">Adapt the model to the Hudson Bay case </h2>
<p>
<div class="panel panel-default">
@@ -988,7 +1071,7 @@ Note:
<p>
<!-- !split -->
<h2 id="___sec18" class="anchor">The program </h2>
<h2 id="___sec19" class="anchor">The program </h2>
<p>
<div class="panel panel-default">
@@ -1049,7 +1132,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split -->
<h2 id="___sec19" class="anchor">The plot </h2>
<h2 id="___sec20" class="anchor">The plot </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_sim.png" align="bottom" width=700></p></center><br /><br />
@@ -1060,7 +1143,7 @@ If we perform a least-square fitting, we can find optimal values for the paramet
<p>
<!-- !split -->
<h2 id="___sec20" class="anchor">Linear regression in Python </h2>
<h2 id="___sec21" class="anchor">Linear regression in Python </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
@@ -1094,7 +1177,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split -->
<h2 id="___sec21" class="anchor">Linear Least squares in R </h2>
<h2 id="___sec22" class="anchor">Linear Least squares in R </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
@@ -1126,41 +1209,6 @@ predict(linearMod,<span style="color: #B00040">data.frame</span>(Year<span style
</div>
<p>
<!-- !split -->
<h2 id="___sec22" class="anchor">Non-Linear Least squares in R </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
<!-- code=r (!bc r) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000">set.seed</span>(<span style="color: #666666">1485</span>)
len <span style="color: #666666">=</span> <span style="color: #666666">24</span>
x <span style="color: #666666">=</span> runif(len)
y <span style="color: #666666">=</span> x<span style="color: #666666">^3+</span>rnorm(len, <span style="color: #666666">0</span>,<span style="color: #666666">0.06</span>)
ds <span style="color: #666666">=</span> <span style="color: #B00040">data.frame</span>(x <span style="color: #666666">=</span> x, y <span style="color: #666666">=</span> y)
str(ds)
plot( y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span><span style="color: #BA2121">&quot;Known cubic with noise&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,length <span style="color: #666666">=100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=2</span>, col <span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>)
m <span style="color: #666666">=</span> nls(y <span style="color: #666666">~</span> <span style="color: #008000">I</span>(x<span style="color: #666666">^</span>power), data <span style="color: #666666">=</span> ds, start <span style="color: #666666">=</span> <span style="color: #B00040">list</span>(power<span style="color: #666666">=1</span>), trace <span style="color: #666666">=</span> <span style="color: #008000">T</span>)
<span style="color: #008000">class</span>(m)
<span style="color: #008000">summary</span>(m)
power <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">1</span>], <span style="color: #666666">3</span>)
power.se <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">2</span>], <span style="color: #666666">3</span>)
plot(y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Fitted power model&quot;</span>, sub <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Blue: fit; green: known&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>, <span style="color: #666666">1</span>, length <span style="color: #666666">=</span> <span style="color: #666666">100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=</span> <span style="color: #666666">2</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #B00040">list</span>(x <span style="color: #666666">=</span> s)), lty <span style="color: #666666">=</span> <span style="color: #666666">1</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;blue&quot;</span>)
text(<span style="color: #666666">0</span>, <span style="color: #666666">0.5</span>, <span style="color: #008000">paste</span>(<span style="color: #BA2121">&quot;y =x^ (&quot;</span>, power, <span style="color: #BA2121">&quot; +/- &quot;</span>, power.se, <span style="color: #BA2121">&quot;)&quot;</span>, sep <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;&quot;</span>), pos <span style="color: #666666">=</span> <span style="color: #666666">4</span>)
</pre></div>
<p>
</div>
</div>
<p>
<!-- !split -->
doc/pub/How2ReadData/html/How2ReadData-reveal.html
+109 -60
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>May 28, 2018</h4></center> <!-- date -->
<center><h4>May 29, 2018</h4></center> <!-- date -->
<br>
<p>
@@ -184,16 +184,20 @@ then use machine learning algorithms included in for example
<b>scikit-learn</b>.
<p>
These examples will serve us the purpose of getting started. Furthermore, they
allow us to catch more than two birds with a stone. They will allow us
to bring in some programming specific topics and tools as well as
showing the power of various Python (and R) packages for machine
learning and statistical data analysis. In the lectures on linear
algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which
are relevant for the various algorithms we will discuss. Here, we will
mainly focus on two specific Python packages for Machine Learning,
scikit-learn and tensorflow (see below for links etc).
Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming.
These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python (and
R) packages for machine learning and statistical data analysis. In the
lectures on linear algebra we cover in more detail various programming
features of languages like Python and C++ (and other), we will also
look into more specific linear functions which are relevant for the
various algorithms we will discuss. Here, we will mainly focus on two
specific Python packages for Machine Learning, scikit-learn and
tensorflow (see below for links etc). Moreover, the examples we
introduce will serve as inputs to many of our discussions later, as
well as allowing you to set up models and produce your own data and
get started with programming.
</section>
@@ -601,7 +605,84 @@ years etc.
<p>
We will discuss in more
detail these and more function in the various lectures.
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.linear_model</span> <span style="color: #8B008B; font-weight: bold">import</span> Ridge
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.pipeline</span> <span style="color: #8B008B; font-weight: bold">import</span> make_pipeline
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">f</span>(x):
<span style="color: #CD5555">&quot;&quot;&quot; function to approximate by polynomial interpolation&quot;&quot;&quot;</span>
<span style="color: #8B008B; font-weight: bold">return</span> x*x*x
<span style="color: #228B22"># generate points used to plot </span>
x_plot = np.linspace(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">10</span>, <span style="color: #B452CD">100</span>)
<span style="color: #228B22"># generate points and keep a subset of them </span>
x = np.linspace(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">10</span>, <span style="color: #B452CD">100</span>)
rng = np.random.RandomState(<span style="color: #B452CD">0</span>)
rng.shuffle(x)
x = np.sort(x[:<span style="color: #B452CD">20</span>])
y = f(x)
<span style="color: #228B22"># create matrix versions of these arrays </span>
X = x[:, np.newaxis]
X_plot = x_plot[:, np.newaxis]
colors = [<span style="color: #CD5555">&#39;teal&#39;</span>, <span style="color: #CD5555">&#39;yellowgreen&#39;</span>, <span style="color: #CD5555">&#39;gold&#39;</span>]
lw = <span style="color: #B452CD">2</span>
plt.plot(x_plot, f(x_plot), color=<span style="color: #CD5555">&#39;cornflowerblue&#39;</span>, linewidth=lw,
label=<span style="color: #CD5555">&quot;ground truth&quot;</span>)
plt.scatter(x, y, color=<span style="color: #CD5555">&#39;navy&#39;</span>, s=<span style="color: #B452CD">30</span>, marker=<span style="color: #CD5555">&#39;o&#39;</span>, label=<span style="color: #CD5555">&quot;training points&quot;</span>)
<span style="color: #8B008B; font-weight: bold">for</span> count, degree <span style="color: #8B008B">in</span> <span style="color: #658b00">enumerate</span>([<span style="color: #B452CD">3</span>, <span style="color: #B452CD">4</span>, <span style="color: #B452CD">5</span>]):
model = make_pipeline(PolynomialFeatures(degree), Ridge())
model.fit(X, y)
y_plot = model.predict(X_plot)
plt.plot(x_plot, y_plot, color=colors[count], linewidth=lw,
label=<span style="color: #CD5555">&quot;degree %d&quot;</span> % degree)
plt.legend(loc=<span style="color: #CD5555">&#39;lower left&#39;</span>)
plt.show()
</pre></div>
</section>
<section>
<h2 id="___sec6">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<!-- code=r (!bc r) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">set.seed</span>(<span style="color: #B452CD">1485</span>)
len = <span style="color: #B452CD">24</span>
x = runif(len)
y = x^<span style="color: #B452CD">3</span>+rnorm(len, <span style="color: #B452CD">0</span>,<span style="color: #B452CD">0.06</span>)
ds = <span style="color: #00688B; font-weight: bold">data.frame</span>(x = x, y = y)
str(ds)
plot( y ~ x, main =<span style="color: #CD5555">&quot;Known cubic with noise&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>,<span style="color: #B452CD">1</span>,length =<span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty =<span style="color: #B452CD">2</span>, col =<span style="color: #CD5555">&quot;green&quot;</span>)
m = nls(y ~ <span style="color: #8B008B; font-weight: bold">I</span>(x^power), data = ds, start = <span style="color: #00688B; font-weight: bold">list</span>(power=<span style="color: #B452CD">1</span>), trace = <span style="color: #658b00">T</span>)
<span style="color: #8B008B; font-weight: bold">class</span>(m)
<span style="color: #8B008B; font-weight: bold">summary</span>(m)
power = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">1</span>], <span style="color: #B452CD">3</span>)
power.se = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">2</span>], <span style="color: #B452CD">3</span>)
plot(y ~ x, main = <span style="color: #CD5555">&quot;Fitted power model&quot;</span>, sub = <span style="color: #CD5555">&quot;Blue: fit; green: known&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">1</span>, length = <span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty = <span style="color: #B452CD">2</span>, col = <span style="color: #CD5555">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #00688B; font-weight: bold">list</span>(x = s)), lty = <span style="color: #B452CD">1</span>, col = <span style="color: #CD5555">&quot;blue&quot;</span>)
text(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">0.5</span>, <span style="color: #8B008B; font-weight: bold">paste</span>(<span style="color: #CD5555">&quot;y =x^ (&quot;</span>, power, <span style="color: #CD5555">&quot; +/- &quot;</span>, power.se, <span style="color: #CD5555">&quot;)&quot;</span>, sep = <span style="color: #CD5555">&quot;&quot;</span>), pos = <span style="color: #B452CD">4</span>)
</pre></div>
</div>
<p>
Another useful Python package is
@@ -622,7 +703,7 @@ display(data_pandas)
<section>
<h2 id="___sec6">Predator-Prey model from ecology </h2>
<h2 id="___sec7">Predator-Prey model from ecology </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -645,7 +726,7 @@ scientific method:
<section>
<h2 id="___sec7">Case study from Hudson bay </h2>
<h2 id="___sec8">Case study from Hudson bay </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -665,7 +746,7 @@ Here we start by
<section>
<h2 id="___sec8">Hudson bay data </h2>
<h2 id="___sec9">Hudson bay data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -711,7 +792,7 @@ One reason that this particular system has been so extensively studied is that t
<section>
<h2 id="___sec9">Plotting the data </h2>
<h2 id="___sec10">Plotting the data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -745,7 +826,7 @@ plt.show()
<section>
<h2 id="___sec10">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<h2 id="___sec11">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_data.png" align="bottom" width=700></p></center><br /><br />
@@ -753,7 +834,7 @@ plt.show()
<section>
<h2 id="___sec11">Why now create a computer model for the hare and lynx populations? </h2>
<h2 id="___sec12">Why now create a computer model for the hare and lynx populations? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -785,7 +866,7 @@ climate and other complicating factors. How significant are these?
<section>
<h2 id="___sec12">The traditional (top-down) approach </h2>
<h2 id="___sec13">The traditional (top-down) approach </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -820,7 +901,7 @@ ODEs</em> (which cannot be solved)
<section>
<h2 id="___sec13">Basic mathematics notation </h2>
<h2 id="___sec14">Basic mathematics notation </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<ul>
@@ -837,7 +918,7 @@ ODEs</em> (which cannot be solved)
<section>
<h2 id="___sec14">Basic dynamics of the population of hares </h2>
<h2 id="___sec15">Basic dynamics of the population of hares </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -868,7 +949,7 @@ $$ \Delta H = a\Delta t H^n - b \Delta t H^nL^n$$
<section>
<h2 id="___sec15">Basic dynamics of the population of lynx </h2>
<h2 id="___sec16">Basic dynamics of the population of lynx </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -897,7 +978,7 @@ $$ \Delta L = d\Delta t H^nL^n - c\Delta t L^n$$
<section>
<h2 id="___sec16">Evolution equations </h2>
<h2 id="___sec17">Evolution equations </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -928,7 +1009,7 @@ Note:
<section>
<h2 id="___sec17">Adapt the model to the Hudson Bay case </h2>
<h2 id="___sec18">Adapt the model to the Hudson Bay case </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -948,7 +1029,7 @@ Note:
<section>
<h2 id="___sec18">The program </h2>
<h2 id="___sec19">The program </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -1006,7 +1087,7 @@ plt.show()
<section>
<h2 id="___sec19">The plot </h2>
<h2 id="___sec20">The plot </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_sim.png" align="bottom" width=700></p></center><br /><br />
@@ -1017,7 +1098,7 @@ If we perform a least-square fitting, we can find optimal values for the paramet
<section>
<h2 id="___sec20">Linear regression in Python </h2>
<h2 id="___sec21">Linear regression in Python </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1048,7 +1129,7 @@ plt.show()
<section>
<h2 id="___sec21">Linear Least squares in R </h2>
<h2 id="___sec22">Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1079,38 +1160,6 @@ predict(linearMod,<span style="color: #00688B; font-weight: bold">data.frame</sp
</section>
<section>
<h2 id="___sec22">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<!-- code=r (!bc r) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">set.seed</span>(<span style="color: #B452CD">1485</span>)
len = <span style="color: #B452CD">24</span>
x = runif(len)
y = x^<span style="color: #B452CD">3</span>+rnorm(len, <span style="color: #B452CD">0</span>,<span style="color: #B452CD">0.06</span>)
ds = <span style="color: #00688B; font-weight: bold">data.frame</span>(x = x, y = y)
str(ds)
plot( y ~ x, main =<span style="color: #CD5555">&quot;Known cubic with noise&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>,<span style="color: #B452CD">1</span>,length =<span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty =<span style="color: #B452CD">2</span>, col =<span style="color: #CD5555">&quot;green&quot;</span>)
m = nls(y ~ <span style="color: #8B008B; font-weight: bold">I</span>(x^power), data = ds, start = <span style="color: #00688B; font-weight: bold">list</span>(power=<span style="color: #B452CD">1</span>), trace = <span style="color: #658b00">T</span>)
<span style="color: #8B008B; font-weight: bold">class</span>(m)
<span style="color: #8B008B; font-weight: bold">summary</span>(m)
power = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">1</span>], <span style="color: #B452CD">3</span>)
power.se = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">2</span>], <span style="color: #B452CD">3</span>)
plot(y ~ x, main = <span style="color: #CD5555">&quot;Fitted power model&quot;</span>, sub = <span style="color: #CD5555">&quot;Blue: fit; green: known&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">1</span>, length = <span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty = <span style="color: #B452CD">2</span>, col = <span style="color: #CD5555">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #00688B; font-weight: bold">list</span>(x = s)), lty = <span style="color: #B452CD">1</span>, col = <span style="color: #CD5555">&quot;blue&quot;</span>)
text(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">0.5</span>, <span style="color: #8B008B; font-weight: bold">paste</span>(<span style="color: #CD5555">&quot;y =x^ (&quot;</span>, power, <span style="color: #CD5555">&quot; +/- &quot;</span>, power.se, <span style="color: #CD5555">&quot;)&quot;</span>, sep = <span style="color: #CD5555">&quot;&quot;</span>), pos = <span style="color: #B452CD">4</span>)
</pre></div>
</div>
</section>
<section>
<h2 id="___sec23">Example: ecoli lab experiment </h2>
doc/pub/How2ReadData/html/How2ReadData-solarized.html
+127 -79
View File
@@ -69,33 +69,33 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'___sec5'),
('Predator-Prey model from ecology', 2, None, '___sec6'),
('Case study from Hudson bay', 2, None, '___sec7'),
('Hudson bay data', 2, None, '___sec8'),
('Plotting the data', 2, None, '___sec9'),
('Non-Linear Least squares in R', 2, None, '___sec6'),
('Predator-Prey model from ecology', 2, None, '___sec7'),
('Case study from Hudson bay', 2, None, '___sec8'),
('Hudson bay data', 2, None, '___sec9'),
('Plotting the data', 2, None, '___sec10'),
('Hares and lynx in Hudson bay from 1900 to 1920',
2,
None,
'___sec10'),
'___sec11'),
('Why now create a computer model for the hare and lynx '
'populations?',
2,
None,
'___sec11'),
('The traditional (top-down) approach', 2, None, '___sec12'),
('Basic mathematics notation', 2, None, '___sec13'),
'___sec12'),
('The traditional (top-down) approach', 2, None, '___sec13'),
('Basic mathematics notation', 2, None, '___sec14'),
('Basic dynamics of the population of hares',
2,
None,
'___sec14'),
('Basic dynamics of the population of lynx', 2, None, '___sec15'),
('Evolution equations', 2, None, '___sec16'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec17'),
('The program', 2, None, '___sec18'),
('The plot', 2, None, '___sec19'),
('Linear regression in Python', 2, None, '___sec20'),
('Linear Least squares in R', 2, None, '___sec21'),
('Non-Linear Least squares in R', 2, None, '___sec22'),
'___sec15'),
('Basic dynamics of the population of lynx', 2, None, '___sec16'),
('Evolution equations', 2, None, '___sec17'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec18'),
('The program', 2, None, '___sec19'),
('The plot', 2, None, '___sec20'),
('Linear regression in Python', 2, None, '___sec21'),
('Linear Least squares in R', 2, None, '___sec22'),
('Example: ecoli lab experiment', 2, None, '___sec23'),
('The program', 2, None, '___sec24'),
('The output', 2, None, '___sec25'),
@@ -151,7 +151,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>May 28, 2018</h4></center> <!-- date -->
<center><h4>May 29, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -181,16 +181,20 @@ then use machine learning algorithms included in for example
<b>scikit-learn</b>.
<p>
These examples will serve us the purpose of getting started. Furthermore, they
allow us to catch more than two birds with a stone. They will allow us
to bring in some programming specific topics and tools as well as
showing the power of various Python (and R) packages for machine
learning and statistical data analysis. In the lectures on linear
algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which
are relevant for the various algorithms we will discuss. Here, we will
mainly focus on two specific Python packages for Machine Learning,
scikit-learn and tensorflow (see below for links etc).
Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming.
These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python (and
R) packages for machine learning and statistical data analysis. In the
lectures on linear algebra we cover in more detail various programming
features of languages like Python and C++ (and other), we will also
look into more specific linear functions which are relevant for the
various algorithms we will discuss. Here, we will mainly focus on two
specific Python packages for Machine Learning, scikit-learn and
tensorflow (see below for links etc). Moreover, the examples we
introduce will serve as inputs to many of our discussions later, as
well as allowing you to set up models and produce your own data and
get started with programming.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -575,7 +579,85 @@ years etc.
<p>
We will discuss in more
detail these and more function in the various lectures.
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.linear_model</span> <span style="color: #8B008B; font-weight: bold">import</span> Ridge
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.pipeline</span> <span style="color: #8B008B; font-weight: bold">import</span> make_pipeline
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">f</span>(x):
<span style="color: #CD5555">&quot;&quot;&quot; function to approximate by polynomial interpolation&quot;&quot;&quot;</span>
<span style="color: #8B008B; font-weight: bold">return</span> x*x*x
<span style="color: #228B22"># generate points used to plot </span>
x_plot = np.linspace(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">10</span>, <span style="color: #B452CD">100</span>)
<span style="color: #228B22"># generate points and keep a subset of them </span>
x = np.linspace(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">10</span>, <span style="color: #B452CD">100</span>)
rng = np.random.RandomState(<span style="color: #B452CD">0</span>)
rng.shuffle(x)
x = np.sort(x[:<span style="color: #B452CD">20</span>])
y = f(x)
<span style="color: #228B22"># create matrix versions of these arrays </span>
X = x[:, np.newaxis]
X_plot = x_plot[:, np.newaxis]
colors = [<span style="color: #CD5555">&#39;teal&#39;</span>, <span style="color: #CD5555">&#39;yellowgreen&#39;</span>, <span style="color: #CD5555">&#39;gold&#39;</span>]
lw = <span style="color: #B452CD">2</span>
plt.plot(x_plot, f(x_plot), color=<span style="color: #CD5555">&#39;cornflowerblue&#39;</span>, linewidth=lw,
label=<span style="color: #CD5555">&quot;ground truth&quot;</span>)
plt.scatter(x, y, color=<span style="color: #CD5555">&#39;navy&#39;</span>, s=<span style="color: #B452CD">30</span>, marker=<span style="color: #CD5555">&#39;o&#39;</span>, label=<span style="color: #CD5555">&quot;training points&quot;</span>)
<span style="color: #8B008B; font-weight: bold">for</span> count, degree <span style="color: #8B008B">in</span> <span style="color: #658b00">enumerate</span>([<span style="color: #B452CD">3</span>, <span style="color: #B452CD">4</span>, <span style="color: #B452CD">5</span>]):
model = make_pipeline(PolynomialFeatures(degree), Ridge())
model.fit(X, y)
y_plot = model.predict(X_plot)
plt.plot(x_plot, y_plot, color=colors[count], linewidth=lw,
label=<span style="color: #CD5555">&quot;degree %d&quot;</span> % degree)
plt.legend(loc=<span style="color: #CD5555">&#39;lower left&#39;</span>)
plt.show()
</pre></div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=r (!bc r) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eee8d5"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">set.seed</span>(<span style="color: #B452CD">1485</span>)
len = <span style="color: #B452CD">24</span>
x = runif(len)
y = x^<span style="color: #B452CD">3</span>+rnorm(len, <span style="color: #B452CD">0</span>,<span style="color: #B452CD">0.06</span>)
ds = <span style="color: #00688B; font-weight: bold">data.frame</span>(x = x, y = y)
str(ds)
plot( y ~ x, main =<span style="color: #CD5555">&quot;Known cubic with noise&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>,<span style="color: #B452CD">1</span>,length =<span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty =<span style="color: #B452CD">2</span>, col =<span style="color: #CD5555">&quot;green&quot;</span>)
m = nls(y ~ <span style="color: #8B008B; font-weight: bold">I</span>(x^power), data = ds, start = <span style="color: #00688B; font-weight: bold">list</span>(power=<span style="color: #B452CD">1</span>), trace = <span style="color: #658b00">T</span>)
<span style="color: #8B008B; font-weight: bold">class</span>(m)
<span style="color: #8B008B; font-weight: bold">summary</span>(m)
power = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">1</span>], <span style="color: #B452CD">3</span>)
power.se = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">2</span>], <span style="color: #B452CD">3</span>)
plot(y ~ x, main = <span style="color: #CD5555">&quot;Fitted power model&quot;</span>, sub = <span style="color: #CD5555">&quot;Blue: fit; green: known&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">1</span>, length = <span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty = <span style="color: #B452CD">2</span>, col = <span style="color: #CD5555">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #00688B; font-weight: bold">list</span>(x = s)), lty = <span style="color: #B452CD">1</span>, col = <span style="color: #CD5555">&quot;blue&quot;</span>)
text(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">0.5</span>, <span style="color: #8B008B; font-weight: bold">paste</span>(<span style="color: #CD5555">&quot;y =x^ (&quot;</span>, power, <span style="color: #CD5555">&quot; +/- &quot;</span>, power.se, <span style="color: #CD5555">&quot;)&quot;</span>, sep = <span style="color: #CD5555">&quot;&quot;</span>), pos = <span style="color: #B452CD">4</span>)
</pre></div>
</div>
<p>
Another useful Python package is
@@ -595,7 +677,7 @@ display(data_pandas)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Predator-Prey model from ecology </h2>
<h2 id="___sec7">Predator-Prey model from ecology </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -619,7 +701,7 @@ scientific method:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">Case study from Hudson bay </h2>
<h2 id="___sec8">Case study from Hudson bay </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -640,7 +722,7 @@ Here we start by
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Hudson bay data </h2>
<h2 id="___sec9">Hudson bay data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -689,7 +771,7 @@ One reason that this particular system has been so extensively studied is that t
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec9">Plotting the data </h2>
<h2 id="___sec10">Plotting the data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -725,7 +807,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec10">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<h2 id="___sec11">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_data.png" align="bottom" width=700></p></center><br /><br />
@@ -733,7 +815,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec11">Why now create a computer model for the hare and lynx populations? </h2>
<h2 id="___sec12">Why now create a computer model for the hare and lynx populations? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -766,7 +848,7 @@ climate and other complicating factors. How significant are these?
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec12">The traditional (top-down) approach </h2>
<h2 id="___sec13">The traditional (top-down) approach </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -799,7 +881,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec13">Basic mathematics notation </h2>
<h2 id="___sec14">Basic mathematics notation </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -819,7 +901,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Basic dynamics of the population of hares </h2>
<h2 id="___sec15">Basic dynamics of the population of hares </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -847,7 +929,7 @@ $$ \Delta H = a\Delta t H^n - b \Delta t H^nL^n$$
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec15">Basic dynamics of the population of lynx </h2>
<h2 id="___sec16">Basic dynamics of the population of lynx </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -876,7 +958,7 @@ $$ \Delta L = d\Delta t H^nL^n - c\Delta t L^n$$
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec16">Evolution equations </h2>
<h2 id="___sec17">Evolution equations </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -905,7 +987,7 @@ Note:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">Adapt the model to the Hudson Bay case </h2>
<h2 id="___sec18">Adapt the model to the Hudson Bay case </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -928,7 +1010,7 @@ Note:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">The program </h2>
<h2 id="___sec19">The program </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -988,7 +1070,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec19">The plot </h2>
<h2 id="___sec20">The plot </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_sim.png" align="bottom" width=700></p></center><br /><br />
@@ -999,7 +1081,7 @@ If we perform a least-square fitting, we can find optimal values for the paramet
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec20">Linear regression in Python </h2>
<h2 id="___sec21">Linear regression in Python </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1032,7 +1114,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">Linear Least squares in R </h2>
<h2 id="___sec22">Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1063,40 +1145,6 @@ predict(linearMod,<span style="color: #00688B; font-weight: bold">data.frame</sp
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec22">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=r (!bc r) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eee8d5"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">set.seed</span>(<span style="color: #B452CD">1485</span>)
len = <span style="color: #B452CD">24</span>
x = runif(len)
y = x^<span style="color: #B452CD">3</span>+rnorm(len, <span style="color: #B452CD">0</span>,<span style="color: #B452CD">0.06</span>)
ds = <span style="color: #00688B; font-weight: bold">data.frame</span>(x = x, y = y)
str(ds)
plot( y ~ x, main =<span style="color: #CD5555">&quot;Known cubic with noise&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>,<span style="color: #B452CD">1</span>,length =<span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty =<span style="color: #B452CD">2</span>, col =<span style="color: #CD5555">&quot;green&quot;</span>)
m = nls(y ~ <span style="color: #8B008B; font-weight: bold">I</span>(x^power), data = ds, start = <span style="color: #00688B; font-weight: bold">list</span>(power=<span style="color: #B452CD">1</span>), trace = <span style="color: #658b00">T</span>)
<span style="color: #8B008B; font-weight: bold">class</span>(m)
<span style="color: #8B008B; font-weight: bold">summary</span>(m)
power = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">1</span>], <span style="color: #B452CD">3</span>)
power.se = <span style="color: #8B008B; font-weight: bold">round</span>(<span style="color: #8B008B; font-weight: bold">summary</span>(m)$coefficients[<span style="color: #B452CD">2</span>], <span style="color: #B452CD">3</span>)
plot(y ~ x, main = <span style="color: #CD5555">&quot;Fitted power model&quot;</span>, sub = <span style="color: #CD5555">&quot;Blue: fit; green: known&quot;</span>)
s = <span style="color: #8B008B; font-weight: bold">seq</span>(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">1</span>, length = <span style="color: #B452CD">100</span>)
lines(s, s^<span style="color: #B452CD">3</span>, lty = <span style="color: #B452CD">2</span>, col = <span style="color: #CD5555">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #00688B; font-weight: bold">list</span>(x = s)), lty = <span style="color: #B452CD">1</span>, col = <span style="color: #CD5555">&quot;blue&quot;</span>)
text(<span style="color: #B452CD">0</span>, <span style="color: #B452CD">0.5</span>, <span style="color: #8B008B; font-weight: bold">paste</span>(<span style="color: #CD5555">&quot;y =x^ (&quot;</span>, power, <span style="color: #CD5555">&quot; +/- &quot;</span>, power.se, <span style="color: #CD5555">&quot;)&quot;</span>, sep = <span style="color: #CD5555">&quot;&quot;</span>), pos = <span style="color: #B452CD">4</span>)
</pre></div>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/How2ReadData/html/How2ReadData.html
+127 -79
View File
@@ -74,33 +74,33 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'___sec5'),
('Predator-Prey model from ecology', 2, None, '___sec6'),
('Case study from Hudson bay', 2, None, '___sec7'),
('Hudson bay data', 2, None, '___sec8'),
('Plotting the data', 2, None, '___sec9'),
('Non-Linear Least squares in R', 2, None, '___sec6'),
('Predator-Prey model from ecology', 2, None, '___sec7'),
('Case study from Hudson bay', 2, None, '___sec8'),
('Hudson bay data', 2, None, '___sec9'),
('Plotting the data', 2, None, '___sec10'),
('Hares and lynx in Hudson bay from 1900 to 1920',
2,
None,
'___sec10'),
'___sec11'),
('Why now create a computer model for the hare and lynx '
'populations?',
2,
None,
'___sec11'),
('The traditional (top-down) approach', 2, None, '___sec12'),
('Basic mathematics notation', 2, None, '___sec13'),
'___sec12'),
('The traditional (top-down) approach', 2, None, '___sec13'),
('Basic mathematics notation', 2, None, '___sec14'),
('Basic dynamics of the population of hares',
2,
None,
'___sec14'),
('Basic dynamics of the population of lynx', 2, None, '___sec15'),
('Evolution equations', 2, None, '___sec16'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec17'),
('The program', 2, None, '___sec18'),
('The plot', 2, None, '___sec19'),
('Linear regression in Python', 2, None, '___sec20'),
('Linear Least squares in R', 2, None, '___sec21'),
('Non-Linear Least squares in R', 2, None, '___sec22'),
'___sec15'),
('Basic dynamics of the population of lynx', 2, None, '___sec16'),
('Evolution equations', 2, None, '___sec17'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec18'),
('The program', 2, None, '___sec19'),
('The plot', 2, None, '___sec20'),
('Linear regression in Python', 2, None, '___sec21'),
('Linear Least squares in R', 2, None, '___sec22'),
('Example: ecoli lab experiment', 2, None, '___sec23'),
('The program', 2, None, '___sec24'),
('The output', 2, None, '___sec25'),
@@ -156,7 +156,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>May 28, 2018</h4></center> <!-- date -->
<center><h4>May 29, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -186,16 +186,20 @@ then use machine learning algorithms included in for example
<b>scikit-learn</b>.
<p>
These examples will serve us the purpose of getting started. Furthermore, they
allow us to catch more than two birds with a stone. They will allow us
to bring in some programming specific topics and tools as well as
showing the power of various Python (and R) packages for machine
learning and statistical data analysis. In the lectures on linear
algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which
are relevant for the various algorithms we will discuss. Here, we will
mainly focus on two specific Python packages for Machine Learning,
scikit-learn and tensorflow (see below for links etc).
Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming.
These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python (and
R) packages for machine learning and statistical data analysis. In the
lectures on linear algebra we cover in more detail various programming
features of languages like Python and C++ (and other), we will also
look into more specific linear functions which are relevant for the
various algorithms we will discuss. Here, we will mainly focus on two
specific Python packages for Machine Learning, scikit-learn and
tensorflow (see below for links etc). Moreover, the examples we
introduce will serve as inputs to many of our discussions later, as
well as allowing you to set up models and produce your own data and
get started with programming.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -580,7 +584,85 @@ years etc.
<p>
We will discuss in more
detail these and more function in the various lectures.
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> Ridge
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> make_pipeline
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">f</span>(x):
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot; function to approximate by polynomial interpolation&quot;&quot;&quot;</span>
<span style="color: #008000; font-weight: bold">return</span> x<span style="color: #666666">*</span>x<span style="color: #666666">*</span>x
<span style="color: #408080; font-style: italic"># generate points used to plot </span>
x_plot <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
<span style="color: #408080; font-style: italic"># generate points and keep a subset of them </span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
rng <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>RandomState(<span style="color: #666666">0</span>)
rng<span style="color: #666666">.</span>shuffle(x)
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sort(x[:<span style="color: #666666">20</span>])
y <span style="color: #666666">=</span> f(x)
<span style="color: #408080; font-style: italic"># create matrix versions of these arrays </span>
X <span style="color: #666666">=</span> x[:, np<span style="color: #666666">.</span>newaxis]
X_plot <span style="color: #666666">=</span> x_plot[:, np<span style="color: #666666">.</span>newaxis]
colors <span style="color: #666666">=</span> [<span style="color: #BA2121">&#39;teal&#39;</span>, <span style="color: #BA2121">&#39;yellowgreen&#39;</span>, <span style="color: #BA2121">&#39;gold&#39;</span>]
lw <span style="color: #666666">=</span> <span style="color: #666666">2</span>
plt<span style="color: #666666">.</span>plot(x_plot, f(x_plot), color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;cornflowerblue&#39;</span>, linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;ground truth&quot;</span>)
plt<span style="color: #666666">.</span>scatter(x, y, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;navy&#39;</span>, s<span style="color: #666666">=30</span>, marker<span style="color: #666666">=</span><span style="color: #BA2121">&#39;o&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;training points&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> count, degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>([<span style="color: #666666">3</span>, <span style="color: #666666">4</span>, <span style="color: #666666">5</span>]):
model <span style="color: #666666">=</span> make_pipeline(PolynomialFeatures(degree), Ridge())
model<span style="color: #666666">.</span>fit(X, y)
y_plot <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_plot)
plt<span style="color: #666666">.</span>plot(x_plot, y_plot, color<span style="color: #666666">=</span>colors[count], linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;degree </span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121">&quot;</span> <span style="color: #666666">%</span> degree)
plt<span style="color: #666666">.</span>legend(loc<span style="color: #666666">=</span><span style="color: #BA2121">&#39;lower left&#39;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=r (!bc r) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000">set.seed</span>(<span style="color: #666666">1485</span>)
len <span style="color: #666666">=</span> <span style="color: #666666">24</span>
x <span style="color: #666666">=</span> runif(len)
y <span style="color: #666666">=</span> x<span style="color: #666666">^3+</span>rnorm(len, <span style="color: #666666">0</span>,<span style="color: #666666">0.06</span>)
ds <span style="color: #666666">=</span> <span style="color: #B00040">data.frame</span>(x <span style="color: #666666">=</span> x, y <span style="color: #666666">=</span> y)
str(ds)
plot( y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span><span style="color: #BA2121">&quot;Known cubic with noise&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,length <span style="color: #666666">=100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=2</span>, col <span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>)
m <span style="color: #666666">=</span> nls(y <span style="color: #666666">~</span> <span style="color: #008000">I</span>(x<span style="color: #666666">^</span>power), data <span style="color: #666666">=</span> ds, start <span style="color: #666666">=</span> <span style="color: #B00040">list</span>(power<span style="color: #666666">=1</span>), trace <span style="color: #666666">=</span> <span style="color: #008000">T</span>)
<span style="color: #008000">class</span>(m)
<span style="color: #008000">summary</span>(m)
power <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">1</span>], <span style="color: #666666">3</span>)
power.se <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">2</span>], <span style="color: #666666">3</span>)
plot(y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Fitted power model&quot;</span>, sub <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Blue: fit; green: known&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>, <span style="color: #666666">1</span>, length <span style="color: #666666">=</span> <span style="color: #666666">100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=</span> <span style="color: #666666">2</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #B00040">list</span>(x <span style="color: #666666">=</span> s)), lty <span style="color: #666666">=</span> <span style="color: #666666">1</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;blue&quot;</span>)
text(<span style="color: #666666">0</span>, <span style="color: #666666">0.5</span>, <span style="color: #008000">paste</span>(<span style="color: #BA2121">&quot;y =x^ (&quot;</span>, power, <span style="color: #BA2121">&quot; +/- &quot;</span>, power.se, <span style="color: #BA2121">&quot;)&quot;</span>, sep <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;&quot;</span>), pos <span style="color: #666666">=</span> <span style="color: #666666">4</span>)
</pre></div>
</div>
<p>
Another useful Python package is
@@ -600,7 +682,7 @@ display(data_pandas)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Predator-Prey model from ecology </h2>
<h2 id="___sec7">Predator-Prey model from ecology </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -624,7 +706,7 @@ scientific method:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">Case study from Hudson bay </h2>
<h2 id="___sec8">Case study from Hudson bay </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -645,7 +727,7 @@ Here we start by
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Hudson bay data </h2>
<h2 id="___sec9">Hudson bay data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -694,7 +776,7 @@ One reason that this particular system has been so extensively studied is that t
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec9">Plotting the data </h2>
<h2 id="___sec10">Plotting the data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -730,7 +812,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec10">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<h2 id="___sec11">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_data.png" align="bottom" width=700></p></center><br /><br />
@@ -738,7 +820,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec11">Why now create a computer model for the hare and lynx populations? </h2>
<h2 id="___sec12">Why now create a computer model for the hare and lynx populations? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -771,7 +853,7 @@ climate and other complicating factors. How significant are these?
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec12">The traditional (top-down) approach </h2>
<h2 id="___sec13">The traditional (top-down) approach </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -804,7 +886,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec13">Basic mathematics notation </h2>
<h2 id="___sec14">Basic mathematics notation </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -824,7 +906,7 @@ ODEs</em> (which cannot be solved)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Basic dynamics of the population of hares </h2>
<h2 id="___sec15">Basic dynamics of the population of hares </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -852,7 +934,7 @@ $$ \Delta H = a\Delta t H^n - b \Delta t H^nL^n$$
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec15">Basic dynamics of the population of lynx </h2>
<h2 id="___sec16">Basic dynamics of the population of lynx </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -881,7 +963,7 @@ $$ \Delta L = d\Delta t H^nL^n - c\Delta t L^n$$
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec16">Evolution equations </h2>
<h2 id="___sec17">Evolution equations </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -910,7 +992,7 @@ Note:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">Adapt the model to the Hudson Bay case </h2>
<h2 id="___sec18">Adapt the model to the Hudson Bay case </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -933,7 +1015,7 @@ Note:
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">The program </h2>
<h2 id="___sec19">The program </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
@@ -993,7 +1075,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec19">The plot </h2>
<h2 id="___sec20">The plot </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_sim.png" align="bottom" width=700></p></center><br /><br />
@@ -1004,7 +1086,7 @@ If we perform a least-square fitting, we can find optimal values for the paramet
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec20">Linear regression in Python </h2>
<h2 id="___sec21">Linear regression in Python </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1037,7 +1119,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">Linear Least squares in R </h2>
<h2 id="___sec22">Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1068,40 +1150,6 @@ predict(linearMod,<span style="color: #B00040">data.frame</span>(Year<span style
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec22">Non-Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=r (!bc r) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000">set.seed</span>(<span style="color: #666666">1485</span>)
len <span style="color: #666666">=</span> <span style="color: #666666">24</span>
x <span style="color: #666666">=</span> runif(len)
y <span style="color: #666666">=</span> x<span style="color: #666666">^3+</span>rnorm(len, <span style="color: #666666">0</span>,<span style="color: #666666">0.06</span>)
ds <span style="color: #666666">=</span> <span style="color: #B00040">data.frame</span>(x <span style="color: #666666">=</span> x, y <span style="color: #666666">=</span> y)
str(ds)
plot( y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span><span style="color: #BA2121">&quot;Known cubic with noise&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,length <span style="color: #666666">=100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=2</span>, col <span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>)
m <span style="color: #666666">=</span> nls(y <span style="color: #666666">~</span> <span style="color: #008000">I</span>(x<span style="color: #666666">^</span>power), data <span style="color: #666666">=</span> ds, start <span style="color: #666666">=</span> <span style="color: #B00040">list</span>(power<span style="color: #666666">=1</span>), trace <span style="color: #666666">=</span> <span style="color: #008000">T</span>)
<span style="color: #008000">class</span>(m)
<span style="color: #008000">summary</span>(m)
power <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">1</span>], <span style="color: #666666">3</span>)
power.se <span style="color: #666666">=</span> <span style="color: #008000">round</span>(<span style="color: #008000">summary</span>(m)<span style="color: #666666">$</span>coefficients[<span style="color: #666666">2</span>], <span style="color: #666666">3</span>)
plot(y <span style="color: #666666">~</span> x, main <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Fitted power model&quot;</span>, sub <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Blue: fit; green: known&quot;</span>)
s <span style="color: #666666">=</span> <span style="color: #008000">seq</span>(<span style="color: #666666">0</span>, <span style="color: #666666">1</span>, length <span style="color: #666666">=</span> <span style="color: #666666">100</span>)
lines(s, s<span style="color: #666666">^3</span>, lty <span style="color: #666666">=</span> <span style="color: #666666">2</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;green&quot;</span>)
lines(s, predict(m, <span style="color: #B00040">list</span>(x <span style="color: #666666">=</span> s)), lty <span style="color: #666666">=</span> <span style="color: #666666">1</span>, col <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;blue&quot;</span>)
text(<span style="color: #666666">0</span>, <span style="color: #666666">0.5</span>, <span style="color: #008000">paste</span>(<span style="color: #BA2121">&quot;y =x^ (&quot;</span>, power, <span style="color: #BA2121">&quot; +/- &quot;</span>, power.se, <span style="color: #BA2121">&quot;)&quot;</span>, sep <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;&quot;</span>), pos <span style="color: #666666">=</span> <span style="color: #666666">4</span>)
</pre></div>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/How2ReadData/ipynb/How2ReadData.ipynb
+111 -54
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **May 28, 2018**\n",
"Date: **May 29, 2018**\n",
"\n",
"Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -19,8 +19,6 @@
"\n",
"## Introduction\n",
"\n",
"\n",
"\n",
"Our emphasis throughout this series of lectures \n",
"is on understanding the mathematical aspects of\n",
"different algorithms used in the fields of data analysis and machine learning. \n",
@@ -41,16 +39,20 @@
"then use machine learning algorithms included in for example\n",
"**scikit-learn**. \n",
"\n",
"These examples will serve us the purpose of getting started. Furthermore, they\n",
"allow us to catch more than two birds with a stone. They will allow us\n",
"to bring in some programming specific topics and tools as well as\n",
"showing the power of various Python (and R) packages for machine\n",
"learning and statistical data analysis. In the lectures on linear\n",
"algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which\n",
"are relevant for the various algorithms we will discuss. Here, we will\n",
"mainly focus on two specific Python packages for Machine Learning,\n",
"scikit-learn and tensorflow (see below for links etc).\n",
"Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming. \n",
"These examples will serve us the purpose of getting\n",
"started. Furthermore, they allow us to catch more than two birds with\n",
"a stone. They will allow us to bring in some programming specific\n",
"topics and tools as well as showing the power of various Python (and\n",
"R) packages for machine learning and statistical data analysis. In the\n",
"lectures on linear algebra we cover in more detail various programming\n",
"features of languages like Python and C++ (and other), we will also\n",
"look into more specific linear functions which are relevant for the\n",
"various algorithms we will discuss. Here, we will mainly focus on two\n",
"specific Python packages for Machine Learning, scikit-learn and\n",
"tensorflow (see below for links etc). Moreover, the examples we\n",
"introduce will serve as inputs to many of our discussions later, as\n",
"well as allowing you to set up models and produce your own data and\n",
"get started with programming.\n",
"\n",
"\n",
"\n",
@@ -531,8 +533,95 @@
"years etc. \n",
"\n",
"We will discuss in more\n",
"detail these and more function in the various lectures.\n",
"detail these and other functions in the various lectures. We conclude this part with another example. Instead of \n",
"a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from sklearn.linear_model import Ridge\n",
"from sklearn.preprocessing import PolynomialFeatures\n",
"from sklearn.pipeline import make_pipeline\n",
"\n",
"def f(x):\n",
" \"\"\" function to approximate by polynomial interpolation\"\"\"\n",
" return x*x*x\n",
"\n",
"# generate points used to plot \n",
"x_plot = np.linspace(0, 10, 100)\n",
"\n",
"# generate points and keep a subset of them \n",
"x = np.linspace(0, 10, 100)\n",
"rng = np.random.RandomState(0)\n",
"rng.shuffle(x)\n",
"x = np.sort(x[:20])\n",
"y = f(x)\n",
"# create matrix versions of these arrays \n",
"X = x[:, np.newaxis]\n",
"X_plot = x_plot[:, np.newaxis]\n",
"\n",
"colors = ['teal', 'yellowgreen', 'gold']\n",
"lw = 2\n",
"plt.plot(x_plot, f(x_plot), color='cornflowerblue', linewidth=lw,\n",
" label=\"ground truth\")\n",
"plt.scatter(x, y, color='navy', s=30, marker='o', label=\"training points\")\n",
"\n",
"for count, degree in enumerate([3, 4, 5]):\n",
" model = make_pipeline(PolynomialFeatures(degree), Ridge())\n",
" model.fit(X, y)\n",
" y_plot = model.predict(X_plot)\n",
" plt.plot(x_plot, y_plot, color=colors[count], linewidth=lw,\n",
" label=\"degree %d\" % degree)\n",
"\n",
"plt.legend(loc='lower left')\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Non-Linear Least squares in R"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" set.seed(1485)\n",
" len = 24\n",
" x = runif(len)\n",
" y = x^3+rnorm(len, 0,0.06)\n",
" ds = data.frame(x = x, y = y)\n",
" str(ds)\n",
" plot( y ~ x, main =\"Known cubic with noise\")\n",
" s = seq(0,1,length =100)\n",
" lines(s, s^3, lty =2, col =\"green\")\n",
" m = nls(y ~ I(x^power), data = ds, start = list(power=1), trace = T)\n",
" class(m)\n",
" summary(m)\n",
" power = round(summary(m)$coefficients[1], 3)\n",
" power.se = round(summary(m)$coefficients[2], 3)\n",
" plot(y ~ x, main = \"Fitted power model\", sub = \"Blue: fit; green: known\")\n",
" s = seq(0, 1, length = 100)\n",
" lines(s, s^3, lty = 2, col = \"green\")\n",
" lines(s, predict(m, list(x = s)), lty = 1, col = \"blue\")\n",
" text(0, 0.5, paste(\"y =x^ (\", power, \" +/- \", power.se, \")\", sep = \"\"), pos = 4)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another useful Python package is\n",
"[pandas](https://pandas.pydata.org/), which is an open source library\n",
"providing high-performance, easy-to-use data structures and data\n",
@@ -541,7 +630,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {
"collapsed": false
},
@@ -640,7 +729,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {
"collapsed": false
},
@@ -897,7 +986,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"metadata": {
"collapsed": false
},
@@ -971,7 +1060,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 8,
"metadata": {
"collapsed": false
},
@@ -1029,38 +1118,6 @@
" predict(linearMod,data.frame(Year=c(1910,1914,1920)),interval=\"confidence\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Non-Linear Least squares in R"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" set.seed(1485)\n",
" len = 24\n",
" x = runif(len)\n",
" y = x^3+rnorm(len, 0,0.06)\n",
" ds = data.frame(x = x, y = y)\n",
" str(ds)\n",
" plot( y ~ x, main =\"Known cubic with noise\")\n",
" s = seq(0,1,length =100)\n",
" lines(s, s^3, lty =2, col =\"green\")\n",
" m = nls(y ~ I(x^power), data = ds, start = list(power=1), trace = T)\n",
" class(m)\n",
" summary(m)\n",
" power = round(summary(m)$coefficients[1], 3)\n",
" power.se = round(summary(m)$coefficients[2], 3)\n",
" plot(y ~ x, main = \"Fitted power model\", sub = \"Blue: fit; green: known\")\n",
" s = seq(0, 1, length = 100)\n",
" lines(s, s^3, lty = 2, col = \"green\")\n",
" lines(s, predict(m, list(x = s)), lty = 1, col = \"blue\")\n",
" text(0, 0.5, paste(\"y =x^ (\", power, \" +/- \", power.se, \")\", sep = \"\"), pos = 4)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -1104,7 +1161,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"metadata": {
"collapsed": false
},
@@ -1215,7 +1272,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 10,
"metadata": {
"collapsed": false
},
@@ -1407,7 +1464,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 11,
"metadata": {
"collapsed": false
},
@@ -1606,7 +1663,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 12,
"metadata": {
"collapsed": false
},
doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/How2ReadData/pdf/How2ReadData-beamer-handouts2x3.pdf
BIN
View File
Binary file not shown.
doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf
BIN
View File
Binary file not shown.
doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf
BIN
View File
Binary file not shown.
doc/src/How2ReadData/How2ReadData.do.txt
+86 -38
View File
@@ -6,8 +6,6 @@ DATE: today
!split
===== Introduction =====
Our emphasis throughout this series of lectures
is on understanding the mathematical aspects of
different algorithms used in the fields of data analysis and machine learning.
@@ -28,16 +26,20 @@ models. These are examples where we can easily set up the data and
then use machine learning algorithms included in for example
_scikit-learn_.
These examples will serve us the purpose of getting started. Furthermore, they
allow us to catch more than two birds with a stone. They will allow us
to bring in some programming specific topics and tools as well as
showing the power of various Python (and R) packages for machine
learning and statistical data analysis. In the lectures on linear
algebra we cover in more detail various programming features of languages like Python and C++ (and other), we will also look into more specific linear functions which
are relevant for the various algorithms we will discuss. Here, we will
mainly focus on two specific Python packages for Machine Learning,
scikit-learn and tensorflow (see below for links etc).
Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming.
These examples will serve us the purpose of getting
started. Furthermore, they allow us to catch more than two birds with
a stone. They will allow us to bring in some programming specific
topics and tools as well as showing the power of various Python (and
R) packages for machine learning and statistical data analysis. In the
lectures on linear algebra we cover in more detail various programming
features of languages like Python and C++ (and other), we will also
look into more specific linear functions which are relevant for the
various algorithms we will discuss. Here, we will mainly focus on two
specific Python packages for Machine Learning, scikit-learn and
tensorflow (see below for links etc). Moreover, the examples we
introduce will serve as inputs to many of our discussions later, as
well as allowing you to set up models and produce your own data and
get started with programming.
@@ -391,7 +393,78 @@ as population counts, average sales of a commodity over a span of
years etc.
We will discuss in more
detail these and more function in the various lectures.
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
!bc pycod
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import Ridge
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
def f(x):
""" function to approximate by polynomial interpolation"""
return x*x*x
# generate points used to plot
x_plot = np.linspace(0, 10, 100)
# generate points and keep a subset of them
x = np.linspace(0, 10, 100)
rng = np.random.RandomState(0)
rng.shuffle(x)
x = np.sort(x[:20])
y = f(x)
# create matrix versions of these arrays
X = x[:, np.newaxis]
X_plot = x_plot[:, np.newaxis]
colors = ['teal', 'yellowgreen', 'gold']
lw = 2
plt.plot(x_plot, f(x_plot), color='cornflowerblue', linewidth=lw,
label="ground truth")
plt.scatter(x, y, color='navy', s=30, marker='o', label="training points")
for count, degree in enumerate([3, 4, 5]):
model = make_pipeline(PolynomialFeatures(degree), Ridge())
model.fit(X, y)
y_plot = model.predict(X_plot)
plt.plot(x_plot, y_plot, color=colors[count], linewidth=lw,
label="degree %d" % degree)
plt.legend(loc='lower left')
plt.show()
!ec
!split
===== Non-Linear Least squares in R =====
!bblock
!bc r
set.seed(1485)
len = 24
x = runif(len)
y = x^3+rnorm(len, 0,0.06)
ds = data.frame(x = x, y = y)
str(ds)
plot( y ~ x, main ="Known cubic with noise")
s = seq(0,1,length =100)
lines(s, s^3, lty =2, col ="green")
m = nls(y ~ I(x^power), data = ds, start = list(power=1), trace = T)
class(m)
summary(m)
power = round(summary(m)$coefficients[1], 3)
power.se = round(summary(m)$coefficients[2], 3)
plot(y ~ x, main = "Fitted power model", sub = "Blue: fit; green: known")
s = seq(0, 1, length = 100)
lines(s, s^3, lty = 2, col = "green")
lines(s, predict(m, list(x = s)), lty = 1, col = "blue")
text(0, 0.5, paste("y =x^ (", power, " +/- ", power.se, ")", sep = ""), pos = 4)
!ec
!eblock
Another useful Python package is
"pandas":"https://pandas.pydata.org/", which is an open source library
@@ -709,31 +782,6 @@ predict(linearMod,data.frame(Year=c(1910,1914,1920)),interval="confidence")
!ec
!eblock
!split
===== Non-Linear Least squares in R =====
!bblock
!bc r
set.seed(1485)
len = 24
x = runif(len)
y = x^3+rnorm(len, 0,0.06)
ds = data.frame(x = x, y = y)
str(ds)
plot( y ~ x, main ="Known cubic with noise")
s = seq(0,1,length =100)
lines(s, s^3, lty =2, col ="green")
m = nls(y ~ I(x^power), data = ds, start = list(power=1), trace = T)
class(m)
summary(m)
power = round(summary(m)$coefficients[1], 3)
power.se = round(summary(m)$coefficients[2], 3)
plot(y ~ x, main = "Fitted power model", sub = "Blue: fit; green: known")
s = seq(0, 1, length = 100)
lines(s, s^3, lty = 2, col = "green")
lines(s, predict(m, list(x = s)), lty = 1, col = "blue")
text(0, 0.5, paste("y =x^ (", power, " +/- ", power.se, ")", sep = ""), pos = 4)
!ec
!eblock
doc/web/course.do.txt
-2
View File
@@ -35,8 +35,6 @@ chapters = {
* LaTeX PDF:
* For printing:
* "Standard one-page format": "${pub_url}/${name}/pdf/${name}-minted.pdf"
* For screen viewing:
* "standard Beamer format": "${pub_url}/${name}/pdf/${name}-beamer.pdf"
* HTML:
* "Plain html": "${pub_url}/${name}/html/${name}.html"
* "reveal.js beige slide style": "${pub_url}/${name}/html/${name}-reveal.html"
doc/web/course.html
-66
View File
@@ -183,12 +183,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Introduction/pdf/Introduction-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Introduction/pdf/Introduction-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -219,12 +213,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -255,12 +243,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Linalg/pdf/Linalg-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Linalg/pdf/Linalg-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -291,12 +273,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Statistics/pdf/Statistics-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Statistics/pdf/Statistics-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -327,12 +303,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Splines/pdf/Splines-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Splines/pdf/Splines-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -363,12 +333,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Regression/pdf/Regression-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Regression/pdf/Regression-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -399,12 +363,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/pdf/NeuralNet-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/pdf/NeuralNet-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -435,12 +393,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Bayesian/pdf/Bayesian-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/Bayesian/pdf/Bayesian-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -471,12 +423,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/DecisionTrees/pdf/DecisionTrees-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -507,12 +453,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/svm/pdf/svm-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/svm/pdf/svm-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>
@@ -543,12 +483,6 @@ formulas in HTML or ipython notebook files.
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/BM/pdf/BM-minted.pdf" target="_self">Standard one-page format</a></li>
</ul>
<li> For screen viewing:</li>
<ul>
<li> <a href="https://compphysics.github.io/MachineLearning/doc/pub/BM/pdf/BM-beamer.pdf" target="_self">standard Beamer format</a></li>
</ul>
</ul>
<li> HTML:</li>