lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

More updates

Browse Source
This commit is contained in:
mhjensen
2018-05-29 11:04:42 -04:00
parent 0d988ad4fc
commit 3a2abcd62b
9 changed files with 1164 additions and 2519 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/How2ReadData/html/How2ReadData.html
+187 -465
View File
@@ -75,46 +75,14 @@ div { text-align: justify; text-justify: inter-word; }
None,
'___sec5'),
('Non-Linear Least squares in R', 2, None, '___sec6'),
('Predator-Prey model from ecology', 2, None, '___sec7'),
('Case study from Hudson bay', 2, None, '___sec8'),
('Hudson bay data', 2, None, '___sec9'),
('Plotting the data', 2, None, '___sec10'),
('Hares and lynx in Hudson bay from 1900 to 1920',
2,
('Examples', 2, None, '___sec7'),
('Ecoli lab experiment', 3, None, '___sec8'),
('Predator-Prey model from ecology', 3, None, '___sec9'),
('Simulating financial transactions', 3, None, '___sec10'),
('Particle in one dimension and velocity distribution',
3,
None,
'___sec11'),
('Why now create a computer model for the hare and lynx '
'populations?',
2,
None,
'___sec12'),
('The traditional (top-down) approach', 2, None, '___sec13'),
('Basic mathematics notation', 2, None, '___sec14'),
('Basic dynamics of the population of hares',
2,
None,
'___sec15'),
('Basic dynamics of the population of lynx', 2, None, '___sec16'),
('Evolution equations', 2, None, '___sec17'),
('Adapt the model to the Hudson Bay case', 2, None, '___sec18'),
('The program', 2, None, '___sec19'),
('The plot', 2, None, '___sec20'),
('Linear regression in Python', 2, None, '___sec21'),
('Linear Least squares in R', 2, None, '___sec22'),
('Example: ecoli lab experiment', 2, None, '___sec23'),
('The program', 2, None, '___sec24'),
('The output', 2, None, '___sec25'),
('Parameter estimation', 2, None, '___sec26'),
('A program relevant for the biological problem',
2,
None,
'___sec27'),
('Simulating financial transactions', 2, None, '___sec28'),
('Simulation of Transactions', 3, None, '___sec29'),
('Particle in one dimension an velocity distribution',
2,
None,
'___sec30')]}
'___sec11')]}
end of tocinfo -->
<body>
@@ -590,47 +558,39 @@ a linear \( x \)-dependence we study now a cubic polynomial and use the polynomi
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">random</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> Ridge
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> make_pipeline
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">f</span>(x):
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot; function to approximate by polynomial interpolation&quot;&quot;&quot;</span>
<span style="color: #008000; font-weight: bold">return</span> x<span style="color: #666666">*</span>x<span style="color: #666666">*</span>x
<span style="color: #408080; font-style: italic"># generate points used to plot </span>
x_plot <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
<span style="color: #408080; font-style: italic"># generate points and keep a subset of them </span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">100</span>)
rng <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>RandomState(<span style="color: #666666">0</span>)
rng<span style="color: #666666">.</span>shuffle(x)
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sort(x[:<span style="color: #666666">20</span>])
y <span style="color: #666666">=</span> f(x)
<span style="color: #408080; font-style: italic"># create matrix versions of these arrays </span>
X <span style="color: #666666">=</span> x[:, np<span style="color: #666666">.</span>newaxis]
X_plot <span style="color: #666666">=</span> x_plot[:, np<span style="color: #666666">.</span>newaxis]
colors <span style="color: #666666">=</span> [<span style="color: #BA2121">&#39;teal&#39;</span>, <span style="color: #BA2121">&#39;yellowgreen&#39;</span>, <span style="color: #BA2121">&#39;gold&#39;</span>]
lw <span style="color: #666666">=</span> <span style="color: #666666">2</span>
plt<span style="color: #666666">.</span>plot(x_plot, f(x_plot), color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;cornflowerblue&#39;</span>, linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;ground truth&quot;</span>)
plt<span style="color: #666666">.</span>scatter(x, y, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;navy&#39;</span>, s<span style="color: #666666">=30</span>, marker<span style="color: #666666">=</span><span style="color: #BA2121">&#39;o&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;training points&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> count, degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>([<span style="color: #666666">3</span>, <span style="color: #666666">4</span>, <span style="color: #666666">5</span>]):
model <span style="color: #666666">=</span> make_pipeline(PolynomialFeatures(degree), Ridge())
model<span style="color: #666666">.</span>fit(X, y)
y_plot <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_plot)
plt<span style="color: #666666">.</span>plot(x_plot, y_plot, color<span style="color: #666666">=</span>colors[count], linewidth<span style="color: #666666">=</span>lw,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;degree </span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121">&quot;</span> <span style="color: #666666">%</span> degree)
plt<span style="color: #666666">.</span>legend(loc<span style="color: #666666">=</span><span style="color: #BA2121">&#39;lower left&#39;</span>)
x<span style="color: #666666">=</span>np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0.02</span>,<span style="color: #666666">0.98</span>,<span style="color: #666666">200</span>)
noise <span style="color: #666666">=</span> np<span style="color: #666666">.</span>asarray(random<span style="color: #666666">.</span>sample((<span style="color: #008000">range</span>(<span style="color: #666666">200</span>)),<span style="color: #666666">200</span>))
y<span style="color: #666666">=</span>x<span style="color: #666666">**3*</span>noise
yn<span style="color: #666666">=</span>x<span style="color: #666666">**3*100</span>
poly3 <span style="color: #666666">=</span> PolynomialFeatures(degree<span style="color: #666666">=3</span>)
X <span style="color: #666666">=</span> poly3<span style="color: #666666">.</span>fit_transform(x[:,np<span style="color: #666666">.</span>newaxis])
clf3 <span style="color: #666666">=</span> LinearRegression()
clf3<span style="color: #666666">.</span>fit(X,y)
Xplot<span style="color: #666666">=</span>poly3<span style="color: #666666">.</span>fit_transform(x[:,np<span style="color: #666666">.</span>newaxis])
poly3_plot<span style="color: #666666">=</span>plt<span style="color: #666666">.</span>plot(x, clf3<span style="color: #666666">.</span>predict(Xplot), label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Cubic Fit&#39;</span>)
plt<span style="color: #666666">.</span>plot(x,yn, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;red&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;True Cubic&quot;</span>)
plt<span style="color: #666666">.</span>scatter(x, y, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Data&#39;</span>, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;orange&#39;</span>, s<span style="color: #666666">=15</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">error</span>(a):
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> y:
err<span style="color: #666666">=</span>(y<span style="color: #666666">-</span>yn)<span style="color: #666666">/</span>yn
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #008000">abs</span>(np<span style="color: #666666">.</span>sum(err))<span style="color: #666666">/</span><span style="color: #008000">len</span>(err)
<span style="color: #008000; font-weight: bold">print</span> (error(y))
</pre></div>
<p>
Similarly, using <b>R</b>, we can perform similar studies. The following <b>R</b> code illustrates this.
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Non-Linear Least squares in R </h2>
@@ -664,6 +624,9 @@ text(<span style="color: #666666">0</span>, <span style="color: #666666">0.5</sp
</div>
<p>
In our lectures on regression analysis (and other ones as well), we will discuss in more details various <b>R</b> functionalities.
<p>
Another useful Python package is
<a href="https://pandas.pydata.org/" target="_blank">pandas</a>, which is an open source library
@@ -682,11 +645,147 @@ display(data_pandas)
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">Predator-Prey model from ecology </h2>
<h2 id="___sec7">Examples </h2>
<p>
We present here several examples, with pertinent Python codes that we
will us to illustrate various machine learning methods and ways to
analyze, from simple to complex, various data sets. Many of these
examples allow us to generate the data we want to analyze, following
much of the same philosophy we discussed above when
fitting various polynomials.
<p>
We start with a simple exponential growth model that is meant to mimick an ecoli lab experiment.
We can easily model this system and then produce the data used to train various machine learning algorithms.
Another model from the life sciences is the so-called predator-prey model from ecology. Thereafter we present
a simple model for financial transactions before moving to a random walk model and ending with
the simulation of velocities of a non-interacting atom or molecule confined to move in a one-dimensional region.
<h3 id="___sec8">Ecoli lab experiment </h3>
<p>
A typical pattern seen in population models is that the population grows faster and faster. <a href="http://www.zo.utexas.edu/courses/Thoc/PopGrowth.html" target="_blank">Why? Is there an underlying (general) mechanism</a>?
Here we will construct a model for cell growth based on a simple difference equation for the growth. We make the following assumptions
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<ol>
<li> Cells divide after \( T \) seconds on average (one generation)</li>
<li> \( 2N \) celles divide into twice as many new cells \( \Delta N \) in a time
interval \( \Delta t \) as \( N \) cells would: \( \Delta N \propto N \)</li>
<li> \( N \) cells result in twice as many new individuals \( \Delta N \) in
time \( 2\Delta t \) as in time \( \Delta t \): \( \Delta N \propto\Delta t \)</li>
<li> Same proportionality wrt death</li>
<li> Proposed model: \( \Delta N = b\Delta t N - d\Delta tN \) for some unknown
constants \( b \) (births) and \( d \) (deaths)</li>
<li> Describe evolution in discrete time: \( t_n=n\Delta t \)</li>
<li> Program-friendly notation: \( N \) at \( t_n \) is \( N^n \)</li>
<li> Math model: \( N^{n+1} = N^n + r\Delta t\, N \) (with \( \ r=b-d \))</li>
<li> Program model: <code>N[n+1] = N[n] + r*dt*N[n]</code></li>
</ol>
</div>
<p>
The difference equation can be programmed in a simple was, and in order to get started we
set \( r=1.5 \), \( N^0=1 \), \( \Delta t=0.5 \). The program reads
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
t <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">21</span>) <span style="color: #408080; font-style: italic"># 20 intervals in [0, 10]</span>
dt <span style="color: #666666">=</span> t[<span style="color: #666666">1</span>] <span style="color: #666666">-</span> t[<span style="color: #666666">0</span>]
N <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(t<span style="color: #666666">.</span>size)
N[<span style="color: #666666">0</span>] <span style="color: #666666">=</span> <span style="color: #666666">1</span>
r <span style="color: #666666">=</span> <span style="color: #666666">0.5</span>
<span style="color: #008000; font-weight: bold">for</span> n <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">0</span>, N<span style="color: #666666">.</span>size<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>):
N[n<span style="color: #666666">+1</span>] <span style="color: #666666">=</span> N[n] <span style="color: #666666">+</span> r<span style="color: #666666">*</span>dt<span style="color: #666666">*</span>N[n]
<span style="color: #008000; font-weight: bold">print</span> <span style="color: #BA2121">&#39;N[</span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121">]=</span><span style="color: #BB6688; font-weight: bold">%.1f</span><span style="color: #BA2121">&#39;</span> <span style="color: #666666">%</span> (n<span style="color: #666666">+1</span>, N[n<span style="color: #666666">+1</span>])
</pre></div>
<p>
and it generates the following output
<p>
<!-- code=text typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>N[1]=1.2
N[2]=1.6
N[3]=2.0
N[4]=2.4
N[5]=3.1
N[6]=3.8
N[7]=4.8
N[8]=6.0
N[9]=7.5
N[10]=9.3
N[11]=11.6
N[12]=14.6
N[13]=18.2
N[14]=22.7
N[15]=28.4
N[16]=35.5
N[17]=44.4
N[18]=55.5
N[19]=69.4
N[20]=86.7
</pre></div>
<p>
This forms our data which later will define our training set.
In this case we defined the value of the parameter \( r \). We could alternatively assume that we just received the
above data file and where asked to use find \( r \). How can we estimate \( r \) from data?
<p>
We can use the difference equation with the experimental data
$$ N^{n+1} = N^n + r\Delta t N^n$$
Suppose now that \( N^{n+1} \) and \( N^n \) are known from data. Then we could solve with respect to \( r \) as follows
$$ r = \frac{N^{n+1}-N^n}{N^n\Delta t} $$
Suppose we set \( t_1=600 \), \( t_2=1200 \),
\( N^1=140 \) and \( N^2=250 \). We obtain then \( r=0.0013 \). The exact value is \( r = 0.000694 \)
The following code plot
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #408080; font-style: italic"># Estimate r</span>
data <span style="color: #666666">=</span> np<span style="color: #666666">.</span>loadtxt(<span style="color: #BA2121">&#39;ecoli.csv&#39;</span>, delimiter<span style="color: #666666">=</span><span style="color: #BA2121">&#39;,&#39;</span>)
t_e <span style="color: #666666">=</span> data[:,<span style="color: #666666">0</span>]
N_e <span style="color: #666666">=</span> data[:,<span style="color: #666666">1</span>]
i <span style="color: #666666">=</span> <span style="color: #666666">2</span> <span style="color: #408080; font-style: italic"># Data point (i,i+1) used to estimate r</span>
r <span style="color: #666666">=</span> (N_e[i<span style="color: #666666">+1</span>] <span style="color: #666666">-</span> N_e[i])<span style="color: #666666">/</span>(N_e[i]<span style="color: #666666">*</span>(t_e[i<span style="color: #666666">+1</span>] <span style="color: #666666">-</span> t_e[i]))
<span style="color: #008000; font-weight: bold">print</span> <span style="color: #BA2121">&#39;Estimated r=</span><span style="color: #BB6688; font-weight: bold">%.5f</span><span style="color: #BA2121">&#39;</span> <span style="color: #666666">%</span> r
<span style="color: #408080; font-style: italic"># Can experiment with r values and see if the model can</span>
<span style="color: #408080; font-style: italic"># match the data better</span>
T <span style="color: #666666">=</span> <span style="color: #666666">1200</span> <span style="color: #408080; font-style: italic"># cell can divide after T sec</span>
t_max <span style="color: #666666">=</span> <span style="color: #666666">5*</span>T <span style="color: #408080; font-style: italic"># 5 generations in experiment</span>
t <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, t_max, <span style="color: #666666">1000</span>)
dt <span style="color: #666666">=</span> t[<span style="color: #666666">1</span>] <span style="color: #666666">-</span> t[<span style="color: #666666">0</span>]
N <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(t<span style="color: #666666">.</span>size)
N[<span style="color: #666666">0</span>] <span style="color: #666666">=</span> <span style="color: #666666">100</span>
<span style="color: #008000; font-weight: bold">for</span> n <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">0</span>, <span style="color: #008000">len</span>(t)<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>):
N[n<span style="color: #666666">+1</span>] <span style="color: #666666">=</span> N[n] <span style="color: #666666">+</span> r<span style="color: #666666">*</span>dt<span style="color: #666666">*</span>N[n]
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
plt<span style="color: #666666">.</span>plot(t, N, <span style="color: #BA2121">&#39;r-&#39;</span>, t_e, N_e, <span style="color: #BA2121">&#39;bo&#39;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&#39;time [s]&#39;</span>); plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&#39;N&#39;</span>)
plt<span style="color: #666666">.</span>legend([<span style="color: #BA2121">&#39;model&#39;</span>, <span style="color: #BA2121">&#39;experiment&#39;</span>], loc<span style="color: #666666">=</span><span style="color: #BA2121">&#39;upper left&#39;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
We can then change the parameter \( r \) in the program and play around to make a better fit. By now we know that this
'search bythe eye' approach is not the most optimal one.
<h3 id="___sec9">Predator-Prey model from ecology </h3>
<p>
The population dynamics of a simple predator-prey system is a
classical example shown in many biology textbooks when ecological
@@ -700,18 +799,7 @@ scientific method:
<li> trying to extract general behaviors and extract eventual laws or patterns</li>
<li> develop mathematical relations for the uncovered regularities/laws and test these by per forming new experiments</li>
</ul>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Case study from Hudson bay </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
Lots of data about populations of hares and lynx collected from furs in Hudson Bay, Canada, are available. It is known that the populations oscillate. Why?
Here we start by
@@ -721,20 +809,7 @@ Here we start by
<li> (fitting parameters in the model to the data)</li>
<li> using the model predict the evolution other predator-pray systems</li>
</ol>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec9">Hudson bay data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
Most mammalian predators rely on a variety of prey, which complicates mathematical modeling; however, a few predators have become highly specialized and seek almost exclusively a single prey species. An example of this simplified predator-prey interaction is seen in Canadian northern forests, where the populations of the lynx and the snowshoe hare are intertwined in a life and death struggle.
<p>
@@ -769,19 +844,6 @@ One reason that this particular system has been so extensively studied is that t
<tr><td align="left"> 1920 </td> <td align="right"> 24.7 </td> <td align="right"> 8.6 </td> </tr>
</tbody>
</table>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec10">Plotting the data </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=python (!bc pypro) typeset with pygments style "default" -->
@@ -805,24 +867,9 @@ plt<span style="color: #666666">.</span>savefig(<span style="color: #BA2121">&#3
plt<span style="color: #666666">.</span>savefig(<span style="color: #BA2121">&#39;Hudson_Bay_data.png&#39;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec11">Hares and lynx in Hudson bay from 1900 to 1920 </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_data.png" align="bottom" width=700></p></center><br /><br />
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec12">Why now create a computer model for the hare and lynx populations? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
We see from the plot that there are indeed fluctuations.
We would like to create a mathematical model that explains these
@@ -844,21 +891,9 @@ climate and other complicating factors. How significant are these?
<li> What causes cycles to slow or speed up?</li>
<li> What affects the amplitude of the oscillation or do you expect to see the oscillations damp to a stable equilibrium?</li>
<li> With a model we can better <em>understand the data</em></li>
<li> More important: we can understand the ecology dynamics of
predator-pray populations</li>
<li> More important: Can we understand the ecology dynamics of predator-pray populations?</li>
</ul>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec13">The traditional (top-down) approach </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
The classical way (in all books) is to present the Lotka-Volterra equations:
$$
@@ -877,41 +912,6 @@ Here,
<li> \( a \), \( b \), \( d \), \( c \) are parameters</li>
</ul>
Most books quickly establish the model and then use considerable space on
discussing the qualitative properties of this <em>nonlinear system of
ODEs</em> (which cannot be solved)
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Basic mathematics notation </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<ul>
<li> Time points: \( t_0,t_1,\ldots,t_m \)</li>
<li> Uniform distribution of time points: \( t_n=n\Delta t \)</li>
<li> \( H^n \): population of hares at time \( t_n \)</li>
<li> \( L^n \): population of lynx at time \( t_n \)</li>
<li> We want to model the changes in populations, \( \Delta H=H^{n+1}-H^n \)
and \( \Delta L=L^{n+1}-L^n \) during a general time interval \( [t_{n+1},t_n] \)
of length \( \Delta t=t_{n+1}-t_n \)</li>
</ul>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec15">Basic dynamics of the population of hares </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
The population of hares evolves due to births and deaths exactly as a bacteria population:
$$
@@ -928,17 +928,7 @@ So in fraction \( b\Delta t HL \), the lynx eat hares. This
loss of hares must be accounted for. Subtracted in the equation for hares:
$$ \Delta H = a\Delta t H^n - b \Delta t H^nL^n$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec16">Basic dynamics of the population of lynx </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
We assume that the primary growth for the lynx population depends on sufficient food for raising lynx kittens, which implies an adequate source of nutrients from predation on hares. Thus, the growth of the lynx population does not only depend of how many lynx there are, but on how many hares they can eat.
In a time interval \( \Delta t HL \) hares and lynx can meet, and in a
@@ -948,26 +938,9 @@ contribute to the growth of lynx, again just a fraction of
\( d\Delta t HL \). In addition, lynx die just as in the population
dynamics with one isolated animal population, leading to a loss
\( -c\Delta t L \).
</div>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
The accounting of lynx then looks like
$$ \Delta L = d\Delta t H^nL^n - c\Delta t L^n$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">Evolution equations </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
By writing up the definition of \( \Delta H \) and \( \Delta L \), and putting
all assumed known terms \( H^n \) and \( L^n \) on the right-hand side, we have
@@ -982,42 +955,16 @@ Note:
<ul>
<li> These equations are ready to be implemented!</li>
<li> But to start, we need \( H^0 \) and \( L^0 \) <br />
(which we can get from the data)</li>
<li> But to start, we need \( H^0 \) and \( L^0 \) (which we can get from the data)</li>
<li> We also need values for \( a \), \( b \), \( d \), \( c \)</li>
</ul>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">Adapt the model to the Hudson Bay case </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<ul>
<li> As always, models tend to be general - as here, applicable
to &quot;all&quot; predator-pray systems</li>
<li> The critical issue is whether the <em>interaction</em> between hares and lynx
is sufficiently well modeled by \( \hbox{const}HL \)</li>
<li> The parameters \( a \), \( b \), \( d \), and \( c \) must be
estimated from data</li>
<li> Measure time in years</li>
<li> \( t_0=1900 \), \( t_m=1920 \)</li>
</ul>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec19">The program </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -1072,24 +1019,19 @@ plt<span style="color: #666666">.</span>show()
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec20">The plot </h2>
<p>
<br /><br /><center><p><img src="fig/Hudson_Bay_sim.png" align="bottom" width=700></p></center><br /><br />
<p>
If we perform a least-square fitting, we can find optimal values for the parameters \( a \), \( b \), \( d \), \( c \). The optimal parameters are \( a=0.4807 \), \( b=0.02482 \), \( d=0.9272 \) and \( c=0.02756 \). These parameters result in a slightly modified initial conditions, namely \( H(0) = 34.91 \) and \( L(0)=3.857 \). With these parameters we are now ready to solve the equations and plot these data together with the experimental values.
We will later perform a least-square fitting. Then we can find optimal
values for the parameters \( a \), \( b \), \( d \), \( c \). In our calculations here
we set \( a=0.4807 \), \( b=0.02482 \), \( d=0.9272 \) and \( c=0.02756 \). These
parameters result in a slightly modified initial conditions, namely
\( H(0) = 34.91 \) and \( L(0)=3.857 \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">Linear regression in Python </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
The following Python demonstrates how we can use linear regression to fit for example the population of lynx.
Similarly, we have also used a decision tree algorithm to fit the lynx population data. As expected, the linear regression is not exactly impressive
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1112,17 +1054,8 @@ plt<span style="color: #666666">.</span>plot(line, regline<span style="color: #6
plt<span style="color: #666666">.</span>plot(x, y, label<span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Linear Regression&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec22">Linear Least squares in R </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
The similar code for linear regression in <b>R</b> reads
<p>
<!-- code=r (!bc r) typeset with pygments style "default" -->
@@ -1147,188 +1080,7 @@ confint(linearMod)
predict(linearMod,<span style="color: #B00040">data.frame</span>(Year<span style="color: #666666">=</span><span style="color: #B00040">c</span>(<span style="color: #666666">1910</span>,<span style="color: #666666">1914</span>,<span style="color: #666666">1920</span>)),interval<span style="color: #666666">=</span><span style="color: #BA2121">&quot;confidence&quot;</span>)
</pre></div>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec23">Example: ecoli lab experiment </h2>
<p>
<div class="alert alert-block alert-notice alert-text-normal">
<b>Typical pattern:</b>
<p>
The population grows faster and faster. <a href="http://www.zo.utexas.edu/courses/Thoc/PopGrowth.html" target="_blank">Why? Is there an underlying (general) mechanism</a>?
</div>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<ol>
<li> Cells divide after \( T \) seconds on average (one generation)</li>
<li> \( 2N \) celles divide into twice as many new cells \( \Delta N \) in a time
interval \( \Delta t \) as \( N \) cells would: \( \Delta N \propto N \)</li>
<li> \( N \) cells result in twice as many new individuals \( \Delta N \) in
time \( 2\Delta t \) as in time \( \Delta t \): \( \Delta N \propto\Delta t \)</li>
<li> Same proportionality wrt death (repeat reasoning)</li>
<li> Proposed model: \( \Delta N = b\Delta t N - d\Delta tN \) for some unknown
constants \( b \) (births) and \( d \) (deaths)</li>
<li> Describe evolution in discrete time: \( t_n=n\Delta t \)</li>
<li> Program-friendly notation: \( N \) at \( t_n \) is \( N^n \)</li>
<li> Math model: \( N^{n+1} = N^n + r\Delta t\, N \) (with \( \ r=b-d \))</li>
<li> Program model: <code>N[n+1] = N[n] + r*dt*N[n]</code></li>
</ol>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec24">The program </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
Let us solve the difference equation in as simple way as possible,
just to train some programming: \( r=1.5 \), \( N^0=1 \), \( \Delta t=0.5 \)
<p>
<!-- code=python (!bc pypro) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
t <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">10</span>, <span style="color: #666666">21</span>) <span style="color: #408080; font-style: italic"># 20 intervals in [0, 10]</span>
dt <span style="color: #666666">=</span> t[<span style="color: #666666">1</span>] <span style="color: #666666">-</span> t[<span style="color: #666666">0</span>]
N <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(t<span style="color: #666666">.</span>size)
N[<span style="color: #666666">0</span>] <span style="color: #666666">=</span> <span style="color: #666666">1</span>
r <span style="color: #666666">=</span> <span style="color: #666666">0.5</span>
<span style="color: #008000; font-weight: bold">for</span> n <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">0</span>, N<span style="color: #666666">.</span>size<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>):
N[n<span style="color: #666666">+1</span>] <span style="color: #666666">=</span> N[n] <span style="color: #666666">+</span> r<span style="color: #666666">*</span>dt<span style="color: #666666">*</span>N[n]
<span style="color: #008000; font-weight: bold">print</span> <span style="color: #BA2121">&#39;N[</span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121">]=</span><span style="color: #BB6688; font-weight: bold">%.1f</span><span style="color: #BA2121">&#39;</span> <span style="color: #666666">%</span> (n<span style="color: #666666">+1</span>, N[n<span style="color: #666666">+1</span>])
</pre></div>
</div>
<p>
% if FORMAT != 'ipynb':
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec25">The output </h2>
<p>
<!-- code=text typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>N[1]=1.2
N[2]=1.6
N[3]=2.0
N[4]=2.4
N[5]=3.1
N[6]=3.8
N[7]=4.8
N[8]=6.0
N[9]=7.5
N[10]=9.3
N[11]=11.6
N[12]=14.6
N[13]=18.2
N[14]=22.7
N[15]=28.4
N[16]=35.5
N[17]=44.4
N[18]=55.5
N[19]=69.4
N[20]=86.7
</pre></div>
<p>
% endif
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec26">Parameter estimation </h2>
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<ul>
<li> We do not know \( r \)</li>
<li> How can we estimate \( r \) from data?</li>
</ul>
We can use the difference equation with the experimental data
$$ N^{n+1} = N^n + r\Delta t N^n$$
Say \( N^{n+1} \) and \( N^n \) are known from data, solve wrt \( r \):
$$ r = \frac{N^{n+1}-N^n}{N^n\Delta t} $$
<p>
Use experimental data in the fraction, say \( t_1=600 \), \( t_2=1200 \),
\( N^1=140 \), \( N^2=250 \): \( r=0.0013 \).
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec27">A program relevant for the biological problem </h2>
<p>
<!-- exact r = 0.000694 -->
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
<!-- code=python (!bc pypro) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #408080; font-style: italic"># Estimate r</span>
data <span style="color: #666666">=</span> np<span style="color: #666666">.</span>loadtxt(<span style="color: #BA2121">&#39;ecoli.csv&#39;</span>, delimiter<span style="color: #666666">=</span><span style="color: #BA2121">&#39;,&#39;</span>)
t_e <span style="color: #666666">=</span> data[:,<span style="color: #666666">0</span>]
N_e <span style="color: #666666">=</span> data[:,<span style="color: #666666">1</span>]
i <span style="color: #666666">=</span> <span style="color: #666666">2</span> <span style="color: #408080; font-style: italic"># Data point (i,i+1) used to estimate r</span>
r <span style="color: #666666">=</span> (N_e[i<span style="color: #666666">+1</span>] <span style="color: #666666">-</span> N_e[i])<span style="color: #666666">/</span>(N_e[i]<span style="color: #666666">*</span>(t_e[i<span style="color: #666666">+1</span>] <span style="color: #666666">-</span> t_e[i]))
<span style="color: #008000; font-weight: bold">print</span> <span style="color: #BA2121">&#39;Estimated r=</span><span style="color: #BB6688; font-weight: bold">%.5f</span><span style="color: #BA2121">&#39;</span> <span style="color: #666666">%</span> r
<span style="color: #408080; font-style: italic"># Can experiment with r values and see if the model can</span>
<span style="color: #408080; font-style: italic"># match the data better</span>
T <span style="color: #666666">=</span> <span style="color: #666666">1200</span> <span style="color: #408080; font-style: italic"># cell can divide after T sec</span>
t_max <span style="color: #666666">=</span> <span style="color: #666666">5*</span>T <span style="color: #408080; font-style: italic"># 5 generations in experiment</span>
t <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, t_max, <span style="color: #666666">1000</span>)
dt <span style="color: #666666">=</span> t[<span style="color: #666666">1</span>] <span style="color: #666666">-</span> t[<span style="color: #666666">0</span>]
N <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(t<span style="color: #666666">.</span>size)
N[<span style="color: #666666">0</span>] <span style="color: #666666">=</span> <span style="color: #666666">100</span>
<span style="color: #008000; font-weight: bold">for</span> n <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">0</span>, <span style="color: #008000">len</span>(t)<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>):
N[n<span style="color: #666666">+1</span>] <span style="color: #666666">=</span> N[n] <span style="color: #666666">+</span> r<span style="color: #666666">*</span>dt<span style="color: #666666">*</span>N[n]
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
plt<span style="color: #666666">.</span>plot(t, N, <span style="color: #BA2121">&#39;r-&#39;</span>, t_e, N_e, <span style="color: #BA2121">&#39;bo&#39;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&#39;time [s]&#39;</span>); plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&#39;N&#39;</span>)
plt<span style="color: #666666">.</span>legend([<span style="color: #BA2121">&#39;model&#39;</span>, <span style="color: #BA2121">&#39;experiment&#39;</span>], loc<span style="color: #666666">=</span><span style="color: #BA2121">&#39;upper left&#39;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Change <code>r</code> in the program and play around to make a better fit!
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec28">Simulating financial transactions </h2>
<h3 id="___sec10">Simulating financial transactions </h3>
<p>
The aim here is to simulate financial transactions among financial agents
@@ -1402,8 +1154,7 @@ exponentially decreases with \( m' \).
We assume that we have \( N=500 \) agents. In each simulation, we need a sufficiently large number of transactions, say \( 10^7 \). Our aim is find the final equilibrium distribution \( w_m \). In order to do that we would need
several runs of the above simulations, at least \( 10^3-10^4 \) runs (experiments).
<h3 id="___sec29">Simulation of Transactions </h3>
<p>
Our task is to first set up an algorithm which simulates the above transactions with an initial
amount \( m_0 \).
The challenge here is to figure out a Monte Carlo simulation based on the
@@ -1502,40 +1253,11 @@ $$
showing how money is conserved during a transaction.
Select values of \( \lambda =0.25,0.5 \) and \( \lambda=0.9 \) and try to extract the corresponding
equilibrium distributions and compare these with the Gibbs distribution. Comment your results.
Extract a parametrization of the above curves, see for example <a href="http://www.sciencedirect.com/science/article/pii/S0378437104004327" target="_blank">Patriarca and collaborators</a> and see if you can parametrize the high-end tails of the distributions in terms of power laws. Comment your results.
equilibrium distributions and compare these with the Gibbs distribution. We will use this model to
extract a parametrization of the above curves, see for example <a href="http://www.sciencedirect.com/science/article/pii/S0378437104004327" target="_blank">Patriarca and collaborators</a>.
<p>
In the studies above the agents were selected randomly, irrespective of whether we allowed for
saving or not during a transaction. What is often observed is that various agents tend to make preferences for for whom to interact with. We will now study the evolution of the distribution of wealth \( w_m \) by assuming that there is a likelihood
$$
p_{ij} \propto \vert m_i-m_j\vert^{-\alpha},
$$
<h3 id="___sec11">Particle in one dimension and velocity distribution </h3>
for an interaction between agents \( i \) and \( j \) with respective wealths \( m_i \) and \( m_j \). The parameter \( \alpha > 0 \). For \( \alpha=0 \) we recover our model from part 5a).
Perform the same analysis as previously with \( N=500 \) as well as with \( N=1000 \) agents and study the distribution of wealth for \( \alpha =0.5 \), \( \alpha =1.0 \), \( \alpha =1.5 \) and \( \alpha =2.0 \).
You should try to reproduce Figure 1 of <a href="http://www.sciencedirect.com/science/article/pii/S0378437114006967" target="_blank">Goswami and Sen</a>.
Extract the tail of the distribution and see if it follows a Pareto distribution
$$
w_m\propto m^{-1-\alpha}.
$$
What happens if \( \alpha \gg 1 \)?
<p>
Perform the analysis with and without a saving \( \lambda \) on each transaction and comment your results.
We add to the previous probability the possibility that two agents who interact have performed similar transactions earlier. That is, in addition to being financially close, we assume that the likelihood for interacting increases if two agents have interacted earlier.
We add this feature by modifying the previous likelihood to
$$
p_{ij} \propto \vert m_i-m_j\vert^{-\alpha}\left(c_{ij}+1\right)^{\gamma},
$$
where \( c_{ij} \) represents the number of previous interactions that have taken place between \( i \) and \( j \). The factor \( 1 \) is added in order to ensure that if they have not interacted earlier they can still interact. Perform similar studies as above with \( N=1000 \), \( \alpha=1.0 \) and \( \alpha=2.0 \) using \( \gamma = 0.0, 1.0, 2.0, 3.0 \) and \( 4.0 \). Plot the wealth distributions for these cases and try to extract eventual power law tails with and without a saving \( \lambda \) in each transaction. Comment your results and compare them with figures 5 and 6 of <a href="http://www.sciencedirect.com/science/article/pii/S0378437114006967" target="_blank">Goswami and Sen</a>.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec30">Particle in one dimension an velocity distribution </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->