lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e3813eeebd2a1af83d3f07cbede285b9fdf1312a
FYS-STK4155/doc/pub/DecisionTrees/html/DecisionTrees-bs.html
T
mhjensen e3813eeebd small update on decision trees, more to come
2018-11-01 06:46:28 +01:00

413 lines
31 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees">
<title>Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Decision trees, overarching aims', 2, None, '___sec0'),
('Nearest Neighbors', 2, None, '___sec1'),
('Decision trees and Regression', 2, None, '___sec2')]}
end of tocinfo -->
<body>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="DecisionTrees-bs.html">Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Decision trees, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Nearest Neighbors</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">Decision trees and Regression</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees</h1></center> <!-- document title -->
<p>
<!-- author(s): Morten Hjorth-Jensen -->
<center>
<b>Morten Hjorth-Jensen</b> [1, 2]
</center>
<p>
<!-- institution(s) -->
<center>[1] <b>Department of Physics, University of Oslo</b></center>
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 1, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- potential-jumbotron-button -->
</div> <!-- end jumbotron -->
<!-- !split -->
<h2 id="___sec0" class="anchor">Decision trees, overarching aims </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
Decision trees are supervised learning algorithms used for both,
classification and regression tasks where we will concentrate on
classification in this first part of our decision tree tutorial.
Decision trees are assigned to the information based learning
algorithms which use different measures of information gain for
learning. We can use decision trees for issues where we have
continuous but also categorical input and target features.
<p>
</div>
</div>
<p>
<!-- !split -->
<h2 id="___sec1" class="anchor">Nearest Neighbors </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">mglearn</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn</span> <span style="color: #008000; font-weight: bold">import</span> linear_model
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> Pipeline
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.neighbors</span> <span style="color: #008000; font-weight: bold">import</span> KNeighborsClassifier
<span style="color: #408080; font-style: italic"># Generate sample data</span>
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sort(<span style="color: #666666">5*</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">40</span>,<span style="color: #666666">1</span>), axis<span style="color: #666666">=0</span>)
y <span style="color: #666666">=</span> X<span style="color: #666666">**3</span>
y<span style="color: #666666">=</span>y<span style="color: #666666">.</span>ravel()
<span style="color: #408080; font-style: italic"># Add noise to targets</span>
X[::<span style="color: #666666">4</span>] <span style="color: #666666">+=</span>(<span style="color: #666666">0.5</span> <span style="color: #666666">-</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">1</span>))
y[::<span style="color: #666666">5</span>] <span style="color: #666666">+=</span>(<span style="color: #666666">0.5</span> <span style="color: #666666">-</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">8</span>))
a<span style="color: #666666">=</span>np<span style="color: #666666">.</span>array(X)
b<span style="color: #666666">=</span>np<span style="color: #666666">.</span>array(y)
X_train<span style="color: #666666">=</span>a[:<span style="color: #666666">19</span>]
X_test<span style="color: #666666">=</span>a[<span style="color: #666666">19</span>:]
y_train<span style="color: #666666">=</span>b[:<span style="color: #666666">19</span>]
y_test<span style="color: #666666">=</span>b[<span style="color: #666666">19</span>:]
model<span style="color: #666666">=</span>Pipeline([(<span style="color: #BA2121">&#39;poly&#39;</span>, PolynomialFeatures(degree<span style="color: #666666">=3</span>)),(<span style="color: #BA2121">&#39;linear&#39;</span>, LinearRegression(fit_intercept<span style="color: #666666">=</span><span style="color: #008000">False</span>))])
model<span style="color: #666666">=</span>model<span style="color: #666666">.</span>fit(X_train, y_train)
pred<span style="color: #666666">=</span>model<span style="color: #666666">.</span>predict(X_test)
poly<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=3</span>)
poly<span style="color: #666666">.</span>fit_transform(X_train, y_train)
plt<span style="color: #666666">.</span>scatter(X_test, y_test)
plt<span style="color: #666666">.</span>plot(X_test, pred, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;green&#39;</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #008000; font-weight: bold">print</span> (model<span style="color: #666666">.</span>score(X_test,y_test))
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;---------K-Nearest Neighbors-------&quot;</span>)
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot;neighbors_settings=range(1,11)</span>
<span style="color: #BA2121; font-style: italic">for n_neighbors in neighbors_settings:</span>
<span style="color: #BA2121; font-style: italic"> clf=KNeighborsClassifier(n_neighbors=n_neighbors)</span>
<span style="color: #BA2121; font-style: italic"> clf.fit(X_train, y_train)</span>
<span style="color: #BA2121; font-style: italic"> training_accuracy.append(clf.score(X_train, y_train))</span>
<span style="color: #BA2121; font-style: italic"> test_accuracy.append(clf.score(X_test, y_test))</span>
<span style="color: #BA2121; font-style: italic">print (mglearn.plots.plot_knn_regression(n_neighbors=3))&quot;&quot;&quot;</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.neighbors</span> <span style="color: #008000; font-weight: bold">import</span> KNeighborsRegressor
X, y<span style="color: #666666">=</span>mglearn<span style="color: #666666">.</span>datasets<span style="color: #666666">.</span>make_wave(n_samples<span style="color: #666666">=40</span>)
reg <span style="color: #666666">=</span> KNeighborsRegressor(n_neighbors<span style="color: #666666">=3</span>)
reg<span style="color: #666666">.</span>fit(X_train, y_train)
</pre></div>
<p>
<!-- !split -->
<h2 id="___sec2" class="anchor">Decision trees and Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
steps<span style="color: #666666">=250</span>
distance<span style="color: #666666">=0</span>
x<span style="color: #666666">=0</span>
distance_list<span style="color: #666666">=</span>[]
steps_list<span style="color: #666666">=</span>[]
<span style="color: #008000; font-weight: bold">while</span> x<span style="color: #666666">&lt;</span>steps:
distance<span style="color: #666666">+=</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>randint(<span style="color: #666666">-1</span>,<span style="color: #666666">2</span>)
distance_list<span style="color: #666666">.</span>append(distance)
x<span style="color: #666666">+=1</span>
steps_list<span style="color: #666666">.</span>append(x)
plt<span style="color: #666666">.</span>plot(steps_list,distance_list, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;green&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;Random Walk Data&quot;</span>)
steps_list<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(steps_list)
distance_list<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(distance_list)
X<span style="color: #666666">=</span>steps_list[:,np<span style="color: #666666">.</span>newaxis]
<span style="color: #408080; font-style: italic">#Polynomial fits</span>
<span style="color: #408080; font-style: italic">#Degree 2</span>
poly_features<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=2</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly<span style="color: #666666">=</span>poly_features<span style="color: #666666">.</span>fit_transform(X)
lin_reg<span style="color: #666666">=</span>LinearRegression()
poly_fit<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly,distance_list)
b<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>coef_
c<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>intercept_
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;2nd degree coefficients:&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;zero power: &quot;</span>,c)
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;first power: &quot;</span>, b[<span style="color: #666666">0</span>])
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;second power: &quot;</span>,b[<span style="color: #666666">1</span>])
z <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0</span>, steps, <span style="color: #666666">.01</span>)
z_mod<span style="color: #666666">=</span>b[<span style="color: #666666">1</span>]<span style="color: #666666">*</span>z<span style="color: #666666">**2+</span>b[<span style="color: #666666">0</span>]<span style="color: #666666">*</span>z<span style="color: #666666">+</span>c
fit_mod<span style="color: #666666">=</span>b[<span style="color: #666666">1</span>]<span style="color: #666666">*</span>X<span style="color: #666666">**2+</span>b[<span style="color: #666666">0</span>]<span style="color: #666666">*</span>X<span style="color: #666666">+</span>c
plt<span style="color: #666666">.</span>plot(z, z_mod, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;r&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;2nd Degree Fit&quot;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Polynomial Regression&quot;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;Steps&quot;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;Distance&quot;</span>)
<span style="color: #408080; font-style: italic">#Degree 10</span>
poly_features10<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=10</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly10<span style="color: #666666">=</span>poly_features10<span style="color: #666666">.</span>fit_transform(X)
poly_fit10<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly10,distance_list)
y_plot<span style="color: #666666">=</span>poly_fit10<span style="color: #666666">.</span>predict(X_poly10)
plt<span style="color: #666666">.</span>plot(X, y_plot, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;black&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;10th Degree Fit&quot;</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Decision Tree Regression</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
regr_1<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=2</span>)
regr_2<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=5</span>)
regr_3<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=7</span>)
regr_1<span style="color: #666666">.</span>fit(X, distance_list)
regr_2<span style="color: #666666">.</span>fit(X, distance_list)
regr_3<span style="color: #666666">.</span>fit(X, distance_list)
X_test <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0.0</span>, steps, <span style="color: #666666">0.01</span>)[:, np<span style="color: #666666">.</span>newaxis]
y_1 <span style="color: #666666">=</span> regr_1<span style="color: #666666">.</span>predict(X_test)
y_2 <span style="color: #666666">=</span> regr_2<span style="color: #666666">.</span>predict(X_test)
y_3<span style="color: #666666">=</span>regr_3<span style="color: #666666">.</span>predict(X_test)
<span style="color: #408080; font-style: italic"># Plot the results</span>
plt<span style="color: #666666">.</span>figure()
plt<span style="color: #666666">.</span>scatter(X, distance_list, s<span style="color: #666666">=2.5</span>, c<span style="color: #666666">=</span><span style="color: #BA2121">&quot;black&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;data&quot;</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_1, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;red&quot;</span>,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=2&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_2, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=5&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_3, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;m&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=7&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;Data&quot;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;Darget&quot;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Decision Tree Regression&quot;</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># Program to test the Metropolis algorithm with one particle at given temp in</span>
<span style="color: #408080; font-style: italic"># one dimension</span>
<span style="color: #408080; font-style: italic">#!/usr/bin/env python</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.mlab</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">mlab</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">random</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">math</span> <span style="color: #008000; font-weight: bold">import</span> sqrt, exp, log
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
<span style="color: #408080; font-style: italic"># initialize the rng with a seed</span>
random<span style="color: #666666">.</span>seed()
<span style="color: #408080; font-style: italic"># Hard coding of input parameters</span>
MCcycles <span style="color: #666666">=</span> <span style="color: #666666">100000</span>
Temperature <span style="color: #666666">=</span> <span style="color: #666666">2.0</span>
beta <span style="color: #666666">=</span> <span style="color: #666666">1./</span>Temperature
InitialVelocity <span style="color: #666666">=</span> <span style="color: #666666">-2.0</span>
CurrentVelocity <span style="color: #666666">=</span> InitialVelocity
Energy <span style="color: #666666">=</span> <span style="color: #666666">0.5*</span>InitialVelocity<span style="color: #666666">*</span>InitialVelocity
VelocityRange <span style="color: #666666">=</span> <span style="color: #666666">10*</span>sqrt(Temperature)
VelocityStep <span style="color: #666666">=</span> <span style="color: #666666">2*</span>VelocityRange<span style="color: #666666">/10.</span>
AverageEnergy <span style="color: #666666">=</span> Energy
AverageEnergy2 <span style="color: #666666">=</span> Energy<span style="color: #666666">*</span>Energy
VelocityValues <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(MCcycles)
<span style="color: #408080; font-style: italic"># The Monte Carlo sampling with Metropolis starts here</span>
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span> (<span style="color: #666666">1</span>, MCcycles, <span style="color: #666666">1</span>):
TrialVelocity <span style="color: #666666">=</span> CurrentVelocity <span style="color: #666666">+</span> (<span style="color: #666666">2.0*</span>random<span style="color: #666666">.</span>random() <span style="color: #666666">-</span> <span style="color: #666666">1.0</span>)<span style="color: #666666">*</span>VelocityStep
EnergyChange <span style="color: #666666">=</span> <span style="color: #666666">0.5*</span>(TrialVelocity<span style="color: #666666">*</span>TrialVelocity <span style="color: #666666">-</span>CurrentVelocity<span style="color: #666666">*</span>CurrentVelocity);
<span style="color: #008000; font-weight: bold">if</span> random<span style="color: #666666">.</span>random() <span style="color: #666666">&lt;=</span> exp(<span style="color: #666666">-</span>beta<span style="color: #666666">*</span>EnergyChange):
CurrentVelocity <span style="color: #666666">=</span> TrialVelocity
Energy <span style="color: #666666">+=</span> EnergyChange
VelocityValues[i] <span style="color: #666666">=</span> CurrentVelocity
AverageEnergy <span style="color: #666666">+=</span> Energy
AverageEnergy2 <span style="color: #666666">+=</span> Energy<span style="color: #666666">*</span>Energy
<span style="color: #408080; font-style: italic">#Final averages</span>
AverageEnergy <span style="color: #666666">=</span> AverageEnergy<span style="color: #666666">/</span>MCcycles
AverageEnergy2 <span style="color: #666666">=</span> AverageEnergy2<span style="color: #666666">/</span>MCcycles
Variance <span style="color: #666666">=</span> AverageEnergy2 <span style="color: #666666">-</span> AverageEnergy<span style="color: #666666">*</span>AverageEnergy
<span style="color: #008000; font-weight: bold">print</span>(AverageEnergy, Variance)
n, bins, patches <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>hist(VelocityValues, <span style="color: #666666">400</span>, facecolor<span style="color: #666666">=</span><span style="color: #BA2121">&#39;green&#39;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&#39;$v$&#39;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&#39;Velocity distribution P(v)&#39;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">r&#39;Velocity histogram at $k_BT=2$&#39;</span>)
plt<span style="color: #666666">.</span>axis([<span style="color: #666666">-5</span>, <span style="color: #666666">5</span>, <span style="color: #666666">0</span>, <span style="color: #666666">600</span>])
plt<span style="color: #666666">.</span>grid(<span style="color: #008000">True</span>)
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">collections</span> <span style="color: #008000; font-weight: bold">import</span> Counter
<span style="color: #408080; font-style: italic">#print (Counter(VelocityValues))</span>
<span style="color: #008000; font-weight: bold">print</span> (VelocityValues[:<span style="color: #666666">20</span>])
VelocityValues<span style="color: #666666">=</span><span style="color: #008000">list</span>(Counter(VelocityValues)<span style="color: #666666">.</span>keys())
d<span style="color: #666666">=</span><span style="color: #008000">list</span>(Counter(VelocityValues)<span style="color: #666666">.</span>values())
VelocityValues<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(VelocityValues)[:, np<span style="color: #666666">.</span>newaxis]
d<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(d)
<span style="color: #008000; font-weight: bold">print</span> (VelocityValues<span style="color: #666666">.</span>shape, d<span style="color: #666666">.</span>shape)
plt<span style="color: #666666">.</span>scatter(VelocityValues, d)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#2nd Degree Polynomial</span>
poly_feat<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=20</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly<span style="color: #666666">=</span>poly_feat<span style="color: #666666">.</span>fit_transform(VelocityValues)
lin_reg<span style="color: #666666">=</span>LinearRegression()
poly_fit<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly,d)
y_plot<span style="color: #666666">=</span>poly_fit<span style="color: #666666">.</span>predict(X_poly)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Polynomial Fit&quot;</span>)
plt<span style="color: #666666">.</span>plot(VelocityValues, y_plot, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;black&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;Fit&quot;</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Decision Trees</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
regr_1<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=2</span>)
regr_2<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=5</span>)
regr_3<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=7</span>)
regr_1<span style="color: #666666">.</span>fit(VelocityValues, d)
regr_2<span style="color: #666666">.</span>fit(VelocityValues, d)
regr_3<span style="color: #666666">.</span>fit(VelocityValues, d)
X_test <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0.0</span>, MCcycles, <span style="color: #666666">0.01</span>)[:, np<span style="color: #666666">.</span>newaxis]
y_1<span style="color: #666666">=</span>regr_1<span style="color: #666666">.</span>predict(X_test)
y_2<span style="color: #666666">=</span>regr_2<span style="color: #666666">.</span>predict(X_test)
y_3<span style="color: #666666">=</span>regr_3<span style="color: #666666">.</span>predict(X_test)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Decision Tree&quot;</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_1, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;red&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=2&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_2, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=5&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_3, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;m&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=7&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Separate each frequency not in one specific velocity, but in a range of values,</span>
<span style="color: #408080; font-style: italic">#i.e. frequency of all velocities in range -5 to -4.9, -4.9 to -4.8, etc...</span>
</pre></div>
<p>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink