lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
efa30bcf9e512b04a00a1a196151e6aa948bd6de
FYS-STK4155/doc/pub/DecisionTrees/html/DecisionTrees-bs.html
T
mhjensen 50471e835b updated material about trees, more to come 
nothing about random forests
2018-11-02 06:05:54 +01:00

407 lines
27 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Trees, forests and all that">
<title>Data Analysis and Machine Learning: Trees, forests and all that</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Decision trees, overarching aims', 2, None, '___sec0'),
('Nodes, leafs, roots and branches', 2, None, '___sec1'),
('How do we set it up?', 2, None, '___sec2'),
('Decision trees and Regression', 2, None, '___sec3'),
('Maxwell-Boltzmann velocity distribution', 2, None, '___sec4')]}
end of tocinfo -->
<body>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="DecisionTrees-bs.html">Data Analysis and Machine Learning: Trees, forests and all that</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Decision trees, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Nodes, leafs, roots and branches</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">How do we set it up?</a></li>
<!-- navigation toc: --> <li><a href="#___sec3" style="font-size: 80%;">Decision trees and Regression</a></li>
<!-- navigation toc: --> <li><a href="#___sec4" style="font-size: 80%;">Maxwell-Boltzmann velocity distribution</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Data Analysis and Machine Learning: Trees, forests and all that</h1></center> <!-- document title -->
<p>
<!-- author(s): Morten Hjorth-Jensen -->
<center>
<b>Morten Hjorth-Jensen</b> [1, 2]
</center>
<p>
<!-- institution(s) -->
<center>[1] <b>Department of Physics, University of Oslo</b></center>
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 2, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- potential-jumbotron-button -->
</div> <!-- end jumbotron -->
<!-- !split -->
<h2 id="___sec0" class="anchor">Decision trees, overarching aims </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<p>
Decision trees are supervised learning algorithms used for both,
classification and regression tasks where we will concentrate on
classification in this first part of our decision tree tutorial.
Decision trees are assigned to the information based learning
algorithms which use different measures of information gain for
learning. We can use decision trees for issues where we have
continuous but also categorical input and target features.
<p>
</div>
</div>
<p>
<!-- !split -->
<h2 id="___sec1" class="anchor">Nodes, leafs, roots and branches </h2>
<p>
The main idea of decision trees
is to find those descriptive features which contain the most
<b>information</b> regarding the target feature and then split the dataset
along the values of these features such that the target feature values
for the resulting sub\_datasets are as pure as possible.
<p>
The descriptive feature which leaves the target feature most purely is said
to be the most informative one. This process of finding the <b>most
informative</b> feature is done until we accomplish a stopping criteria
where we then finally end up in so called <b>leaf nodes</b>.
<p>
The leaf nodes
contain the predictions we will make for new query instances presented
to our trained model. This is possible since the model has kind of
learned the underlying structure of the training data and hence can,
given some assumptions, make predictions about the target feature value
(class) of unseen query instances.
<p>
A decision tree mainly contains of a <b>root node</b>, <b>interior nodes</b>,
and <b>leaf nodes</b> which are then connected by <b>branches</b>.
<p>
<!-- !split -->
<h2 id="___sec2" class="anchor">How do we set it up? </h2>
<p>
In simplified terms, the process of training a decision tree and
predicting the target features of query instances is as follows:
<ol>
<li> Present a dataset containing of a number of training instances characterized by a number of descriptive features and a target feature</li>
<li> Train the decision tree model by continuously splitting the target feature along the values of the descriptive features using a measure of information gain during the training process</li>
<li> Grow the tree until we accomplish a stopping criteria create leaf nodes which represent the <em>predictions</em> we want to make for new query instances</li>
<li> Show query instances to the tree and run down the tree until we arrive at leaf nodes</li>
</ol>
Then we are essentially done!
<p>
<!-- !split -->
<h2 id="___sec3" class="anchor">Decision trees and Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
steps<span style="color: #666666">=250</span>
distance<span style="color: #666666">=0</span>
x<span style="color: #666666">=0</span>
distance_list<span style="color: #666666">=</span>[]
steps_list<span style="color: #666666">=</span>[]
<span style="color: #008000; font-weight: bold">while</span> x<span style="color: #666666">&lt;</span>steps:
distance<span style="color: #666666">+=</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>randint(<span style="color: #666666">-1</span>,<span style="color: #666666">2</span>)
distance_list<span style="color: #666666">.</span>append(distance)
x<span style="color: #666666">+=1</span>
steps_list<span style="color: #666666">.</span>append(x)
plt<span style="color: #666666">.</span>plot(steps_list,distance_list, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;green&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;Random Walk Data&quot;</span>)
steps_list<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(steps_list)
distance_list<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(distance_list)
X<span style="color: #666666">=</span>steps_list[:,np<span style="color: #666666">.</span>newaxis]
<span style="color: #408080; font-style: italic">#Polynomial fits</span>
<span style="color: #408080; font-style: italic">#Degree 2</span>
poly_features<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=2</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly<span style="color: #666666">=</span>poly_features<span style="color: #666666">.</span>fit_transform(X)
lin_reg<span style="color: #666666">=</span>LinearRegression()
poly_fit<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly,distance_list)
b<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>coef_
c<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>intercept_
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;2nd degree coefficients:&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;zero power: &quot;</span>,c)
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;first power: &quot;</span>, b[<span style="color: #666666">0</span>])
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;second power: &quot;</span>,b[<span style="color: #666666">1</span>])
z <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0</span>, steps, <span style="color: #666666">.01</span>)
z_mod<span style="color: #666666">=</span>b[<span style="color: #666666">1</span>]<span style="color: #666666">*</span>z<span style="color: #666666">**2+</span>b[<span style="color: #666666">0</span>]<span style="color: #666666">*</span>z<span style="color: #666666">+</span>c
fit_mod<span style="color: #666666">=</span>b[<span style="color: #666666">1</span>]<span style="color: #666666">*</span>X<span style="color: #666666">**2+</span>b[<span style="color: #666666">0</span>]<span style="color: #666666">*</span>X<span style="color: #666666">+</span>c
plt<span style="color: #666666">.</span>plot(z, z_mod, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;r&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;2nd Degree Fit&quot;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Polynomial Regression&quot;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;Steps&quot;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;Distance&quot;</span>)
<span style="color: #408080; font-style: italic">#Degree 10</span>
poly_features10<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=10</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly10<span style="color: #666666">=</span>poly_features10<span style="color: #666666">.</span>fit_transform(X)
poly_fit10<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly10,distance_list)
y_plot<span style="color: #666666">=</span>poly_fit10<span style="color: #666666">.</span>predict(X_poly10)
plt<span style="color: #666666">.</span>plot(X, y_plot, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;black&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;10th Degree Fit&quot;</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Decision Tree Regression</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
regr_1<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=2</span>)
regr_2<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=5</span>)
regr_3<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=7</span>)
regr_1<span style="color: #666666">.</span>fit(X, distance_list)
regr_2<span style="color: #666666">.</span>fit(X, distance_list)
regr_3<span style="color: #666666">.</span>fit(X, distance_list)
X_test <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0.0</span>, steps, <span style="color: #666666">0.01</span>)[:, np<span style="color: #666666">.</span>newaxis]
y_1 <span style="color: #666666">=</span> regr_1<span style="color: #666666">.</span>predict(X_test)
y_2 <span style="color: #666666">=</span> regr_2<span style="color: #666666">.</span>predict(X_test)
y_3<span style="color: #666666">=</span>regr_3<span style="color: #666666">.</span>predict(X_test)
<span style="color: #408080; font-style: italic"># Plot the results</span>
plt<span style="color: #666666">.</span>figure()
plt<span style="color: #666666">.</span>scatter(X, distance_list, s<span style="color: #666666">=2.5</span>, c<span style="color: #666666">=</span><span style="color: #BA2121">&quot;black&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;data&quot;</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_1, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;red&quot;</span>,
label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=2&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_2, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=5&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_3, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;m&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=7&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;Data&quot;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;Darget&quot;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Decision Tree Regression&quot;</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<!-- !split -->
<h2 id="___sec4" class="anchor">Maxwell-Boltzmann velocity distribution </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># Program to test the Metropolis algorithm with one particle at given temp in</span>
<span style="color: #408080; font-style: italic"># one dimension</span>
<span style="color: #408080; font-style: italic">#!/usr/bin/env python</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.mlab</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">mlab</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">random</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">math</span> <span style="color: #008000; font-weight: bold">import</span> sqrt, exp, log
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> LinearRegression
<span style="color: #408080; font-style: italic"># initialize the rng with a seed</span>
random<span style="color: #666666">.</span>seed()
<span style="color: #408080; font-style: italic"># Hard coding of input parameters</span>
MCcycles <span style="color: #666666">=</span> <span style="color: #666666">100000</span>
Temperature <span style="color: #666666">=</span> <span style="color: #666666">2.0</span>
beta <span style="color: #666666">=</span> <span style="color: #666666">1./</span>Temperature
InitialVelocity <span style="color: #666666">=</span> <span style="color: #666666">-2.0</span>
CurrentVelocity <span style="color: #666666">=</span> InitialVelocity
Energy <span style="color: #666666">=</span> <span style="color: #666666">0.5*</span>InitialVelocity<span style="color: #666666">*</span>InitialVelocity
VelocityRange <span style="color: #666666">=</span> <span style="color: #666666">10*</span>sqrt(Temperature)
VelocityStep <span style="color: #666666">=</span> <span style="color: #666666">2*</span>VelocityRange<span style="color: #666666">/10.</span>
AverageEnergy <span style="color: #666666">=</span> Energy
AverageEnergy2 <span style="color: #666666">=</span> Energy<span style="color: #666666">*</span>Energy
VelocityValues <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(MCcycles)
<span style="color: #408080; font-style: italic"># The Monte Carlo sampling with Metropolis starts here</span>
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span> (<span style="color: #666666">1</span>, MCcycles, <span style="color: #666666">1</span>):
TrialVelocity <span style="color: #666666">=</span> CurrentVelocity <span style="color: #666666">+</span> (<span style="color: #666666">2.0*</span>random<span style="color: #666666">.</span>random() <span style="color: #666666">-</span> <span style="color: #666666">1.0</span>)<span style="color: #666666">*</span>VelocityStep
EnergyChange <span style="color: #666666">=</span> <span style="color: #666666">0.5*</span>(TrialVelocity<span style="color: #666666">*</span>TrialVelocity <span style="color: #666666">-</span>CurrentVelocity<span style="color: #666666">*</span>CurrentVelocity);
<span style="color: #008000; font-weight: bold">if</span> random<span style="color: #666666">.</span>random() <span style="color: #666666">&lt;=</span> exp(<span style="color: #666666">-</span>beta<span style="color: #666666">*</span>EnergyChange):
CurrentVelocity <span style="color: #666666">=</span> TrialVelocity
Energy <span style="color: #666666">+=</span> EnergyChange
VelocityValues[i] <span style="color: #666666">=</span> CurrentVelocity
AverageEnergy <span style="color: #666666">+=</span> Energy
AverageEnergy2 <span style="color: #666666">+=</span> Energy<span style="color: #666666">*</span>Energy
<span style="color: #408080; font-style: italic">#Final averages</span>
AverageEnergy <span style="color: #666666">=</span> AverageEnergy<span style="color: #666666">/</span>MCcycles
AverageEnergy2 <span style="color: #666666">=</span> AverageEnergy2<span style="color: #666666">/</span>MCcycles
Variance <span style="color: #666666">=</span> AverageEnergy2 <span style="color: #666666">-</span> AverageEnergy<span style="color: #666666">*</span>AverageEnergy
<span style="color: #008000; font-weight: bold">print</span>(AverageEnergy, Variance)
n, bins, patches <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>hist(VelocityValues, <span style="color: #666666">400</span>, facecolor<span style="color: #666666">=</span><span style="color: #BA2121">&#39;green&#39;</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&#39;$v$&#39;</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&#39;Velocity distribution P(v)&#39;</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">r&#39;Velocity histogram at $k_BT=2$&#39;</span>)
plt<span style="color: #666666">.</span>axis([<span style="color: #666666">-5</span>, <span style="color: #666666">5</span>, <span style="color: #666666">0</span>, <span style="color: #666666">600</span>])
plt<span style="color: #666666">.</span>grid(<span style="color: #008000">True</span>)
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">collections</span> <span style="color: #008000; font-weight: bold">import</span> Counter
<span style="color: #408080; font-style: italic">#print (Counter(VelocityValues))</span>
<span style="color: #008000; font-weight: bold">print</span> (VelocityValues[:<span style="color: #666666">20</span>])
VelocityValues<span style="color: #666666">=</span><span style="color: #008000">list</span>(Counter(VelocityValues)<span style="color: #666666">.</span>keys())
d<span style="color: #666666">=</span><span style="color: #008000">list</span>(Counter(VelocityValues)<span style="color: #666666">.</span>values())
VelocityValues<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(VelocityValues)[:, np<span style="color: #666666">.</span>newaxis]
d<span style="color: #666666">=</span>np<span style="color: #666666">.</span>asarray(d)
<span style="color: #008000; font-weight: bold">print</span> (VelocityValues<span style="color: #666666">.</span>shape, d<span style="color: #666666">.</span>shape)
plt<span style="color: #666666">.</span>scatter(VelocityValues, d)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#2nd Degree Polynomial</span>
poly_feat<span style="color: #666666">=</span>PolynomialFeatures(degree<span style="color: #666666">=20</span>, include_bias<span style="color: #666666">=</span><span style="color: #008000">False</span>)
X_poly<span style="color: #666666">=</span>poly_feat<span style="color: #666666">.</span>fit_transform(VelocityValues)
lin_reg<span style="color: #666666">=</span>LinearRegression()
poly_fit<span style="color: #666666">=</span>lin_reg<span style="color: #666666">.</span>fit(X_poly,d)
y_plot<span style="color: #666666">=</span>poly_fit<span style="color: #666666">.</span>predict(X_poly)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Polynomial Fit&quot;</span>)
plt<span style="color: #666666">.</span>plot(VelocityValues, y_plot, color<span style="color: #666666">=</span><span style="color: #BA2121">&#39;black&#39;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;Fit&quot;</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Decision Trees</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
regr_1<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=2</span>)
regr_2<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=5</span>)
regr_3<span style="color: #666666">=</span>DecisionTreeRegressor(max_depth<span style="color: #666666">=7</span>)
regr_1<span style="color: #666666">.</span>fit(VelocityValues, d)
regr_2<span style="color: #666666">.</span>fit(VelocityValues, d)
regr_3<span style="color: #666666">.</span>fit(VelocityValues, d)
X_test <span style="color: #666666">=</span> np<span style="color: #666666">.</span>arange(<span style="color: #666666">0.0</span>, MCcycles, <span style="color: #666666">0.01</span>)[:, np<span style="color: #666666">.</span>newaxis]
y_1<span style="color: #666666">=</span>regr_1<span style="color: #666666">.</span>predict(X_test)
y_2<span style="color: #666666">=</span>regr_2<span style="color: #666666">.</span>predict(X_test)
y_3<span style="color: #666666">=</span>regr_3<span style="color: #666666">.</span>predict(X_test)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Decision Tree&quot;</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_1, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;red&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=2&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_2, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;green&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=5&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>plot(X_test, y_3, color<span style="color: #666666">=</span><span style="color: #BA2121">&quot;m&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;max_depth=7&quot;</span>, linewidth<span style="color: #666666">=2</span>)
plt<span style="color: #666666">.</span>show()
<span style="color: #408080; font-style: italic">#Separate each frequency not in one specific velocity, but in a range of values,</span>
<span style="color: #408080; font-style: italic">#i.e. frequency of all velocities in range -5 to -4.9, -4.9 to -4.8, etc...</span>
</pre></div>
<p>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink