lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4f4c4e69464b1b4bc304ff84c9054e5ffdb7a4ee
FYS-STK4155/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
T
mhjensen 8bf7984604 Updating various chapters with codes
2018-05-30 23:00:43 -04:00

278 lines
14 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees">
<title>Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees</title>
<link href="https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_styles/style_solarized_box/css/solarized_light_code.css" rel="stylesheet" type="text/css" title="light"/>
<script src="https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_styles/style_solarized_box/js/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<link href="https://thomasf.github.io/solarized-css/solarized-light.min.css" rel="stylesheet">
<style type="text/css">
h1 {color: #b58900;} /* yellow */
/* h1 {color: #cb4b16;} orange */
/* h1 {color: #d33682;} magenta, the original choice of thomasf */
code { padding: 0px; background-color: inherit; }
pre {
border: 0pt solid #93a1a1;
box-shadow: none;
}
.alert-text-small { font-size: 80%; }
.alert-text-large { font-size: 130%; }
.alert-text-normal { font-size: 90%; }
.alert {
padding:8px 35px 8px 14px; margin-bottom:18px;
text-shadow:0 1px 0 rgba(255,255,255,0.5);
border:1px solid #93a1a1;
border-radius: 4px;
-webkit-border-radius: 4px;
-moz-border-radius: 4px;
color: #555;
background-color: #eee8d5;
background-position: 10px 5px;
background-repeat: no-repeat;
background-size: 38px;
padding-left: 55px;
width: 75%;
}
.alert-block {padding-top:14px; padding-bottom:14px}
.alert-block > p, .alert-block > ul {margin-bottom:1em}
.alert li {margin-top: 1em}
.alert-block p+p {margin-top:5px}
.alert-notice { background-image: url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_notice.png); }
.alert-summary { background-image:url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_summary.png); }
.alert-warning { background-image: url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_warning.png); }
.alert-question {background-image:url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_question.png); }
div { text-align: justify; text-justify: inter-word; }
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Decision trees, overarching aims', 2, None, '___sec0'),
('Nearest Neighbors', 2, None, '___sec1'),
('Decision trees and Regression', 2, None, '___sec2')]}
end of tocinfo -->
<body>
<!-- ------------------- main content ---------------------- -->
<center><h1>Data Analysis and Machine Learning: Nearest Neighbors and Decision Trees</h1></center> <!-- document title -->
<p>
<!-- author(s): Morten Hjorth-Jensen -->
<center>
<b>Morten Hjorth-Jensen</b> [1, 2]
</center>
<p>
<!-- institution(s) -->
<center>[1] <b>Department of Physics, University of Oslo</b></center>
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>May 30, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec0">Decision trees, overarching aims </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec1">Nearest Neighbors </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">mglearn</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn</span> <span style="color: #8B008B; font-weight: bold">import</span> linear_model
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.linear_model</span> <span style="color: #8B008B; font-weight: bold">import</span> LinearRegression
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.pipeline</span> <span style="color: #8B008B; font-weight: bold">import</span> Pipeline
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.neighbors</span> <span style="color: #8B008B; font-weight: bold">import</span> KNeighborsClassifier
<span style="color: #228B22"># Generate sample data</span>
X = np.sort(<span style="color: #B452CD">5</span>*np.random.rand(<span style="color: #B452CD">40</span>,<span style="color: #B452CD">1</span>), axis=<span style="color: #B452CD">0</span>)
y = X**<span style="color: #B452CD">3</span>
y=y.ravel()
<span style="color: #228B22"># Add noise to targets</span>
X[::<span style="color: #B452CD">4</span>] +=(<span style="color: #B452CD">0.5</span> - np.random.rand(<span style="color: #B452CD">1</span>))
y[::<span style="color: #B452CD">5</span>] +=(<span style="color: #B452CD">0.5</span> - np.random.rand(<span style="color: #B452CD">8</span>))
a=np.array(X)
b=np.array(y)
X_train=a[:<span style="color: #B452CD">19</span>]
X_test=a[<span style="color: #B452CD">19</span>:]
y_train=b[:<span style="color: #B452CD">19</span>]
y_test=b[<span style="color: #B452CD">19</span>:]
model=Pipeline([(<span style="color: #CD5555">&#39;poly&#39;</span>, PolynomialFeatures(degree=<span style="color: #B452CD">3</span>)),(<span style="color: #CD5555">&#39;linear&#39;</span>, LinearRegression(fit_intercept=<span style="color: #658b00">False</span>))])
model=model.fit(X_train, y_train)
pred=model.predict(X_test)
poly=PolynomialFeatures(degree=<span style="color: #B452CD">3</span>)
poly.fit_transform(X_train, y_train)
plt.scatter(X_test, y_test)
plt.plot(X_test, pred, color=<span style="color: #CD5555">&#39;green&#39;</span>)
plt.show()
<span style="color: #8B008B; font-weight: bold">print</span> (model.score(X_test,y_test))
<span style="color: #8B008B; font-weight: bold">print</span> (<span style="color: #CD5555">&quot;---------K-Nearest Neighbors-------&quot;</span>)
<span style="color: #CD5555">&quot;&quot;&quot;neighbors_settings=range(1,11)</span>
<span style="color: #CD5555">for n_neighbors in neighbors_settings:</span>
<span style="color: #CD5555"> clf=KNeighborsClassifier(n_neighbors=n_neighbors)</span>
<span style="color: #CD5555"> clf.fit(X_train, y_train)</span>
<span style="color: #CD5555"> training_accuracy.append(clf.score(X_train, y_train))</span>
<span style="color: #CD5555"> test_accuracy.append(clf.score(X_test, y_test))</span>
<span style="color: #CD5555">print (mglearn.plots.plot_knn_regression(n_neighbors=3))&quot;&quot;&quot;</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.neighbors</span> <span style="color: #8B008B; font-weight: bold">import</span> KNeighborsRegressor
X, y=mglearn.datasets.make_wave(n_samples=<span style="color: #B452CD">40</span>)
reg = KNeighborsRegressor(n_neighbors=<span style="color: #B452CD">3</span>)
reg.fit(X_train, y_train)
</pre></div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec2">Decision trees and Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> PolynomialFeatures
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.linear_model</span> <span style="color: #8B008B; font-weight: bold">import</span> LinearRegression
steps=<span style="color: #B452CD">250</span>
distance=<span style="color: #B452CD">0</span>
x=<span style="color: #B452CD">0</span>
distance_list=[]
steps_list=[]
<span style="color: #8B008B; font-weight: bold">while</span> x&lt;steps:
distance+=np.random.randint(-<span style="color: #B452CD">1</span>,<span style="color: #B452CD">2</span>)
distance_list.append(distance)
x+=<span style="color: #B452CD">1</span>
steps_list.append(x)
plt.plot(steps_list,distance_list, color=<span style="color: #CD5555">&#39;green&#39;</span>, label=<span style="color: #CD5555">&quot;Random Walk Data&quot;</span>)
steps_list=np.asarray(steps_list)
distance_list=np.asarray(distance_list)
X=steps_list[:,np.newaxis]
<span style="color: #228B22">#Polynomial fits</span>
<span style="color: #228B22">#Degree 2</span>
poly_features=PolynomialFeatures(degree=<span style="color: #B452CD">2</span>, include_bias=<span style="color: #658b00">False</span>)
X_poly=poly_features.fit_transform(X)
lin_reg=LinearRegression()
poly_fit=lin_reg.fit(X_poly,distance_list)
b=lin_reg.coef_
c=lin_reg.intercept_
<span style="color: #8B008B; font-weight: bold">print</span> (<span style="color: #CD5555">&quot;2nd degree coefficients:&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span> (<span style="color: #CD5555">&quot;zero power: &quot;</span>,c)
<span style="color: #8B008B; font-weight: bold">print</span> (<span style="color: #CD5555">&quot;first power: &quot;</span>, b[<span style="color: #B452CD">0</span>])
<span style="color: #8B008B; font-weight: bold">print</span> (<span style="color: #CD5555">&quot;second power: &quot;</span>,b[<span style="color: #B452CD">1</span>])
z = np.arange(<span style="color: #B452CD">0</span>, steps, .<span style="color: #B452CD">01</span>)
z_mod=b[<span style="color: #B452CD">1</span>]*z**<span style="color: #B452CD">2</span>+b[<span style="color: #B452CD">0</span>]*z+c
fit_mod=b[<span style="color: #B452CD">1</span>]*X**<span style="color: #B452CD">2</span>+b[<span style="color: #B452CD">0</span>]*X+c
plt.plot(z, z_mod, color=<span style="color: #CD5555">&#39;r&#39;</span>, label=<span style="color: #CD5555">&quot;2nd Degree Fit&quot;</span>)
plt.title(<span style="color: #CD5555">&quot;Polynomial Regression&quot;</span>)
plt.xlabel(<span style="color: #CD5555">&quot;Steps&quot;</span>)
plt.ylabel(<span style="color: #CD5555">&quot;Distance&quot;</span>)
<span style="color: #228B22">#Degree 10</span>
poly_features10=PolynomialFeatures(degree=<span style="color: #B452CD">10</span>, include_bias=<span style="color: #658b00">False</span>)
X_poly10=poly_features10.fit_transform(X)
poly_fit10=lin_reg.fit(X_poly10,distance_list)
y_plot=poly_fit10.predict(X_poly10)
plt.plot(X, y_plot, color=<span style="color: #CD5555">&#39;black&#39;</span>, label=<span style="color: #CD5555">&quot;10th Degree Fit&quot;</span>)
plt.legend()
plt.show()
<span style="color: #228B22">#Decision Tree Regression</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeRegressor
regr_1=DecisionTreeRegressor(max_depth=<span style="color: #B452CD">2</span>)
regr_2=DecisionTreeRegressor(max_depth=<span style="color: #B452CD">5</span>)
regr_3=DecisionTreeRegressor(max_depth=<span style="color: #B452CD">7</span>)
regr_1.fit(X, distance_list)
regr_2.fit(X, distance_list)
regr_3.fit(X, distance_list)
X_test = np.arange(<span style="color: #B452CD">0.0</span>, steps, <span style="color: #B452CD">0.01</span>)[:, np.newaxis]
y_1 = regr_1.predict(X_test)
y_2 = regr_2.predict(X_test)
y_3=regr_3.predict(X_test)
<span style="color: #228B22"># Plot the results</span>
plt.figure()
plt.scatter(X, distance_list, s=<span style="color: #B452CD">2.5</span>, c=<span style="color: #CD5555">&quot;black&quot;</span>, label=<span style="color: #CD5555">&quot;data&quot;</span>)
plt.plot(X_test, y_1, color=<span style="color: #CD5555">&quot;red&quot;</span>,
label=<span style="color: #CD5555">&quot;max_depth=2&quot;</span>, linewidth=<span style="color: #B452CD">2</span>)
plt.plot(X_test, y_2, color=<span style="color: #CD5555">&quot;green&quot;</span>, label=<span style="color: #CD5555">&quot;max_depth=5&quot;</span>, linewidth=<span style="color: #B452CD">2</span>)
plt.plot(X_test, y_3, color=<span style="color: #CD5555">&quot;m&quot;</span>, label=<span style="color: #CD5555">&quot;max_depth=7&quot;</span>, linewidth=<span style="color: #B452CD">2</span>)
plt.xlabel(<span style="color: #CD5555">&quot;Data&quot;</span>)
plt.ylabel(<span style="color: #CD5555">&quot;Darget&quot;</span>)
plt.title(<span style="color: #CD5555">&quot;Decision Tree Regression&quot;</span>)
plt.legend()
plt.show()
</pre></div>
<p>
<!-- ------------------- end of main content --------------- -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink