mhjensen
317 lines
16 KiB
HTML
317 lines
16 KiB
HTML
<!--
|
|
Automatically generated HTML file from DocOnce source
|
|
(https://github.com/hplgit/doconce/)
|
|
-->
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
|
|
<meta name="description" content="Data Analysis and Machine Learning: Introduction and Representing data">
|
|
|
|
<title>Data Analysis and Machine Learning: Introduction and Representing data</title>
|
|
|
|
|
|
<link href="https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_styles/style_solarized_box/css/solarized_light_code.css" rel="stylesheet" type="text/css" title="light"/>
|
|
<script src="https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_styles/style_solarized_box/js/highlight.pack.js"></script>
|
|
<script>hljs.initHighlightingOnLoad();</script>
|
|
|
|
<link href="https://thomasf.github.io/solarized-css/solarized-light.min.css" rel="stylesheet">
|
|
<style type="text/css">
|
|
h1 {color: #b58900;} /* yellow */
|
|
/* h1 {color: #cb4b16;} orange */
|
|
/* h1 {color: #d33682;} magenta, the original choice of thomasf */
|
|
code { padding: 0px; background-color: inherit; }
|
|
pre {
|
|
border: 0pt solid #93a1a1;
|
|
box-shadow: none;
|
|
}
|
|
.alert-text-small { font-size: 80%; }
|
|
.alert-text-large { font-size: 130%; }
|
|
.alert-text-normal { font-size: 90%; }
|
|
.alert {
|
|
padding:8px 35px 8px 14px; margin-bottom:18px;
|
|
text-shadow:0 1px 0 rgba(255,255,255,0.5);
|
|
border:1px solid #93a1a1;
|
|
border-radius: 4px;
|
|
-webkit-border-radius: 4px;
|
|
-moz-border-radius: 4px;
|
|
color: #555;
|
|
background-color: #eee8d5;
|
|
background-position: 10px 5px;
|
|
background-repeat: no-repeat;
|
|
background-size: 38px;
|
|
padding-left: 55px;
|
|
width: 75%;
|
|
}
|
|
.alert-block {padding-top:14px; padding-bottom:14px}
|
|
.alert-block > p, .alert-block > ul {margin-bottom:1em}
|
|
.alert li {margin-top: 1em}
|
|
.alert-block p+p {margin-top:5px}
|
|
.alert-notice { background-image: url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_notice.png); }
|
|
.alert-summary { background-image:url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_summary.png); }
|
|
.alert-warning { background-image: url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_warning.png); }
|
|
.alert-question {background-image:url(https://cdn.rawgit.com/hplgit/doconce/master/bundled/html_images/small_yellow_question.png); }
|
|
|
|
div { text-align: justify; text-justify: inter-word; }
|
|
</style>
|
|
|
|
|
|
</head>
|
|
|
|
<!-- tocinfo
|
|
{'highest level': 2,
|
|
'sections': [('What is Machine Learning?', 2, None, '___sec0'),
|
|
('Types of Machine Learning', 2, None, '___sec1'),
|
|
('Different algorithms', 2, None, '___sec2'),
|
|
('Software and needed installations', 2, None, '___sec3'),
|
|
('Python installers', 2, None, '___sec4'),
|
|
('Installing R and C++', 2, None, '___sec5'),
|
|
('Introduction to Jupyter notebook and available tools',
|
|
2,
|
|
None,
|
|
'___sec6'),
|
|
('Representing data, overarching aims', 2, None, '___sec7'),
|
|
('Representing data, more examples', 2, None, '___sec8')]}
|
|
end of tocinfo -->
|
|
|
|
<body>
|
|
|
|
|
|
<!-- ------------------- main content ---------------------- -->
|
|
|
|
|
|
|
|
<center><h1>Data Analysis and Machine Learning: Introduction and Representing data</h1></center> <!-- document title -->
|
|
|
|
<p>
|
|
<!-- author(s): Morten Hjorth-Jensen -->
|
|
|
|
<center>
|
|
<b>Morten Hjorth-Jensen</b> [1, 2]
|
|
</center>
|
|
|
|
<p>
|
|
<!-- institution(s) -->
|
|
|
|
<center>[1] <b>Department of Physics, University of Oslo</b></center>
|
|
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
|
<br>
|
|
<p>
|
|
<center><h4>Nov 27, 2017</h4></center> <!-- date -->
|
|
<br>
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec0">What is Machine Learning? </h2>
|
|
|
|
<p>
|
|
Machine learning is the science of giving computers the ability to
|
|
learn without being explicitly programmed. The idea is that there
|
|
exist generic algorithms which can be used to find patterns in a broad
|
|
class of data sets without having to write code specifically for each
|
|
problem. The algorithm will build its own logic based on the data.
|
|
|
|
<p>
|
|
Machine learning is a subfield of computer science, and is closely
|
|
related to computational statistics. It evolved from the study of
|
|
pattern recognition in artificial intelligence (AI) research, and has
|
|
made contributions to AI tasks like computer vision, natural language
|
|
processing and speech recognition. It has also, especially in later
|
|
years, found applications in a wide variety of other areas, including
|
|
bioinformatics, economy, physics, finance and marketing.
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec1">Types of Machine Learning </h2>
|
|
|
|
<p>
|
|
The approaches to machine learning are many, but are often split into two main categories.
|
|
In <em>supervised learning</em> we know the answer to a problem,
|
|
and let the computer deduce the logic behind it. On the other hand, <em>unsupervised learning</em>
|
|
is a method for finding patterns and relationship in data sets without any prior knowledge of the system.
|
|
Some authours also operate with a third category, namely <em>reinforcement learning</em>. This is a paradigm
|
|
of learning inspired by behavioural psychology, where learning is achieved by trial-and-error,
|
|
solely from rewards and punishment.
|
|
|
|
<p>
|
|
Another way to categorize machine learning tasks is to consider the desired output of a system.
|
|
Some of the most common tasks are:
|
|
|
|
<ul>
|
|
<li> Classification: Outputs are divided into two or more classes. The goal is to produce a model that assigns inputs into one of these classes. An example is to identify digits based on pictures of hand-written ones. Classification is typically supervised learning.</li>
|
|
<li> Regression: Finding a functional relationship between an input data set and a reference data set. The goal is to construct a function that maps input data to continuous output values.</li>
|
|
<li> Clustering: Data are divided into groups with certain common traits, without knowing the different groups beforehand. It is thus a form of unsupervised learning.</li>
|
|
</ul>
|
|
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec2">Different algorithms </h2>
|
|
In this course we will build our machine learning approach on a statistical foundation, with elements
|
|
from data analysis, stochastic processes etc before we proceed with the following machine learning algorithms
|
|
|
|
<ol>
|
|
<li> Linear regression and its variants</li>
|
|
<li> Decision tree algorithms, from simpler to more complex ones</li>
|
|
<li> Nearest neighbors models</li>
|
|
<li> Bayesian statistics</li>
|
|
<li> Support vector machines and finally various variants of</li>
|
|
<li> Artifical neural networks</li>
|
|
</ol>
|
|
|
|
Before we proceed however, there are several practicalities with data analysis and software tools we would
|
|
like to present. These tools will help us in our understanding of various machine learning algorithms.
|
|
|
|
<p>
|
|
Our emphasis here is on understanding the mathematical aspects of different algorithms, however, where possible
|
|
we will emphasize the importance of using available software.
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec3">Software and needed installations </h2>
|
|
We will make intensive use of python as programming language and the myriad of available libraries.
|
|
Furthermore, you will find IPython/Jupyter notebooks invaluable in your work.
|
|
You can run <b>R</b> codes in the Jupyter/IPython notebooks, with the immediate benefit of visualizing your data.
|
|
|
|
<p>
|
|
If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages,
|
|
we recommend that you install the following Python packages via <b>pip</b> as
|
|
|
|
<ol>
|
|
<li> pip install numpy scipy matplotlib ipython scikit-learn mglearn sympy pandas pillow</li>
|
|
</ol>
|
|
|
|
For Python3, replace <b>pip</b> with <b>pip3</b>.
|
|
|
|
<p>
|
|
For OSX user we recommend also, after having installed Xcode, to install <b>brew</b>. Brew allows
|
|
for a seamless installation of additional software via for example
|
|
|
|
<ol>
|
|
<li> brew install python3</li>
|
|
</ol>
|
|
|
|
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution
|
|
you can use <b>pip</b> as well and simply install Python as
|
|
|
|
<ol>
|
|
<li> sudo apt-get install python3 (or python for pyhton2.7)</li>
|
|
</ol>
|
|
|
|
etc etc.
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec4">Python installers </h2>
|
|
If you don't want to perform these operations separately, we recommend two widely used distrubutions which set up
|
|
all relevant dependencies for Python, namely
|
|
|
|
<ol>
|
|
<li> anaconda</li>
|
|
<li> Enthought canopy</li>
|
|
</ol>
|
|
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec5">Installing R and C++ </h2>
|
|
|
|
<p>
|
|
You will also find it convenient to utilize R. Say more about R.
|
|
Jupyter/Ipython notebook allows you run <b>R</b> code interactively in your browser. The software library <b>R</b> is
|
|
tuned to statistically analysis and allows for an easy usage of the tools we will discuss in these texts.
|
|
|
|
<p>
|
|
For the C++ affecianodas, Jupyter/IPython notebook allows you also to install C++ and run codes written in this language
|
|
interactively in the browser. Since we will emphasize writing many of the algorithms yourself, you can thus opt for
|
|
either Python or C++ as programming languages.
|
|
|
|
<p>
|
|
To add more entropy, <b>cython</b> can also be used when running your notebooks. It means that Python with the Jupyter/IPython notebook
|
|
setup allows you to integrate widely popular softwares and tools for scientific computing. With its versatility,
|
|
including symbolic operations, Python offers a unique computational environment. Your Jupyter/IPython notebook
|
|
can easily be converted into a nicely rendered <b>PDF</b> file or a Latex file for further processing.
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec6">Introduction to Jupyter notebook and available tools </h2>
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec7">Representing data, overarching aims </h2>
|
|
<div class="alert alert-block alert-block alert-text-normal">
|
|
<b></b>
|
|
<p>
|
|
<p>
|
|
|
|
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
|
<div class="highlight" style="background: #eee8d5"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">scipy</span> <span style="color: #8B008B; font-weight: bold">import</span> sparse
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
|
|
eye = np.eye(<span style="color: #B452CD">4</span>)
|
|
<span style="color: #8B008B; font-weight: bold">print</span>(eye)
|
|
sparse_mtx = sparse.csr_matrix(eye)
|
|
<span style="color: #8B008B; font-weight: bold">print</span>(sparse_mtx)
|
|
x = np.linspace(-<span style="color: #B452CD">10</span>,<span style="color: #B452CD">10</span>,<span style="color: #B452CD">100</span>)
|
|
y = np.sin(x)
|
|
plt.plot(x,y,marker=<span style="color: #CD5555">'x'</span>)
|
|
plt.show()
|
|
data = {<span style="color: #CD5555">'Name'</span>: [<span style="color: #CD5555">"John"</span>, <span style="color: #CD5555">"Anna"</span>, <span style="color: #CD5555">"Peter"</span>, <span style="color: #CD5555">"Linda"</span>], <span style="color: #CD5555">'Location'</span>: [<span style="color: #CD5555">"Roma"</span>, <span style="color: #CD5555">"Napoli"</span>, <span style="color: #CD5555">"Torino"</span>, <span style="color: #CD5555">"Milano"</span>], <span style="color: #CD5555">'Age'</span>:[<span style="color: #B452CD">51</span>, <span style="color: #B452CD">21</span>, <span style="color: #B452CD">34</span>, <span style="color: #B452CD">45</span>]}
|
|
data_pandas = pd.DataFrame(data)
|
|
display(data_pandas)
|
|
</pre></div>
|
|
|
|
</div>
|
|
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec8">Representing data, more examples </h2>
|
|
<div class="alert alert-block alert-block alert-text-normal">
|
|
<b></b>
|
|
<p>
|
|
<p>
|
|
|
|
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
|
<div class="highlight" style="background: #eee8d5"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">scipy</span> <span style="color: #8B008B; font-weight: bold">import</span> sparse
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">mglearn</span>
|
|
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">sklearn</span>
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.linear_model</span> <span style="color: #8B008B; font-weight: bold">import</span> LinearRegression
|
|
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeRegressor
|
|
x, y = mglearn.datasets.make_wave(n_samples=<span style="color: #B452CD">100</span>)
|
|
line = np.linspace(-<span style="color: #B452CD">3</span>,<span style="color: #B452CD">3</span>,<span style="color: #B452CD">1000</span>,endpoint=<span style="color: #658b00">False</span>).reshape(-<span style="color: #B452CD">1</span>,<span style="color: #B452CD">1</span>)
|
|
reg = DecisionTreeRegressor(min_samples_split=<span style="color: #B452CD">3</span>).fit(x,y)
|
|
plt.plot(line, reg.predict(line), label=<span style="color: #CD5555">"decision tree"</span>)
|
|
regline = LinearRegression().fit(x,y)
|
|
plt.plot(line, regline.predict(line), label= <span style="color: #CD5555">"Linear Rgression"</span>)
|
|
plt.show()
|
|
</pre></div>
|
|
|
|
</div>
|
|
|
|
|
|
<p>
|
|
|
|
<!-- ------------------- end of main content --------------- -->
|
|
|
|
|
|
<center style="font-size:80%">
|
|
<!-- copyright --> © 1999-2017, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
|
|
</center>
|
|
|
|
|
|
</body>
|
|
</html>
|
|
|
|
|