added introduction chapter
This commit is contained in:
mhjensen
@@ -0,0 +1,157 @@
|
||||
TITLE: Data Analysis and Machine Learning: Representing data
|
||||
AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
|
||||
DATE: today
|
||||
|
||||
|
||||
!split
|
||||
===== Introduction =====
|
||||
|
||||
Statistics, data science and machine learning form important fields of
|
||||
research in modern science. They describe how to learn and make
|
||||
predictions from data, as well as allowing us to extract important
|
||||
correlations about physical process and the underlying laws of motion
|
||||
in large data sets. The latter, big data sets, appear
|
||||
frequently in essentially all disciplines, from the traditional Science,
|
||||
Technology, Mathematics and Engineering fields to Life Science, Law, education research,
|
||||
the Humanities and
|
||||
the Social Sciences. It has become more and more common to see
|
||||
research projects on big data in for example the Social
|
||||
Sciences where extracting patterns from complicated survey data is one of many research directions.
|
||||
Having a solid grasp of data analysis and machine learning
|
||||
is thus becoming central to scientific computing in many
|
||||
fields, and competences and skills within the fields of machine learning
|
||||
and scientific computing are nowadays strongly requested by many
|
||||
potential employers. The latter cannot be overstated, familiarity with
|
||||
machine learning has almost become a prerequisite for many of the most
|
||||
exciting employment opportunities, whether they are in bioinformatics,
|
||||
life science, physics or finance, in the private or the public
|
||||
sector. This author has had several students or met students who have
|
||||
been hired recently based on their skills and competences in
|
||||
scientific computing and data science, often with marginal knowledge
|
||||
of machine learning.
|
||||
|
||||
Machine learning is a subfield of computer science, and is closely
|
||||
related to computational statistics. It evolved from the study of
|
||||
pattern recognition in artificial intelligence (AI) research, and has
|
||||
made contributions to AI tasks like computer vision, natural language
|
||||
processing and speech recognition.
|
||||
Machine learning represents the
|
||||
science of giving computers the ability to learn without being
|
||||
explicitly programmed. The idea is that there exist generic
|
||||
algorithms which can be used to find patterns in a broad class of data
|
||||
sets without having to write code specifically for each problem. The
|
||||
algorithm will build its own logic based on the data.
|
||||
|
||||
Machine learning is an extremely rich field, in spite of its young age. The
|
||||
increases we have seen during the last three decades in computational
|
||||
capabilities have been followed by developments of methods and
|
||||
techniques for analyzing and handling large date sets, relying heavily
|
||||
on statistics, computer science and mathematics. The field is rather
|
||||
new and developing rapidly. Popular software packages written in
|
||||
Python for machine learning like "Scikit-learn":"http://scikit-learn.org/stable/", "Tensorflow":"https://www.tensorflow.org/",
|
||||
"PyTorch":"http://pytorch.org/" and "Keras":"https://keras.io/", all freely available at their respective GitHub sites,
|
||||
encompass communities of developers in the thousands or more. And the number
|
||||
of code developers and contributors keeps increasing. Not all the
|
||||
algorithms and methods can be given a rigorous mathematical
|
||||
justification, opening up thereby large rooms for experimenting
|
||||
and trial and error and thereby exciting new developments.
|
||||
However, a solid command of linear algebra, multivariate theory,
|
||||
probability theory, statistical data analysis,
|
||||
understanding errors and Monte Carlo methods are central elements in a proper understanding of many of
|
||||
algorithms and methods we will discuss.
|
||||
|
||||
!split
|
||||
===== Learning outcomes =====
|
||||
|
||||
These lectures aim at giving you an overview of central aspects of
|
||||
statistical data analysis as well as some of the central algorithms
|
||||
used in machine learning. We will introduce a variety of central
|
||||
algorithms and methods essential for studies of data analysis and
|
||||
machine learning.
|
||||
|
||||
Hands-on projects and experimenting with data and algorithms plays a central role in
|
||||
these lectures, and our hope is, through the various
|
||||
projects and exercies, to expose you to fundamental
|
||||
research problems in these fields, with the aim to reproduce state of
|
||||
the art scientific results. You will learn to develop and
|
||||
structure large codes for studying these systems, get acquainted with
|
||||
computing facilities and learn to handle large scientific projects. A
|
||||
good scientific and ethical conduct is emphasized throughout the
|
||||
course. More specifically, you will
|
||||
|
||||
o learn about basic data analysis, Bayesian statistics, Monte Carlo methods, data optimization and machine learning;
|
||||
o be capable of extending the acquired knowledge to other systems and cases;
|
||||
o Have an understanding of central algorithms used in data analysis and machine learning;
|
||||
o Gain knowledge of central aspects of Monte Carlo methods, Markov chains, Gibbs samplers and their possible applications, from numerical integration to simulation of stock markets;
|
||||
o Understand methods for regression and classification;
|
||||
o Learn about neural network, genetic algorithms and Boltzmann machines;
|
||||
o Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++, in addition to a basic knowledge of linear algebra (typically taught during the first one or two years of undergraduate studies).
|
||||
|
||||
There are several topics we will cover here, spanning from a
|
||||
statistical data analysis and its basic concepts such expectation
|
||||
values, variance, covariance, correlation functions and errors, via
|
||||
well-known probability distribution functions like uniform
|
||||
distribution, the binomial distribution, the Poisson distribution and
|
||||
simple and multivariate normal distributions to central elements of
|
||||
Bayesian statistics and modeling. We will also remind the reader about
|
||||
central elements from linear algebra and standard methods based on
|
||||
linear algebra used to fit functions such Cubic splines and gradient
|
||||
methods for data optimization and the Singular-value decomposition and
|
||||
least square methods for parameterizing data.
|
||||
|
||||
We will also cover Monte Carlo methods, Markov chains, well-known
|
||||
algorithms for sampling stochastic events like the Metropolis-Hastings
|
||||
and Gibbs sampling methods. An important aspect of all our
|
||||
calculations is a proper estimation of errors. Here we will also
|
||||
discuss famous resampling techniques like the blocking, bootstrapping
|
||||
and jackknife methods.
|
||||
|
||||
The second part of the material covers several algorithms used in
|
||||
machine learning.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
!split
|
||||
===== Types of Machine Learning =====
|
||||
|
||||
|
||||
The approaches to machine learning are many, but are often split into two main categories.
|
||||
In *supervised learning* we know the answer to a problem,
|
||||
and let the computer deduce the logic behind it. On the other hand, *unsupervised learning*
|
||||
is a method for finding patterns and relationship in data sets without any prior knowledge of the system.
|
||||
Some authours also operate with a third category, namely *reinforcement learning*. This is a paradigm
|
||||
of learning inspired by behavioral psychology, where learning is achieved by trial-and-error,
|
||||
solely from rewards and punishment.
|
||||
|
||||
Another way to categorize machine learning tasks is to consider the desired output of a system.
|
||||
Some of the most common tasks are:
|
||||
|
||||
* Classification: Outputs are divided into two or more classes. The goal is to produce a model that assigns inputs into one of these classes. An example is to identify digits based on pictures of hand-written ones. Classification is typically supervised learning.
|
||||
|
||||
* Regression: Finding a functional relationship between an input data set and a reference data set. The goal is to construct a function that maps input data to continuous output values.
|
||||
|
||||
* Clustering: Data are divided into groups with certain common traits, without knowing the different groups beforehand. It is thus a form of unsupervised learning.
|
||||
|
||||
|
||||
The methods we cover have three main topics in common, irrespective of
|
||||
whether we deal with supervised or unsupervised learning. The first
|
||||
ingredient is normally our data set, the second is a model which is
|
||||
normally a function of some parameters. The last ingredient is a
|
||||
so-called _cost_ function which allows us to present an estimate on
|
||||
how good our model is in reproducing the data it is supposed to train.
|
||||
|
||||
Here we will build our machine learning approach on elements of the
|
||||
statistical foundation discussed above, with elements from data
|
||||
analysis, stochastic processes etc. We will discuss the following
|
||||
machine learning algorithms
|
||||
|
||||
o Linear regression and its variants, in essence polynomial regression
|
||||
o Decision tree algorithms, from simpler to more complex ones
|
||||
o Nearest neighbors models
|
||||
o Bayesian statistics and regression
|
||||
o Support vector machines and finally various variants of
|
||||
o Artifical neural networks and deep learning
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
\mode<presentation>
|
||||
\usecolortheme[rgb={0.8, 0.2, 0}]{structure}
|
||||
\usefonttheme[onlysmall]{structurebold}
|
||||
|
||||
\setbeamertemplate{navigation symbols}{}
|
||||
%\setbeamertemplate{footline}[frame number]
|
||||
|
||||
\usepackage{tikz}
|
||||
\usetikzlibrary{arrows,shapes,backgrounds,decorations,mindmap}
|
||||
|
||||
\mode
|
||||
<all>
|
||||
@@ -0,0 +1,15 @@
|
||||
\mode<presentation>
|
||||
|
||||
\useoutertheme{smoothbars}
|
||||
\useinnertheme[shadow=true]{rounded}
|
||||
\usecolortheme{orchid}
|
||||
\usecolortheme{whale}
|
||||
\usecolortheme[rgb={0.7, 0.2, 0}]{structure} % (darker red)
|
||||
\useoutertheme{shadow}
|
||||
\usefonttheme[onlysmall]{structurebold}
|
||||
|
||||
\setbeamercolor{title}{use=structure,fg=white,bg=structure.fg}
|
||||
\setbeamerfont{block title}{size={}}
|
||||
|
||||
\mode
|
||||
<all>
|
||||
Executable
+3
@@ -0,0 +1,3 @@
|
||||
#!/bin/sh
|
||||
doconce clean
|
||||
rm -rf *.pdf *.tex ipynb*.tar.gz *.html ._*.html *~ reveal.js Trash README.txt
|
||||
Executable
+118
@@ -0,0 +1,118 @@
|
||||
#!/bin/sh
|
||||
set -x
|
||||
|
||||
function system {
|
||||
"$@"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "make.sh: unsuccessful command $@"
|
||||
echo "abort!"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
if [ $# -eq 0 ]; then
|
||||
echo 'bash make.sh slides1|slides2'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
name=$1
|
||||
rm -f *.tar.gz
|
||||
|
||||
opt="--encoding=utf-8"
|
||||
# Note: Makefile examples contain constructions like ${PROG} which
|
||||
# looks like Mako constructions, but they are not. Use --no_mako
|
||||
# to turn off Mako processing.
|
||||
opt="--no_mako"
|
||||
|
||||
rm -f *.aux
|
||||
|
||||
|
||||
html=${name}-reveal
|
||||
system doconce format html $name --pygments_html_style=perldoc --keep_pygments_html_bg --html_links_in_new_window --html_output=$html $opt
|
||||
system doconce slides_html $html reveal --html_slide_theme=beige
|
||||
|
||||
# Plain HTML documents
|
||||
|
||||
html=${name}-solarized
|
||||
system doconce format html $name --pygments_html_style=perldoc --html_style=solarized3 --html_links_in_new_window --html_output=$html $opt
|
||||
system doconce split_html $html.html --method=space10
|
||||
|
||||
html=${name}
|
||||
system doconce format html $name --pygments_html_style=default --html_style=bloodish --html_links_in_new_window --html_output=$html $opt
|
||||
system doconce split_html $html.html --method=space10
|
||||
|
||||
# Bootstrap style
|
||||
html=${name}-bs
|
||||
system doconce format html $name --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=$html $opt
|
||||
#system doconce split_html $html.html --method=split --pagination --nav_button=bottom
|
||||
|
||||
# IPython notebook
|
||||
system doconce format ipynb $name $opt
|
||||
|
||||
# LaTeX Beamer slides
|
||||
beamertheme=red_plain
|
||||
system doconce format pdflatex $name --latex_title_layout=beamer --latex_table_format=footnotesize $opt
|
||||
system doconce ptex2tex $name envir=minted
|
||||
# Add special packages
|
||||
doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex
|
||||
system doconce slides_beamer $name --beamer_slide_theme=$beamertheme
|
||||
system pdflatex -shell-escape ${name}
|
||||
system pdflatex -shell-escape ${name}
|
||||
cp $name.pdf ${name}-beamer.pdf
|
||||
cp $name.tex ${name}-beamer.tex
|
||||
|
||||
# Handouts
|
||||
system doconce format pdflatex $name --latex_title_layout=beamer --latex_table_format=footnotesize $opt
|
||||
system doconce ptex2tex $name envir=minted
|
||||
# Add special packages
|
||||
doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex
|
||||
system doconce slides_beamer $name --beamer_slide_theme=red_shadow --handout
|
||||
system pdflatex -shell-escape $name
|
||||
pdflatex -shell-escape $name
|
||||
pdflatex -shell-escape $name
|
||||
pdfnup --nup 2x3 --frame true --delta "1cm 1cm" --scale 0.9 --outfile ${name}-beamer-handouts2x3.pdf ${name}.pdf
|
||||
rm -f ${name}.pdf
|
||||
|
||||
# Ordinary plain LaTeX document
|
||||
rm -f *.aux # important after beamer
|
||||
system doconce format pdflatex $name --minted_latex_style=trac --latex_admon=paragraph $opt
|
||||
system doconce ptex2tex $name envir=minted
|
||||
# Add special packages
|
||||
doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex
|
||||
doconce replace 'section{' 'section*{' $name.tex
|
||||
pdflatex -shell-escape $name
|
||||
pdflatex -shell-escape $name
|
||||
mv -f $name.pdf ${name}-minted.pdf
|
||||
cp $name.tex ${name}-plain-minted.tex
|
||||
|
||||
|
||||
|
||||
# Publish
|
||||
dest=../../pub
|
||||
if [ ! -d $dest/$name ]; then
|
||||
mkdir $dest/$name
|
||||
mkdir $dest/$name/pdf
|
||||
mkdir $dest/$name/html
|
||||
mkdir $dest/$name/ipynb
|
||||
fi
|
||||
cp ${name}*.pdf $dest/$name/pdf
|
||||
cp -r ${name}*.html ._${name}*.html reveal.js $dest/$name/html
|
||||
|
||||
# Figures: cannot just copy link, need to physically copy the files
|
||||
if [ -d fig-${name} ]; then
|
||||
if [ ! -d $dest/$name/html/fig-$name ]; then
|
||||
mkdir $dest/$name/html/fig-$name
|
||||
fi
|
||||
cp -r fig-${name}/* $dest/$name/html/fig-$name
|
||||
fi
|
||||
|
||||
cp ${name}.ipynb $dest/$name/ipynb
|
||||
ipynb_tarfile=ipynb-${name}-src.tar.gz
|
||||
if [ ! -f ${ipynb_tarfile} ]; then
|
||||
cat > README.txt <<EOF
|
||||
This IPython notebook ${name}.ipynb does not require any additional
|
||||
programs.
|
||||
EOF
|
||||
tar czf ${ipynb_tarfile} README.txt
|
||||
fi
|
||||
cp ${ipynb_tarfile} $dest/$name/ipynb
|
||||
Reference in New Issue
Block a user