lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

More text on how to read

Browse Source
This commit is contained in:
mhjensen
2018-05-26 13:05:30 -04:00
parent ccccc6ed1b
commit f90a232c25
10 changed files with 1844 additions and 1473 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/How2ReadData/How2ReadData.do.txt
+148 -44
View File
@@ -143,21 +143,40 @@ We start with perhaps our simplest possible example, using _scikit-learn_ to per
What follows is a simple Python code where we have defined function $y$ in terms of the variable $x$. Both are defined as vectors of dimension $1\times 100$. The entries to the vector $\hat{x}$ are given by random numbers generated with a uniform distribution with entries $x_i \in [0,1]$ (more about probability distribution functions later). These values are then used to define a function $y(x)$ (tabulated again as a vector) with a linear dependence on $x$ plus a random noise added via the normal distribution.
The Numpy functions are imported used the _import numpy as np_ statement and the
random number generator for the uniform distribution is called using the function
_np.random.rand()_, where we specificy that we want $100$ random variables.
Using Numpy we define automatically an array with the specified number of elements, $100$ in our case.
Using the Numpy function _randn()_ we can compute random numbers with the normal distribution (mean value equal to zero and variance set to one) and produce the values of $y$
assuming a linear dependence as function of $x$
The Numpy functions are imported used the _import numpy as np_
statement and the random number generator for the uniform distribution
is called using the function _np.random.rand()_, where we specificy
that we want $100$ random variables. Using Numpy we define
automatically an array with the specified number of elements, $100$ in
our case. With the Numpy function _randn()_ we can compute random
numbers with the normal distribution (mean value $\mu$ equal to zero and
variance $\sigma^2$ set to one) and produce the values of $y$ assuming a linear
dependence as function of $x$
!bt
\[
y = 2*x+N(0,1),
y = 2x+N(0,1),
\]
!et
where $N(0,1)$ represents random numbers generated by the normal distribution.
From _scikit_learn_ we import then the _LinearRegression_ functionality and make a prediction $\tilde{y} = \alpha + \beta x$ using the function _fit(x,y)_
We make also a prediction
where $N(0,1)$ represents random numbers generated by the normal
distribution. From _scikit-learn_ we import then the
_LinearRegression_ functionality and make a prediction $\tilde{y} =
\alpha + \beta x$ using the function _fit(x,y)_. We call the set of
data $(\hat{x},\hat{y})$ for our training data. The Python package
_scikit-learn_ has also a functionality which extracts the above
fitting parameters $\alpha$ and $\beta$ (see below). Later we will
distinguish between training data and test data.
For plotting we use the Python package
"matplotlib":"https://matplotlib.org/" which produces publication
quality figures. Feel free to explore the extensive
"gallery":"https://matplotlib.org/gallery/index.html" of examples. In
this example we plot our original values of $x$ and $y$ as well as the
prediction _ypredict_ ($\tilde{y}$), which attempts at fitting our
data with a straight line.
The Python follows here.
!bc pycod
# Importing various packages
import numpy as np
@@ -168,22 +187,135 @@ x = np.random.rand(100,1)
y = 2*x+np.random.randn(100,1)
linreg = LinearRegression()
linreg.fit(x,y)
xnew = np.array([[0],[2]])
xnew = np.array([[0],[1]])
ypredict = linreg.predict(xnew)
plt.plot(xnew, ypredict, "r-")
plt.plot(x, y ,'ro')
plt.axis([0,2.0,0, 15.0])
plt.axis([0,1.0,0, 5.0])
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.title(r'Random numbers ')
plt.title(r'Simple Linear Regression')
plt.show()
!ec
Add paly around with various forms, also relative error, plot that and other ways to estimate by the eye the qaulity of the fit
This example serves several aims. It allows us to demonstrate several
aspects of data analysis and later machine learning algorithms. The
immediate visualization shows that our linear fit is not
impressive. It goes through the data points, but there are many
outliers which are not reproduced by our linear regression. We could
now play around with this small program and change for example the
factor in front of $x$ and the normal distribution. Try to change the
function $y$ to
!split
===== Introduction to Jupyter notebook and available tools =====
!bt
\[
y = 10x+0.01N(0,1),
\]
!et
where $x$ is defined as before. Does the fit look better? Indeed, by
reducing the role of the normal distribution we see immediately that
our linear prediction seemingly reproduces better the training
set. However, this testing 'by the eye' is obviouly not satisfactory in the
long run. Here we have only defined the training data and our model, and
have not discussed a more rigorous approach to the _cost_ function.
We need more rigorous criteria in defining whether we have succeed or
not in modeling our training data. You will be surprised to see that
many scientists seldomly venture beyond this 'by the eye' approach. A
standard approach for the *cost* function is the so-called $\chi^2$
function
!bt
\[ \chi^2 = \frac{1}{n}
\sum_{i=0}^{n-1}\frac{(y_i-\tilde{y}_i)^2}{\sigma_i^2},
\]
!et
where $\sigma_i^2$ is the variance (to be defined later) of the entry
$y_i$. We may not know the explicit value of $\sigma_i^2$, it serves
however the aim of scaling the equations and make the cost function
dimensionless.
Minimizing the cost function is a central aspect of
our discussions to come. Finding its minima as function of the model
parameters ($\alpha$ and $\beta$ in our case) will be a recurring
theme in these series of lectures. Essentially all machine learning
algorithms we will discuss center around the minimization of the
chosen cost function. This depends in turn on our specific
model for describing the data, a typical situation in supervised
learning. Automatizing the search for the minima of the cost function is a
central ingredient in all algorithms. Typical methods which are
employed are various variants of _gradient_ methods. These will be
discussed in more detail later. Again, you'll be surprised to hear that
many practitioners minimize the above function ''by the eye', popularly dubbed as
'chi by the eye'. That is, change a parameter and see (visually and numerically) that
the your $\chi^2$ function becomes smaller.
There are many ways to define the cost function. A simpler approach is to look at the relative difference between the training data and the predicted data, that is we define
the relative error as
!bt
\[
\epsilon_{\mathrm{relative}}= \frac{\vert \hat{y} -\hat{\tilde{y}}\vert}{\vert \hat{y}\vert}.
\]
!et
We can modify easily the above Python code and plot the relative instead
!bc pycod
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
x = np.random.rand(100,1)
y = 5*x+0.01*np.random.randn(100,1)
linreg = LinearRegression()
linreg.fit(x,y)
ypredict = linreg.predict(x)
plt.plot(x, np.abs(ypredict-y)/abs(y), "ro")
plt.axis([0,1.0,0.0, 0.5])
plt.xlabel(r'$x$')
plt.ylabel(r'$\epsilon_{\mathrm{relative}}$')
plt.title(r'Relative error')
plt.show()
!ec
Depending on the parameter in front of the normal distribution, we may have a small or larger relative error. Try to play around with different training data sets and study (graphically) the value of the relative error.
As mentioned above, _scikit-learn_ has an impressive functionality.
We can for example extract the values of $\alpha$ and $\beta$ and their error estimates,
or the variance and standard deviation and many other properties from the statistical data analysis. Here we show an example of the functionality of scikit-learn.
!bc pycod
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
x = np.random.rand(100,1)
y = 2*x+np.random.randn(100,1)
linreg = LinearRegression()
linreg.fit(x,y)
ypredict = linreg.predict(x)
print('Coefficients: \n', linreg.coef_)
# The mean squared error
print("Mean squared error: %.2f" % mean_squared_error(y, ypredict))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % r2_score(y, ypredict))
plt.plot(x, ypredict, "r-")
plt.plot(x, y ,'ro')
plt.axis([0,1.0,0, 5.0])
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.title(r'Linear Regression fit ')
plt.show()
!ec
We will come to the definition of these outputs later.
Another useful Python package is
"pandas":"https://pandas.pydata.org/", which is an open source library
providing high-performance, easy-to-use data structures and data
analysis tools for Python. The following simple example shows an example on how we can, in an easy way make tables of our data. Here we define a data set which includes names, city of residence and age, and displays the data in an easy to read way. We will see repeated use of _pandas_, in particular in connection with classification of data.
!bblock
!bc pycod
@@ -208,34 +340,6 @@ display(data_pandas)
!split
===== Simple linear regression model using _scikit-learn_ =====
Add info about the equations
!bc pycod
# Importing various packages
from random import random, seed
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
x = 2*np.random.rand(100,1)
y = 4+3*x+np.random.randn(100,1)
linreg = LinearRegression()
linreg.fit(x,y)
xnew = np.array([[0],[2]])
ypredict = linreg.predict(xnew)
plt.plot(xnew, ypredict, "r-")
plt.plot(x, y ,'ro')
plt.axis([0,2.0,0, 15.0])
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.title(r'Random numbers ')
plt.show()
!ec
!split
===== Predator-Prey model from ecology =====