We start with perhaps our simplest possible example, using scikit-learn to perform linear regression analysis on a data set produced by us. What follows is a simple Python code where we have defined function \( y \) in terms of the variable \( x \). Both are defined as vectors of dimension \( 1\times 100 \). The entries to the vector \( \hat{x} \) are given by random numbers generated with a uniform distribution with entries \( x_i \in [0,1] \) (more about probability distribution functions later). These values are then used to define a function \( y(x) \) (tabulated again as a vector) with a linear dependence on \( x \) plus a random noise added via the normal distribution.
The Numpy functions are imported used the import numpy as np statement and the random number generator for the uniform distribution is called using the function np.random.rand(), where we specificy that we want \( 100 \) random variables. Using Numpy we define automatically an array with the specified number of elements, \( 100 \) in our case. With the Numpy function randn() we can compute random numbers with the normal distribution (mean value \( \mu \) equal to zero and variance \( \sigma^2 \) set to one) and produce the values of \( y \) assuming a linear dependence as function of \( x \) $$ y = 2x+N(0,1), $$
where \( N(0,1) \) represents random numbers generated by the normal distribution. From scikit-learn we import then the LinearRegression functionality and make a prediction \( \tilde{y} = \alpha + \beta x \) using the function fit(x,y). We call the set of data \( (\hat{x},\hat{y}) \) for our training data. The Python package scikit-learn has also a functionality which extracts the above fitting parameters \( \alpha \) and \( \beta \) (see below). Later we will distinguish between training data and test data.
For plotting we use the Python package matplotlib which produces publication quality figures. Feel free to explore the extensive gallery of examples. In this example we plot our original values of \( x \) and \( y \) as well as the prediction ypredict (\( \tilde{y} \)), which attempts at fitting our data with a straight line.
The Python code follows here.
# Importing various packages
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
x = np.random.rand(100,1)
y = 2*x+np.random.randn(100,1)
linreg = LinearRegression()
linreg.fit(x,y)
xnew = np.array([[0],[1]])
ypredict = linreg.predict(xnew)
plt.plot(xnew, ypredict, "r-")
plt.plot(x, y ,'ro')
plt.axis([0,1.0,0, 5.0])
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.title(r'Simple Linear Regression')
plt.show()