lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-08-25 12:55:11 +02:00
parent 262fca1d27
commit e831542c43
79 changed files with 4174 additions and 6638 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/week35/week35.do.txt
+3 -157
View File
@@ -18,16 +18,15 @@ o Monday: Ridge and Lasso regression and Singular Value Decomposition
o See lecture notes for week 35 at URL:"https://compphysics.github.io/MachineLearning/doc/web/course.html"
o Goodfellow, Bengio and Courville, Deep Learning, chapter 2 on linear algebra and sections 3.1-3.10 on elements of statistics (background)
o Raschka et al on preprocessing of data, relevant for exercise 3 this week, see chapter 4.
o For exercise 1 of week 35, the book by A. Aldo Faisal, Cheng Soon Ong, and Marc Peter Deisenroth on the Mathematics of Machine Learning, may be very relevant. In particular chapter 5 at URL"https://mml-book.github.io/" (section 5.5 on derivatives) is very useful for exercise 1 this coming week.
!split
===== Why Linear Regression (aka Ordinary Least Squares and family), repeat from last week =====
===== For exercise sessions: Why Linear Regression (aka Ordinary Least Squares and family), repeat from last week =====
We need first a reminder from last week about linear regression.
@@ -1376,159 +1375,6 @@ regression next week.
!split
===== The Boston housing data example =====
The Boston housing
data set was originally a part of UCI Machine Learning Repository
and has been removed now. The data set is now included in _Scikit-Learn_'s
library. There are 506 samples and 13 feature (predictor) variables
in this data set. The objective is to predict the value of prices of
the house using the features (predictors) listed here.
The features/predictors are
o CRIM: Per capita crime rate by town
o ZN: Proportion of residential land zoned for lots over 25000 square feet
o INDUS: Proportion of non-retail business acres per town
o CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
o NOX: Nitric oxide concentration (parts per 10 million)
o RM: Average number of rooms per dwelling
o AGE: Proportion of owner-occupied units built prior to 1940
o DIS: Weighted distances to five Boston employment centers
o RAD: Index of accessibility to radial highways
o TAX: Full-value property tax rate per USD10000
o B: $1000(Bk - 0.63)^2$, where $Bk$ is the proportion of [people of African American descent] by town
o LSTAT: Percentage of lower status of the population
o MEDV: Median value of owner-occupied homes in USD 1000s
!split
===== Housing data, the code =====
We start by importing the libraries
!bc pycod
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
!ec
and load the Boston Housing DataSet from _Scikit-Learn_
!bc pycod
from sklearn.datasets import load_boston
boston_dataset = load_boston()
# boston_dataset is a dictionary
# let's check what it contains
boston_dataset.keys()
!ec
Then we invoke Pandas
!bc pycod
boston = pd.DataFrame(boston_dataset.data, columns=boston_dataset.feature_names)
boston.head()
boston['MEDV'] = boston_dataset.target
!ec
and preprocess the data
!bc pycod
# check for missing values in all the columns
boston.isnull().sum()
!ec
We can then visualize the data
!bc pycod
# set the size of the figure
sns.set(rc={'figure.figsize':(11.7,8.27)})
# plot a histogram showing the distribution of the target values
sns.distplot(boston['MEDV'], bins=30)
plt.show()
!ec
It is now useful to look at the correlation matrix
!bc pycod
# compute the pair wise correlation for all columns
correlation_matrix = boston.corr().round(2)
# use the heatmap function from seaborn to plot the correlation matrix
# annot = True to print the values inside the square
sns.heatmap(data=correlation_matrix, annot=True)
!ec
From the above coorelation plot we can see that _MEDV_ is strongly correlated to _LSTAT_ and _RM_. We see also that _RAD_ and _TAX_ are stronly correlated, but we don't include this in our features together to avoid multi-colinearity
!bc pycod
plt.figure(figsize=(20, 5))
features = ['LSTAT', 'RM']
target = boston['MEDV']
for i, col in enumerate(features):
plt.subplot(1, len(features) , i+1)
x = boston[col]
y = target
plt.scatter(x, y, marker='o')
plt.title(col)
plt.xlabel(col)
plt.ylabel('MEDV')
!ec
Now we start training our model
!bc pycod
X = pd.DataFrame(np.c_[boston['LSTAT'], boston['RM']], columns = ['LSTAT','RM'])
Y = boston['MEDV']
!ec
We split the data into training and test sets
!bc pycod
from sklearn.model_selection import train_test_split
# splits the training and test data set in 80% : 20%
# assign random_state to any value.This ensures consistency.
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, random_state=5)
print(X_train.shape)
print(X_test.shape)
print(Y_train.shape)
print(Y_test.shape)
!ec
Then we use the linear regression functionality from _Scikit-Learn_
!bc pycod
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
lin_model = LinearRegression()
lin_model.fit(X_train, Y_train)
# model evaluation for training set
y_train_predict = lin_model.predict(X_train)
rmse = (np.sqrt(mean_squared_error(Y_train, y_train_predict)))
r2 = r2_score(Y_train, y_train_predict)
print("The model performance for training set")
print("--------------------------------------")
print('RMSE is {}'.format(rmse))
print('R2 score is {}'.format(r2))
print("\n")
# model evaluation for testing set
y_test_predict = lin_model.predict(X_test)
# root mean square error of the model
rmse = (np.sqrt(mean_squared_error(Y_test, y_test_predict)))
# r-squared score of the model
r2 = r2_score(Y_test, y_test_predict)
print("The model performance for testing set")
print("--------------------------------------")
print('RMSE is {}'.format(rmse))
print('R2 score is {}'.format(r2))
!ec
!bc pycod
# plotting the y_test vs y_pred
# ideally should have been a straight line
plt.scatter(Y_test, y_test_predict)
plt.show()
!ec