lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cleaning up typos

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-10-04 20:48:46 +02:00
parent 405a0f0348
commit 623ed51668
10 changed files with 315 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/Projects/2021/Project1/html/._Project1-bs000.html
+6 -6
View File
@@ -168,7 +168,7 @@ MathJax.Hub.Config({
<center><b>University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 14, 2021</h4></center> <!-- date -->
<center><h4>Oct 4, 2021</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
@@ -284,21 +284,21 @@ analysis using polynomials in \( x \) and \( y \) up to fifth order. Find the
<a href="https://en.wikipedia.org/wiki/Confidence_interval" target="_self">confidence intervals</a> of the parameters (estimators) \( \beta \) by computing their
variances, evaluate the Mean Squared error (MSE)
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
$$ MSE(\boldsymbol{y},\tilde{\boldsymbol{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
<p>
and the \( R^2 \) score function. If \( \tilde{\hat{y}}_i \) is the predicted
and the \( R^2 \) score function. If \( \tilde{\boldsymbol{y}}_i \) is the predicted
value of the \( i-th \) sample and \( y_i \) is the corresponding true value,
then the score \( R^2 \) is defined as
$$
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\boldsymbol{y}, \tilde{\boldsymbol{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
$$
<p>
where we have defined the mean value of \( \hat{y} \) as
where we have defined the mean value of \( \boldsymbol{y} \) as
$$
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
@@ -406,7 +406,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
<p>
Implement the \( k \)-fold cross-validation algorithm (write your own
doc/Projects/2021/Project1/html/Project1-bs.html
+6 -6
View File
@@ -168,7 +168,7 @@ MathJax.Hub.Config({
<center><b>University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 14, 2021</h4></center> <!-- date -->
<center><h4>Oct 4, 2021</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
@@ -284,21 +284,21 @@ analysis using polynomials in \( x \) and \( y \) up to fifth order. Find the
<a href="https://en.wikipedia.org/wiki/Confidence_interval" target="_self">confidence intervals</a> of the parameters (estimators) \( \beta \) by computing their
variances, evaluate the Mean Squared error (MSE)
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
$$ MSE(\boldsymbol{y},\tilde{\boldsymbol{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
<p>
and the \( R^2 \) score function. If \( \tilde{\hat{y}}_i \) is the predicted
and the \( R^2 \) score function. If \( \tilde{\boldsymbol{y}}_i \) is the predicted
value of the \( i-th \) sample and \( y_i \) is the corresponding true value,
then the score \( R^2 \) is defined as
$$
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\boldsymbol{y}, \tilde{\boldsymbol{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
$$
<p>
where we have defined the mean value of \( \hat{y} \) as
where we have defined the mean value of \( \boldsymbol{y} \) as
$$
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
@@ -406,7 +406,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
<p>
Implement the \( k \)-fold cross-validation algorithm (write your own
doc/Projects/2021/Project1/html/Project1.html
+6 -6
View File
@@ -126,7 +126,7 @@ MathJax.Hub.Config({
<center><b>University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 14, 2021</h4></center> <!-- date -->
<center><h4>Oct 4, 2021</h4></center> <!-- date -->
<br>
<h2 id="regression-analysis-and-resampling-methods">Regression analysis and resampling methods </h2>
@@ -240,21 +240,21 @@ analysis using polynomials in \( x \) and \( y \) up to fifth order. Find the
<a href="https://en.wikipedia.org/wiki/Confidence_interval" target="_blank">confidence intervals</a> of the parameters (estimators) \( \beta \) by computing their
variances, evaluate the Mean Squared error (MSE)
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
$$ MSE(\boldsymbol{y},\tilde{\boldsymbol{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
<p>
and the \( R^2 \) score function. If \( \tilde{\hat{y}}_i \) is the predicted
and the \( R^2 \) score function. If \( \tilde{\boldsymbol{y}}_i \) is the predicted
value of the \( i-th \) sample and \( y_i \) is the corresponding true value,
then the score \( R^2 \) is defined as
$$
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\boldsymbol{y}, \tilde{\boldsymbol{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
$$
<p>
where we have defined the mean value of \( \hat{y} \) as
where we have defined the mean value of \( \boldsymbol{y} \) as
$$
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
@@ -362,7 +362,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
<p>
Implement the \( k \)-fold cross-validation algorithm (write your own
doc/Projects/2021/Project1/ipynb/Project1.ipynb
+7 -6
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**[Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html)**, University of Oslo, Norway\n",
"\n",
"Date: **Sep 14, 2021**\n",
"Date: **Oct 4, 2021**\n",
"\n",
"## Regression analysis and resampling methods\n",
"\n",
@@ -149,7 +149,7 @@
"metadata": {},
"source": [
"$$\n",
"MSE(\\hat{y},\\hat{\\tilde{y}}) = \\frac{1}{n}\n",
"MSE(\\boldsymbol{y},\\tilde{\\boldsymbol{y}}) = \\frac{1}{n}\n",
"\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n",
"$$"
]
@@ -158,7 +158,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"and the $R^2$ score function. If $\\tilde{\\hat{y}}_i$ is the predicted\n",
"and the $R^2$ score function. If $\\tilde{\\boldsymbol{y}}_i$ is the predicted\n",
"value of the $i-th$ sample and $y_i$ is the corresponding true value,\n",
"then the score $R^2$ is defined as"
]
@@ -168,7 +168,7 @@
"metadata": {},
"source": [
"$$\n",
"R^2(\\hat{y}, \\tilde{\\hat{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
"R^2(\\boldsymbol{y}, \\tilde{\\boldsymbol{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
"$$"
]
},
@@ -176,7 +176,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"where we have defined the mean value of $\\hat{y}$ as"
"where we have defined the mean value of $\\boldsymbol{y}$ as"
]
},
{
@@ -200,6 +200,7 @@
"sure you have installed it). This function is called\n",
"$train\\_test\\_split$. **You should present a critical discussion of why and how you have scaled or not scaled the data**.\n",
"\n",
"\n",
"It is normal in essentially all Machine Learning studies to split the\n",
"data in a training set and a test set (eventually also an additional\n",
"validation set). There\n",
@@ -311,7 +312,7 @@
"The aim here is to write your own code for another widely popular\n",
"resampling technique, the so-called cross-validation method. Again,\n",
"before you start with cross-validation approach, you should scale your\n",
"data.\n",
"data if you think this is needed.\n",
"\n",
"Implement the $k$-fold cross-validation algorithm (write your own\n",
"code) and evaluate again the MSE function resulting\n",
doc/Projects/2021/Project1/ipynb/ipynb-Project1-src.tar.gz
BIN
View File
Binary file not shown.
doc/Projects/2021/Project1/pdf/Project1.p.tex
+7 -6
View File
@@ -132,7 +132,7 @@ Project 1 on Machine Learning, deadline October 4, 2021
% --- begin date ---
\begin{center}
Sep 14, 2021
Oct 4, 2021
\end{center}
% --- end date ---
@@ -242,19 +242,19 @@ analysis using polynomials in $x$ and $y$ up to fifth order. Find the
\href{{https://en.wikipedia.org/wiki/Confidence_interval}}{confidence intervals} of the parameters (estimators) $\beta$ by computing their
variances, evaluate the Mean Squared error (MSE)
\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\[ MSE(\bm{y},\tilde{\bm{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]
and the $R^2$ score function. If $\tilde{\hat{y}}_i$ is the predicted
and the $R^2$ score function. If $\tilde{\bm{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as
\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\bm{y}, \tilde{\bm{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]
where we have defined the mean value of $\hat{y}$ as
where we have defined the mean value of $\bm{y}$ as
\[
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
@@ -268,6 +268,7 @@ splitting training data provided by the library \textbf{Scikit-Learn} (make
sure you have installed it). This function is called
$train\_test\_split$. \textbf{You should present a critical discussion of why and how you have scaled or not scaled the data}.
It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (eventually also an additional
validation set). There
@@ -345,7 +346,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
Implement the $k$-fold cross-validation algorithm (write your own
code) and evaluate again the MSE function resulting
doc/Projects/2021/Project1/pdf/Project1.pdf
BIN
View File
Binary file not shown.
doc/Projects/2021/Project1/pdf/Project1.tex
+7 -6
View File
@@ -102,7 +102,7 @@ Project 1 on Machine Learning, deadline October 4, 2021
% --- begin date ---
\begin{center}
Sep 14, 2021
Oct 4, 2021
\end{center}
% --- end date ---
@@ -212,19 +212,19 @@ analysis using polynomials in $x$ and $y$ up to fifth order. Find the
\href{{https://en.wikipedia.org/wiki/Confidence_interval}}{confidence intervals} of the parameters (estimators) $\beta$ by computing their
variances, evaluate the Mean Squared error (MSE)
\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\[ MSE(\bm{y},\tilde{\bm{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]
and the $R^2$ score function. If $\tilde{\hat{y}}_i$ is the predicted
and the $R^2$ score function. If $\tilde{\bm{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as
\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\bm{y}, \tilde{\bm{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]
where we have defined the mean value of $\hat{y}$ as
where we have defined the mean value of $\bm{y}$ as
\[
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
@@ -238,6 +238,7 @@ splitting training data provided by the library \textbf{Scikit-Learn} (make
sure you have installed it). This function is called
$train\_test\_split$. \textbf{You should present a critical discussion of why and how you have scaled or not scaled the data}.
It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (eventually also an additional
validation set). There
@@ -315,7 +316,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
Implement the $k$-fold cross-validation algorithm (write your own
code) and evaluate again the MSE function resulting
doc/src/Projects/2021/Project1/Project1.do.txt
+6 -5
View File
@@ -111,22 +111,22 @@ analysis using polynomials in $x$ and $y$ up to fifth order. Find the
variances, evaluate the Mean Squared error (MSE)
!bt
\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\[ MSE(\bm{y},\tilde{\bm{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]
!et
and the $R^2$ score function. If $\tilde{\hat{y}}_i$ is the predicted
and the $R^2$ score function. If $\tilde{\bm{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as
!bt
\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
R^2(\bm{y}, \tilde{\bm{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]
!et
where we have defined the mean value of $\hat{y}$ as
where we have defined the mean value of $\bm{y}$ as
!bt
\[
@@ -142,6 +142,7 @@ splitting training data provided by the library _Scikit-Learn_ (make
sure you have installed it). This function is called
$train\_test\_split$. _You should present a critical discussion of why and how you have scaled or not scaled the data_.
It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (eventually also an additional
validation set). There
@@ -228,7 +229,7 @@ Note also that when you calculate the bias, in all applications you don't know t
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method. Again,
before you start with cross-validation approach, you should scale your
data.
data if you think this is needed.
Implement the $k$-fold cross-validation algorithm (write your own
code) and evaluate again the MSE function resulting
doc/src/week37/programs/LinearReg.py
+270
View File
@@ -0,0 +1,270 @@
import numpy as np
from sklearn import linear_model, metrics
from sklearn.model_selection import train_test_split
class linregOwn:
"""
A class of linear regressions. Perform ordinarly least squares (OLS) and Ridge regression manually. Lasso
is performed using scikit-learn functionality.
"""
def __init__(self, method = 'ols'):
"""
Constructor
Determines the method used in the fitting
Arguments:
method: string type. Accepts either 'ols', 'ridge' or 'lasso'.
"""
self.method = method
self.yHat = None
self.X = None
self.y = None
self.beta = None
self._MSE = None
self._R2 = None
self._betaVariance = None
self.lambda_ = None
def fit(self, X_train, y_train, lambda_ = 0):
"""
Performs the fit of OLS, Ridge or Lasso, depending on the argument provided initially.
Arguments:
X_train: Covariate matrix of the train data set, i.e. design matrix of
the shape m x p where m is the number of rows and p is the number of columns
(i.e. p is the complexity parameter).
y_train: Outcome variable, 1D numpy array
lambda_: float type. Shrinkage parameter for ridge and lasso methods. The higher value, higher shrinkage.
lambda_ is set to 0 for the OLS regression
"""
self.X_train = X_train
self.y_train = y_train
self.lambda_ = lambda_
if self.method == 'ols':
self._olsFit(X_train, y_train)
if self.method == 'ridge':
self._ridgeFit(X_train, y_train, lambda_)
if self.method == 'lasso':
self._lassoFitSKL(X_train, y_train, lambda_)
return self.beta
def _olsFit(self, X_train, y_train):
"""
Performs the ordinary least squares (OLS) fit on the provided data using singular value decomposition(SVD).
Arguments:
X_train: Covariate matrix of the train data set, i.e. design matrix of
the shape m x p where m is the number of rows and p is the number of columns
(i.e. p is the complexity parameter).
y_train: Outcome variable, 1D numpy array
Returns:
beta : numpy.array
The beta parameters from the performed fit
"""
self.X_train = X_train
self.y_test = y_train
U, S, VT = np.linalg.svd(self.X_train, full_matrices=True)
S_inverse = np.zeros(shape=self.X_train.shape)
##S is a vector, with shape of the number of columns
S_inverse[:S.shape[0], :S.shape[0]] = np.diag(1/S)
self.beta = np.dot(VT.T, np.dot(S_inverse.T, np.dot(U.T, self.y_train)))
#self.beta = np.linalg.inv(np.dot(X.T,X)).dot(X.T, y)
def _ridgeFit(self, X_train, y_train, lambda_):
"""
Performs the ridge regression fit
Arguments:
X_train: Covariate matrix of the train data set, design matrix of
the shape m x p (m_train_rows, p_columns).
y_train: Outcome variable, 1D numpy array, dimension m x 1
lambda_: Integer type. The shrinkage parameter
Returns:
beta : numpy.array
The beta parameters from the performed fit
"""
self.X_train = X_train
self.y_train = y_train
self.lambda_ = lambda_
self.beta = np.dot(np.linalg.inv(np.dot(X_train.T,X_train) + self.lambda_ * np.eye(X_train.shape[1])), np.dot(X_train.T,y_train))
def _lassoFitSKL(self, X_train, y_train, lambda_):
"""
Performs lasso fit using scikit-learn functionality.
Arguments:
X_train: Covariate matrix of the train data set, design matrix of
the shape m x p (m_train_datapoints, p_parameters).
y_train: Outcome variable, 1D numpy array, dimension m x 1
lambda_: Integer type. The shrinkage parameter
Returns:
self.beta : numpy.array
The beta parameters from the performed fit
"""
self.regression = linear_model.Lasso(fit_intercept=True, max_iter=1000000, alpha=self.lambda_)
self.regression.fit(X_train,y_train)
self.beta = self.regression.coef_
self.beta[0] = self.regression.intercept_
def predict(self, X_test):
"""
Performs prediction of the fitted model on the provided test data set.
Arguments:
X_test: Design matrix, covariate matrix, dimension k x p (k_test_rows, p_columns)
Returns: self.yHat
numpy 1D array, prediction values of dimension k x p
"""
self.X_test = X_test
self._predictOwntest(X_test)
return self.yHat
def _predictOwntest(self, X_test):
"""
Performs manual prediction of the given model on the train data.
"""
self.X_test = X_test
self.yHat = np.dot(self.X_test, self.beta)
def MSE(self, y_test):
"""
Calculates the mean squared error (MSE) manually after the fit and prediction have been implemented.
Arguments:
y_test: Outcome variable, 1D numpy array, dimension k x 1 (k_test_rows, 1_column)
Returns: self._MSE
The mean squared error of the predicted model
"""
self.y_test = y_test
if self.yHat is None :
self._predictOwntest(X_test)
N = self.yHat.size
self._MSE = (np.sum((self.y_test - self.yHat)**2))/N
return self._MSE
def R2(self, y_test):
"""
Calculates R2 score manually after the fit and prediction have been implemented.
Arguments:
y_test: Outcome variable, 1D numpy array, dimension k x 1 (k_test_rows, 1_column)
Returns: self._R2
The R2 score of the predicted model
"""
self.y_test = y_test
if self.yHat is None:
self._predictOwntest(X_test)
yMean = (1.0 / self.y_test.size) * np.sum(self.y_test)
self._R2 = 1.0 - np.sum((self.y_test - self.yHat)**2) / np.sum((self.y_test - yMean)**2)
return self._R2
def CI(self, y_test):
"""
Calculates confidence intervals manually after the fit and prediction have been implemented.
Arguments:
y_test: Outcome variable, 1D numpy array, dimension k x 1 (k_test_rows, 1_column)
Returns: var, Lower, Upper
Variance, Lower and Upper bounds of the confidence intervals for the parameter self.beta
"""
self.y_test = y_test
if self.yHat is None:
self._predictOwntest(X_test)
sigma2 = np.sum(((self.y_test - self.yHat)**2))/(self.y_test.size - self.beta.size)
var = np.diag(np.linalg.inv(np.dot(self.X_test.T, self.X_test))) * sigma2
Lower = self.beta - 1.96*np.sqrt(var)
Upper = self.beta + 1.96*np.sqrt(var)
return var, Lower, Upper
###Implementation through scikitlearn
class linregSKL:
def __init__(self, method = 'ols'):
"""
A class of linear regressions. Perform ordinarly least squares (OLS) and Ridge and Lasso
using scikit-learn functionality.
"""
self.method = method
self.yHat = None
self.X = None
self.y = None
self.beta = None
self._MSE = None
self._R2 = None
self._betaVariance = None
def fit(self, X_train, y_train, lambda_ = 0):
self.X_train = X_train
self.y_train = y_train
if self.method == 'ols':
self._olsSKLfit(X_train, y_train)
if self.method == 'ridge':
self._sklRidgeFit(X_train, y_train, lambda_)
if self.method == 'lasso':
self._SKLlassoFit(X_train, y_train, lambda_)
return self.beta
def _olsSKLfit(self, X_train, y_train):
self.X_train = X_train
self.y_train = y_train
##We already have standardized data from design matrix
self.ols = linear_model.LinearRegression().fit(self.X_train, self.y_train)
self.beta = self.ols.coef_
self.beta[0] = self.ols.intercept_
def _SKLlassoFit(self, X_train, y_train, lambda_):
self.regression = linear_model.Lasso(fit_intercept=True, max_iter=100000, alpha=self.lambda_)
self.regression.fit(X_train,y_train)
self.beta = self.regression.coef_
self.beta[0] = self.regression.intercept_
def _sklRidgeFit(self, X_train, y_train, lambda_):
self.regression = linear_model.Ridge(fit_intercept=True, alpha=self.lambda_)
self.regression.fit(X,y)
self.beta = self.regression.coef_
self.beta[0] = self.regression.intercept_
def predict(self, X_test):
self.X_test = X_test
if self.method == 'ols':
self._sklPredict(X_test)
return self.yHat
def _sklPredict(self, X_test):
self.X_test = X_test
## Since our data contains 1-s, we should subtract intercept, since scikit learn additionally
##generates the 1-s
self.yHat = self.ols.predict(self.X_test) - self.beta[0]
def MSE(self, y_test):
self.y_test = y_test
if self.yHat is None :
self._sklPredict(X_test)
self._MSE = metrics.mean_squared_error(self.y_test, self.yHat)
return self._MSE
def R2(self, y_test):
self.y_test = y_test
if self.yHat is None :
self._sklPredict()
self._R2 = metrics.r2_score(self.y_test, self.yHat)
return self._R2