updated hw2

This commit is contained in:
mhjensen
2020-09-10 11:03:20 +02:00
parent fd91b20596
commit 300bf26d4f
2 changed files with 20 additions and 20 deletions
Binary file not shown.
+20 -20
View File
@@ -167,10 +167,10 @@ distribution. The function $y$ is a quadratic polynomial in $x$ with
added stochastic noise according to the normal distribution $\cal{N}(0,1)$.
The following simple Python instructions define our $x$ and $y$ values (with 100 data points).
\begin{print}
\begin{verbatim}
x = np.random.rand(100)
y = 2.0+5*x*x+0.1*np.random.randn(100)
\end{print}
\end{verbatim}
\subex{a)}
@@ -180,7 +180,7 @@ Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.
% --- begin solution of exercise ---
\paragraph{Solution.}
The code here allows you to perform your own Ridge calculation and perform calculations for various values of the regularization parameter $\lambda$. This program can easily be extended upon.
\begin{print}
\begin{verbatim}
import os
import numpy as np
import pandas as pd
@@ -254,7 +254,7 @@ plt.xlabel('log10(lambda)')
plt.ylabel('MSE')
plt.legend()
plt.show()
\end{print}
\end{verbatim}
% --- end solution of exercise ---
@@ -265,7 +265,7 @@ Repeat the above but using the functionality of \textbf{Scikit-Learn}. Compare y
% --- begin solution of exercise ---
\paragraph{Solution.}
To use \textbf{scikit-learn} with Ridge, we simply need to add the relevant function \textbf{Ridge()}, as done in the code here.
\begin{print}
\begin{verbatim}
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
@@ -345,7 +345,7 @@ plt.xlabel('log10(lambda)')
plt.ylabel('MSE')
plt.legend()
plt.show()
\end{print}
\end{verbatim}
% --- end solution of exercise ---
@@ -355,7 +355,7 @@ Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$
% --- begin solution of exercise ---
\paragraph{Solution.}
\begin{print}
\begin{verbatim}
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
@@ -412,7 +412,7 @@ for i in range(nlambdas):
\end{print}
\end{verbatim}
% --- end solution of exercise ---
@@ -422,7 +422,7 @@ Repeat the previous step but add now the Lasso method, see equation (3.53) of Ha
% --- begin solution of exercise ---
\paragraph{Solution.}
\begin{print}
\begin{verbatim}
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
@@ -497,7 +497,7 @@ plt.xlabel('log10(lambda)')
plt.ylabel('MSE')
plt.legend()
plt.show()
\end{print}
\end{verbatim}
% --- end solution of exercise ---
@@ -574,17 +574,17 @@ techniques.
It also common to split the data in a \textbf{training} set and a \textbf{testing} set. A typical split is to use $80\%$ of the data for training and the rest
for testing. This can be done as follows with our design matrix $\bm{X}$ and data $\bm{y}$ (remember to import \textbf{scikit-learn})
\begin{print}
\begin{verbatim}
# split in training and test data
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
\end{print}
\end{verbatim}
Then we can use the standard scaler to scale our data as
\begin{print}
\begin{verbatim}
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
\end{print}
\end{verbatim}
In this exercise we want you to to compute the MSE for the training
@@ -597,14 +597,14 @@ We will also use Ridge and Lasso regression.
Our data is defined by $x\in [-3,3]$ with a total of for example $100$ data points.
\begin{print}
\begin{verbatim}
np.random.seed()
n = 100
maxdegree = 14
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
\end{print}
\end{verbatim}
where $y$ is the function we want to fit with a given polynomial.
@@ -614,7 +614,7 @@ Write a first code which sets up a design matrix $X$ defined by a fifth-order po
% --- begin solution of exercise ---
\paragraph{Solution.}
\begin{print}
\begin{verbatim}
import matplotlib.pyplot as plt
import numpy as np
from sklearn.linear_model import LinearRegression, Ridge, Lasso
@@ -651,7 +651,7 @@ plt.plot(polydegree, TestError, label='Test Error')
plt.plot(polydegree, TrainError, label='Train Error')
plt.legend()
plt.show()
\end{print}
\end{verbatim}
% --- end solution of exercise ---
@@ -682,7 +682,7 @@ Repeat part (2c) but now using Ridge regressions with various hyperparameters $\
% --- begin solution of exercise ---
\paragraph{Solution.}
Here you need to add for example the same loop over the parameters $\lambda$ as you did in the first exercise, that is add
\begin{print}
\begin{verbatim}
nlambdas = 100
MSEPredictRidge = np.zeros(nlambdas)
lambdas = np.logspace(-4, 0, nlambdas)
@@ -691,7 +691,7 @@ for i in range(nlambdas):
# add ridge
clf_ridge = skl.Ridge(alpha=lmb).fit(X_train_scaled, y_train)
\end{print}
\end{verbatim}
The plotting functionality of the first exercise can be reused here as well.
% --- end solution of exercise ---