diff --git a/doc/Projects/2020/hw2/pdf/hw2.pdf b/doc/Projects/2020/hw2/pdf/hw2.pdf index bc2cdf423..3e90b859d 100644 Binary files a/doc/Projects/2020/hw2/pdf/hw2.pdf and b/doc/Projects/2020/hw2/pdf/hw2.pdf differ diff --git a/doc/Projects/2020/hw2/pdf/hw2.tex b/doc/Projects/2020/hw2/pdf/hw2.tex index 68d65edce..2bc82a788 100644 --- a/doc/Projects/2020/hw2/pdf/hw2.tex +++ b/doc/Projects/2020/hw2/pdf/hw2.tex @@ -167,10 +167,10 @@ distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\cal{N}(0,1)$. The following simple Python instructions define our $x$ and $y$ values (with 100 data points). -\begin{print} +\begin{verbatim} x = np.random.rand(100) y = 2.0+5*x*x+0.1*np.random.randn(100) -\end{print} +\end{verbatim} \subex{a)} @@ -180,7 +180,7 @@ Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al. % --- begin solution of exercise --- \paragraph{Solution.} The code here allows you to perform your own Ridge calculation and perform calculations for various values of the regularization parameter $\lambda$. This program can easily be extended upon. -\begin{print} +\begin{verbatim} import os import numpy as np import pandas as pd @@ -254,7 +254,7 @@ plt.xlabel('log10(lambda)') plt.ylabel('MSE') plt.legend() plt.show() -\end{print} +\end{verbatim} % --- end solution of exercise --- @@ -265,7 +265,7 @@ Repeat the above but using the functionality of \textbf{Scikit-Learn}. Compare y % --- begin solution of exercise --- \paragraph{Solution.} To use \textbf{scikit-learn} with Ridge, we simply need to add the relevant function \textbf{Ridge()}, as done in the code here. -\begin{print} +\begin{verbatim} import numpy as np import pandas as pd import matplotlib.pyplot as plt @@ -345,7 +345,7 @@ plt.xlabel('log10(lambda)') plt.ylabel('MSE') plt.legend() plt.show() -\end{print} +\end{verbatim} % --- end solution of exercise --- @@ -355,7 +355,7 @@ Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ % --- begin solution of exercise --- \paragraph{Solution.} -\begin{print} +\begin{verbatim} import numpy as np import pandas as pd import matplotlib.pyplot as plt @@ -412,7 +412,7 @@ for i in range(nlambdas): -\end{print} +\end{verbatim} % --- end solution of exercise --- @@ -422,7 +422,7 @@ Repeat the previous step but add now the Lasso method, see equation (3.53) of Ha % --- begin solution of exercise --- \paragraph{Solution.} -\begin{print} +\begin{verbatim} import numpy as np import pandas as pd import matplotlib.pyplot as plt @@ -497,7 +497,7 @@ plt.xlabel('log10(lambda)') plt.ylabel('MSE') plt.legend() plt.show() -\end{print} +\end{verbatim} % --- end solution of exercise --- @@ -574,17 +574,17 @@ techniques. It also common to split the data in a \textbf{training} set and a \textbf{testing} set. A typical split is to use $80\%$ of the data for training and the rest for testing. This can be done as follows with our design matrix $\bm{X}$ and data $\bm{y}$ (remember to import \textbf{scikit-learn}) -\begin{print} +\begin{verbatim} # split in training and test data X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2) -\end{print} +\end{verbatim} Then we can use the standard scaler to scale our data as -\begin{print} +\begin{verbatim} scaler = StandardScaler() scaler.fit(X_train) X_train_scaled = scaler.transform(X_train) X_test_scaled = scaler.transform(X_test) -\end{print} +\end{verbatim} In this exercise we want you to to compute the MSE for the training @@ -597,14 +597,14 @@ We will also use Ridge and Lasso regression. Our data is defined by $x\in [-3,3]$ with a total of for example $100$ data points. -\begin{print} +\begin{verbatim} np.random.seed() n = 100 maxdegree = 14 # Make data set. x = np.linspace(-3, 3, n).reshape(-1, 1) y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape) -\end{print} +\end{verbatim} where $y$ is the function we want to fit with a given polynomial. @@ -614,7 +614,7 @@ Write a first code which sets up a design matrix $X$ defined by a fifth-order po % --- begin solution of exercise --- \paragraph{Solution.} -\begin{print} +\begin{verbatim} import matplotlib.pyplot as plt import numpy as np from sklearn.linear_model import LinearRegression, Ridge, Lasso @@ -651,7 +651,7 @@ plt.plot(polydegree, TestError, label='Test Error') plt.plot(polydegree, TrainError, label='Train Error') plt.legend() plt.show() -\end{print} +\end{verbatim} % --- end solution of exercise --- @@ -682,7 +682,7 @@ Repeat part (2c) but now using Ridge regressions with various hyperparameters $\ % --- begin solution of exercise --- \paragraph{Solution.} Here you need to add for example the same loop over the parameters $\lambda$ as you did in the first exercise, that is add -\begin{print} +\begin{verbatim} nlambdas = 100 MSEPredictRidge = np.zeros(nlambdas) lambdas = np.logspace(-4, 0, nlambdas) @@ -691,7 +691,7 @@ for i in range(nlambdas): # add ridge clf_ridge = skl.Ridge(alpha=lmb).fit(X_train_scaled, y_train) -\end{print} +\end{verbatim} The plotting functionality of the first exercise can be reused here as well. % --- end solution of exercise ---