diff --git a/doc/Projects/2018/hw2/html/hw2-bs.html b/doc/Projects/2018/hw2/html/hw2-bs.html index 367aaa066..7e1d9a41e 100644 --- a/doc/Projects/2018/hw2/html/hw2-bs.html +++ b/doc/Projects/2018/hw2/html/hw2-bs.html @@ -143,14 +143,9 @@ The following simple Python instructions define our \( x \) and \( y \) values ( y = 5*x*x+0.1*np.random.randn(100,1)
    -
  1. Write your own code for the Ridge method (see chapter 3.4 of Hastie et al., equations (3.43) and (3.44)) and compute the parametrization for different values of \( \lambda \).
    2. -
- -Compare and analyze your results with those from exercise 2. Study the dependence on \( \lambda \) while also varying the strength of the noise in your expression for \( y(x) \). - -
    +
  1. Write your own code for the Ridge method (see chapter 3.4 of Hastie et al., equations (3.43) and (3.44)) and compute the parametrization for different values of \( \lambda \). Compare and analyze your results with those from exercise 2. Study the dependence on \( \lambda \) while also varying the strength of the noise in your expression for \( y(x) \).
  2. Repeat the above but using the functionality of scikit-learn. Compare your code with the results from scikit-learn. Remember to run with the same random numbers for generating \( x \) and \( y \).
    3. -
  3. Our next step is to study the variance of the parameters \( \beta_1 \) and \( \beta_2 \) (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix \( (\hat{X}^T\hat{X})^{-1}+\lambda\hat{I} \)) or use the functionality of scikit-learn and computetheir variances. Discuss the results of these variances as functions of \( \lambda \). In particular, try to link your discussion with the discussion in Hastie et al. and their figure 3.11.
    4. +
  4. Our next step is to study the variance of the parameters \( \beta_1 \) and \( \beta_2 \) (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix \( (\hat{X}^T\hat{X})^{-1}+\lambda\hat{I} \)) or use the functionality of scikit-learn and compute their variances. Discuss the results of these variances as functions of \( \lambda \). In particular, try to link your discussion with the discussion in Hastie et al. and their figure 3.11.
  5. Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie et al.. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of scikit-learn.
  6. Finally, using scikit-learn or your own code, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
diff --git a/doc/Projects/2018/hw2/html/hw2.html b/doc/Projects/2018/hw2/html/hw2.html index 81b326f5d..e68f82db2 100644 --- a/doc/Projects/2018/hw2/html/hw2.html +++ b/doc/Projects/2018/hw2/html/hw2.html @@ -108,14 +108,9 @@ The following simple Python instructions define our \( x \) and \( y \) values ( y = 5*x*x+0.1*np.random.randn(100,1)
    -
  1. Write your own code for the Ridge method (see chapter 3.4 of Hastie et al., equations (3.43) and (3.44)) and compute the parametrization for different values of \( \lambda \).
    2. -
- -Compare and analyze your results with those from exercise 2. Study the dependence on \( \lambda \) while also varying the strength of the noise in your expression for \( y(x) \). - -
    +
  1. Write your own code for the Ridge method (see chapter 3.4 of Hastie et al., equations (3.43) and (3.44)) and compute the parametrization for different values of \( \lambda \). Compare and analyze your results with those from exercise 2. Study the dependence on \( \lambda \) while also varying the strength of the noise in your expression for \( y(x) \).
  2. Repeat the above but using the functionality of scikit-learn. Compare your code with the results from scikit-learn. Remember to run with the same random numbers for generating \( x \) and \( y \).
    3. -
  3. Our next step is to study the variance of the parameters \( \beta_1 \) and \( \beta_2 \) (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix \( (\hat{X}^T\hat{X})^{-1}+\lambda\hat{I} \)) or use the functionality of scikit-learn and computetheir variances. Discuss the results of these variances as functions of \( \lambda \). In particular, try to link your discussion with the discussion in Hastie et al. and their figure 3.11.
    4. +
  4. Our next step is to study the variance of the parameters \( \beta_1 \) and \( \beta_2 \) (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix \( (\hat{X}^T\hat{X})^{-1}+\lambda\hat{I} \)) or use the functionality of scikit-learn and compute their variances. Discuss the results of these variances as functions of \( \lambda \). In particular, try to link your discussion with the discussion in Hastie et al. and their figure 3.11.
  5. Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie et al.. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of scikit-learn.
  6. Finally, using scikit-learn or your own code, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
diff --git a/doc/Projects/2018/hw2/ipynb/ipynb-hw2-src.tar.gz b/doc/Projects/2018/hw2/ipynb/ipynb-hw2-src.tar.gz index b45bada77..6bb2b7ebf 100644 Binary files a/doc/Projects/2018/hw2/ipynb/ipynb-hw2-src.tar.gz and b/doc/Projects/2018/hw2/ipynb/ipynb-hw2-src.tar.gz differ diff --git a/doc/Projects/2018/hw2/pdf/hw2.p.tex b/doc/Projects/2018/hw2/pdf/hw2.p.tex index dd6529e49..03165d7b8 100644 --- a/doc/Projects/2018/hw2/pdf/hw2.p.tex +++ b/doc/Projects/2018/hw2/pdf/hw2.p.tex @@ -182,16 +182,11 @@ y = 5*x*x+0.1*np.random.randn(100,1) \epycod \begin{enumerate} -\item Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.}, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. -\end{enumerate} +\item Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.}, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. -\noindent -Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. - -\begin{enumerate} \item Repeat the above but using the functionality of \textbf{scikit-learn}. Compare your code with the results from \textbf{scikit-learn}. Remember to run with the same random numbers for generating $x$ and $y$. -\item Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of \textbf{scikit-learn} and computetheir variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie \emph{et al.} and their figure 3.11. +\item Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of \textbf{scikit-learn} and compute their variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie \emph{et al.} and their figure 3.11. \item Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie \emph{et al.}. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of \textbf{scikit-learn}. diff --git a/doc/Projects/2018/hw2/pdf/hw2.pdf b/doc/Projects/2018/hw2/pdf/hw2.pdf index 1fbd9008a..c4daaa3b4 100644 Binary files a/doc/Projects/2018/hw2/pdf/hw2.pdf and b/doc/Projects/2018/hw2/pdf/hw2.pdf differ diff --git a/doc/Projects/2018/hw2/pdf/hw2.tex b/doc/Projects/2018/hw2/pdf/hw2.tex index 05027f064..99574c061 100644 --- a/doc/Projects/2018/hw2/pdf/hw2.tex +++ b/doc/Projects/2018/hw2/pdf/hw2.tex @@ -152,16 +152,11 @@ y = 5*x*x+0.1*np.random.randn(100,1) \end{print} \begin{enumerate} -\item Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.}, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. -\end{enumerate} +\item Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.}, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. -\noindent -Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. - -\begin{enumerate} \item Repeat the above but using the functionality of \textbf{scikit-learn}. Compare your code with the results from \textbf{scikit-learn}. Remember to run with the same random numbers for generating $x$ and $y$. -\item Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of \textbf{scikit-learn} and computetheir variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie \emph{et al.} and their figure 3.11. +\item Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of \textbf{scikit-learn} and compute their variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie \emph{et al.} and their figure 3.11. \item Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie \emph{et al.}. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of \textbf{scikit-learn}. diff --git a/doc/src/Projects/2018/Exercises/hw2.do.txt b/doc/src/Projects/2018/Exercises/hw2.do.txt index add3555f6..593182104 100644 --- a/doc/src/Projects/2018/Exercises/hw2.do.txt +++ b/doc/src/Projects/2018/Exercises/hw2.do.txt @@ -22,12 +22,11 @@ x = np.random.rand(100,1) y = 5*x*x+0.1*np.random.randn(100,1) !ec -o Write your own code for the Ridge method (see chapter 3.4 of Hastie *et al.*, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. -Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. +o Write your own code for the Ridge method (see chapter 3.4 of Hastie *et al.*, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$. o Repeat the above but using the functionality of _scikit-learn_. Compare your code with the results from _scikit-learn_. Remember to run with the same random numbers for generating $x$ and $y$. -o Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of _scikit-learn_ and computetheir variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie *et al.* and their figure 3.11. +o Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})^{-1}+\lambda\hat{I}$) or use the functionality of _scikit-learn_ and compute their variances. Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie *et al.* and their figure 3.11. o Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie *et al.*. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of _scikit-learn_.