-
-
How to set up the cross-validation for Ridge and/or Lasso
-
-
-- Define a range of interest for the penalty parameter.
-- Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.
-- Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \boldsymbol{\sigma}_{-i}^2(\lambda) \), as
-
-$$
-\begin{align*}
-\boldsymbol{\beta}_{-i}(\lambda) & = ( \boldsymbol{X}_{-i, \ast}^{T}
-\boldsymbol{X}_{-i, \ast} + \lambda \boldsymbol{I}_{pp})^{-1}
-\boldsymbol{X}_{-i, \ast}^{T} \boldsymbol{y}_{-i}
-\end{align*}
-$$
-
-
-
-- Evaluate the prediction performance of these models on the test set by \( [y_i, \boldsymbol{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)] \). Or, by the prediction error \( |y_i - \boldsymbol{X}_{i, \ast} \boldsymbol{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.
-- Repeat the first three steps such that each sample plays the role of the test set once.
-- Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
-
-$$
-\begin{align*}
-\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)]\}.
-\end{align*}
-$$
+
+
Cross-validation in brief
+
For the various values of \( k \)
+
+- shuffle the dataset randomly.
+- Split the dataset into \( k \) groups.
+- For each unique group:
+
+- Decide which group to use as set for test data
+- Take the remaining groups as a training data set
+- Fit a model on the training set and evaluate it on the test set
+- Retain the evaluation score and discard the model
+
+
Summarize the model using the sample of model evaluation scores
+
diff --git a/doc/pub/week37/html/._week37-bs045.html b/doc/pub/week37/html/._week37-bs045.html
index 8c1d51079..be88f20c9 100644
--- a/doc/pub/week37/html/._week37-bs045.html
+++ b/doc/pub/week37/html/._week37-bs045.html
@@ -152,10 +152,6 @@ doconce format html week37.do.txt --html_style=bootstrap --pygments_html_style=d
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -250,11 +246,10 @@ MathJax.Hub.Config({
Summing up
Another Example from Scikit-Learn's Repository
Various steps in cross-validation
-
How to set up the cross-validation for Ridge and/or Lasso
-
Cross-validation in brief
-
Code Example for Cross-validation and \( k \)-fold Cross-validation
-
More examples on bootstrap and cross-validation and errors
-
The same example but now with cross-validation
+
Cross-validation in brief
+
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
More examples on bootstrap and cross-validation and errors
+
The same example but now with cross-validation
@@ -266,22 +261,121 @@ MathJax.Hub.Config({
-
Cross-validation in brief
+
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
+
The code here uses Ridge regression with cross-validation (CV) resampling and \( k \)-fold CV in order to fit a specific polynomial.
+
+
+
-
For the various values of \( k \)
-
-- shuffle the dataset randomly.
-- Split the dataset into \( k \) groups.
-- For each unique group:
-
-- Decide which group to use as set for test data
-- Take the remaining groups as a training data set
-- Fit a model on the training set and evaluate it on the test set
-- Retain the evaluation score and discard the model
-
-
Summarize the model using the sample of model evaluation scores
-
diff --git a/doc/pub/week37/html/._week37-bs046.html b/doc/pub/week37/html/._week37-bs046.html
index 48d02a96a..da9a7aba4 100644
--- a/doc/pub/week37/html/._week37-bs046.html
+++ b/doc/pub/week37/html/._week37-bs046.html
@@ -152,10 +152,6 @@ doconce format html week37.do.txt --html_style=bootstrap --pygments_html_style=d
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -250,11 +246,10 @@ MathJax.Hub.Config({
Summing up
Another Example from Scikit-Learn's Repository
Various steps in cross-validation
-
How to set up the cross-validation for Ridge and/or Lasso
-
Cross-validation in brief
-
Code Example for Cross-validation and \( k \)-fold Cross-validation
-
More examples on bootstrap and cross-validation and errors
-
The same example but now with cross-validation
+
Cross-validation in brief
+
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
More examples on bootstrap and cross-validation and errors
+
The same example but now with cross-validation
@@ -266,9 +261,8 @@ MathJax.Hub.Config({
-
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
More examples on bootstrap and cross-validation and errors
-
The code here uses Ridge regression with cross-validation (CV) resampling and \( k \)-fold CV in order to fit a specific polynomial.
@@ -276,94 +270,83 @@ MathJax.Hub.Config({
+
Note that we kept the intercept column in the fitting here. This means that we need to set the intercept in the call to the Scikit-Learn function as False. Alternatively, we could have set up the design matrix \( X \) without the first column of ones.
@@ -397,7 +381,6 @@ plt.show()
46
47
48
-
49
»
diff --git a/doc/pub/week37/html/._week37-bs047.html b/doc/pub/week37/html/._week37-bs047.html
index ee7a0b53f..16f0a1e98 100644
--- a/doc/pub/week37/html/._week37-bs047.html
+++ b/doc/pub/week37/html/._week37-bs047.html
@@ -152,10 +152,6 @@ doconce format html week37.do.txt --html_style=bootstrap --pygments_html_style=d
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -250,11 +246,10 @@ MathJax.Hub.Config({
Summing up
Another Example from Scikit-Learn's Repository
Various steps in cross-validation
-
How to set up the cross-validation for Ridge and/or Lasso
-
Cross-validation in brief
-
Code Example for Cross-validation and \( k \)-fold Cross-validation
-
More examples on bootstrap and cross-validation and errors
-
The same example but now with cross-validation
+
Cross-validation in brief
+
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
More examples on bootstrap and cross-validation and errors
+
The same example but now with cross-validation
@@ -265,9 +260,10 @@ MathJax.Hub.Config({
-
-
More examples on bootstrap and cross-validation and errors
+
+
The same example but now with cross-validation
+
In this example we keep the intercept column again but add cross-validation in order to estimate the best possible value of the means squared error.
@@ -281,9 +277,11 @@ MathJax.Hub.Config({
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression, Ridge, Lasso
-from sklearn.model_selection import train_test_split
-from sklearn.utils import resample
from sklearn.metrics import mean_squared_error
+from sklearn.model_selection import KFold
+from sklearn.model_selection import cross_val_score
+
+
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
@@ -320,35 +318,22 @@ Density = EoS[&
Maxpolydegree = 30
X = np.zeros((len(Density),Maxpolydegree))
X[:,0] = 1.0
-testerror = np.zeros(Maxpolydegree)
-trainingerror = np.zeros(Maxpolydegree)
+estimated_mse_sklearn = np.zeros(Maxpolydegree)
polynomial = np.zeros(Maxpolydegree)
+k =5
+kfold = KFold(n_splits = k)
-trials = 100
for polydegree in range(1, Maxpolydegree):
polynomial[polydegree] = polydegree
for degree in range(polydegree):
X[:,degree] = Density**(degree/3.0)
-
+ OLS = LinearRegression(fit_intercept=False)
# loop over trials in order to estimate the expectation value of the MSE
- testerror[polydegree] = 0.0
- trainingerror[polydegree] = 0.0
- for samples in range(trials):
- x_train, x_test, y_train, y_test = train_test_split(X, Energies, test_size=0.2)
- model = LinearRegression(fit_intercept=False).fit(x_train, y_train)
- ypred = model.predict(x_train)
- ytilde = model.predict(x_test)
- testerror[polydegree] += mean_squared_error(y_test, ytilde)
- trainingerror[polydegree] += mean_squared_error(y_train, ypred)
+ estimated_mse_folds = cross_val_score(OLS, X, Energies, scoring='neg_mean_squared_error', cv=kfold)
+#[:, np.newaxis]
+ estimated_mse_sklearn[polydegree] = np.mean(-estimated_mse_folds)
- testerror[polydegree] /= trials
- trainingerror[polydegree] /= trials
- print("Degree of polynomial: %3d"% polynomial[polydegree])
- print("Mean squared error on training data: %.8f" % trainingerror[polydegree])
- print("Mean squared error on test data: %.8f" % testerror[polydegree])
-
-plt.plot(polynomial, np.log10(trainingerror), label='Training Error')
-plt.plot(polynomial, np.log10(testerror), label='Test Error')
+plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')
plt.xlabel('Polynomial degree')
plt.ylabel('log10[MSE]')
plt.legend()
@@ -368,8 +353,6 @@ plt.show()
-
Note that we kept the intercept column in the fitting here. This means that we need to set the intercept in the call to the Scikit-Learn function as False. Alternatively, we could have set up the design matrix \( X \) without the first column of ones.
-
diff --git a/doc/pub/week37/html/week37-bs.html b/doc/pub/week37/html/week37-bs.html
index da2d4f749..909297b04 100644
--- a/doc/pub/week37/html/week37-bs.html
+++ b/doc/pub/week37/html/week37-bs.html
@@ -152,10 +152,6 @@ doconce format html week37.do.txt --html_style=bootstrap --pygments_html_style=d
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -250,11 +246,10 @@ MathJax.Hub.Config({
Summing up
Another Example from Scikit-Learn's Repository
Various steps in cross-validation
-
How to set up the cross-validation for Ridge and/or Lasso
-
Cross-validation in brief
-
Code Example for Cross-validation and \( k \)-fold Cross-validation
-
More examples on bootstrap and cross-validation and errors
-
The same example but now with cross-validation
+
Cross-validation in brief
+
Code Example for Cross-validation and \( k \)-fold Cross-validation
+
More examples on bootstrap and cross-validation and errors
+
The same example but now with cross-validation
@@ -309,7 +304,7 @@ MathJax.Hub.Config({
9
10
...
-
49
+
48
»
diff --git a/doc/pub/week37/html/week37-reveal.html b/doc/pub/week37/html/week37-reveal.html
index 605886141..b70a1e09a 100644
--- a/doc/pub/week37/html/week37-reveal.html
+++ b/doc/pub/week37/html/week37-reveal.html
@@ -1721,41 +1721,6 @@ cross-validation (LOOCV).
-
-How to set up the cross-validation for Ridge and/or Lasso
-
-
-- Define a range of interest for the penalty parameter.
-- Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.
-- Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \boldsymbol{\sigma}_{-i}^2(\lambda) \), as
-
-
-
-$$
-\begin{align*}
-\boldsymbol{\beta}_{-i}(\lambda) & = ( \boldsymbol{X}_{-i, \ast}^{T}
-\boldsymbol{X}_{-i, \ast} + \lambda \boldsymbol{I}_{pp})^{-1}
-\boldsymbol{X}_{-i, \ast}^{T} \boldsymbol{y}_{-i}
-\end{align*}
-$$
-
-
-
-
-- Evaluate the prediction performance of these models on the test set by \( [y_i, \boldsymbol{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)] \). Or, by the prediction error \( |y_i - \boldsymbol{X}_{i, \ast} \boldsymbol{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.
-- Repeat the first three steps such that each sample plays the role of the test set once.
-- Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
-
-
-
-$$
-\begin{align*}
-\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)]\}.
-\end{align*}
-$$
-
-
-
Cross-validation in brief
diff --git a/doc/pub/week37/html/week37-solarized.html b/doc/pub/week37/html/week37-solarized.html
index 13b2d648a..fbba8964d 100644
--- a/doc/pub/week37/html/week37-solarized.html
+++ b/doc/pub/week37/html/week37-solarized.html
@@ -179,10 +179,6 @@ div.toc p,a {
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -1643,35 +1639,6 @@ choosing \( k=n \). This particular case is referred to as leave-one-out
cross-validation (LOOCV).
-
-How to set up the cross-validation for Ridge and/or Lasso
-
-
-- Define a range of interest for the penalty parameter.
-- Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.
-- Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \boldsymbol{\sigma}_{-i}^2(\lambda) \), as
-
-$$
-\begin{align*}
-\boldsymbol{\beta}_{-i}(\lambda) & = ( \boldsymbol{X}_{-i, \ast}^{T}
-\boldsymbol{X}_{-i, \ast} + \lambda \boldsymbol{I}_{pp})^{-1}
-\boldsymbol{X}_{-i, \ast}^{T} \boldsymbol{y}_{-i}
-\end{align*}
-$$
-
-
-
-- Evaluate the prediction performance of these models on the test set by \( [y_i, \boldsymbol{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)] \). Or, by the prediction error \( |y_i - \boldsymbol{X}_{i, \ast} \boldsymbol{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.
-- Repeat the first three steps such that each sample plays the role of the test set once.
-- Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
-
-$$
-\begin{align*}
-\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)]\}.
-\end{align*}
-$$
-
-
Cross-validation in brief
diff --git a/doc/pub/week37/html/week37.html b/doc/pub/week37/html/week37.html
index b32c6c36c..851920ee8 100644
--- a/doc/pub/week37/html/week37.html
+++ b/doc/pub/week37/html/week37.html
@@ -256,10 +256,6 @@ div.toc p,a {
2,
None,
'various-steps-in-cross-validation'),
- ('How to set up the cross-validation for Ridge and/or Lasso',
- 2,
- None,
- 'how-to-set-up-the-cross-validation-for-ridge-and-or-lasso'),
('Cross-validation in brief',
2,
None,
@@ -1720,35 +1716,6 @@ choosing \( k=n \). This particular case is referred to as leave-one-out
cross-validation (LOOCV).
-
-How to set up the cross-validation for Ridge and/or Lasso
-
-
-- Define a range of interest for the penalty parameter.
-- Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.
-- Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \boldsymbol{\sigma}_{-i}^2(\lambda) \), as
-
-$$
-\begin{align*}
-\boldsymbol{\beta}_{-i}(\lambda) & = ( \boldsymbol{X}_{-i, \ast}^{T}
-\boldsymbol{X}_{-i, \ast} + \lambda \boldsymbol{I}_{pp})^{-1}
-\boldsymbol{X}_{-i, \ast}^{T} \boldsymbol{y}_{-i}
-\end{align*}
-$$
-
-
-
-- Evaluate the prediction performance of these models on the test set by \( [y_i, \boldsymbol{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)] \). Or, by the prediction error \( |y_i - \boldsymbol{X}_{i, \ast} \boldsymbol{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.
-- Repeat the first three steps such that each sample plays the role of the test set once.
-- Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
-
-$$
-\begin{align*}
-\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \boldsymbol{\beta}_{-i}(\lambda), \boldsymbol{\sigma}_{-i}^2(\lambda)]\}.
-\end{align*}
-$$
-
-
Cross-validation in brief
diff --git a/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz b/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz
index b2f81f95a..ea57fbcbc 100644
Binary files a/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz and b/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz differ
diff --git a/doc/pub/week37/ipynb/week37.ipynb b/doc/pub/week37/ipynb/week37.ipynb
index 2687a3a87..29c0793ab 100644
--- a/doc/pub/week37/ipynb/week37.ipynb
+++ b/doc/pub/week37/ipynb/week37.ipynb
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
- "id": "3413b194",
+ "id": "83e91ce3",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
- "id": "ecf8eb8c",
+ "id": "88f778c9",
"metadata": {
"editable": true
},
@@ -29,7 +29,7 @@
},
{
"cell_type": "markdown",
- "id": "b873b4b3",
+ "id": "8cad2f0c",
"metadata": {
"editable": true
},
@@ -54,7 +54,7 @@
},
{
"cell_type": "markdown",
- "id": "6f7dcfc4",
+ "id": "197e7446",
"metadata": {
"editable": true
},
@@ -64,7 +64,7 @@
},
{
"cell_type": "markdown",
- "id": "f578dcee",
+ "id": "04951cec",
"metadata": {
"editable": true
},
@@ -87,7 +87,7 @@
},
{
"cell_type": "markdown",
- "id": "7c52f047",
+ "id": "ce4380d8",
"metadata": {
"editable": true
},
@@ -99,7 +99,7 @@
},
{
"cell_type": "markdown",
- "id": "eb27f88a",
+ "id": "206592ba",
"metadata": {
"editable": true
},
@@ -112,7 +112,7 @@
},
{
"cell_type": "markdown",
- "id": "99bd0787",
+ "id": "4a0d88d0",
"metadata": {
"editable": true
},
@@ -124,7 +124,7 @@
},
{
"cell_type": "markdown",
- "id": "323d053d",
+ "id": "e6c7ba75",
"metadata": {
"editable": true
},
@@ -136,7 +136,7 @@
},
{
"cell_type": "markdown",
- "id": "c5bcb8bb",
+ "id": "665e0170",
"metadata": {
"editable": true
},
@@ -148,7 +148,7 @@
},
{
"cell_type": "markdown",
- "id": "00c07681",
+ "id": "73636714",
"metadata": {
"editable": true
},
@@ -159,7 +159,7 @@
},
{
"cell_type": "markdown",
- "id": "24fec3d9",
+ "id": "ad90bb50",
"metadata": {
"editable": true
},
@@ -171,7 +171,7 @@
},
{
"cell_type": "markdown",
- "id": "dc03828d",
+ "id": "24493b42",
"metadata": {
"editable": true
},
@@ -182,7 +182,7 @@
},
{
"cell_type": "markdown",
- "id": "ee74422b",
+ "id": "fd50e415",
"metadata": {
"editable": true
},
@@ -194,7 +194,7 @@
},
{
"cell_type": "markdown",
- "id": "7dcdd285",
+ "id": "3d8e865e",
"metadata": {
"editable": true
},
@@ -204,7 +204,7 @@
},
{
"cell_type": "markdown",
- "id": "39107290",
+ "id": "17f93455",
"metadata": {
"editable": true
},
@@ -235,7 +235,7 @@
},
{
"cell_type": "markdown",
- "id": "378cc7e7",
+ "id": "9d6b2b2a",
"metadata": {
"editable": true
},
@@ -247,7 +247,7 @@
},
{
"cell_type": "markdown",
- "id": "f55bf648",
+ "id": "03ca6196",
"metadata": {
"editable": true
},
@@ -259,7 +259,7 @@
},
{
"cell_type": "markdown",
- "id": "982e45f8",
+ "id": "a9f157c0",
"metadata": {
"editable": true
},
@@ -269,7 +269,7 @@
},
{
"cell_type": "markdown",
- "id": "52050954",
+ "id": "7a0b2baa",
"metadata": {
"editable": true
},
@@ -281,7 +281,7 @@
},
{
"cell_type": "markdown",
- "id": "401762d1",
+ "id": "0500cb7f",
"metadata": {
"editable": true
},
@@ -291,7 +291,7 @@
},
{
"cell_type": "markdown",
- "id": "c39377bd",
+ "id": "98951cc5",
"metadata": {
"editable": true
},
@@ -303,7 +303,7 @@
},
{
"cell_type": "markdown",
- "id": "b16491e9",
+ "id": "a168a53a",
"metadata": {
"editable": true
},
@@ -313,7 +313,7 @@
},
{
"cell_type": "markdown",
- "id": "fed7e2df",
+ "id": "40c3e355",
"metadata": {
"editable": true
},
@@ -325,7 +325,7 @@
},
{
"cell_type": "markdown",
- "id": "9beafdd9",
+ "id": "690e5ebd",
"metadata": {
"editable": true
},
@@ -337,7 +337,7 @@
},
{
"cell_type": "markdown",
- "id": "ad2ee7bf",
+ "id": "d5e4d063",
"metadata": {
"editable": true
},
@@ -349,7 +349,7 @@
},
{
"cell_type": "markdown",
- "id": "10eb9160",
+ "id": "f3c99110",
"metadata": {
"editable": true
},
@@ -359,7 +359,7 @@
},
{
"cell_type": "markdown",
- "id": "18cda10f",
+ "id": "8b94286b",
"metadata": {
"editable": true
},
@@ -371,7 +371,7 @@
},
{
"cell_type": "markdown",
- "id": "fb6f2e9d",
+ "id": "274f358c",
"metadata": {
"editable": true
},
@@ -381,7 +381,7 @@
},
{
"cell_type": "markdown",
- "id": "3d415353",
+ "id": "75c6245e",
"metadata": {
"editable": true
},
@@ -398,7 +398,7 @@
},
{
"cell_type": "markdown",
- "id": "9dd02f04",
+ "id": "c813eb77",
"metadata": {
"editable": true
},
@@ -417,7 +417,7 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "f6d8eab0",
+ "id": "6bef25fe",
"metadata": {
"collapsed": false,
"editable": true
@@ -493,7 +493,7 @@
},
{
"cell_type": "markdown",
- "id": "dd37b16d",
+ "id": "0ff10c31",
"metadata": {
"editable": true
},
@@ -503,7 +503,7 @@
},
{
"cell_type": "markdown",
- "id": "cc8c954c",
+ "id": "c072a6dd",
"metadata": {
"editable": true
},
@@ -533,7 +533,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "e631888e",
+ "id": "71231be2",
"metadata": {
"collapsed": false,
"editable": true
@@ -582,7 +582,7 @@
},
{
"cell_type": "markdown",
- "id": "e8a1b23b",
+ "id": "b9475a90",
"metadata": {
"editable": true
},
@@ -596,7 +596,7 @@
},
{
"cell_type": "markdown",
- "id": "6462ab01",
+ "id": "1b3063f0",
"metadata": {
"editable": true
},
@@ -608,7 +608,7 @@
},
{
"cell_type": "markdown",
- "id": "b1c5c4f0",
+ "id": "e48ff5b1",
"metadata": {
"editable": true
},
@@ -618,7 +618,7 @@
},
{
"cell_type": "markdown",
- "id": "3b4b3365",
+ "id": "d731fabe",
"metadata": {
"editable": true
},
@@ -630,7 +630,7 @@
},
{
"cell_type": "markdown",
- "id": "a87967e1",
+ "id": "4cfd7f0e",
"metadata": {
"editable": true
},
@@ -640,7 +640,7 @@
},
{
"cell_type": "markdown",
- "id": "a995bc0d",
+ "id": "8f7e06da",
"metadata": {
"editable": true
},
@@ -652,7 +652,7 @@
},
{
"cell_type": "markdown",
- "id": "cde16d8a",
+ "id": "951c18fa",
"metadata": {
"editable": true
},
@@ -662,7 +662,7 @@
},
{
"cell_type": "markdown",
- "id": "40002307",
+ "id": "f830a42e",
"metadata": {
"editable": true
},
@@ -674,7 +674,7 @@
},
{
"cell_type": "markdown",
- "id": "ad1f47f2",
+ "id": "dc3bb676",
"metadata": {
"editable": true
},
@@ -684,7 +684,7 @@
},
{
"cell_type": "markdown",
- "id": "44205dca",
+ "id": "36b4dc39",
"metadata": {
"editable": true
},
@@ -700,7 +700,7 @@
},
{
"cell_type": "markdown",
- "id": "5911811f",
+ "id": "e58db2db",
"metadata": {
"editable": true
},
@@ -712,7 +712,7 @@
},
{
"cell_type": "markdown",
- "id": "1a29cb1b",
+ "id": "f9d2d731",
"metadata": {
"editable": true
},
@@ -722,7 +722,7 @@
},
{
"cell_type": "markdown",
- "id": "3c8baa81",
+ "id": "4285c782",
"metadata": {
"editable": true
},
@@ -734,7 +734,7 @@
},
{
"cell_type": "markdown",
- "id": "99c31881",
+ "id": "860c78f8",
"metadata": {
"editable": true
},
@@ -747,7 +747,7 @@
},
{
"cell_type": "markdown",
- "id": "7b0c2d78",
+ "id": "5f736b0d",
"metadata": {
"editable": true
},
@@ -759,7 +759,7 @@
},
{
"cell_type": "markdown",
- "id": "5ab49aec",
+ "id": "a8336ec5",
"metadata": {
"editable": true
},
@@ -769,7 +769,7 @@
},
{
"cell_type": "markdown",
- "id": "c64df7b2",
+ "id": "d29a5d9d",
"metadata": {
"editable": true
},
@@ -781,7 +781,7 @@
},
{
"cell_type": "markdown",
- "id": "7848825a",
+ "id": "b33925a8",
"metadata": {
"editable": true
},
@@ -791,7 +791,7 @@
},
{
"cell_type": "markdown",
- "id": "99512677",
+ "id": "7af8aa06",
"metadata": {
"editable": true
},
@@ -803,7 +803,7 @@
},
{
"cell_type": "markdown",
- "id": "d2415ce3",
+ "id": "3b620f9e",
"metadata": {
"editable": true
},
@@ -815,7 +815,7 @@
},
{
"cell_type": "markdown",
- "id": "ff4c9a96",
+ "id": "8dda9b19",
"metadata": {
"editable": true
},
@@ -825,7 +825,7 @@
},
{
"cell_type": "markdown",
- "id": "32566e9b",
+ "id": "9094fb73",
"metadata": {
"editable": true
},
@@ -837,7 +837,7 @@
},
{
"cell_type": "markdown",
- "id": "5305fbde",
+ "id": "3566a77d",
"metadata": {
"editable": true
},
@@ -849,7 +849,7 @@
},
{
"cell_type": "markdown",
- "id": "3689be86",
+ "id": "afd5b520",
"metadata": {
"editable": true
},
@@ -861,7 +861,7 @@
},
{
"cell_type": "markdown",
- "id": "408012c9",
+ "id": "fedb062e",
"metadata": {
"editable": true
},
@@ -871,7 +871,7 @@
},
{
"cell_type": "markdown",
- "id": "64de98b1",
+ "id": "1c402711",
"metadata": {
"editable": true
},
@@ -883,7 +883,7 @@
},
{
"cell_type": "markdown",
- "id": "95077ce1",
+ "id": "ca3ffbe8",
"metadata": {
"editable": true
},
@@ -893,7 +893,7 @@
},
{
"cell_type": "markdown",
- "id": "5ae72845",
+ "id": "ef5649aa",
"metadata": {
"editable": true
},
@@ -912,7 +912,7 @@
},
{
"cell_type": "markdown",
- "id": "37f35940",
+ "id": "733c5a10",
"metadata": {
"editable": true
},
@@ -940,7 +940,7 @@
},
{
"cell_type": "markdown",
- "id": "cec4ccbf",
+ "id": "8ab3bfdd",
"metadata": {
"editable": true
},
@@ -966,7 +966,7 @@
},
{
"cell_type": "markdown",
- "id": "91ebab74",
+ "id": "a9ec244e",
"metadata": {
"editable": true
},
@@ -983,7 +983,7 @@
},
{
"cell_type": "markdown",
- "id": "f3ac7279",
+ "id": "c48eb948",
"metadata": {
"editable": true
},
@@ -1003,7 +1003,7 @@
},
{
"cell_type": "markdown",
- "id": "ac4c5cdb",
+ "id": "00971958",
"metadata": {
"editable": true
},
@@ -1032,7 +1032,7 @@
},
{
"cell_type": "markdown",
- "id": "83ae82a8",
+ "id": "65d9f81b",
"metadata": {
"editable": true
},
@@ -1057,7 +1057,7 @@
},
{
"cell_type": "markdown",
- "id": "4f9d0cf9",
+ "id": "cadd98e1",
"metadata": {
"editable": true
},
@@ -1071,7 +1071,7 @@
},
{
"cell_type": "markdown",
- "id": "bc98c140",
+ "id": "e652a034",
"metadata": {
"editable": true
},
@@ -1083,7 +1083,7 @@
},
{
"cell_type": "markdown",
- "id": "5ed9625e",
+ "id": "286616bf",
"metadata": {
"editable": true
},
@@ -1096,7 +1096,7 @@
},
{
"cell_type": "markdown",
- "id": "1937dce6",
+ "id": "62d35d9a",
"metadata": {
"editable": true
},
@@ -1107,7 +1107,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "49b5827a",
+ "id": "b452c740",
"metadata": {
"collapsed": false,
"editable": true
@@ -1146,7 +1146,7 @@
},
{
"cell_type": "markdown",
- "id": "d59c7cb3",
+ "id": "4423761d",
"metadata": {
"editable": true
},
@@ -1171,7 +1171,7 @@
},
{
"cell_type": "markdown",
- "id": "dc98f057",
+ "id": "9e042d22",
"metadata": {
"editable": true
},
@@ -1191,7 +1191,7 @@
},
{
"cell_type": "markdown",
- "id": "16792734",
+ "id": "17dabd08",
"metadata": {
"editable": true
},
@@ -1203,7 +1203,7 @@
},
{
"cell_type": "markdown",
- "id": "a9191ac1",
+ "id": "29decc08",
"metadata": {
"editable": true
},
@@ -1213,7 +1213,7 @@
},
{
"cell_type": "markdown",
- "id": "b512964a",
+ "id": "74dded9c",
"metadata": {
"editable": true
},
@@ -1228,7 +1228,7 @@
},
{
"cell_type": "markdown",
- "id": "ad322dc6",
+ "id": "ae938950",
"metadata": {
"editable": true
},
@@ -1241,7 +1241,7 @@
},
{
"cell_type": "markdown",
- "id": "d1149979",
+ "id": "fe91bf5a",
"metadata": {
"editable": true
},
@@ -1254,7 +1254,7 @@
},
{
"cell_type": "markdown",
- "id": "bae2c22e",
+ "id": "3e0ba223",
"metadata": {
"editable": true
},
@@ -1266,7 +1266,7 @@
},
{
"cell_type": "markdown",
- "id": "7815160c",
+ "id": "5f44dec4",
"metadata": {
"editable": true
},
@@ -1279,7 +1279,7 @@
},
{
"cell_type": "markdown",
- "id": "dac5fb75",
+ "id": "04f24465",
"metadata": {
"editable": true
},
@@ -1290,7 +1290,7 @@
},
{
"cell_type": "markdown",
- "id": "8b5e65ba",
+ "id": "c4d46b5b",
"metadata": {
"editable": true
},
@@ -1304,7 +1304,7 @@
},
{
"cell_type": "markdown",
- "id": "1b98fbe5",
+ "id": "75f7562f",
"metadata": {
"editable": true
},
@@ -1314,7 +1314,7 @@
},
{
"cell_type": "markdown",
- "id": "e7409503",
+ "id": "1b4315d0",
"metadata": {
"editable": true
},
@@ -1328,7 +1328,7 @@
},
{
"cell_type": "markdown",
- "id": "7adc92e9",
+ "id": "904ccefa",
"metadata": {
"editable": true
},
@@ -1341,7 +1341,7 @@
},
{
"cell_type": "markdown",
- "id": "9d58e53e",
+ "id": "947e35c0",
"metadata": {
"editable": true
},
@@ -1354,7 +1354,7 @@
},
{
"cell_type": "markdown",
- "id": "d63c1e78",
+ "id": "e8c626e4",
"metadata": {
"editable": true
},
@@ -1364,7 +1364,7 @@
},
{
"cell_type": "markdown",
- "id": "3ab9990a",
+ "id": "c51cb4dd",
"metadata": {
"editable": true
},
@@ -1377,7 +1377,7 @@
},
{
"cell_type": "markdown",
- "id": "385e8e52",
+ "id": "d5b1a803",
"metadata": {
"editable": true
},
@@ -1387,7 +1387,7 @@
},
{
"cell_type": "markdown",
- "id": "4dbefb62",
+ "id": "1a142b5a",
"metadata": {
"editable": true
},
@@ -1400,7 +1400,7 @@
},
{
"cell_type": "markdown",
- "id": "23f4e746",
+ "id": "0dc30002",
"metadata": {
"editable": true
},
@@ -1412,7 +1412,7 @@
},
{
"cell_type": "markdown",
- "id": "2cffab06",
+ "id": "26f23bd8",
"metadata": {
"editable": true
},
@@ -1431,7 +1431,7 @@
},
{
"cell_type": "markdown",
- "id": "04ba02de",
+ "id": "99e316c5",
"metadata": {
"editable": true
},
@@ -1444,7 +1444,7 @@
},
{
"cell_type": "markdown",
- "id": "53de7c66",
+ "id": "35830d99",
"metadata": {
"editable": true
},
@@ -1456,7 +1456,7 @@
},
{
"cell_type": "markdown",
- "id": "f5054463",
+ "id": "989bfe6a",
"metadata": {
"editable": true
},
@@ -1469,7 +1469,7 @@
},
{
"cell_type": "markdown",
- "id": "07569301",
+ "id": "9cb57daf",
"metadata": {
"editable": true
},
@@ -1489,7 +1489,7 @@
},
{
"cell_type": "markdown",
- "id": "8f6dd2ff",
+ "id": "311a55d9",
"metadata": {
"editable": true
},
@@ -1512,7 +1512,7 @@
},
{
"cell_type": "markdown",
- "id": "18e63889",
+ "id": "209dfe12",
"metadata": {
"editable": true
},
@@ -1527,7 +1527,7 @@
},
{
"cell_type": "markdown",
- "id": "66177c23",
+ "id": "9ddf4ef4",
"metadata": {
"editable": true
},
@@ -1539,7 +1539,7 @@
},
{
"cell_type": "markdown",
- "id": "10312f61",
+ "id": "4b2b7bfc",
"metadata": {
"editable": true
},
@@ -1559,7 +1559,7 @@
},
{
"cell_type": "markdown",
- "id": "12b94aae",
+ "id": "8999dff6",
"metadata": {
"editable": true
},
@@ -1579,7 +1579,7 @@
},
{
"cell_type": "markdown",
- "id": "c3f7b485",
+ "id": "982d5fbc",
"metadata": {
"editable": true
},
@@ -1603,7 +1603,7 @@
},
{
"cell_type": "markdown",
- "id": "e08528e6",
+ "id": "2ae38e48",
"metadata": {
"editable": true
},
@@ -1624,7 +1624,7 @@
},
{
"cell_type": "markdown",
- "id": "c9ac007c",
+ "id": "bd20d60a",
"metadata": {
"editable": true
},
@@ -1654,7 +1654,7 @@
},
{
"cell_type": "markdown",
- "id": "0f4a3620",
+ "id": "0c72fdb0",
"metadata": {
"editable": true
},
@@ -1678,7 +1678,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "097242cb",
+ "id": "49104a67",
"metadata": {
"collapsed": false,
"editable": true
@@ -1715,7 +1715,7 @@
},
{
"cell_type": "markdown",
- "id": "751ffe74",
+ "id": "27642741",
"metadata": {
"editable": true
},
@@ -1725,7 +1725,7 @@
},
{
"cell_type": "markdown",
- "id": "dd1cb382",
+ "id": "e91dfb35",
"metadata": {
"editable": true
},
@@ -1736,7 +1736,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "c127ea22",
+ "id": "3f088b19",
"metadata": {
"collapsed": false,
"editable": true
@@ -1756,7 +1756,7 @@
},
{
"cell_type": "markdown",
- "id": "03706aaa",
+ "id": "1d36aae1",
"metadata": {
"editable": true
},
@@ -1774,7 +1774,7 @@
},
{
"cell_type": "markdown",
- "id": "2bd565a3",
+ "id": "b1ed3d14",
"metadata": {
"editable": true
},
@@ -1786,7 +1786,7 @@
},
{
"cell_type": "markdown",
- "id": "b5ba3c44",
+ "id": "d05df5cd",
"metadata": {
"editable": true
},
@@ -1803,7 +1803,7 @@
},
{
"cell_type": "markdown",
- "id": "fd0d03ce",
+ "id": "757ce9bd",
"metadata": {
"editable": true
},
@@ -1815,7 +1815,7 @@
},
{
"cell_type": "markdown",
- "id": "9a06d7b1",
+ "id": "0e176bde",
"metadata": {
"editable": true
},
@@ -1825,7 +1825,7 @@
},
{
"cell_type": "markdown",
- "id": "f0884966",
+ "id": "6227b309",
"metadata": {
"editable": true
},
@@ -1837,7 +1837,7 @@
},
{
"cell_type": "markdown",
- "id": "9fcfd8e4",
+ "id": "ad1dab0e",
"metadata": {
"editable": true
},
@@ -1854,7 +1854,7 @@
},
{
"cell_type": "markdown",
- "id": "72786344",
+ "id": "37753004",
"metadata": {
"editable": true
},
@@ -1866,7 +1866,7 @@
},
{
"cell_type": "markdown",
- "id": "9cbe3bd1",
+ "id": "59e6e439",
"metadata": {
"editable": true
},
@@ -1876,7 +1876,7 @@
},
{
"cell_type": "markdown",
- "id": "a9f1b60c",
+ "id": "5bd100b3",
"metadata": {
"editable": true
},
@@ -1888,7 +1888,7 @@
},
{
"cell_type": "markdown",
- "id": "df279145",
+ "id": "c3a847fc",
"metadata": {
"editable": true
},
@@ -1898,7 +1898,7 @@
},
{
"cell_type": "markdown",
- "id": "cc5b4571",
+ "id": "a52ccb28",
"metadata": {
"editable": true
},
@@ -1910,7 +1910,7 @@
},
{
"cell_type": "markdown",
- "id": "0b73c25e",
+ "id": "9d662c2b",
"metadata": {
"editable": true
},
@@ -1920,7 +1920,7 @@
},
{
"cell_type": "markdown",
- "id": "160fb1bb",
+ "id": "fa90fc6f",
"metadata": {
"editable": true
},
@@ -1936,7 +1936,7 @@
},
{
"cell_type": "markdown",
- "id": "a6436245",
+ "id": "7338e614",
"metadata": {
"editable": true
},
@@ -1947,7 +1947,7 @@
{
"cell_type": "code",
"execution_count": 6,
- "id": "a51b5594",
+ "id": "af3464ee",
"metadata": {
"collapsed": false,
"editable": true
@@ -2012,7 +2012,7 @@
},
{
"cell_type": "markdown",
- "id": "81a7b921",
+ "id": "aa9520b0",
"metadata": {
"editable": true
},
@@ -2023,7 +2023,7 @@
{
"cell_type": "code",
"execution_count": 7,
- "id": "83c3d0e1",
+ "id": "73f2eca3",
"metadata": {
"collapsed": false,
"editable": true
@@ -2080,7 +2080,7 @@
},
{
"cell_type": "markdown",
- "id": "a5fccc44",
+ "id": "e8b3e361",
"metadata": {
"editable": true
},
@@ -2118,7 +2118,7 @@
},
{
"cell_type": "markdown",
- "id": "32bb1864",
+ "id": "122aad6f",
"metadata": {
"editable": true
},
@@ -2129,7 +2129,7 @@
{
"cell_type": "code",
"execution_count": 8,
- "id": "be84f020",
+ "id": "f8996c67",
"metadata": {
"collapsed": false,
"editable": true
@@ -2211,7 +2211,7 @@
},
{
"cell_type": "markdown",
- "id": "93f4f24b",
+ "id": "b877b018",
"metadata": {
"editable": true
},
@@ -2236,67 +2236,7 @@
},
{
"cell_type": "markdown",
- "id": "4775ea37",
- "metadata": {
- "editable": true
- },
- "source": [
- "## How to set up the cross-validation for Ridge and/or Lasso\n",
- "\n",
- "* Define a range of interest for the penalty parameter.\n",
- "\n",
- "* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n",
- "\n",
- "* Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\boldsymbol{\\sigma}_{-i}^2(\\lambda)$, as"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d6dc5694",
- "metadata": {
- "editable": true
- },
- "source": [
- "$$\n",
- "\\begin{align*}\n",
- "\\boldsymbol{\\beta}_{-i}(\\lambda) & = ( \\boldsymbol{X}_{-i, \\ast}^{T}\n",
- "\\boldsymbol{X}_{-i, \\ast} + \\lambda \\boldsymbol{I}_{pp})^{-1}\n",
- "\\boldsymbol{X}_{-i, \\ast}^{T} \\boldsymbol{y}_{-i}\n",
- "\\end{align*}\n",
- "$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "6f9343ce",
- "metadata": {
- "editable": true
- },
- "source": [
- "* Evaluate the prediction performance of these models on the test set by $[y_i, \\boldsymbol{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]$. Or, by the prediction error $|y_i - \\boldsymbol{X}_{i, \\ast} \\boldsymbol{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n",
- "\n",
- "* Repeat the first three steps such that each sample plays the role of the test set once.\n",
- "\n",
- "* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "c1180543",
- "metadata": {
- "editable": true
- },
- "source": [
- "$$\n",
- "\\begin{align*}\n",
- "\\frac{1}{n} \\sum_{i = 1}^n \\log\\{L[y_i, \\mathbf{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}.\n",
- "\\end{align*}\n",
- "$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "8d7e0832",
+ "id": "4ba845d0",
"metadata": {
"editable": true
},
@@ -2324,7 +2264,7 @@
},
{
"cell_type": "markdown",
- "id": "1236691a",
+ "id": "dbfb73ea",
"metadata": {
"editable": true
},
@@ -2337,7 +2277,7 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "f3da8566",
+ "id": "67a992a9",
"metadata": {
"collapsed": false,
"editable": true
@@ -2437,7 +2377,7 @@
},
{
"cell_type": "markdown",
- "id": "af0e6ba4",
+ "id": "388aeee5",
"metadata": {
"editable": true
},
@@ -2448,7 +2388,7 @@
{
"cell_type": "code",
"execution_count": 10,
- "id": "fddda52e",
+ "id": "e39bfca9",
"metadata": {
"collapsed": false,
"editable": true
@@ -2537,7 +2477,7 @@
},
{
"cell_type": "markdown",
- "id": "5fee654f",
+ "id": "10dd42f3",
"metadata": {
"editable": true
},
@@ -2547,7 +2487,7 @@
},
{
"cell_type": "markdown",
- "id": "c24b5918",
+ "id": "094bf65d",
"metadata": {
"editable": true
},
@@ -2560,7 +2500,7 @@
{
"cell_type": "code",
"execution_count": 11,
- "id": "974f9c58",
+ "id": "ea273fff",
"metadata": {
"collapsed": false,
"editable": true
diff --git a/doc/src/week37/week37.do.txt b/doc/src/week37/week37.do.txt
index b0f732ace..e16d0ee9b 100644
--- a/doc/src/week37/week37.do.txt
+++ b/doc/src/week37/week37.do.txt
@@ -1278,32 +1278,6 @@ involves a degree of randomness. This may be fully excluded when
choosing $k=n$. This particular case is referred to as leave-one-out
cross-validation (LOOCV).
-!split
-===== How to set up the cross-validation for Ridge and/or Lasso =====
-
-* Define a range of interest for the penalty parameter.
-
-* Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively.
-
-* Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\bm{\sigma}_{-i}^2(\lambda)$, as
-!bt
-\begin{align*}
-\bm{\beta}_{-i}(\lambda) & = ( \bm{X}_{-i, \ast}^{T}
-\bm{X}_{-i, \ast} + \lambda \bm{I}_{pp})^{-1}
-\bm{X}_{-i, \ast}^{T} \bm{y}_{-i}
-\end{align*}
-!et
-
-* Evaluate the prediction performance of these models on the test set by $[y_i, \bm{X}_{i, \ast}; \bm{\beta}_{-i}(\lambda), \bm{\sigma}_{-i}^2(\lambda)]$. Or, by the prediction error $|y_i - \bm{X}_{i, \ast} \bm{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function.
-
-* Repeat the first three steps such that each sample plays the role of the test set once.
-
-* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
-!bt
-\begin{align*}
-\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \bm{\beta}_{-i}(\lambda), \bm{\sigma}_{-i}^2(\lambda)]\}.
-\end{align*}
-!et
!split
===== Cross-validation in brief =====