update on html slides for regression
This commit is contained in:
@@ -262,11 +262,11 @@ MathJax.Hub.Config({
|
||||
|
||||
<h2 id="___sec50" class="anchor">How to set up the cross-validation for Ridge and/or Lasso </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Define a range of interest for the penalty parameter.</li>
|
||||
<li> Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.</li>
|
||||
<li> Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \hat{\sigma}_{-i}^2(\lambda) \), as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -277,11 +277,11 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Evaluate the prediction performance of these models on the test set by \( \log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\} \). Or, by the prediction error \( |y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.</li>
|
||||
<li> Repeat steps 1) to 3) such that each sample plays the role of the test set once.</li>
|
||||
<li> Repeat the first three steps such that each sample plays the role of the test set once.</li>
|
||||
<li> Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the <em>cross-validated log-likelihood</em>. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -290,9 +290,9 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
<p>
|
||||
<!-- navigation buttons at the bottom of the page -->
|
||||
|
||||
@@ -1904,11 +1904,11 @@ cross-validation (LOOCV).
|
||||
<section>
|
||||
<h2 id="___sec50">How to set up the cross-validation for Ridge and/or Lasso </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> Define a range of interest for the penalty parameter.</li>
|
||||
<p><li> Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.</li>
|
||||
<p><li> Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \hat{\sigma}_{-i}^2(\lambda) \), as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p> <br>
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1920,11 +1920,11 @@ $$
|
||||
<p> <br>
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> Evaluate the prediction performance of these models on the test set by \( \log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\} \). Or, by the prediction error \( |y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.</li>
|
||||
<p><li> Repeat steps 1) to 3) such that each sample plays the role of the test set once.</li>
|
||||
<p><li> Repeat the first three steps such that each sample plays the role of the test set once.</li>
|
||||
<p><li> Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the <em>cross-validated log-likelihood</em>. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p> <br>
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1934,9 +1934,9 @@ $$
|
||||
<p> <br>
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
@@ -1863,11 +1863,11 @@ cross-validation (LOOCV).
|
||||
|
||||
<h2 id="___sec50">How to set up the cross-validation for Ridge and/or Lasso </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Define a range of interest for the penalty parameter.</li>
|
||||
<li> Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.</li>
|
||||
<li> Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \hat{\sigma}_{-i}^2(\lambda) \), as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1878,11 +1878,11 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Evaluate the prediction performance of these models on the test set by \( \log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\} \). Or, by the prediction error \( |y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.</li>
|
||||
<li> Repeat steps 1) to 3) such that each sample plays the role of the test set once.</li>
|
||||
<li> Repeat the first three steps such that each sample plays the role of the test set once.</li>
|
||||
<li> Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the <em>cross-validated log-likelihood</em>. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1891,9 +1891,9 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
|
||||
|
||||
@@ -1868,11 +1868,11 @@ cross-validation (LOOCV).
|
||||
|
||||
<h2 id="___sec50">How to set up the cross-validation for Ridge and/or Lasso </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Define a range of interest for the penalty parameter.</li>
|
||||
<li> Divide the data set into training and test set comprising samples \( \{1, \ldots, n\} \setminus i \) and \( \{ i \} \), respectively.</li>
|
||||
<li> Fit the linear regression model by means of ridge estimation for each \( \lambda \) in the grid using the training set, and the corresponding estimate of the error variance \( \hat{\sigma}_{-i}^2(\lambda) \), as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1883,11 +1883,11 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Evaluate the prediction performance of these models on the test set by \( \log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\} \). Or, by the prediction error \( |y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)| \), the relative error, the error squared or the R2 score function.</li>
|
||||
<li> Repeat steps 1) to 3) such that each sample plays the role of the test set once.</li>
|
||||
<li> Repeat the first three steps such that each sample plays the role of the test set once.</li>
|
||||
<li> Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the <em>cross-validated log-likelihood</em>. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
@@ -1896,9 +1896,9 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
|
||||
|
||||
@@ -2267,11 +2267,11 @@
|
||||
"<!-- !split -->\n",
|
||||
"## How to set up the cross-validation for Ridge and/or Lasso\n",
|
||||
"\n",
|
||||
"1. Define a range of interest for the penalty parameter.\n",
|
||||
"* Define a range of interest for the penalty parameter.\n",
|
||||
"\n",
|
||||
"2. Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n",
|
||||
"* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n",
|
||||
"\n",
|
||||
"3. Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\hat{\\sigma}_{-i}^2(\\lambda)$, as"
|
||||
"* Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\hat{\\sigma}_{-i}^2(\\lambda)$, as"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -2291,11 +2291,11 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"1. Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\hat{X}_{i, \\ast}; \\hat{\\beta}_{-i}(\\lambda), \\hat{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\hat{X}_{i, \\ast} \\hat{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n",
|
||||
"* Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\hat{X}_{i, \\ast}; \\hat{\\beta}_{-i}(\\lambda), \\hat{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\hat{X}_{i, \\ast} \\hat{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n",
|
||||
"\n",
|
||||
"2. Repeat steps 1) to 3) such that each sample plays the role of the test set once.\n",
|
||||
"* Repeat the first three steps such that each sample plays the role of the test set once.\n",
|
||||
"\n",
|
||||
"3. Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as"
|
||||
"* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -2313,7 +2313,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"1. The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.\n",
|
||||
"* The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.\n",
|
||||
"\n",
|
||||
"## Predicted Residual Error Sum of Squares\n",
|
||||
"Another approach in the LOOCV scheme is to the use the so-called Predicted Residual Error Sum of Squares (PRESS). \n",
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1539,11 +1539,11 @@ cross-validation (LOOCV).
|
||||
!split
|
||||
===== How to set up the cross-validation for Ridge and/or Lasso =====
|
||||
|
||||
o Define a range of interest for the penalty parameter.
|
||||
* Define a range of interest for the penalty parameter.
|
||||
|
||||
o Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively.
|
||||
* Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively.
|
||||
|
||||
o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as
|
||||
* Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as
|
||||
!bt
|
||||
\begin{align*}
|
||||
\hat{\beta}_{-i}(\lambda) & = ( \hat{X}_{-i, \ast}^{\top}
|
||||
@@ -1552,18 +1552,18 @@ o Fit the linear regression model by means of ridge estimation for each $\lambd
|
||||
\end{align*}
|
||||
!et
|
||||
|
||||
o Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function.
|
||||
* Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function.
|
||||
|
||||
o Repeat steps 1) to 3) such that each sample plays the role of the test set once.
|
||||
* Repeat the first three steps such that each sample plays the role of the test set once.
|
||||
|
||||
o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
|
||||
* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
|
||||
!bt
|
||||
\begin{align*}
|
||||
\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}.
|
||||
\end{align*}
|
||||
!et
|
||||
|
||||
o The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.
|
||||
* The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.
|
||||
|
||||
!split
|
||||
===== Predicted Residual Error Sum of Squares =====
|
||||
|
||||
Reference in New Issue
Block a user