diff --git a/doc/pub/Regression/html/._Regression-bs051.html b/doc/pub/Regression/html/._Regression-bs051.html index 7542dbe4a..26b39e68a 100644 --- a/doc/pub/Regression/html/._Regression-bs051.html +++ b/doc/pub/Regression/html/._Regression-bs051.html @@ -262,11 +262,11 @@ MathJax.Hub.Config({

How to set up the cross-validation for Ridge and/or Lasso

-
    +
+ $$ \begin{align*} @@ -277,11 +277,11 @@ $$ $$ -
    +
+ $$ \begin{align*} @@ -290,9 +290,9 @@ $$ $$ -
    +
+

diff --git a/doc/pub/Regression/html/Regression-reveal.html b/doc/pub/Regression/html/Regression-reveal.html index 63a356a89..df03e6c5f 100644 --- a/doc/pub/Regression/html/Regression-reveal.html +++ b/doc/pub/Regression/html/Regression-reveal.html @@ -1904,11 +1904,11 @@ cross-validation (LOOCV).

How to set up the cross-validation for Ridge and/or Lasso

-
    +
+

 
$$ \begin{align*} @@ -1920,11 +1920,11 @@ $$

 
-

    +
+

 
$$ \begin{align*} @@ -1934,9 +1934,9 @@ $$

 
-

    +
+
diff --git a/doc/pub/Regression/html/Regression-solarized.html b/doc/pub/Regression/html/Regression-solarized.html index 88feb089c..575b33a94 100644 --- a/doc/pub/Regression/html/Regression-solarized.html +++ b/doc/pub/Regression/html/Regression-solarized.html @@ -1863,11 +1863,11 @@ cross-validation (LOOCV).

How to set up the cross-validation for Ridge and/or Lasso

-
    +
+ $$ \begin{align*} @@ -1878,11 +1878,11 @@ $$ $$ -
    +
+ $$ \begin{align*} @@ -1891,9 +1891,9 @@ $$ $$ -
    +
+









diff --git a/doc/pub/Regression/html/Regression.html b/doc/pub/Regression/html/Regression.html index a53c0e587..a6ee19e22 100644 --- a/doc/pub/Regression/html/Regression.html +++ b/doc/pub/Regression/html/Regression.html @@ -1868,11 +1868,11 @@ cross-validation (LOOCV).

How to set up the cross-validation for Ridge and/or Lasso

-
    +
+ $$ \begin{align*} @@ -1883,11 +1883,11 @@ $$ $$ -
    +
+ $$ \begin{align*} @@ -1896,9 +1896,9 @@ $$ $$ -
    +
+









diff --git a/doc/pub/Regression/ipynb/Regression.ipynb b/doc/pub/Regression/ipynb/Regression.ipynb index 09033bf6f..f86beff59 100644 --- a/doc/pub/Regression/ipynb/Regression.ipynb +++ b/doc/pub/Regression/ipynb/Regression.ipynb @@ -2267,11 +2267,11 @@ "\n", "## How to set up the cross-validation for Ridge and/or Lasso\n", "\n", - "1. Define a range of interest for the penalty parameter.\n", + "* Define a range of interest for the penalty parameter.\n", "\n", - "2. Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n", + "* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n", "\n", - "3. Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\hat{\\sigma}_{-i}^2(\\lambda)$, as" + "* Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\hat{\\sigma}_{-i}^2(\\lambda)$, as" ] }, { @@ -2291,11 +2291,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "1. Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\hat{X}_{i, \\ast}; \\hat{\\beta}_{-i}(\\lambda), \\hat{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\hat{X}_{i, \\ast} \\hat{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n", + "* Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\hat{X}_{i, \\ast}; \\hat{\\beta}_{-i}(\\lambda), \\hat{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\hat{X}_{i, \\ast} \\hat{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n", "\n", - "2. Repeat steps 1) to 3) such that each sample plays the role of the test set once.\n", + "* Repeat the first three steps such that each sample plays the role of the test set once.\n", "\n", - "3. Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as" + "* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as" ] }, { @@ -2313,7 +2313,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "1. The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.\n", + "* The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.\n", "\n", "## Predicted Residual Error Sum of Squares\n", "Another approach in the LOOCV scheme is to the use the so-called Predicted Residual Error Sum of Squares (PRESS). \n", diff --git a/doc/pub/Regression/ipynb/ipynb-Regression-src.tar.gz b/doc/pub/Regression/ipynb/ipynb-Regression-src.tar.gz index c0e43e801..0e95f5af2 100644 Binary files a/doc/pub/Regression/ipynb/ipynb-Regression-src.tar.gz and b/doc/pub/Regression/ipynb/ipynb-Regression-src.tar.gz differ diff --git a/doc/pub/Regression/pdf/Regression-beamer-handouts2x3.pdf b/doc/pub/Regression/pdf/Regression-beamer-handouts2x3.pdf index 38f2d7194..406466c7a 100644 Binary files a/doc/pub/Regression/pdf/Regression-beamer-handouts2x3.pdf and b/doc/pub/Regression/pdf/Regression-beamer-handouts2x3.pdf differ diff --git a/doc/pub/Regression/pdf/Regression-beamer.pdf b/doc/pub/Regression/pdf/Regression-beamer.pdf index 3b5a3ea7d..aa07d3550 100644 Binary files a/doc/pub/Regression/pdf/Regression-beamer.pdf and b/doc/pub/Regression/pdf/Regression-beamer.pdf differ diff --git a/doc/pub/Regression/pdf/Regression-minted.pdf b/doc/pub/Regression/pdf/Regression-minted.pdf index 25211c937..84c286073 100644 Binary files a/doc/pub/Regression/pdf/Regression-minted.pdf and b/doc/pub/Regression/pdf/Regression-minted.pdf differ diff --git a/doc/src/Regression/Regression.do.txt b/doc/src/Regression/Regression.do.txt index 605835410..914eed14e 100644 --- a/doc/src/Regression/Regression.do.txt +++ b/doc/src/Regression/Regression.do.txt @@ -1539,11 +1539,11 @@ cross-validation (LOOCV). !split ===== How to set up the cross-validation for Ridge and/or Lasso ===== -o Define a range of interest for the penalty parameter. +* Define a range of interest for the penalty parameter. -o Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively. +* Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively. -o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as +* Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as !bt \begin{align*} \hat{\beta}_{-i}(\lambda) & = ( \hat{X}_{-i, \ast}^{\top} @@ -1552,18 +1552,18 @@ o Fit the linear regression model by means of ridge estimation for each $\lambd \end{align*} !et -o Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function. +* Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function. -o Repeat steps 1) to 3) such that each sample plays the role of the test set once. +* Repeat the first three steps such that each sample plays the role of the test set once. -o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as +* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as !bt \begin{align*} \frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}. \end{align*} !et -o The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions. +* The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions. !split ===== Predicted Residual Error Sum of Squares =====