update on html slides for regression

This commit is contained in:
mhjensen
2018-09-07 05:40:44 +02:00
parent 9d1de4186e
commit 4291b46bcb
17 changed files with 2648 additions and 28 deletions
+4 -6
View File
@@ -1543,27 +1543,25 @@ o Define a range of interest for the penalty parameter.
o Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively.
o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set as
o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as
!bt
\begin{align*}
\hat{\beta}_{-i}(\lambda) & = ( \hat{X}_{-i, \ast}^{\top}
\hat{X}_{-i, \ast} + \lambda \hat{I}_{pp})^{-1}
\hat{X}_{-i, \ast}^{\top} \hat{y}_{-i}
\end{align*}
!et
and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$.
!et
o Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function.
o Repeat steps 1) to 3) such that each sample plays the role of the test set once.
o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter
o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
!bt
\begin{align*}
\frac{1}{n} \sum_{i = 1}^n \log\{L[Y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}.
\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}.
\end{align*}
!et
The quantity above is called the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data.
o The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.