update on html slides for regression
This commit is contained in:
@@ -1543,27 +1543,25 @@ o Define a range of interest for the penalty parameter.
|
||||
|
||||
o Divide the data set into training and test set comprising samples $\{1, \ldots, n\} \setminus i$ and $\{ i \}$, respectively.
|
||||
|
||||
o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set as
|
||||
o Fit the linear regression model by means of ridge estimation for each $\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$, as
|
||||
!bt
|
||||
\begin{align*}
|
||||
\hat{\beta}_{-i}(\lambda) & = ( \hat{X}_{-i, \ast}^{\top}
|
||||
\hat{X}_{-i, \ast} + \lambda \hat{I}_{pp})^{-1}
|
||||
\hat{X}_{-i, \ast}^{\top} \hat{y}_{-i}
|
||||
\end{align*}
|
||||
!et
|
||||
and the corresponding estimate of the error variance $\hat{\sigma}_{-i}^2(\lambda)$.
|
||||
!et
|
||||
|
||||
o Evaluate the prediction performance of these models on the test set by $\log\{L[y_i, \hat{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}$. Or, by the prediction error $|y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)|$, the relative error, the error squared or the R2 score function.
|
||||
|
||||
o Repeat steps 1) to 3) such that each sample plays the role of the test set once.
|
||||
|
||||
o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter
|
||||
o Average the prediction performances of the test sets at each grid point of the penalty bias/parameter by computing the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as
|
||||
!bt
|
||||
\begin{align*}
|
||||
\frac{1}{n} \sum_{i = 1}^n \log\{L[Y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}.
|
||||
\frac{1}{n} \sum_{i = 1}^n \log\{L[y_i, \mathbf{X}_{i, \ast}; \hat{\beta}_{-i}(\lambda), \hat{\sigma}_{-i}^2(\lambda)]\}.
|
||||
\end{align*}
|
||||
!et
|
||||
The quantity above is called the *cross-validated log-likelihood*. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data.
|
||||
|
||||
o The value of the penalty parameter that maximizes the cross-validated log-likelihood is the value of choice. Or we can use the MSE or the R2 score functions.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user