Another approach in the LOOCV scheme is to the use the so-called Predicted Residual Error Sum of Squares (PRESS).

We can define the optimal penalty parameter to minimize $$ \begin{align*} \lambda_{\mbox{{\tiny opt}}} = \arg \min_{\lambda} \frac{1}{n} \sum_{i=1}^n [y_i - \hat{X}_{i, \ast} \hat{\beta}_{-i}(\lambda)]^2. \end{align*} $$

The LOOCV prediction performance can be expressed analytically in terms of the known quantities derived from the design matrix and the parameters \( \beta \).