lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating week 36

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-09-08 13:49:10 +02:00
parent b8ca339713
commit 0f1d69e035
7 changed files with 214 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week36/html/week36-bs.html
+35 -33
View File
@@ -87,6 +87,7 @@ Automatically generated HTML file from DocOnce source
('Lasso Regression', 2, None, 'lasso-regression'),
('Yet another Example', 2, None, 'yet-another-example'),
('The OLS case', 2, None, 'the-ols-case'),
('The Ridge case', 2, None, 'the-ridge-case'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
@@ -245,38 +246,39 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week36-bs016.html#lasso-regression" style="font-size: 80%;">Lasso Regression</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs017.html#yet-another-example" style="font-size: 80%;">Yet another Example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs018.html#the-ols-case" style="font-size: 80%;">The OLS case</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs019.html#linking-the-regression-analysis-with-a-statistical-interpretation" style="font-size: 80%;">Linking the regression analysis with a statistical interpretation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs020.html#assumptions-made" style="font-size: 80%;">Assumptions made</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs021.html#expectation-value-and-variance" style="font-size: 80%;">Expectation value and variance</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs022.html#expectation-value-and-variance-for-boldsymbol-beta" style="font-size: 80%;">Expectation value and variance for \( \boldsymbol{\beta} \)</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs023.html#friday-september-10" style="font-size: 80%;">Friday September 10</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs027.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs029.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs026.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs027.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs028.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs029.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs030.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs031.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs032.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs033.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs034.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs035.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs036.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs037.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs038.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs041.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs045.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs019.html#the-ridge-case" style="font-size: 80%;">The Ridge case</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs020.html#linking-the-regression-analysis-with-a-statistical-interpretation" style="font-size: 80%;">Linking the regression analysis with a statistical interpretation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs021.html#assumptions-made" style="font-size: 80%;">Assumptions made</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs022.html#expectation-value-and-variance" style="font-size: 80%;">Expectation value and variance</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs023.html#expectation-value-and-variance-for-boldsymbol-beta" style="font-size: 80%;">Expectation value and variance for \( \boldsymbol{\beta} \)</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs024.html#friday-september-10" style="font-size: 80%;">Friday September 10</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs028.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs030.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs027.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs028.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs029.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs030.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs031.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs032.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs033.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs034.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs035.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs036.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs037.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs038.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs041.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs045.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
</ul>
</li>
@@ -335,7 +337,7 @@ MathJax.Hub.Config({
<li><a href="._week36-bs008.html">9</a></li>
<li><a href="._week36-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week36-bs050.html">51</a></li>
<li><a href="._week36-bs051.html">52</a></li>
<li><a href="._week36-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week36/html/week36-reveal.html
+37 -2
View File
@@ -693,7 +693,7 @@ For ordinary least squares (OLS) we know that the optimal solution is
<p>&nbsp;<br>
$$
\hat{\boldsymbol{\beta}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
<p>&nbsp;<br>
@@ -701,9 +701,44 @@ Inserting the above values we obtain that
<p>&nbsp;<br>
$$
\hat{\boldsymbol{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
$$
<p>&nbsp;<br>
<p>
Computing the mean squared error we obtian a value of \( 0.27 \).
<p>
The code which implements this simpler case is presented after the discussion of Ridge and Lasso.
</section>
<section>
<h2 id="the-ridge-case">The Ridge case </h2>
<p>
For Ridge regression we have
<p>&nbsp;<br>
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\left( \boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
<p>&nbsp;<br>
Inserting the above values we obtain that
<p>&nbsp;<br>
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\begin{matrix}\frac{11}{5+\lambda} \\ \frac{2}{1+\lambda}\end{bmatrix},
$$
<p>&nbsp;<br>
<p>
There is normally a constraint on the value of $\vert\vert \boldsymbol{\beta}\vert\vert_2 via the parameter \( \lambda \).
Let us for simplicity assume now that \( \beta_0^2+\beta_1^2=1 \) as constraint. This will allow us to find an expression with the optimal values of \( \beta \) and \( \lambda \).
<p>
To see this, let us write the cost function for Ridge regression.
</section>
doc/pub/week36/html/week36-solarized.html
+34 -2
View File
@@ -107,6 +107,7 @@ div { text-align: justify; text-justify: inter-word; }
('Lasso Regression', 2, None, 'lasso-regression'),
('Yet another Example', 2, None, 'yet-another-example'),
('The OLS case', 2, None, 'the-ols-case'),
('The Ridge case', 2, None, 'the-ridge-case'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
@@ -706,15 +707,46 @@ meaning that we have two features and two unknown parameters \( \beta_0 \) and \
For ordinary least squares (OLS) we know that the optimal solution is
$$
\hat{\boldsymbol{\beta}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
Inserting the above values we obtain that
$$
\hat{\boldsymbol{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
$$
<p>
Computing the mean squared error we obtian a value of \( 0.27 \).
<p>
The code which implements this simpler case is presented after the discussion of Ridge and Lasso.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="the-ridge-case">The Ridge case </h2>
<p>
For Ridge regression we have
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\left( \boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
Inserting the above values we obtain that
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\begin{matrix}\frac{11}{5+\lambda} \\ \frac{2}{1+\lambda}\end{bmatrix},
$$
<p>
There is normally a constraint on the value of $\vert\vert \boldsymbol{\beta}\vert\vert_2 via the parameter \( \lambda \).
Let us for simplicity assume now that \( \beta_0^2+\beta_1^2=1 \) as constraint. This will allow us to find an expression with the optimal values of \( \beta \) and \( \lambda \).
<p>
To see this, let us write the cost function for Ridge regression.
<p>
<!-- !split -->
doc/pub/week36/html/week36.html
+34 -2
View File
@@ -112,6 +112,7 @@ div { text-align: justify; text-justify: inter-word; }
('Lasso Regression', 2, None, 'lasso-regression'),
('Yet another Example', 2, None, 'yet-another-example'),
('The OLS case', 2, None, 'the-ols-case'),
('The Ridge case', 2, None, 'the-ridge-case'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
@@ -711,15 +712,46 @@ meaning that we have two features and two unknown parameters \( \beta_0 \) and \
For ordinary least squares (OLS) we know that the optimal solution is
$$
\hat{\boldsymbol{\beta}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\left( \boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
Inserting the above values we obtain that
$$
\hat{\boldsymbol{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
\hat{\boldsymbol{\beta}}^{\mathrm{OLS}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
$$
<p>
Computing the mean squared error we obtian a value of \( 0.27 \).
<p>
The code which implements this simpler case is presented after the discussion of Ridge and Lasso.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="the-ridge-case">The Ridge case </h2>
<p>
For Ridge regression we have
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\left( \boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
$$
Inserting the above values we obtain that
$$
\hat{\boldsymbol{\beta}}^{\mathrm{Ridge}}=\begin{matrix}\frac{11}{5+\lambda} \\ \frac{2}{1+\lambda}\end{bmatrix},
$$
<p>
There is normally a constraint on the value of $\vert\vert \boldsymbol{\beta}\vert\vert_2 via the parameter \( \lambda \).
Let us for simplicity assume now that \( \beta_0^2+\beta_1^2=1 \) as constraint. This will allow us to find an expression with the optimal values of \( \beta \) and \( \lambda \).
<p>
To see this, let us write the cost function for Ridge regression.
<p>
<!-- !split -->
doc/pub/week36/ipynb/ipynb-week36-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week36/ipynb/week36.ipynb
+46 -2
View File
@@ -824,7 +824,7 @@
"metadata": {},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}=\\left( \\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
"\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}=\\left( \\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
"$$"
]
},
@@ -840,7 +840,7 @@
"metadata": {},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}=\\begin{matrix}\\frac{11}{5} \\\\ 2\\end{bmatrix},\n",
"\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}=\\begin{matrix}\\frac{11}{5} \\\\ 2\\end{bmatrix},\n",
"$$"
]
},
@@ -848,6 +848,50 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Computing the mean squared error we obtian a value of $0.27$. \n",
"\n",
"The code which implements this simpler case is presented after the discussion of Ridge and Lasso.\n",
"\n",
"## The Ridge case\n",
"\n",
"For Ridge regression we have"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}=\\left( \\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Inserting the above values we obtain that"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}=\\begin{matrix}\\frac{11}{5+\\lambda} \\\\ \\frac{2}{1+\\lambda}\\end{bmatrix},\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There is normally a constraint on the value of $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2 via the parameter $\\lambda$.\n",
"Let us for simplicity assume now that $\\beta_0^2+\\beta_1^2=1$ as constraint. This will allow us to find an expression with the optimal values of $\\beta$ and $\\lambda$.\n",
"\n",
"To see this, let us write the cost function for Ridge regression. \n",
"\n",
"\n",
"<!-- !split -->\n",
"## Linking the regression analysis with a statistical interpretation\n",
"\n",
doc/src/week36/week36.do.txt
+28 -2
View File
@@ -454,17 +454,43 @@ For ordinary least squares (OLS) we know that the optimal solution is
!bt
\[
\hat{\bm{\beta}}=\left( \bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
\hat{\bm{\beta}}^{\mathrm{OLS}}=\left( \bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
\]
!et
Inserting the above values we obtain that
!bt
\[
\hat{\bm{\beta}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
\hat{\bm{\beta}}^{\mathrm{OLS}}=\begin{matrix}\frac{11}{5} \\ 2\end{bmatrix},
\]
!et
Computing the mean squared error we obtian a value of $0.27$.
The code which implements this simpler case is presented after the discussion of Ridge and Lasso.
!split
===== The Ridge case =====
For Ridge regression we have
!bt
\[
\hat{\bm{\beta}}^{\mathrm{Ridge}}=\left( \bm{X}^T\bm{X}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}.
\]
!et
Inserting the above values we obtain that
!bt
\[
\hat{\bm{\beta}}^{\mathrm{Ridge}}=\begin{matrix}\frac{11}{5+\lambda} \\ \frac{2}{1+\lambda}\end{bmatrix},
\]
!et
There is normally a constraint on the value of $\vert\vert \bm{\beta}\vert\vert_2 via the parameter $\lambda$.
Let us for simplicity assume now that $\beta_0^2+\beta_1^2=1$ as constraint. This will allow us to find an expression with the optimal values of $\beta$ and $\lambda$.
To see this, let us write the cost function for Ridge regression.
!split