-
@@ -319,7 +324,7 @@ MathJax.Hub.Config({
-
@@ -162,7 +162,7 @@ MathJax.Hub.Config({
+In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +
+An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. +
-
@@ -241,7 +245,7 @@ MathJax.Hub.Config({
+
+In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +
+An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. + +
+
+
diff --git a/doc/pub/week39/html/week39.html b/doc/pub/week39/html/week39.html index 5fe3536e3..8e2873538 100644 --- a/doc/pub/week39/html/week39.html +++ b/doc/pub/week39/html/week39.html @@ -68,6 +68,10 @@ div { text-align: justify; text-justify: inter-word; } {'highest level': 2, 'sections': [('Plan for week 39', 2, None, 'plan-for-week-39'), ('Thursday September 30', 2, None, 'thursday-september-30'), + ('Searching for Optimal Regularization Parameters $\\lambda$', + 2, + None, + 'searching-for-optimal-regularization-parameters-lambda'), ('Optimization, the central part of any Machine Learning ' 'algortithm', 2, @@ -238,7 +242,7 @@ MathJax.Hub.Config({
-
@@ -246,7 +250,7 @@ MathJax.Hub.Config({
+
+In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +
+An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. + +
+
+
diff --git a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz index 7c5812663..f85447f81 100644 Binary files a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz and b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz differ diff --git a/doc/pub/week39/ipynb/week39.ipynb b/doc/pub/week39/ipynb/week39.ipynb index d193807f0..01746bceb 100644 --- a/doc/pub/week39/ipynb/week39.ipynb +++ b/doc/pub/week39/ipynb/week39.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Sep 26, 2021**\n", + "Date: **Sep 27, 2021**\n", "\n", "Copyright 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -18,7 +18,7 @@ "\n", "## Plan for week 39\n", "\n", - "* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods\n", + "* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Discussion of project 1\n", "\n", "* Friday: Stochastic Gradient descent with examples and automatic differentiation\n", "\n", @@ -32,6 +32,21 @@ "[Overview Video, why do we care about gradient methods?](https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h20/forelesningsvideoer/OverarchingAimsWeek39.mp4?vrtx=view-as-webpage)\n", "\n", "\n", + "## Searching for Optimal Regularization Parameters $\\lambda$\n", + "\n", + "In project 1 when using Ridge and Lasso regression, we end up\n", + "searching for the optimal parameter $\\lambda$ which minimizes our\n", + "selected scores (MSE or $R2$ values for example). The brute force\n", + "approach, as discussed in the code here for Ridge regression consists\n", + "in evaluating the MSE as function of different $\\lambda$ values.\n", + "\n", + "An alternative is to use the so-called grid search functionality included with the library **Scikit-Learn**, as demonstrated for the same example here.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", "## Optimization, the central part of any Machine Learning algortithm\n", "\n", "Almost every problem in machine learning and data science starts with\n", diff --git a/doc/src/week39/week39.do.txt b/doc/src/week39/week39.do.txt index e6aceb7aa..7a6448ada 100644 --- a/doc/src/week39/week39.do.txt +++ b/doc/src/week39/week39.do.txt @@ -5,7 +5,7 @@ DATE: today !split ===== Plan for week 39 ===== -* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods +* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Discussion of project 1 * Friday: Stochastic Gradient descent with examples and automatic differentiation @@ -20,6 +20,22 @@ For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 "Overview Video, why do we care about gradient methods?":"https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h20/forelesningsvideoer/OverarchingAimsWeek39.mp4?vrtx=view-as-webpage" +!split +===== Searching for Optimal Regularization Parameters $\lambda$ ===== + +In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter $\lambda$ which minimizes our +selected scores (MSE or $R2$ values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different $\lambda$ values. + +An alternative is to use the so-called grid search functionality included with the library _Scikit-Learn_, as demonstrated for the same example here. + + + + + + !split ===== Optimization, the central part of any Machine Learning algortithm =====