From 1b7471118989373b64f19231ff53e8b88f39ba0e Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Mon, 27 Sep 2021 17:32:18 +0200 Subject: [PATCH] added code on grid search --- doc/pub/week39/html/week39-bs.html | 109 ++++++++++--------- doc/pub/week39/html/week39-reveal.html | 19 +++- doc/pub/week39/html/week39-solarized.html | 23 +++- doc/pub/week39/html/week39.html | 23 +++- doc/pub/week39/ipynb/ipynb-week39-src.tar.gz | Bin 192 -> 193 bytes doc/pub/week39/ipynb/week39.ipynb | 19 +++- doc/src/week39/week39.do.txt | 18 ++- 7 files changed, 150 insertions(+), 61 deletions(-) diff --git a/doc/pub/week39/html/week39-bs.html b/doc/pub/week39/html/week39-bs.html index 67b36ade0..225bb300e 100644 --- a/doc/pub/week39/html/week39-bs.html +++ b/doc/pub/week39/html/week39-bs.html @@ -43,6 +43,10 @@ Automatically generated HTML file from DocOnce source {'highest level': 2, 'sections': [('Plan for week 39', 2, None, 'plan-for-week-39'), ('Thursday September 30', 2, None, 'thursday-september-30'), + ('Searching for Optimal Regularization Parameters $\\lambda$', + 2, + None, + 'searching-for-optimal-regularization-parameters-lambda'), ('Optimization, the central part of any Machine Learning ' 'algortithm', 2, @@ -211,56 +215,57 @@ MathJax.Hub.Config({ @@ -295,7 +300,7 @@ MathJax.Hub.Config({
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Sep 26, 2021

+

Sep 27, 2021


@@ -319,7 +324,7 @@ MathJax.Hub.Config({

  • 9
  • 10
  • ...
    • -
  • 53
    • +
  • 54
  • »
    • diff --git a/doc/pub/week39/html/week39-reveal.html b/doc/pub/week39/html/week39-reveal.html index ca4ae8057..07c2ef8af 100644 --- a/doc/pub/week39/html/week39-reveal.html +++ b/doc/pub/week39/html/week39-reveal.html @@ -148,7 +148,7 @@ MathJax.Hub.Config({
    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

     
    -

    Sep 26, 2021

    +

    Sep 27, 2021


    @@ -162,7 +162,7 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -181,6 +181,21 @@ For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 +
    +

    Searching for Optimal Regularization Parameters \( \lambda \)

    + +

    +In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +

    +An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. +

    + +

    Optimization, the central part of any Machine Learning algortithm

    diff --git a/doc/pub/week39/html/week39-solarized.html b/doc/pub/week39/html/week39-solarized.html index 60a47724f..d8b7b7c6c 100644 --- a/doc/pub/week39/html/week39-solarized.html +++ b/doc/pub/week39/html/week39-solarized.html @@ -63,6 +63,10 @@ div { text-align: justify; text-justify: inter-word; } {'highest level': 2, 'sections': [('Plan for week 39', 2, None, 'plan-for-week-39'), ('Thursday September 30', 2, None, 'thursday-september-30'), + ('Searching for Optimal Regularization Parameters $\\lambda$', + 2, + None, + 'searching-for-optimal-regularization-parameters-lambda'), ('Optimization, the central part of any Machine Learning ' 'algortithm', 2, @@ -233,7 +237,7 @@ MathJax.Hub.Config({
    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

    -

    Sep 26, 2021

    +

    Sep 27, 2021












    @@ -241,7 +245,7 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -260,6 +264,21 @@ For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5











    +

    Searching for Optimal Regularization Parameters \( \lambda \)

    + +

    +In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +

    +An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. + +

    +









    +

    Optimization, the central part of any Machine Learning algortithm

    diff --git a/doc/pub/week39/html/week39.html b/doc/pub/week39/html/week39.html index 5fe3536e3..8e2873538 100644 --- a/doc/pub/week39/html/week39.html +++ b/doc/pub/week39/html/week39.html @@ -68,6 +68,10 @@ div { text-align: justify; text-justify: inter-word; } {'highest level': 2, 'sections': [('Plan for week 39', 2, None, 'plan-for-week-39'), ('Thursday September 30', 2, None, 'thursday-september-30'), + ('Searching for Optimal Regularization Parameters $\\lambda$', + 2, + None, + 'searching-for-optimal-regularization-parameters-lambda'), ('Optimization, the central part of any Machine Learning ' 'algortithm', 2, @@ -238,7 +242,7 @@ MathJax.Hub.Config({

    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

    -

    Sep 26, 2021

    +

    Sep 27, 2021












    @@ -246,7 +250,7 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -265,6 +269,21 @@ For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5











    +

    Searching for Optimal Regularization Parameters \( \lambda \)

    + +

    +In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter \( \lambda \) which minimizes our +selected scores (MSE or \( R2 \) values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different \( \lambda \) values. + +

    +An alternative is to use the so-called grid search functionality included with the library Scikit-Learn, as demonstrated for the same example here. + +

    +









    +

    Optimization, the central part of any Machine Learning algortithm

    diff --git a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz index 7c581266348d2ccbf15d3295561b3bdc8072cafc..f85447f81a01035fd050c43bce7d08f1dd731633 100644 GIT binary patch literal 193 zcmV;y06za8iwFSi<56J%1MSaC3c@fD2H>uHia9|^nxw_0U>7a~5igL^)W+JRCMnw6 z+Xv`MaZ^OdxA_@n7-kOHdb7(ScXz>J5JCy1FlL&tDN#Jv6O0*PiepL=oH7Oo6Glh? zXt|SKI_tO@PHC#MP*$jSbHi9!e%Ld=0?+&t$4Xk*?t52h1xh=SYhA-lh?=Y-+4d@j vLJK>#z=&%njR3BC;6)*w)QVri*65Srjg7)zKjV3x=Y8z~4+xPG00;m8*2q;% literal 192 zcmV;x06+g9iwFQ*&`@Cj1MSaC3c@fD2H>uHia9|^+N9!AunQN0h!;rd)JAPmlN9ak z?E`eBxG5s!Tg*?$FwE?;)q0yn?(Txw5JFPMV8}(v$0YIe9#P7HG8!|=2~8OEh0` \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Sep 26, 2021**\n", + "Date: **Sep 27, 2021**\n", "\n", "Copyright 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -18,7 +18,7 @@ "\n", "## Plan for week 39\n", "\n", - "* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods\n", + "* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Discussion of project 1\n", "\n", "* Friday: Stochastic Gradient descent with examples and automatic differentiation\n", "\n", @@ -32,6 +32,21 @@ "[Overview Video, why do we care about gradient methods?](https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h20/forelesningsvideoer/OverarchingAimsWeek39.mp4?vrtx=view-as-webpage)\n", "\n", "\n", + "## Searching for Optimal Regularization Parameters $\\lambda$\n", + "\n", + "In project 1 when using Ridge and Lasso regression, we end up\n", + "searching for the optimal parameter $\\lambda$ which minimizes our\n", + "selected scores (MSE or $R2$ values for example). The brute force\n", + "approach, as discussed in the code here for Ridge regression consists\n", + "in evaluating the MSE as function of different $\\lambda$ values.\n", + "\n", + "An alternative is to use the so-called grid search functionality included with the library **Scikit-Learn**, as demonstrated for the same example here.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", "## Optimization, the central part of any Machine Learning algortithm\n", "\n", "Almost every problem in machine learning and data science starts with\n", diff --git a/doc/src/week39/week39.do.txt b/doc/src/week39/week39.do.txt index e6aceb7aa..7a6448ada 100644 --- a/doc/src/week39/week39.do.txt +++ b/doc/src/week39/week39.do.txt @@ -5,7 +5,7 @@ DATE: today !split ===== Plan for week 39 ===== -* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods +* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Discussion of project 1 * Friday: Stochastic Gradient descent with examples and automatic differentiation @@ -20,6 +20,22 @@ For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 "Overview Video, why do we care about gradient methods?":"https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h20/forelesningsvideoer/OverarchingAimsWeek39.mp4?vrtx=view-as-webpage" +!split +===== Searching for Optimal Regularization Parameters $\lambda$ ===== + +In project 1 when using Ridge and Lasso regression, we end up +searching for the optimal parameter $\lambda$ which minimizes our +selected scores (MSE or $R2$ values for example). The brute force +approach, as discussed in the code here for Ridge regression consists +in evaluating the MSE as function of different $\lambda$ values. + +An alternative is to use the so-called grid search functionality included with the library _Scikit-Learn_, as demonstrated for the same example here. + + + + + + !split ===== Optimization, the central part of any Machine Learning algortithm =====