From b393687a0149ab0b410237447956b98db30c3456 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Sun, 26 Sep 2021 21:56:28 +0200 Subject: [PATCH] updating week39 --- doc/pub/week39/html/week39-bs.html | 54 +++++++++---------- doc/pub/week39/html/week39-reveal.html | 16 +++--- doc/pub/week39/html/week39-solarized.html | 18 +++---- doc/pub/week39/html/week39.html | 18 +++---- doc/pub/week39/ipynb/ipynb-week39-src.tar.gz | Bin 193 -> 192 bytes doc/pub/week39/ipynb/week39.ipynb | 10 ++-- doc/src/week39/week39.do.txt | 8 +-- 7 files changed, 63 insertions(+), 61 deletions(-) diff --git a/doc/pub/week39/html/week39-bs.html b/doc/pub/week39/html/week39-bs.html index 875b934cb..67b36ade0 100644 --- a/doc/pub/week39/html/week39-bs.html +++ b/doc/pub/week39/html/week39-bs.html @@ -85,7 +85,6 @@ Automatically generated HTML file from DocOnce source 'conditions-on-convex-functions'), ('More on convex functions', 2, None, 'more-on-convex-functions'), ('Some simple problems', 2, None, 'some-simple-problems'), - ('Friday October 1', 2, None, 'friday-october-1'), ('Standard steepest descent', 2, None, @@ -155,6 +154,7 @@ Automatically generated HTML file from DocOnce source 2, None, 'using-gradient-descent-methods-limitations'), + ('Friday October 1', 2, None, 'friday-october-1'), ('Stochastic Gradient Descent', 2, None, @@ -228,31 +228,31 @@ MathJax.Hub.Config({
  • Conditions on convex functions
  • More on convex functions
  • Some simple problems
    • -
  • Friday October 1
    • -
  • Standard steepest descent
    • -
  • Gradient method
    • -
  • Steepest descent method
    • -
  • Steepest descent method
    • -
  • Final expressions
    • -
  • Steepest descent example
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method and iterations
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method
    • -
  • Conjugate gradient method
    • -
  • Revisiting our first homework
    • -
  • Gradient descent example
    • -
  • The derivative of the cost/loss function
    • -
  • The Hessian matrix
    • -
  • Simple program
    • -
  • Gradient Descent Example
    • -
  • And a corresponding example using scikit-learn
    • -
  • Gradient descent and Ridge
    • -
  • Program example for gradient descent with Ridge Regression
    • -
  • Using gradient descent methods, limitations
    • +
  • Standard steepest descent
    • +
  • Gradient method
    • +
  • Steepest descent method
    • +
  • Steepest descent method
    • +
  • Final expressions
    • +
  • Steepest descent example
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method and iterations
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method
    • +
  • Conjugate gradient method
    • +
  • Revisiting our first homework
    • +
  • Gradient descent example
    • +
  • The derivative of the cost/loss function
    • +
  • The Hessian matrix
    • +
  • Simple program
    • +
  • Gradient Descent Example
    • +
  • And a corresponding example using scikit-learn
    • +
  • Gradient descent and Ridge
    • +
  • Program example for gradient descent with Ridge Regression
    • +
  • Using gradient descent methods, limitations
    • +
  • Friday October 1
  • Stochastic Gradient Descent
  • Computation of gradients
  • SGD example
    • @@ -295,7 +295,7 @@ MathJax.Hub.Config({
    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

    -

    Sep 25, 2021

    +

    Sep 26, 2021


    diff --git a/doc/pub/week39/html/week39-reveal.html b/doc/pub/week39/html/week39-reveal.html index 2797742cb..ca4ae8057 100644 --- a/doc/pub/week39/html/week39-reveal.html +++ b/doc/pub/week39/html/week39-reveal.html @@ -148,7 +148,7 @@ MathJax.Hub.Config({

    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

     
    -

    Sep 25, 2021

    +

    Sep 26, 2021


    @@ -162,8 +162,8 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -644,11 +644,6 @@ Using the definition of convexity, try to show that a function satisfying the pr -

    -

    Friday October 1

    -
    - -

    Standard steepest descent

    @@ -1462,6 +1457,11 @@ plt.show()
    +
    +

    Friday October 1

    +
    + +

    Stochastic Gradient Descent

    diff --git a/doc/pub/week39/html/week39-solarized.html b/doc/pub/week39/html/week39-solarized.html index 74c599f9d..60a47724f 100644 --- a/doc/pub/week39/html/week39-solarized.html +++ b/doc/pub/week39/html/week39-solarized.html @@ -105,7 +105,6 @@ div { text-align: justify; text-justify: inter-word; } 'conditions-on-convex-functions'), ('More on convex functions', 2, None, 'more-on-convex-functions'), ('Some simple problems', 2, None, 'some-simple-problems'), - ('Friday October 1', 2, None, 'friday-october-1'), ('Standard steepest descent', 2, None, @@ -175,6 +174,7 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'using-gradient-descent-methods-limitations'), + ('Friday October 1', 2, None, 'friday-october-1'), ('Stochastic Gradient Descent', 2, None, @@ -233,7 +233,7 @@ MathJax.Hub.Config({
    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

    -

    Sep 25, 2021

    +

    Sep 26, 2021












    @@ -241,8 +241,8 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -690,11 +690,6 @@ Using the definition of convexity, try to show that a function satisfying the pr











    -

    Friday October 1

    - -

    -









    -

    Standard steepest descent

    @@ -1434,6 +1429,11 @@ plt.show()









    +

    Friday October 1

    + +

    +









    +

    Stochastic Gradient Descent

    diff --git a/doc/pub/week39/html/week39.html b/doc/pub/week39/html/week39.html index 727aeabc3..5fe3536e3 100644 --- a/doc/pub/week39/html/week39.html +++ b/doc/pub/week39/html/week39.html @@ -110,7 +110,6 @@ div { text-align: justify; text-justify: inter-word; } 'conditions-on-convex-functions'), ('More on convex functions', 2, None, 'more-on-convex-functions'), ('Some simple problems', 2, None, 'some-simple-problems'), - ('Friday October 1', 2, None, 'friday-october-1'), ('Standard steepest descent', 2, None, @@ -180,6 +179,7 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'using-gradient-descent-methods-limitations'), + ('Friday October 1', 2, None, 'friday-october-1'), ('Stochastic Gradient Descent', 2, None, @@ -238,7 +238,7 @@ MathJax.Hub.Config({

    [2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

    -

    Sep 25, 2021

    +

    Sep 26, 2021












    @@ -246,8 +246,8 @@ MathJax.Hub.Config({

    Plan for week 39

    @@ -695,11 +695,6 @@ Using the definition of convexity, try to show that a function satisfying the pr











    -

    Friday October 1

    - -

    -









    -

    Standard steepest descent

    @@ -1439,6 +1434,11 @@ plt.show()









    +

    Friday October 1

    + +

    +









    +

    Stochastic Gradient Descent

    diff --git a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz index 3bc88f1acab5bb79eb8e15799a63ea50c0d1296b..7c581266348d2ccbf15d3295561b3bdc8072cafc 100644 GIT binary patch literal 192 zcmV;x06+g9iwFQ*&`@Cj1MSaC3c@fD2H>uHia9|^+N9!AunQN0h!;rd)JAPmlN9ak z?E`eBxG5s!Tg*?$FwE?;)q0yn?(Txw5JFPMV8}(v$0YIe9#P7HG8!|=2~8OEh0`mlGia9|^nxw_0U>7a~5igL^)W+JRCMnw6 z+aJ)C;--j@Z}SNA2s4Llz1d}vySrd92%&^h7&A@SlqjCx6O0*Pier)vy_5mMgb^A5 zT5hG6&N^;}RhsH7loje-Z>W{!hkxc%;F*8oSV;@peeWu*Kxrp(t!uamQIl09+g{~R vXko_@7;){S5x`XsyeOoTTJcNR8htXnu~GQzV?59Eystd~ba!F+00;m837%Nv diff --git a/doc/pub/week39/ipynb/week39.ipynb b/doc/pub/week39/ipynb/week39.ipynb index d5d27c80b..d193807f0 100644 --- a/doc/pub/week39/ipynb/week39.ipynb +++ b/doc/pub/week39/ipynb/week39.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Sep 25, 2021**\n", + "Date: **Sep 26, 2021**\n", "\n", "Copyright 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -18,9 +18,9 @@ "\n", "## Plan for week 39\n", "\n", - "* Thursday: Repetition of Logistic regression equations and discussion of Gradient methods\n", + "* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods\n", "\n", - "* Friday: Stochastic Gradient descent with examples and automatic differeantion\n", + "* Friday: Stochastic Gradient descent with examples and automatic differentiation\n", "\n", "* Reading recommendations:\n", "\n", @@ -553,7 +553,6 @@ "Using the definition of convexity, try to show that a function satisfying the properties above is convex (the third condition is not needed to show this).\n", "\n", "\n", - "## Friday October 1\n", "\n", "\n", "## Standard steepest descent\n", @@ -1665,6 +1664,9 @@ "\n", "* GD can take exponential time to escape saddle points, even with random initialization. As we mentioned, GD is extremely sensitive to initial condition since it determines the particular local minimum GD would eventually reach. However, even with a good initialization scheme, through the introduction of randomness, GD can still take exponential time to escape saddle points.\n", "\n", + "## Friday October 1\n", + "\n", + "\n", "## Stochastic Gradient Descent\n", "\n", "Stochastic gradient descent (SGD) and variants thereof address some of\n", diff --git a/doc/src/week39/week39.do.txt b/doc/src/week39/week39.do.txt index 37766540d..e6aceb7aa 100644 --- a/doc/src/week39/week39.do.txt +++ b/doc/src/week39/week39.do.txt @@ -5,8 +5,8 @@ DATE: today !split ===== Plan for week 39 ===== -* Thursday: Repetition of Logistic regression equations and discussion of Gradient methods -* Friday: Stochastic Gradient descent with examples and automatic differeantion +* Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods +* Friday: Stochastic Gradient descent with examples and automatic differentiation * Reading recommendations: @@ -391,8 +391,6 @@ o A norm is any function that satisfy the following properties Using the definition of convexity, try to show that a function satisfying the properties above is convex (the third condition is not needed to show this). -!split -===== Friday October 1 ===== !split @@ -1036,6 +1034,8 @@ plt.show() * GD can take exponential time to escape saddle points, even with random initialization. As we mentioned, GD is extremely sensitive to initial condition since it determines the particular local minimum GD would eventually reach. However, even with a good initialization scheme, through the introduction of randomness, GD can still take exponential time to escape saddle points. +!split +===== Friday October 1 ===== !split