From af7737872cb2a5bede490afd494002e4df33ee7f Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Sat, 23 Nov 2024 22:44:33 +0100 Subject: [PATCH] typos --- doc/pub/week48/html/week48-reveal.html | 61 ++-- doc/pub/week48/html/week48-solarized.html | 52 ++- doc/pub/week48/html/week48.html | 52 ++- doc/pub/week48/ipynb/ipynb-week48-src.tar.gz | Bin 823660 -> 823660 bytes doc/pub/week48/ipynb/week48.ipynb | 357 +++++++++---------- doc/src/week48/week48.do.txt | 52 ++- 6 files changed, 271 insertions(+), 303 deletions(-) diff --git a/doc/pub/week48/html/week48-reveal.html b/doc/pub/week48/html/week48-reveal.html index f77f43024..1ceb2e043 100644 --- a/doc/pub/week48/html/week48-reveal.html +++ b/doc/pub/week48/html/week48-reveal.html @@ -243,7 +243,7 @@ MathJax.Hub.Config({

The algorithm described here can be applied to both classification and regression problems.

We will grow of forest of say \( B \) trees.

-
    +
+ +

+

Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.

@@ -1361,8 +1362,7 @@ plt.show()

Machine learning

-

The following topics will be covered

-
    +

    • Linear methods for regression and classification:

      1. Ordinary Least Squares
      2. @@ -1393,7 +1393,7 @@ plt.show()

      3. Regression

      -

+
@@ -1468,17 +1468,17 @@ Neural Networks, etc.

Choose a Model and Algorithm

-
    +

    • Supervised?
    • Start with the simplest model that fits your problem
    • Start with minimal processing of data
    • -
+

Preparing Your Data

-
    +

    • Shuffle your data
    • Mean center your data
      • @@ -1500,51 +1500,38 @@ Neural Networks, etc.

      • Can be hit or miss

      -

    • When to do train/test split?
    • -
+

  • When to do train/test split?
  • +

    Which activation and weights to choose in neural networks

    -
      +

      • RELU? ELU? GELU? etc
      • Sigmoid or Tanh?
      • -

      • Set all weights to 0?
      • -
          - -

        • Terrible idea
        • +

        • Set all weights to 0? Terrible idea
        • +

        • Set all weights to random values? Small random values
        -

        -

      • Set all weights to random values?
      • -
          - -

        • Small random values
        • -
        -

        -

    Optimization Methods and Hyperparameters

    -
      -

    1. Stochastic gradient descent -
        -

      1. Stochastic gradient descent + momentum
      2. -
      -

      -

    2. State-of-the-art approaches:
      • - -

      • RMSProp
      • - -

      • Adam
      • - -

      • and more
      • +

      • Stochastic gradient descent
      • +
          +

        • Stochastic gradient descent + momentum

        +

      • State-of-the-art approaches: +
          +

        1. RMSProp
        2. +

        3. Adam
        4. +

        5. and more

        +

      +

      Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods. diff --git a/doc/pub/week48/html/week48-solarized.html b/doc/pub/week48/html/week48-solarized.html index bff9c030e..18f9c74c8 100644 --- a/doc/pub/week48/html/week48-solarized.html +++ b/doc/pub/week48/html/week48-solarized.html @@ -353,7 +353,7 @@ MathJax.Hub.Config({

      The algorithm described here can be applied to both classification and regression problems.

      We will grow of forest of say \( B \) trees.

      -
        +
        • For \( b=1:B \)
          1. Draw a bootstrap sample from the training data organized in our \( \boldsymbol{X} \) matrix.
          2. @@ -364,8 +364,9 @@ MathJax.Hub.Config({
          3. split the node into daughter nodes
      -
    3. Output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.
    4. -
    + +

    Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.

    +









    Random Forests Compared with other Methods on the Cancer Data

    @@ -1377,8 +1378,7 @@ plt.show()









    Machine learning

    -

    The following topics will be covered

    -
      +
      • Linear methods for regression and classification:
        1. Ordinary Least Squares
        2. @@ -1405,7 +1405,7 @@ plt.show()
        3. Kernel methods
        4. Regression
        -
    +









    Learning outcomes and overarching aims of this course

    @@ -1470,15 +1470,15 @@ Neural Networks, etc.









    Choose a Model and Algorithm

    -
      +
      • Supervised?
      • Start with the simplest model that fits your problem
      • Start with minimal processing of data
      • -
    +









    Preparing Your Data

    -
      +
      • Shuffle your data
      • Mean center your data
        • @@ -1493,37 +1493,31 @@ Neural Networks, etc.
        • Decorrelates data
        • Can be hit or miss
        -
      • When to do train/test split?
      • -
    +
  • When to do train/test split?
  • +









    Which activation and weights to choose in neural networks

    -
      +
      • RELU? ELU? GELU? etc
      • Sigmoid or Tanh?
      • -
      • Set all weights to 0?
      • -
          -
        • Terrible idea
        • +
        • Set all weights to 0? Terrible idea
        • +
        • Set all weights to random values? Small random values
        -
      • Set all weights to random values?
      • -
          -
        • Small random values
        • -
        -










    Optimization Methods and Hyperparameters

    -
      -
    1. Stochastic gradient descent -
        -
      1. Stochastic gradient descent + momentum
      2. -
      -
    2. State-of-the-art approaches:
      • -
      • RMSProp
      • -
      • Adam
      • -
      • and more
      • +
      • Stochastic gradient descent
      • +
          +
        • Stochastic gradient descent + momentum
        +
      • State-of-the-art approaches: +
          +
        1. RMSProp
        2. +
        3. Adam
        4. +
        5. and more
        +

      Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods. diff --git a/doc/pub/week48/html/week48.html b/doc/pub/week48/html/week48.html index 010721e6d..a0d63e6fe 100644 --- a/doc/pub/week48/html/week48.html +++ b/doc/pub/week48/html/week48.html @@ -430,7 +430,7 @@ MathJax.Hub.Config({

      The algorithm described here can be applied to both classification and regression problems.

      We will grow of forest of say \( B \) trees.

      -
        +
        • For \( b=1:B \)
          1. Draw a bootstrap sample from the training data organized in our \( \boldsymbol{X} \) matrix.
          2. @@ -441,8 +441,9 @@ MathJax.Hub.Config({
          3. split the node into daughter nodes
      -
    3. Output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.
    4. -
    + +

    Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.

    +









    Random Forests Compared with other Methods on the Cancer Data

    @@ -1454,8 +1455,7 @@ plt.show()









    Machine learning

    -

    The following topics will be covered

    -
      +
      • Linear methods for regression and classification:
        1. Ordinary Least Squares
        2. @@ -1482,7 +1482,7 @@ plt.show()
        3. Kernel methods
        4. Regression
        -
    +









    Learning outcomes and overarching aims of this course

    @@ -1547,15 +1547,15 @@ Neural Networks, etc.









    Choose a Model and Algorithm

    -
      +
      • Supervised?
      • Start with the simplest model that fits your problem
      • Start with minimal processing of data
      • -
    +









    Preparing Your Data

    -
      +
      • Shuffle your data
      • Mean center your data
        • @@ -1570,37 +1570,31 @@ Neural Networks, etc.
        • Decorrelates data
        • Can be hit or miss
        -
      • When to do train/test split?
      • -
    +
  • When to do train/test split?
  • +









    Which activation and weights to choose in neural networks

    -
      +
      • RELU? ELU? GELU? etc
      • Sigmoid or Tanh?
      • -
      • Set all weights to 0?
      • -
          -
        • Terrible idea
        • +
        • Set all weights to 0? Terrible idea
        • +
        • Set all weights to random values? Small random values
        -
      • Set all weights to random values?
      • -
          -
        • Small random values
        • -
        -










    Optimization Methods and Hyperparameters

    -
      -
    1. Stochastic gradient descent -
        -
      1. Stochastic gradient descent + momentum
      2. -
      -
    2. State-of-the-art approaches:
      • -
      • RMSProp
      • -
      • Adam
      • -
      • and more
      • +
      • Stochastic gradient descent
      • +
          +
        • Stochastic gradient descent + momentum
        +
      • State-of-the-art approaches: +
          +
        1. RMSProp
        2. +
        3. Adam
        4. +
        5. and more
        +

      Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods. diff --git a/doc/pub/week48/ipynb/ipynb-week48-src.tar.gz b/doc/pub/week48/ipynb/ipynb-week48-src.tar.gz index ba9af317afb7adf07e95743b00eaf6d0ba2f8984..d6def3c343811dc416a2fbce45749cd6eb41f6e2 100644 GIT binary patch delta 53 zcmaE}$mq=?BR2VN4u%~*jcl!KjIC@;t!&J#Y%Hy8tgUQ75%yL#j#f6#RyM9yHtwx# HJZTdFq`(cm delta 53 zcmaE}$mq=?BR2VN4u*N&jcl!KjIC@;t!&J#Y%Hy8tgUQ75%yL#j#f6#RyM9yHtwx# HJZTdFpi>Q^ diff --git a/doc/pub/week48/ipynb/week48.ipynb b/doc/pub/week48/ipynb/week48.ipynb index f289f46c8..486829beb 100644 --- a/doc/pub/week48/ipynb/week48.ipynb +++ b/doc/pub/week48/ipynb/week48.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "fe7fd644", + "id": "3b64c66e", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "c407135f", + "id": "45ba1703", "metadata": { "editable": true }, @@ -29,7 +29,7 @@ }, { "cell_type": "markdown", - "id": "1894285d", + "id": "7258309e", "metadata": { "editable": true }, @@ -39,7 +39,7 @@ }, { "cell_type": "markdown", - "id": "bceac1c4", + "id": "8a4ef0a3", "metadata": { "editable": true }, @@ -85,7 +85,7 @@ }, { "cell_type": "markdown", - "id": "eaaa2de8", + "id": "e01d37bc", "metadata": { "editable": true }, @@ -95,7 +95,7 @@ "The algorithm described here can be applied to both classification and regression problems.\n", "\n", "We will grow of forest of say $B$ trees.\n", - "1. For $b=1:B$\n", + "* For $b=1:B$\n", "\n", "a. Draw a bootstrap sample from the training data organized in our $\\boldsymbol{X}$ matrix.\n", "\n", @@ -107,12 +107,12 @@ "\n", "3. split the node into daughter nodes\n", "\n", - "4. Output then the ensemble of trees $\\{T_b\\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem." + "Finally we output then the ensemble of trees $\\{T_b\\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem." ] }, { "cell_type": "markdown", - "id": "36809e3f", + "id": "1160253d", "metadata": { "editable": true }, @@ -123,7 +123,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "ef24e54c", + "id": "ac77382f", "metadata": { "collapsed": false, "editable": true @@ -197,7 +197,7 @@ }, { "cell_type": "markdown", - "id": "0472d908", + "id": "254c4495", "metadata": { "editable": true }, @@ -213,7 +213,7 @@ }, { "cell_type": "markdown", - "id": "08e636dd", + "id": "5633e7a9", "metadata": { "editable": true }, @@ -224,7 +224,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "90e2c43c", + "id": "250bca6d", "metadata": { "collapsed": false, "editable": true @@ -239,7 +239,7 @@ { "cell_type": "code", "execution_count": 3, - "id": "929962a3", + "id": "a8506102", "metadata": { "collapsed": false, "editable": true @@ -257,7 +257,7 @@ }, { "cell_type": "markdown", - "id": "a2039540", + "id": "8713c888", "metadata": { "editable": true }, @@ -277,7 +277,7 @@ }, { "cell_type": "markdown", - "id": "140e30ce", + "id": "efc5e7e7", "metadata": { "editable": true }, @@ -291,7 +291,7 @@ }, { "cell_type": "markdown", - "id": "02cb3a63", + "id": "ee68d9b3", "metadata": { "editable": true }, @@ -303,7 +303,7 @@ }, { "cell_type": "markdown", - "id": "9924d331", + "id": "6efcd942", "metadata": { "editable": true }, @@ -320,7 +320,7 @@ }, { "cell_type": "markdown", - "id": "6e00565f", + "id": "8b2a4e15", "metadata": { "editable": true }, @@ -332,7 +332,7 @@ }, { "cell_type": "markdown", - "id": "de290cb8", + "id": "77d4f75d", "metadata": { "editable": true }, @@ -346,7 +346,7 @@ }, { "cell_type": "markdown", - "id": "fd4fac8f", + "id": "9d0680d2", "metadata": { "editable": true }, @@ -358,7 +358,7 @@ }, { "cell_type": "markdown", - "id": "1ddfce8e", + "id": "a5f51b6a", "metadata": { "editable": true }, @@ -371,7 +371,7 @@ }, { "cell_type": "markdown", - "id": "9e18b5ba", + "id": "c6eafc82", "metadata": { "editable": true }, @@ -383,7 +383,7 @@ }, { "cell_type": "markdown", - "id": "52663df1", + "id": "d42a3420", "metadata": { "editable": true }, @@ -393,7 +393,7 @@ }, { "cell_type": "markdown", - "id": "a6bfc5a8", + "id": "ecac5a19", "metadata": { "editable": true }, @@ -421,7 +421,7 @@ }, { "cell_type": "markdown", - "id": "bd232678", + "id": "632d7379", "metadata": { "editable": true }, @@ -437,7 +437,7 @@ }, { "cell_type": "markdown", - "id": "8f945bbd", + "id": "3a2dc7cd", "metadata": { "editable": true }, @@ -449,7 +449,7 @@ }, { "cell_type": "markdown", - "id": "14894a48", + "id": "d1dae1f7", "metadata": { "editable": true }, @@ -460,7 +460,7 @@ }, { "cell_type": "markdown", - "id": "31d3d404", + "id": "db8e52c8", "metadata": { "editable": true }, @@ -472,7 +472,7 @@ }, { "cell_type": "markdown", - "id": "0d86e417", + "id": "b71e4562", "metadata": { "editable": true }, @@ -482,7 +482,7 @@ }, { "cell_type": "markdown", - "id": "dba5d81b", + "id": "8b287320", "metadata": { "editable": true }, @@ -494,7 +494,7 @@ }, { "cell_type": "markdown", - "id": "9946eed4", + "id": "444b31c7", "metadata": { "editable": true }, @@ -504,7 +504,7 @@ }, { "cell_type": "markdown", - "id": "671b57e9", + "id": "975f549b", "metadata": { "editable": true }, @@ -516,7 +516,7 @@ }, { "cell_type": "markdown", - "id": "2f1afb73", + "id": "49707e43", "metadata": { "editable": true }, @@ -526,7 +526,7 @@ }, { "cell_type": "markdown", - "id": "eb06d7bb", + "id": "30fff982", "metadata": { "editable": true }, @@ -538,7 +538,7 @@ }, { "cell_type": "markdown", - "id": "bd75f76e", + "id": "8936f422", "metadata": { "editable": true }, @@ -552,7 +552,7 @@ }, { "cell_type": "markdown", - "id": "7a2457ae", + "id": "373a4a0c", "metadata": { "editable": true }, @@ -568,7 +568,7 @@ }, { "cell_type": "markdown", - "id": "88fa4f5c", + "id": "c412ab0e", "metadata": { "editable": true }, @@ -580,7 +580,7 @@ }, { "cell_type": "markdown", - "id": "e100093d", + "id": "0018a744", "metadata": { "editable": true }, @@ -596,7 +596,7 @@ }, { "cell_type": "markdown", - "id": "a96c6add", + "id": "d092eb4b", "metadata": { "editable": true }, @@ -608,7 +608,7 @@ }, { "cell_type": "markdown", - "id": "4ee0fb37", + "id": "17820826", "metadata": { "editable": true }, @@ -618,7 +618,7 @@ }, { "cell_type": "markdown", - "id": "b7206f37", + "id": "f4910537", "metadata": { "editable": true }, @@ -630,7 +630,7 @@ }, { "cell_type": "markdown", - "id": "ca864761", + "id": "bfae7a07", "metadata": { "editable": true }, @@ -642,7 +642,7 @@ }, { "cell_type": "markdown", - "id": "b8776574", + "id": "f00bee3c", "metadata": { "editable": true }, @@ -654,7 +654,7 @@ }, { "cell_type": "markdown", - "id": "7afe77b7", + "id": "bf267f5b", "metadata": { "editable": true }, @@ -665,7 +665,7 @@ }, { "cell_type": "markdown", - "id": "1915c382", + "id": "a6eb9909", "metadata": { "editable": true }, @@ -677,7 +677,7 @@ }, { "cell_type": "markdown", - "id": "57398dbb", + "id": "2bbceaad", "metadata": { "editable": true }, @@ -688,7 +688,7 @@ }, { "cell_type": "markdown", - "id": "b63693b2", + "id": "9ff79322", "metadata": { "editable": true }, @@ -700,7 +700,7 @@ }, { "cell_type": "markdown", - "id": "09e68eb6", + "id": "6c4299e2", "metadata": { "editable": true }, @@ -710,7 +710,7 @@ }, { "cell_type": "markdown", - "id": "bf86c857", + "id": "068e2930", "metadata": { "editable": true }, @@ -722,7 +722,7 @@ }, { "cell_type": "markdown", - "id": "cb3d645c", + "id": "53eb174c", "metadata": { "editable": true }, @@ -734,7 +734,7 @@ }, { "cell_type": "markdown", - "id": "2157edce", + "id": "3baeddb4", "metadata": { "editable": true }, @@ -746,7 +746,7 @@ }, { "cell_type": "markdown", - "id": "b9e39b81", + "id": "87178662", "metadata": { "editable": true }, @@ -758,7 +758,7 @@ }, { "cell_type": "markdown", - "id": "941fa37d", + "id": "c48316bd", "metadata": { "editable": true }, @@ -768,7 +768,7 @@ }, { "cell_type": "markdown", - "id": "3ce9f347", + "id": "6ccbebe4", "metadata": { "editable": true }, @@ -780,7 +780,7 @@ }, { "cell_type": "markdown", - "id": "d26d3711", + "id": "526ec37c", "metadata": { "editable": true }, @@ -790,7 +790,7 @@ }, { "cell_type": "markdown", - "id": "44bce843", + "id": "84e556c0", "metadata": { "editable": true }, @@ -802,7 +802,7 @@ }, { "cell_type": "markdown", - "id": "1c997239", + "id": "931a10f7", "metadata": { "editable": true }, @@ -812,7 +812,7 @@ }, { "cell_type": "markdown", - "id": "216bdcdb", + "id": "45b79d8b", "metadata": { "editable": true }, @@ -824,7 +824,7 @@ }, { "cell_type": "markdown", - "id": "373c405a", + "id": "e30b9dee", "metadata": { "editable": true }, @@ -834,7 +834,7 @@ }, { "cell_type": "markdown", - "id": "c57bddad", + "id": "c8d42197", "metadata": { "editable": true }, @@ -846,7 +846,7 @@ }, { "cell_type": "markdown", - "id": "5e936ef6", + "id": "56d6acfd", "metadata": { "editable": true }, @@ -856,7 +856,7 @@ }, { "cell_type": "markdown", - "id": "2c9eaf83", + "id": "df837af9", "metadata": { "editable": true }, @@ -868,7 +868,7 @@ }, { "cell_type": "markdown", - "id": "bdecca1e", + "id": "1fafddb8", "metadata": { "editable": true }, @@ -888,7 +888,7 @@ }, { "cell_type": "markdown", - "id": "fea06c34", + "id": "8a898c02", "metadata": { "editable": true }, @@ -900,7 +900,7 @@ }, { "cell_type": "markdown", - "id": "0025dfdf", + "id": "03435f7c", "metadata": { "editable": true }, @@ -910,7 +910,7 @@ }, { "cell_type": "markdown", - "id": "13f97440", + "id": "ac433aeb", "metadata": { "editable": true }, @@ -926,7 +926,7 @@ }, { "cell_type": "markdown", - "id": "ee2b62a1", + "id": "37e79583", "metadata": { "editable": true }, @@ -938,7 +938,7 @@ }, { "cell_type": "markdown", - "id": "35cc8d6f", + "id": "6eff57bb", "metadata": { "editable": true }, @@ -966,7 +966,7 @@ }, { "cell_type": "markdown", - "id": "90ca86cd", + "id": "143fddc5", "metadata": { "editable": true }, @@ -979,7 +979,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "d3803033", + "id": "627dcde1", "metadata": { "collapsed": false, "editable": true @@ -1004,7 +1004,7 @@ }, { "cell_type": "markdown", - "id": "ac68d8bc", + "id": "0586f2f5", "metadata": { "editable": true }, @@ -1015,7 +1015,7 @@ { "cell_type": "code", "execution_count": 5, - "id": "679b3966", + "id": "57f72ee8", "metadata": { "collapsed": false, "editable": true @@ -1104,7 +1104,7 @@ }, { "cell_type": "markdown", - "id": "58adfdb1", + "id": "98bcb1fd", "metadata": { "editable": true }, @@ -1122,7 +1122,7 @@ }, { "cell_type": "markdown", - "id": "ca89c2b5", + "id": "adfad730", "metadata": { "editable": true }, @@ -1135,7 +1135,7 @@ }, { "cell_type": "markdown", - "id": "bee751db", + "id": "32e3c207", "metadata": { "editable": true }, @@ -1147,7 +1147,7 @@ }, { "cell_type": "markdown", - "id": "64ec846c", + "id": "460df33e", "metadata": { "editable": true }, @@ -1157,7 +1157,7 @@ }, { "cell_type": "markdown", - "id": "2af461db", + "id": "ec1e9c33", "metadata": { "editable": true }, @@ -1169,7 +1169,7 @@ }, { "cell_type": "markdown", - "id": "b6502394", + "id": "5f517f3c", "metadata": { "editable": true }, @@ -1179,7 +1179,7 @@ }, { "cell_type": "markdown", - "id": "48d243dc", + "id": "8352e0e4", "metadata": { "editable": true }, @@ -1191,7 +1191,7 @@ }, { "cell_type": "markdown", - "id": "e03e872f", + "id": "b158a9ff", "metadata": { "editable": true }, @@ -1204,7 +1204,7 @@ }, { "cell_type": "markdown", - "id": "d1613021", + "id": "17fb6c35", "metadata": { "editable": true }, @@ -1216,7 +1216,7 @@ }, { "cell_type": "markdown", - "id": "d5e7fe64", + "id": "64c802de", "metadata": { "editable": true }, @@ -1228,7 +1228,7 @@ }, { "cell_type": "markdown", - "id": "0bd5bef5", + "id": "f4b660f8", "metadata": { "editable": true }, @@ -1240,7 +1240,7 @@ }, { "cell_type": "markdown", - "id": "ef5252a7", + "id": "d592d84b", "metadata": { "editable": true }, @@ -1250,7 +1250,7 @@ }, { "cell_type": "markdown", - "id": "7cb83769", + "id": "265f57f6", "metadata": { "editable": true }, @@ -1262,7 +1262,7 @@ }, { "cell_type": "markdown", - "id": "6b0d28d0", + "id": "149a5a32", "metadata": { "editable": true }, @@ -1272,7 +1272,7 @@ }, { "cell_type": "markdown", - "id": "4762090e", + "id": "fb347cb0", "metadata": { "editable": true }, @@ -1288,7 +1288,7 @@ }, { "cell_type": "markdown", - "id": "59124920", + "id": "66b696bb", "metadata": { "editable": true }, @@ -1300,7 +1300,7 @@ }, { "cell_type": "markdown", - "id": "70bba454", + "id": "6c776311", "metadata": { "editable": true }, @@ -1321,7 +1321,7 @@ }, { "cell_type": "markdown", - "id": "97c28672", + "id": "b7cecfb0", "metadata": { "editable": true }, @@ -1332,7 +1332,7 @@ { "cell_type": "code", "execution_count": 6, - "id": "380a6f12", + "id": "7b1eb376", "metadata": { "collapsed": false, "editable": true @@ -1383,7 +1383,7 @@ }, { "cell_type": "markdown", - "id": "02a1a349", + "id": "9d97b3d7", "metadata": { "editable": true }, @@ -1394,7 +1394,7 @@ { "cell_type": "code", "execution_count": 7, - "id": "931c8836", + "id": "7e853c55", "metadata": { "collapsed": false, "editable": true @@ -1442,7 +1442,7 @@ }, { "cell_type": "markdown", - "id": "ccd968ba", + "id": "b458d799", "metadata": { "editable": true }, @@ -1465,7 +1465,7 @@ }, { "cell_type": "markdown", - "id": "6ec7c62b", + "id": "48d1de5e", "metadata": { "editable": true }, @@ -1478,7 +1478,7 @@ { "cell_type": "code", "execution_count": 8, - "id": "b18ceffd", + "id": "4059e0f6", "metadata": { "collapsed": false, "editable": true @@ -1536,7 +1536,7 @@ }, { "cell_type": "markdown", - "id": "9d4b045f", + "id": "da6ac07a", "metadata": { "editable": true }, @@ -1547,7 +1547,7 @@ { "cell_type": "code", "execution_count": 9, - "id": "2bf430ed", + "id": "8ab70463", "metadata": { "collapsed": false, "editable": true @@ -1634,7 +1634,7 @@ }, { "cell_type": "markdown", - "id": "2c3330d7", + "id": "d3761d54", "metadata": { "editable": true }, @@ -1644,7 +1644,7 @@ }, { "cell_type": "markdown", - "id": "a67ab616", + "id": "780df2c0", "metadata": { "editable": true }, @@ -1659,7 +1659,7 @@ }, { "cell_type": "markdown", - "id": "00a62442", + "id": "8ead86e5", "metadata": { "editable": true }, @@ -1675,7 +1675,7 @@ }, { "cell_type": "markdown", - "id": "483051ba", + "id": "31266704", "metadata": { "editable": true }, @@ -1700,15 +1700,14 @@ }, { "cell_type": "markdown", - "id": "7429c058", + "id": "2021a43b", "metadata": { "editable": true }, "source": [ "## Machine learning\n", "\n", - "The following topics will be covered\n", - "1. Linear methods for regression and classification:\n", + "* Linear methods for regression and classification:\n", "\n", "a. Ordinary Least Squares\n", "\n", @@ -1718,7 +1717,7 @@ "\n", "d. Logistic regression\n", "\n", - "5. Neural networks and deep learning:\n", + "* Neural networks and deep learning:\n", "\n", "a. Feed Forward Neural Networks\n", "\n", @@ -1726,7 +1725,7 @@ "\n", "c. Recurrent Neural Networks\n", "\n", - "4. Decisions trees and ensemble methods:\n", + "* Decisions trees and ensemble methods:\n", "\n", "a. Decision trees\n", "\n", @@ -1736,7 +1735,7 @@ "\n", "d. Boosting and gradient boosting\n", "\n", - "5. Not discussed this year: Support vector machines\n", + "* Not discussed this year: Support vector machines\n", "\n", "a. Binary classification and multiclass classification\n", "\n", @@ -1747,7 +1746,7 @@ }, { "cell_type": "markdown", - "id": "3ab2e37a", + "id": "170e1be1", "metadata": { "editable": true }, @@ -1782,7 +1781,7 @@ }, { "cell_type": "markdown", - "id": "b4a54e3f", + "id": "07f2b380", "metadata": { "editable": true }, @@ -1805,7 +1804,7 @@ }, { "cell_type": "markdown", - "id": "68e0ad8c", + "id": "4408cc2b", "metadata": { "editable": true }, @@ -1828,7 +1827,7 @@ }, { "cell_type": "markdown", - "id": "319b20fd", + "id": "13833ad1", "metadata": { "editable": true }, @@ -1848,89 +1847,85 @@ }, { "cell_type": "markdown", - "id": "8060a087", + "id": "690b357b", "metadata": { "editable": true }, "source": [ "## Choose a Model and Algorithm\n", "\n", - "1. Supervised?\n", + "* Supervised?\n", "\n", - "2. Start with the simplest model that fits your problem\n", + "* Start with the simplest model that fits your problem\n", "\n", - "3. Start with minimal processing of data" + "* Start with minimal processing of data" ] }, { "cell_type": "markdown", - "id": "be25b51b", + "id": "a382da4c", "metadata": { "editable": true }, "source": [ "## Preparing Your Data\n", "\n", - "1. Shuffle your data\n", + "* Shuffle your data\n", "\n", - "2. Mean center your data\n", + "* Mean center your data\n", "\n", " * Why?\n", "\n", - "3. Normalize the variance\n", + "* Normalize the variance\n", "\n", " * Why?\n", "\n", - "4. **Whitening**\n", + "* **Whitening**\n", "\n", " * Decorrelates data\n", "\n", " * Can be hit or miss\n", "\n", - "5. When to do train/test split?" + " * When to do train/test split?" ] }, { "cell_type": "markdown", - "id": "2c67fffd", + "id": "aef19b0c", "metadata": { "editable": true }, "source": [ "## Which activation and weights to choose in neural networks\n", "\n", - "1. RELU? ELU? GELU? etc\n", + "* RELU? ELU? GELU? etc\n", "\n", - "2. Sigmoid or Tanh?\n", + "* Sigmoid or Tanh?\n", "\n", - "3. Set all weights to 0?\n", + "* Set all weights to 0? Terrible idea\n", "\n", - " * Terrible idea\n", - "\n", - "4. Set all weights to random values?\n", - "\n", - " * Small random values" + "* Set all weights to random values? Small random values" ] }, { "cell_type": "markdown", - "id": "6ab2cdf4", + "id": "f301c68e", "metadata": { "editable": true }, "source": [ "## Optimization Methods and Hyperparameters\n", - "1. Stochastic gradient descent\n", + "* Stochastic gradient descent\n", "\n", - "a. Stochastic gradient descent + momentum\n", + " * Stochastic gradient descent + momentum\n", "\n", - "2. State-of-the-art approaches:\n", + "* State-of-the-art approaches:\n", "\n", - " * RMSProp\n", + "a. RMSProp\n", "\n", - " * Adam\n", + "b. Adam\n", "\n", - " * and more\n", + "c. and more\n", "\n", "Which regularization and hyperparameters? $L_1$ or $L_2$, soft\n", "classifiers, depths of trees and many other. Need to explore a large\n", @@ -1939,7 +1934,7 @@ }, { "cell_type": "markdown", - "id": "8a5e039f", + "id": "7f82dc43", "metadata": { "editable": true }, @@ -1957,7 +1952,7 @@ }, { "cell_type": "markdown", - "id": "9a3946d1", + "id": "7a1117b1", "metadata": { "editable": true }, @@ -1987,7 +1982,7 @@ }, { "cell_type": "markdown", - "id": "d9e6d355", + "id": "90fe2084", "metadata": { "editable": true }, @@ -2001,7 +1996,7 @@ }, { "cell_type": "markdown", - "id": "7e99ee2d", + "id": "bc8ad0a3", "metadata": { "editable": true }, @@ -2027,7 +2022,7 @@ }, { "cell_type": "markdown", - "id": "1f94c42f", + "id": "bea1957b", "metadata": { "editable": true }, @@ -2056,7 +2051,7 @@ }, { "cell_type": "markdown", - "id": "f5e8269a", + "id": "c1347ce6", "metadata": { "editable": true }, @@ -2073,7 +2068,7 @@ }, { "cell_type": "markdown", - "id": "6174662e", + "id": "5bcb69f0", "metadata": { "editable": true }, @@ -2095,7 +2090,7 @@ }, { "cell_type": "markdown", - "id": "36165057", + "id": "39597386", "metadata": { "editable": true }, @@ -2113,7 +2108,7 @@ }, { "cell_type": "markdown", - "id": "659b2e21", + "id": "24cb4d77", "metadata": { "editable": true }, @@ -2136,7 +2131,7 @@ }, { "cell_type": "markdown", - "id": "925e9817", + "id": "abe0e029", "metadata": { "editable": true }, @@ -2155,7 +2150,7 @@ }, { "cell_type": "markdown", - "id": "0bc5fe3d", + "id": "be9ea72c", "metadata": { "editable": true }, @@ -2171,7 +2166,7 @@ }, { "cell_type": "markdown", - "id": "68a1f231", + "id": "a85f3d50", "metadata": { "editable": true }, @@ -2186,7 +2181,7 @@ }, { "cell_type": "markdown", - "id": "4aa0e3cf", + "id": "3b68f9bc", "metadata": { "editable": true }, @@ -2210,7 +2205,7 @@ }, { "cell_type": "markdown", - "id": "e440589a", + "id": "eeb46a9a", "metadata": { "editable": true }, @@ -2222,7 +2217,7 @@ }, { "cell_type": "markdown", - "id": "dd9c38a0", + "id": "911fc5be", "metadata": { "editable": true }, @@ -2240,7 +2235,7 @@ }, { "cell_type": "markdown", - "id": "5c393934", + "id": "44398f76", "metadata": { "editable": true }, @@ -2250,7 +2245,7 @@ }, { "cell_type": "markdown", - "id": "ab552000", + "id": "d22991a2", "metadata": { "editable": true }, @@ -2268,7 +2263,7 @@ }, { "cell_type": "markdown", - "id": "878ae3c8", + "id": "c645e455", "metadata": { "editable": true }, @@ -2278,7 +2273,7 @@ }, { "cell_type": "markdown", - "id": "b830eb6d", + "id": "99a56dc1", "metadata": { "editable": true }, @@ -2297,7 +2292,7 @@ }, { "cell_type": "markdown", - "id": "683b809a", + "id": "9a39b491", "metadata": { "editable": true }, @@ -2309,7 +2304,7 @@ }, { "cell_type": "markdown", - "id": "adec988a", + "id": "286d0ab5", "metadata": { "editable": true }, @@ -2323,7 +2318,7 @@ }, { "cell_type": "markdown", - "id": "7c370896", + "id": "e21e58ac", "metadata": { "editable": true }, @@ -2338,7 +2333,7 @@ }, { "cell_type": "markdown", - "id": "1b07308c", + "id": "d9e99856", "metadata": { "editable": true }, @@ -2356,7 +2351,7 @@ }, { "cell_type": "markdown", - "id": "0e443612", + "id": "da27eff3", "metadata": { "editable": true }, @@ -2370,7 +2365,7 @@ }, { "cell_type": "markdown", - "id": "13f2d272", + "id": "ad466922", "metadata": { "editable": true }, @@ -2388,7 +2383,7 @@ }, { "cell_type": "markdown", - "id": "8c9ac9f3", + "id": "e3e1a9ee", "metadata": { "editable": true }, @@ -2412,7 +2407,7 @@ }, { "cell_type": "markdown", - "id": "610c29b7", + "id": "8a463ba1", "metadata": { "editable": true }, @@ -2450,7 +2445,7 @@ }, { "cell_type": "markdown", - "id": "096e7470", + "id": "8b88bdc9", "metadata": { "editable": true }, @@ -2473,7 +2468,7 @@ }, { "cell_type": "markdown", - "id": "fbf18ec6", + "id": "dcf0e541", "metadata": { "editable": true }, @@ -2509,7 +2504,7 @@ }, { "cell_type": "markdown", - "id": "30ca6ef5", + "id": "9d1c6243", "metadata": { "editable": true }, @@ -2530,7 +2525,7 @@ }, { "cell_type": "markdown", - "id": "99d1df35", + "id": "b06e1c4c", "metadata": { "editable": true }, @@ -2552,7 +2547,7 @@ }, { "cell_type": "markdown", - "id": "9abcc525", + "id": "8bd5d6c4", "metadata": { "editable": true }, @@ -2572,7 +2567,7 @@ }, { "cell_type": "markdown", - "id": "ca2124ca", + "id": "40c7ca7e", "metadata": { "editable": true }, @@ -2587,7 +2582,7 @@ }, { "cell_type": "markdown", - "id": "5858078e", + "id": "bc167c7d", "metadata": { "editable": true }, @@ -2605,7 +2600,7 @@ }, { "cell_type": "markdown", - "id": "174ee0c8", + "id": "06e830c9", "metadata": { "editable": true }, @@ -2635,7 +2630,7 @@ }, { "cell_type": "markdown", - "id": "e2c44b50", + "id": "51b7b00d", "metadata": { "editable": true }, @@ -2664,7 +2659,7 @@ }, { "cell_type": "markdown", - "id": "6f4f1c4b", + "id": "265badfe", "metadata": { "editable": true }, @@ -2695,7 +2690,7 @@ }, { "cell_type": "markdown", - "id": "3b86535f", + "id": "28cda775", "metadata": { "editable": true }, @@ -2718,7 +2713,7 @@ }, { "cell_type": "markdown", - "id": "51363de6", + "id": "eab731bc", "metadata": { "editable": true }, @@ -2736,7 +2731,7 @@ }, { "cell_type": "markdown", - "id": "e8a5a5bf", + "id": "105015b2", "metadata": { "editable": true }, @@ -2759,7 +2754,7 @@ }, { "cell_type": "markdown", - "id": "0249293e", + "id": "ddbe105b", "metadata": { "editable": true }, @@ -2780,7 +2775,7 @@ }, { "cell_type": "markdown", - "id": "bfcd921f", + "id": "3e53243c", "metadata": { "editable": true }, @@ -2796,7 +2791,7 @@ }, { "cell_type": "markdown", - "id": "57427c59", + "id": "011485c3", "metadata": { "editable": true }, diff --git a/doc/src/week48/week48.do.txt b/doc/src/week48/week48.do.txt index 1022ae3e2..abdaa7f1f 100644 --- a/doc/src/week48/week48.do.txt +++ b/doc/src/week48/week48.do.txt @@ -38,13 +38,14 @@ o Readings and Videos: The algorithm described here can be applied to both classification and regression problems. We will grow of forest of say $B$ trees. -o For $b=1:B$ +* For $b=1:B$ o Draw a bootstrap sample from the training data organized in our $\bm{X}$ matrix. o We grow then a random forest tree $T_b$ based on the bootstrapped data by repeating the steps outlined till we reach the maximum node size is reached o we select $m \le p$ variables at random from the $p$ predictors/features o pick the best split point among the $m$ features using for example the CART algorithm and create a new node o split the node into daughter nodes -o Output then the ensemble of trees $\{T_b\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem. + +Finally we output then the ensemble of trees $\{T_b\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem. @@ -925,22 +926,21 @@ o Not discussed: Principal Component Analysis to reduce the number of features. !split ===== Machine learning ===== -The following topics will be covered -o Linear methods for regression and classification: +* Linear methods for regression and classification: o Ordinary Least Squares o Ridge regression o Lasso regression o Logistic regression -o Neural networks and deep learning: +* Neural networks and deep learning: o Feed Forward Neural Networks o Convolutional Neural Networks o Recurrent Neural Networks -o Decisions trees and ensemble methods: +* Decisions trees and ensemble methods: o Decision trees o Bagging and voting o Random forests o Boosting and gradient boosting -o Not discussed this year: Support vector machines +* Not discussed this year: Support vector machines o Binary classification and multiclass classification o Kernel methods o Regression @@ -1011,43 +1011,41 @@ o Based on your results, feedback loop to earliest possible point !split ===== Choose a Model and Algorithm ===== -o Supervised? -o Start with the simplest model that fits your problem -o Start with minimal processing of data +* Supervised? +* Start with the simplest model that fits your problem +* Start with minimal processing of data !split ===== Preparing Your Data ===== -o Shuffle your data -o Mean center your data +* Shuffle your data +* Mean center your data * Why? -o Normalize the variance +* Normalize the variance * Why? -o _Whitening_ +* _Whitening_ * Decorrelates data * Can be hit or miss -o When to do train/test split? +* When to do train/test split? !split ===== Which activation and weights to choose in neural networks ===== -o RELU? ELU? GELU? etc -o Sigmoid or Tanh? -o Set all weights to 0? - * Terrible idea -o Set all weights to random values? - * Small random values +* RELU? ELU? GELU? etc +* Sigmoid or Tanh? +* Set all weights to 0? Terrible idea +* Set all weights to random values? Small random values !split ===== Optimization Methods and Hyperparameters ===== -o Stochastic gradient descent -o Stochastic gradient descent + momentum -o State-of-the-art approaches: - * RMSProp - * Adam - * and more +* Stochastic gradient descent +* Stochastic gradient descent + momentum +* State-of-the-art approaches: + o RMSProp + o Adam + o and more Which regularization and hyperparameters? $L_1$ or $L_2$, soft classifiers, depths of trees and many other. Need to explore a large