diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html index 5dbe0927e..856348d11 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html @@ -299,7 +299,7 @@ MathJax.Hub.Config({
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 27, 2019


diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html index 8d3597dbb..4da8edbaa 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html @@ -293,6 +293,18 @@ depends on the performance of the prior machines on that example. The following is a modification of Adaboost by Drucker. +

+Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows: + +

    +
  1. We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).
    2. +
  2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.
    3. +
  3. Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).
    4. +
  4. We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value
    5. +
+ +\( L_i\in [0,1] \). +

diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html index 5dbe0927e..856348d11 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html @@ -299,7 +299,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 27, 2019


diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html index 9ab557f8e..4477c5a25 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html @@ -148,7 +148,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

 
-

Dec 26, 2019

+

Dec 27, 2019


@@ -2437,6 +2437,19 @@ example being in the training set of a particular machine depends on the performance of the prior machines on that example. The following is a modification of Adaboost by Drucker. + +

+Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows: + +

    +

  1. We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).
    2. +

  2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.
    3. +

  3. Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).
    4. +

  4. We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value
    5. +
+

+ +\( L_i\in [0,1] \). diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html index 4bc0e2534..eb79de46c 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html @@ -224,7 +224,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 27, 2019












@@ -2402,6 +2402,18 @@ depends on the performance of the prior machines on that example. The following is a modification of Adaboost by Drucker. +

+Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows: + +

    +
  1. We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).
    2. +
  2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.
    3. +
  3. Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).
    4. +
  4. We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value
    5. +
+ +\( L_i\in [0,1] \). +











diff --git a/doc/pub/DecisionTrees/html/DecisionTrees.html b/doc/pub/DecisionTrees/html/DecisionTrees.html index 38742eeb9..5825b63a4 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees.html @@ -229,7 +229,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 27, 2019












@@ -2407,6 +2407,18 @@ depends on the performance of the prior machines on that example. The following is a modification of Adaboost by Drucker. +

+Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows: + +

    +
  1. We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).
    2. +
  2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.
    3. +
  3. Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).
    4. +
  4. We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value
    5. +
+ +\( L_i\in [0,1] \). +











diff --git a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb index a5c9bc63c..83f02c822 100644 --- a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb +++ b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Dec 26, 2019**\n", + "Date: **Dec 27, 2019**\n", "\n", "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -2666,6 +2666,16 @@ "that example. The following is a modification of\n", "Adaboost by Drucker.\n", "\n", + "Start by selecting a set of training data $n$ and assign to each entry a weight $w_i=1$ for $i=1,2,\\dots,n$. As we have done earlier, we could pick say $80\\%$ of the data set for training. The algorithm runs as follows:\n", + "1. We define the probability that the training sample $i$ is in the set by $p_i = w_i/\\sum_iw_i$. We pick $n$ samples (with replacement) to form our training set. We pick a number uniformly in the range $[0,\\sum_iw_i]$.\n", + "\n", + "2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.\n", + "\n", + "3. Using every member of the training set with the chosen regression machine we obtain then a prediction $\\tilde{y}_i$.\n", + "\n", + "4. We calculate then the loss function $L_i$ for each training sample. We can use various types of loss function as long as we have a value\n", + "\n", + "$L_i\\in [0,1]$. \n", "\n", "## Gradient boosting: Basics with Steepest Descent\n", "\n", diff --git a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz index bcfd4eaf8..5c2a156ac 100644 Binary files a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz and b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz differ diff --git a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf index 6f40060f3..067a42a1f 100644 Binary files a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf and b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf differ diff --git a/doc/pub/DimRed/html/._DimRed-bs000.html b/doc/pub/DimRed/html/._DimRed-bs000.html index 8da1ff958..c0e2e1fdb 100644 --- a/doc/pub/DimRed/html/._DimRed-bs000.html +++ b/doc/pub/DimRed/html/._DimRed-bs000.html @@ -218,7 +218,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 28, 2019


diff --git a/doc/pub/DimRed/html/._DimRed-bs001.html b/doc/pub/DimRed/html/._DimRed-bs001.html index 026609651..604451fe3 100644 --- a/doc/pub/DimRed/html/._DimRed-bs001.html +++ b/doc/pub/DimRed/html/._DimRed-bs001.html @@ -215,11 +215,19 @@ an intractable problem into a tractable one.

Here we will discuss some of the most popular dimensionality reduction -techniques: the principal component analysis PCA, Kernel PCA, and +techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. +

+Principal component analysis and its various variants deal with the +problem of fitting a low-dimensional affine +subspace to a set of of +data points in a high-dimensional space. With its family of methods it +is one of the most used tools in data modeling, compression and +visualization. +

diff --git a/doc/pub/DimRed/html/DimRed-bs.html b/doc/pub/DimRed/html/DimRed-bs.html index 8da1ff958..c0e2e1fdb 100644 --- a/doc/pub/DimRed/html/DimRed-bs.html +++ b/doc/pub/DimRed/html/DimRed-bs.html @@ -218,7 +218,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 28, 2019


diff --git a/doc/pub/DimRed/html/DimRed-reveal.html b/doc/pub/DimRed/html/DimRed-reveal.html index 5b1cbcdce..589de6f51 100644 --- a/doc/pub/DimRed/html/DimRed-reveal.html +++ b/doc/pub/DimRed/html/DimRed-reveal.html @@ -148,7 +148,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

 
-

Dec 26, 2019

+

Dec 28, 2019


@@ -173,11 +173,19 @@ an intractable problem into a tractable one.

Here we will discuss some of the most popular dimensionality reduction -techniques: the principal component analysis PCA, Kernel PCA, and +techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. +

+Principal component analysis and its various variants deal with the +problem of fitting a low-dimensional affine +subspace to a set of of +data points in a high-dimensional space. With its family of methods it +is one of the most used tools in data modeling, compression and +visualization. + diff --git a/doc/pub/DimRed/html/DimRed-solarized.html b/doc/pub/DimRed/html/DimRed-solarized.html index 44ddd27fa..a70ae5c1a 100644 --- a/doc/pub/DimRed/html/DimRed-solarized.html +++ b/doc/pub/DimRed/html/DimRed-solarized.html @@ -174,7 +174,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 28, 2019












@@ -195,11 +195,19 @@ an intractable problem into a tractable one.

Here we will discuss some of the most popular dimensionality reduction -techniques: the principal component analysis PCA, Kernel PCA, and +techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. +

+Principal component analysis and its various variants deal with the +problem of fitting a low-dimensional affine +subspace to a set of of +data points in a high-dimensional space. With its family of methods it +is one of the most used tools in data modeling, compression and +visualization. + diff --git a/doc/pub/DimRed/html/DimRed.html b/doc/pub/DimRed/html/DimRed.html index 5be5508b6..d300d5422 100644 --- a/doc/pub/DimRed/html/DimRed.html +++ b/doc/pub/DimRed/html/DimRed.html @@ -179,7 +179,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 26, 2019

+

Dec 28, 2019












@@ -200,11 +200,19 @@ an intractable problem into a tractable one.

Here we will discuss some of the most popular dimensionality reduction -techniques: the principal component analysis PCA, Kernel PCA, and +techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. +

+Principal component analysis and its various variants deal with the +problem of fitting a low-dimensional affine +subspace to a set of of +data points in a high-dimensional space. With its family of methods it +is one of the most used tools in data modeling, compression and +visualization. + diff --git a/doc/pub/DimRed/ipynb/DimRed.ipynb b/doc/pub/DimRed/ipynb/DimRed.ipynb index 0c2ba7990..f688683cf 100644 --- a/doc/pub/DimRed/ipynb/DimRed.ipynb +++ b/doc/pub/DimRed/ipynb/DimRed.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Dec 26, 2019**\n", + "Date: **Dec 28, 2019**\n", "\n", "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -28,11 +28,18 @@ "an intractable problem into a tractable one.\n", "\n", "Here we will discuss some of the most popular dimensionality reduction\n", - "techniques: the principal component analysis PCA, Kernel PCA, and\n", + "techniques: the principal component analysis (PCA), Kernel PCA, and\n", "Locally Linear Embedding (LLE). Furthermore, we will start by looking\n", "at some simple preprocessing of the data which allow us to rescale the\n", "data.\n", "\n", + "Principal component analysis and its various variants deal with the\n", + "problem of fitting a low-dimensional [affine\n", + "subspace](https://en.wikipedia.org/wiki/Affine_space) to a set of of\n", + "data points in a high-dimensional space. With its family of methods it\n", + "is one of the most used tools in data modeling, compression and\n", + "visualization.\n", + "\n", "\n", "\n", "\n", diff --git a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz index ba174a3e8..2d4521bc7 100644 Binary files a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz and b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz differ diff --git a/doc/pub/DimRed/pdf/DimRed-minted.pdf b/doc/pub/DimRed/pdf/DimRed-minted.pdf index 10daefc17..4e25cee27 100644 Binary files a/doc/pub/DimRed/pdf/DimRed-minted.pdf and b/doc/pub/DimRed/pdf/DimRed-minted.pdf differ diff --git a/doc/src/DecisionTrees/DecisionTrees.do.txt b/doc/src/DecisionTrees/DecisionTrees.do.txt index 39c02a529..b9eeba15b 100644 --- a/doc/src/DecisionTrees/DecisionTrees.do.txt +++ b/doc/src/DecisionTrees/DecisionTrees.do.txt @@ -2011,6 +2011,12 @@ depends on the performance of the prior machines on that example. The following is a modification of Adaboost by Drucker. +Start by selecting a set of training data $n$ and assign to each entry a weight $w_i=1$ for $i=1,2,\dots,n$. As we have done earlier, we could pick say $80\%$ of the data set for training. The algorithm runs as follows: +o We define the probability that the training sample $i$ is in the set by $p_i = w_i/\sum_iw_i$. We pick $n$ samples (with replacement) to form our training set. We pick a number uniformly in the range $[0,\sum_iw_i]$. +o We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis. +o Using every member of the training set with the chosen regression machine we obtain then a prediction $\tilde{y}_i$. +o We calculate then the loss function $L_i$ for each training sample. We can use various types of loss function as long as we have a value +$L_i\in [0,1]$. !split ===== Gradient boosting: Basics with Steepest Descent ===== diff --git a/doc/src/DimRed/DimRed.do.txt b/doc/src/DimRed/DimRed.do.txt index 8955b100e..077e1be6c 100644 --- a/doc/src/DimRed/DimRed.do.txt +++ b/doc/src/DimRed/DimRed.do.txt @@ -16,11 +16,17 @@ often possible to reduce the number of features considerably, turning an intractable problem into a tractable one. Here we will discuss some of the most popular dimensionality reduction -techniques: the principal component analysis PCA, Kernel PCA, and +techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. +Principal component analysis and its various variants deal with the +problem of fitting a low-dimensional "affine +subspace":"https://en.wikipedia.org/wiki/Affine_space" to a set of of +data points in a high-dimensional space. With its family of methods it +is one of the most used tools in data modeling, compression and +visualization. !eblock