lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating dim red

Browse Source
This commit is contained in:
mhjensen
2019-12-28 22:34:10 +01:00
parent df0951f3f9
commit c6fc54e6ff
20 changed files with 128 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html
+1 -1
View File
@@ -299,7 +299,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 27, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html
+12
View File
@@ -293,6 +293,18 @@ depends on the performance of the prior machines on
that example. The following is a modification of
Adaboost by Drucker.
<p>
Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows:
<ol>
<li> We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).</li>
<li> We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.</li>
<li> Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).</li>
<li> We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value</li>
</ol>
\( L_i\in [0,1] \).
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/DecisionTrees/html/DecisionTrees-bs.html
+1 -1
View File
@@ -299,7 +299,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 27, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DecisionTrees/html/DecisionTrees-reveal.html
+14 -1
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 27, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -2437,6 +2437,19 @@ example being in the training set of a particular machine
depends on the performance of the prior machines on
that example. The following is a modification of
Adaboost by Drucker.
<p>
Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows:
<ol>
<p><li> We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).</li>
<p><li> We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.</li>
<p><li> Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).</li>
<p><li> We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value</li>
</ol>
<p>
\( L_i\in [0,1] \).
</section>
doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
+13 -1
View File
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 27, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -2402,6 +2402,18 @@ depends on the performance of the prior machines on
that example. The following is a modification of
Adaboost by Drucker.
<p>
Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows:
<ol>
<li> We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).</li>
<li> We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.</li>
<li> Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).</li>
<li> We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value</li>
</ol>
\( L_i\in [0,1] \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DecisionTrees/html/DecisionTrees.html
+13 -1
View File
@@ -229,7 +229,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 27, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -2407,6 +2407,18 @@ depends on the performance of the prior machines on
that example. The following is a modification of
Adaboost by Drucker.
<p>
Start by selecting a set of training data \( n \) and assign to each entry a weight \( w_i=1 \) for \( i=1,2,\dots,n \). As we have done earlier, we could pick say \( 80\% \) of the data set for training. The algorithm runs as follows:
<ol>
<li> We define the probability that the training sample \( i \) is in the set by \( p_i = w_i/\sum_iw_i \). We pick \( n \) samples (with replacement) to form our training set. We pick a number uniformly in the range \( [0,\sum_iw_i] \).</li>
<li> We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.</li>
<li> Using every member of the training set with the chosen regression machine we obtain then a prediction \( \tilde{y}_i \).</li>
<li> We calculate then the loss function \( L_i \) for each training sample. We can use various types of loss function as long as we have a value</li>
</ol>
\( L_i\in [0,1] \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
+11 -1
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Dec 26, 2019**\n",
"Date: **Dec 27, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -2666,6 +2666,16 @@
"that example. The following is a modification of\n",
"Adaboost by Drucker.\n",
"\n",
"Start by selecting a set of training data $n$ and assign to each entry a weight $w_i=1$ for $i=1,2,\\dots,n$. As we have done earlier, we could pick say $80\\%$ of the data set for training. The algorithm runs as follows:\n",
"1. We define the probability that the training sample $i$ is in the set by $p_i = w_i/\\sum_iw_i$. We pick $n$ samples (with replacement) to form our training set. We pick a number uniformly in the range $[0,\\sum_iw_i]$.\n",
"\n",
"2. We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.\n",
"\n",
"3. Using every member of the training set with the chosen regression machine we obtain then a prediction $\\tilde{y}_i$.\n",
"\n",
"4. We calculate then the loss function $L_i$ for each training sample. We can use various types of loss function as long as we have a value\n",
"\n",
"$L_i\\in [0,1]$. \n",
"\n",
"## Gradient boosting: Basics with Steepest Descent\n",
"\n",
doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf
BIN
View File
Binary file not shown.
doc/pub/DimRed/html/._DimRed-bs000.html
+1 -1
View File
@@ -218,7 +218,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 28, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DimRed/html/._DimRed-bs001.html
+9 -1
View File
@@ -215,11 +215,19 @@ an intractable problem into a tractable one.
<p>
Here we will discuss some of the most popular dimensionality reduction
techniques: the principal component analysis PCA, Kernel PCA, and
techniques: the principal component analysis (PCA), Kernel PCA, and
Locally Linear Embedding (LLE). Furthermore, we will start by looking
at some simple preprocessing of the data which allow us to rescale the
data.
<p>
Principal component analysis and its various variants deal with the
problem of fitting a low-dimensional <a href="https://en.wikipedia.org/wiki/Affine_space" target="_self">affine
subspace</a> to a set of of
data points in a high-dimensional space. With its family of methods it
is one of the most used tools in data modeling, compression and
visualization.
<p>
</div>
</div>
doc/pub/DimRed/html/DimRed-bs.html
+1 -1
View File
@@ -218,7 +218,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 28, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DimRed/html/DimRed-reveal.html
+10 -2
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 28, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -173,11 +173,19 @@ an intractable problem into a tractable one.
<p>
Here we will discuss some of the most popular dimensionality reduction
techniques: the principal component analysis PCA, Kernel PCA, and
techniques: the principal component analysis (PCA), Kernel PCA, and
Locally Linear Embedding (LLE). Furthermore, we will start by looking
at some simple preprocessing of the data which allow us to rescale the
data.
<p>
Principal component analysis and its various variants deal with the
problem of fitting a low-dimensional <a href="https://en.wikipedia.org/wiki/Affine_space" target="_blank">affine
subspace</a> to a set of of
data points in a high-dimensional space. With its family of methods it
is one of the most used tools in data modeling, compression and
visualization.
</div>
</section>
doc/pub/DimRed/html/DimRed-solarized.html
+10 -2
View File
@@ -174,7 +174,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 28, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -195,11 +195,19 @@ an intractable problem into a tractable one.
<p>
Here we will discuss some of the most popular dimensionality reduction
techniques: the principal component analysis PCA, Kernel PCA, and
techniques: the principal component analysis (PCA), Kernel PCA, and
Locally Linear Embedding (LLE). Furthermore, we will start by looking
at some simple preprocessing of the data which allow us to rescale the
data.
<p>
Principal component analysis and its various variants deal with the
problem of fitting a low-dimensional <a href="https://en.wikipedia.org/wiki/Affine_space" target="_blank">affine
subspace</a> to a set of of
data points in a high-dimensional space. With its family of methods it
is one of the most used tools in data modeling, compression and
visualization.
</div>
doc/pub/DimRed/html/DimRed.html
+10 -2
View File
@@ -179,7 +179,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 26, 2019</h4></center> <!-- date -->
<center><h4>Dec 28, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -200,11 +200,19 @@ an intractable problem into a tractable one.
<p>
Here we will discuss some of the most popular dimensionality reduction
techniques: the principal component analysis PCA, Kernel PCA, and
techniques: the principal component analysis (PCA), Kernel PCA, and
Locally Linear Embedding (LLE). Furthermore, we will start by looking
at some simple preprocessing of the data which allow us to rescale the
data.
<p>
Principal component analysis and its various variants deal with the
problem of fitting a low-dimensional <a href="https://en.wikipedia.org/wiki/Affine_space" target="_blank">affine
subspace</a> to a set of of
data points in a high-dimensional space. With its family of methods it
is one of the most used tools in data modeling, compression and
visualization.
</div>
doc/pub/DimRed/ipynb/DimRed.ipynb
+9 -2
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Dec 26, 2019**\n",
"Date: **Dec 28, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -28,11 +28,18 @@
"an intractable problem into a tractable one.\n",
"\n",
"Here we will discuss some of the most popular dimensionality reduction\n",
"techniques: the principal component analysis PCA, Kernel PCA, and\n",
"techniques: the principal component analysis (PCA), Kernel PCA, and\n",
"Locally Linear Embedding (LLE). Furthermore, we will start by looking\n",
"at some simple preprocessing of the data which allow us to rescale the\n",
"data.\n",
"\n",
"Principal component analysis and its various variants deal with the\n",
"problem of fitting a low-dimensional [affine\n",
"subspace](https://en.wikipedia.org/wiki/Affine_space) to a set of of\n",
"data points in a high-dimensional space. With its family of methods it\n",
"is one of the most used tools in data modeling, compression and\n",
"visualization.\n",
"\n",
"\n",
"\n",
"\n",
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DecisionTrees/DecisionTrees.do.txt
+6
View File
@@ -2011,6 +2011,12 @@ depends on the performance of the prior machines on
that example. The following is a modification of
Adaboost by Drucker.
Start by selecting a set of training data $n$ and assign to each entry a weight $w_i=1$ for $i=1,2,\dots,n$. As we have done earlier, we could pick say $80\%$ of the data set for training. The algorithm runs as follows:
o We define the probability that the training sample $i$ is in the set by $p_i = w_i/\sum_iw_i$. We pick $n$ samples (with replacement) to form our training set. We pick a number uniformly in the range $[0,\sum_iw_i]$.
o We choose then a regression machine (for example plain linear regression or a simple decision tree). A given regression machine makes then a hypothesis.
o Using every member of the training set with the chosen regression machine we obtain then a prediction $\tilde{y}_i$.
o We calculate then the loss function $L_i$ for each training sample. We can use various types of loss function as long as we have a value
$L_i\in [0,1]$.
!split
===== Gradient boosting: Basics with Steepest Descent =====
doc/src/DimRed/DimRed.do.txt
+7 -1
View File
@@ -16,11 +16,17 @@ often possible to reduce the number of features considerably, turning
an intractable problem into a tractable one.
Here we will discuss some of the most popular dimensionality reduction
techniques: the principal component analysis PCA, Kernel PCA, and
techniques: the principal component analysis (PCA), Kernel PCA, and
Locally Linear Embedding (LLE). Furthermore, we will start by looking
at some simple preprocessing of the data which allow us to rescale the
data.
Principal component analysis and its various variants deal with the
problem of fitting a low-dimensional "affine
subspace":"https://en.wikipedia.org/wiki/Affine_space" to a set of of
data points in a high-dimensional space. With its family of methods it
is one of the most used tools in data modeling, compression and
visualization.
!eblock