typos
This commit is contained in:
Morten Hjorth-Jensen
@@ -243,7 +243,7 @@ MathJax.Hub.Config({
|
||||
<p>The algorithm described here can be applied to both classification and regression problems.</p>
|
||||
|
||||
<p>We will grow of forest of say \( B \) trees.</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> For \( b=1:B \)
|
||||
<ol type="a"></li>
|
||||
<p><li> Draw a bootstrap sample from the training data organized in our \( \boldsymbol{X} \) matrix.</li>
|
||||
@@ -259,8 +259,9 @@ MathJax.Hub.Config({
|
||||
<p>
|
||||
</ol>
|
||||
<p>
|
||||
<p><li> Output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p>
|
||||
<p>Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem. </p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
@@ -1361,8 +1362,7 @@ plt.show()
|
||||
<section>
|
||||
<h2 id="machine-learning">Machine learning </h2>
|
||||
|
||||
<p>The following topics will be covered</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> Linear methods for regression and classification:
|
||||
<ol type="a"></li>
|
||||
<p><li> Ordinary Least Squares</li>
|
||||
@@ -1393,7 +1393,7 @@ plt.show()
|
||||
<p><li> Regression</li>
|
||||
</ol>
|
||||
<p>
|
||||
</ol>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
@@ -1468,17 +1468,17 @@ Neural Networks, etc.</a>
|
||||
<section>
|
||||
<h2 id="choose-a-model-and-algorithm">Choose a Model and Algorithm </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> Supervised?</li>
|
||||
<p><li> Start with the simplest model that fits your problem</li>
|
||||
<p><li> Start with minimal processing of data</li>
|
||||
</ol>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2 id="preparing-your-data">Preparing Your Data </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> Shuffle your data</li>
|
||||
<p><li> Mean center your data</li>
|
||||
<ul>
|
||||
@@ -1500,51 +1500,38 @@ Neural Networks, etc.</a>
|
||||
<p><li> Can be hit or miss</li>
|
||||
</ul>
|
||||
<p>
|
||||
<p><li> When to do train/test split?</li>
|
||||
</ol>
|
||||
<p><li> When to do train/test split?</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<p><li> RELU? ELU? GELU? etc</li>
|
||||
<p><li> Sigmoid or Tanh?</li>
|
||||
<p><li> Set all weights to 0?</li>
|
||||
<ul>
|
||||
|
||||
<p><li> Terrible idea</li>
|
||||
<p><li> Set all weights to 0? Terrible idea</li>
|
||||
<p><li> Set all weights to random values? Small random values</li>
|
||||
</ul>
|
||||
<p>
|
||||
<p><li> Set all weights to random values?</li>
|
||||
<ul>
|
||||
|
||||
<p><li> Small random values</li>
|
||||
</ul>
|
||||
<p>
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2 id="optimization-methods-and-hyperparameters">Optimization Methods and Hyperparameters </h2>
|
||||
<ol>
|
||||
<p><li> Stochastic gradient descent
|
||||
<ol type="a"></li>
|
||||
<p><li> Stochastic gradient descent + momentum</li>
|
||||
</ol>
|
||||
<p>
|
||||
<p><li> State-of-the-art approaches:</li>
|
||||
<ul>
|
||||
|
||||
<p><li> RMSProp</li>
|
||||
|
||||
<p><li> Adam</li>
|
||||
|
||||
<p><li> and more</li>
|
||||
<p><li> Stochastic gradient descent</li>
|
||||
<ul>
|
||||
<p><li> Stochastic gradient descent + momentum</li>
|
||||
</ul>
|
||||
<p>
|
||||
<p><li> State-of-the-art approaches:
|
||||
<ol type="a"></li>
|
||||
<p><li> RMSProp</li>
|
||||
<p><li> Adam</li>
|
||||
<p><li> and more</li>
|
||||
</ol>
|
||||
<p>
|
||||
</ul>
|
||||
<p>
|
||||
<p>Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft
|
||||
classifiers, depths of trees and many other. Need to explore a large
|
||||
set of hyperparameters and regularization methods.
|
||||
|
||||
@@ -353,7 +353,7 @@ MathJax.Hub.Config({
|
||||
<p>The algorithm described here can be applied to both classification and regression problems.</p>
|
||||
|
||||
<p>We will grow of forest of say \( B \) trees.</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<li> For \( b=1:B \)
|
||||
<ol type="a"></li>
|
||||
<li> Draw a bootstrap sample from the training data organized in our \( \boldsymbol{X} \) matrix.</li>
|
||||
@@ -364,8 +364,9 @@ MathJax.Hub.Config({
|
||||
<li> split the node into daughter nodes</li>
|
||||
</ol>
|
||||
</ol>
|
||||
<li> Output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p>Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem. </p>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="random-forests-compared-with-other-methods-on-the-cancer-data">Random Forests Compared with other Methods on the Cancer Data </h2>
|
||||
|
||||
@@ -1377,8 +1378,7 @@ plt.show()
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="machine-learning">Machine learning </h2>
|
||||
|
||||
<p>The following topics will be covered</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Linear methods for regression and classification:
|
||||
<ol type="a"></li>
|
||||
<li> Ordinary Least Squares</li>
|
||||
@@ -1405,7 +1405,7 @@ plt.show()
|
||||
<li> Kernel methods</li>
|
||||
<li> Regression</li>
|
||||
</ol>
|
||||
</ol>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="learning-outcomes-and-overarching-aims-of-this-course">Learning outcomes and overarching aims of this course </h2>
|
||||
|
||||
@@ -1470,15 +1470,15 @@ Neural Networks, etc.</a>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="choose-a-model-and-algorithm">Choose a Model and Algorithm </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Supervised?</li>
|
||||
<li> Start with the simplest model that fits your problem</li>
|
||||
<li> Start with minimal processing of data</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="preparing-your-data">Preparing Your Data </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Shuffle your data</li>
|
||||
<li> Mean center your data</li>
|
||||
<ul>
|
||||
@@ -1493,37 +1493,31 @@ Neural Networks, etc.</a>
|
||||
<li> Decorrelates data</li>
|
||||
<li> Can be hit or miss</li>
|
||||
</ul>
|
||||
<li> When to do train/test split?</li>
|
||||
</ol>
|
||||
<li> When to do train/test split?</li>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> RELU? ELU? GELU? etc</li>
|
||||
<li> Sigmoid or Tanh?</li>
|
||||
<li> Set all weights to 0?</li>
|
||||
<ul>
|
||||
<li> Terrible idea</li>
|
||||
<li> Set all weights to 0? Terrible idea</li>
|
||||
<li> Set all weights to random values? Small random values</li>
|
||||
</ul>
|
||||
<li> Set all weights to random values?</li>
|
||||
<ul>
|
||||
<li> Small random values</li>
|
||||
</ul>
|
||||
</ol>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="optimization-methods-and-hyperparameters">Optimization Methods and Hyperparameters </h2>
|
||||
<ol>
|
||||
<li> Stochastic gradient descent
|
||||
<ol type="a"></li>
|
||||
<li> Stochastic gradient descent + momentum</li>
|
||||
</ol>
|
||||
<li> State-of-the-art approaches:</li>
|
||||
<ul>
|
||||
<li> RMSProp</li>
|
||||
<li> Adam</li>
|
||||
<li> and more</li>
|
||||
<li> Stochastic gradient descent</li>
|
||||
<ul>
|
||||
<li> Stochastic gradient descent + momentum</li>
|
||||
</ul>
|
||||
<li> State-of-the-art approaches:
|
||||
<ol type="a"></li>
|
||||
<li> RMSProp</li>
|
||||
<li> Adam</li>
|
||||
<li> and more</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p>Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft
|
||||
classifiers, depths of trees and many other. Need to explore a large
|
||||
set of hyperparameters and regularization methods.
|
||||
|
||||
@@ -430,7 +430,7 @@ MathJax.Hub.Config({
|
||||
<p>The algorithm described here can be applied to both classification and regression problems.</p>
|
||||
|
||||
<p>We will grow of forest of say \( B \) trees.</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<li> For \( b=1:B \)
|
||||
<ol type="a"></li>
|
||||
<li> Draw a bootstrap sample from the training data organized in our \( \boldsymbol{X} \) matrix.</li>
|
||||
@@ -441,8 +441,9 @@ MathJax.Hub.Config({
|
||||
<li> split the node into daughter nodes</li>
|
||||
</ol>
|
||||
</ol>
|
||||
<li> Output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem.</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p>Finally we output then the ensemble of trees \( \{T_b\}_1^{B} \) and make predictions for either a regression type of problem or a classification type of problem. </p>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="random-forests-compared-with-other-methods-on-the-cancer-data">Random Forests Compared with other Methods on the Cancer Data </h2>
|
||||
|
||||
@@ -1454,8 +1455,7 @@ plt<span style="color: #666666">.</span>show()
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="machine-learning">Machine learning </h2>
|
||||
|
||||
<p>The following topics will be covered</p>
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Linear methods for regression and classification:
|
||||
<ol type="a"></li>
|
||||
<li> Ordinary Least Squares</li>
|
||||
@@ -1482,7 +1482,7 @@ plt<span style="color: #666666">.</span>show()
|
||||
<li> Kernel methods</li>
|
||||
<li> Regression</li>
|
||||
</ol>
|
||||
</ol>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="learning-outcomes-and-overarching-aims-of-this-course">Learning outcomes and overarching aims of this course </h2>
|
||||
|
||||
@@ -1547,15 +1547,15 @@ Neural Networks, etc.</a>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="choose-a-model-and-algorithm">Choose a Model and Algorithm </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Supervised?</li>
|
||||
<li> Start with the simplest model that fits your problem</li>
|
||||
<li> Start with minimal processing of data</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="preparing-your-data">Preparing Your Data </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> Shuffle your data</li>
|
||||
<li> Mean center your data</li>
|
||||
<ul>
|
||||
@@ -1570,37 +1570,31 @@ Neural Networks, etc.</a>
|
||||
<li> Decorrelates data</li>
|
||||
<li> Can be hit or miss</li>
|
||||
</ul>
|
||||
<li> When to do train/test split?</li>
|
||||
</ol>
|
||||
<li> When to do train/test split?</li>
|
||||
</ul>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
|
||||
|
||||
<ol>
|
||||
<ul>
|
||||
<li> RELU? ELU? GELU? etc</li>
|
||||
<li> Sigmoid or Tanh?</li>
|
||||
<li> Set all weights to 0?</li>
|
||||
<ul>
|
||||
<li> Terrible idea</li>
|
||||
<li> Set all weights to 0? Terrible idea</li>
|
||||
<li> Set all weights to random values? Small random values</li>
|
||||
</ul>
|
||||
<li> Set all weights to random values?</li>
|
||||
<ul>
|
||||
<li> Small random values</li>
|
||||
</ul>
|
||||
</ol>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="optimization-methods-and-hyperparameters">Optimization Methods and Hyperparameters </h2>
|
||||
<ol>
|
||||
<li> Stochastic gradient descent
|
||||
<ol type="a"></li>
|
||||
<li> Stochastic gradient descent + momentum</li>
|
||||
</ol>
|
||||
<li> State-of-the-art approaches:</li>
|
||||
<ul>
|
||||
<li> RMSProp</li>
|
||||
<li> Adam</li>
|
||||
<li> and more</li>
|
||||
<li> Stochastic gradient descent</li>
|
||||
<ul>
|
||||
<li> Stochastic gradient descent + momentum</li>
|
||||
</ul>
|
||||
<li> State-of-the-art approaches:
|
||||
<ol type="a"></li>
|
||||
<li> RMSProp</li>
|
||||
<li> Adam</li>
|
||||
<li> and more</li>
|
||||
</ol>
|
||||
</ul>
|
||||
<p>Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft
|
||||
classifiers, depths of trees and many other. Need to explore a large
|
||||
set of hyperparameters and regularization methods.
|
||||
|
||||
Binary file not shown.
+176
-181
File diff suppressed because it is too large
Load Diff
@@ -38,13 +38,14 @@ o Readings and Videos:
|
||||
The algorithm described here can be applied to both classification and regression problems.
|
||||
|
||||
We will grow of forest of say $B$ trees.
|
||||
o For $b=1:B$
|
||||
* For $b=1:B$
|
||||
o Draw a bootstrap sample from the training data organized in our $\bm{X}$ matrix.
|
||||
o We grow then a random forest tree $T_b$ based on the bootstrapped data by repeating the steps outlined till we reach the maximum node size is reached
|
||||
o we select $m \le p$ variables at random from the $p$ predictors/features
|
||||
o pick the best split point among the $m$ features using for example the CART algorithm and create a new node
|
||||
o split the node into daughter nodes
|
||||
o Output then the ensemble of trees $\{T_b\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem.
|
||||
|
||||
Finally we output then the ensemble of trees $\{T_b\}_1^{B}$ and make predictions for either a regression type of problem or a classification type of problem.
|
||||
|
||||
|
||||
|
||||
@@ -925,22 +926,21 @@ o Not discussed: Principal Component Analysis to reduce the number of features.
|
||||
!split
|
||||
===== Machine learning =====
|
||||
|
||||
The following topics will be covered
|
||||
o Linear methods for regression and classification:
|
||||
* Linear methods for regression and classification:
|
||||
o Ordinary Least Squares
|
||||
o Ridge regression
|
||||
o Lasso regression
|
||||
o Logistic regression
|
||||
o Neural networks and deep learning:
|
||||
* Neural networks and deep learning:
|
||||
o Feed Forward Neural Networks
|
||||
o Convolutional Neural Networks
|
||||
o Recurrent Neural Networks
|
||||
o Decisions trees and ensemble methods:
|
||||
* Decisions trees and ensemble methods:
|
||||
o Decision trees
|
||||
o Bagging and voting
|
||||
o Random forests
|
||||
o Boosting and gradient boosting
|
||||
o Not discussed this year: Support vector machines
|
||||
* Not discussed this year: Support vector machines
|
||||
o Binary classification and multiclass classification
|
||||
o Kernel methods
|
||||
o Regression
|
||||
@@ -1011,43 +1011,41 @@ o Based on your results, feedback loop to earliest possible point
|
||||
!split
|
||||
===== Choose a Model and Algorithm =====
|
||||
|
||||
o Supervised?
|
||||
o Start with the simplest model that fits your problem
|
||||
o Start with minimal processing of data
|
||||
* Supervised?
|
||||
* Start with the simplest model that fits your problem
|
||||
* Start with minimal processing of data
|
||||
|
||||
!split
|
||||
===== Preparing Your Data =====
|
||||
|
||||
o Shuffle your data
|
||||
o Mean center your data
|
||||
* Shuffle your data
|
||||
* Mean center your data
|
||||
* Why?
|
||||
o Normalize the variance
|
||||
* Normalize the variance
|
||||
* Why?
|
||||
o _Whitening_
|
||||
* _Whitening_
|
||||
* Decorrelates data
|
||||
* Can be hit or miss
|
||||
o When to do train/test split?
|
||||
* When to do train/test split?
|
||||
|
||||
|
||||
!split
|
||||
===== Which activation and weights to choose in neural networks =====
|
||||
|
||||
o RELU? ELU? GELU? etc
|
||||
o Sigmoid or Tanh?
|
||||
o Set all weights to 0?
|
||||
* Terrible idea
|
||||
o Set all weights to random values?
|
||||
* Small random values
|
||||
* RELU? ELU? GELU? etc
|
||||
* Sigmoid or Tanh?
|
||||
* Set all weights to 0? Terrible idea
|
||||
* Set all weights to random values? Small random values
|
||||
|
||||
|
||||
!split
|
||||
===== Optimization Methods and Hyperparameters =====
|
||||
o Stochastic gradient descent
|
||||
o Stochastic gradient descent + momentum
|
||||
o State-of-the-art approaches:
|
||||
* RMSProp
|
||||
* Adam
|
||||
* and more
|
||||
* Stochastic gradient descent
|
||||
* Stochastic gradient descent + momentum
|
||||
* State-of-the-art approaches:
|
||||
o RMSProp
|
||||
o Adam
|
||||
o and more
|
||||
|
||||
Which regularization and hyperparameters? $L_1$ or $L_2$, soft
|
||||
classifiers, depths of trees and many other. Need to explore a large
|
||||
|
||||
Reference in New Issue
Block a user