updating summary slides

doc/src/Summary/summary.do.txt

@@ -35,7 +35,6 @@ o Simpler models, binomial distribution, the Poisson distribution, simple and mu
 o Central elements of Bayesian statistics and modeling;
 o Central elements from linear algebra
 o Gradient methods for data optimization
-o Monte Carlo methods, Markov chains, Metropolis-Hastings algorithm;
 o Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;
 o Practical optimization using Singular-value decomposition and least squares for parameterizing data.
 o Principal Component Analysis.
@@ -45,9 +44,8 @@ o Principal Component Analysis.
 
 The following topics will be covered
 o Linear methods for regression and classification;
-o Boltzmann machines;
 o Neural networks;
-o Decisions trees and nearest neighbor algorithms
+o Decisions trees, random forests, boosting and bagging
 o Support vector machines
 
 
@@ -66,15 +64,112 @@ ethical conduct is emphasized throughout the course.
 
 * Understand linear methods for regression and classification;
 * Learn about neural network;
-* Learn about basic data analysis, Bayesian statistics, Monte Carlo methods, data optimization and machine learning;
+* Learn about baggin, boosting and trees
+* Support vector machines
+* Learn about basic data analysis;
 * Be capable of extending the acquired knowledge to other systems and cases;
 * Have an understanding of central algorithms used in data analysis and machine learning;
-* Gain knowledge of central aspects of Monte Carlo methods, Markov chains, Metropolis and Gibbs samplers and their possible applications;
-* Work on numerical projects to illustrate the theory. The projects play a central role and students are expected to know modern programming languages like Python or C++.
+* Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++.
+
+
+
+!split
+===== Perspective on Machine Learning =====
+
+o Rapidly emerging application area
+o Experiment AND theory are evolving in many many fields. Still many low-hanging fruits.
+o Requires education/retraining for more widespread adoption
+o A lot of “word-of-mouth” development methods
+
+Huge amounts of data sets require automation, classical analysis tools often inadequate. 
+High energy physics hit this wall in the 90’s.
+In 2009 single top quark production was determined via "Boosted decision trees, Bayesian
+Neural Networks, etc.":"https://arxiv.org/pdf/0903.0850.pdf"
+
+
+!split
+===== Machine Learning Research =====
+
+Where to find recent results:
+o Conference proceedings, arXiv and blog posts!
+o _NIPS_: "Neural Information Processing Systems":"https://papers.nips.cc"
+o _ICLR_: "International Conference on Learning Representations":"https://openreview.net/group?id=ICLR.cc/2018/Conference#accepted-oral-papers"
+o _ICML_: International Conference on Machine Learning
+o "Journal of Machine Learning Research":"http://www.jmlr.org/papers/v19/" 
+
+!split
+===== Hot Topics Now =====
+
+o Boosting techniques and complex neural networks
+o "Adversarial examples":"https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8"
+o "Zero shot learning":"https://arxiv.org/pdf/1707.00600"
+o Transfer learning
+o "Model interpretability":"https://christophm.github.io/interpretable-mlbook/interpretability.html"
+
+
+!split
+===== Starting your Machine Learning Project  =====
+
+o Identify problem type: classification, generation, regression
+o Consider your data carefully
+o Choose a simple model that fits 1. and 2.
+o Consider your data carefully again… data representation
+o Based on results, feedback loop to earliest possible point
 
 
 
 
+!split
+===== Choose a Model and Algorithm  =====
+
+o Supervised?
+o Start with the simplest model that fits your problem
+o Start with minimal processing of data
+
+!split
+===== Preparing Your Data =====
+
+o Shuffle your data
+o Mean center your data
+  * Why?
+o Normalize the variance
+  * Why?
+o _Whitening_
+  * Decorrelates data
+  * Can be hit or miss
+o  When to do train/test split?
+
+
+!split
+=====  Which Activation and Weights to Choose in Neural Networks =====
+
+o RELU? ELU?
+o Sigmoid or Tanh?
+o Set all weights to 0?
+  * Terrible idea
+o Set all weights to random values?
+  * Small random values
+
+
+!split
+===== Optimization Methods and Hyperparameters =====
+o Stochastic gradient descent
+o  Stochastic gradient descent + momentum
+o State-of-the-art approaches:
+  * RMSProp
+  * Adam
+
+Which regularization and hyperparameters? $L_1$ or $L_2$, soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods. 
+
+
+!split
+===== Resampling =====
+
+When do we resample?
+
+o Bootstrap
+o Cross-validation
+o Jackknife and many other
 
 
 !split