lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating summary slides

Browse Source
This commit is contained in:
mhjensen
2019-11-27 12:14:01 +01:00
parent 2289180c2e
commit 25880e6fa8
58 changed files with 4448 additions and 3228 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/summary/html/summary.html
+193 -14
View File
@@ -6,6 +6,7 @@ Automatically generated HTML file from DocOnce source
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Summary of course">
<title>Summary of course</title>
@@ -54,20 +55,48 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'___sec5'),
('Perspective on Machine Learning', 2, None, '___sec6'),
('Machine Learning Research', 2, None, '___sec7'),
('Hot Topics Now', 2, None, '___sec8'),
('Starting your Machine Learning Project', 2, None, '___sec9'),
('Choose a Model and Algorithm', 2, None, '___sec10'),
('Preparing Your Data', 2, None, '___sec11'),
('Which Activation and Weights to Choose in Neural Networks',
2,
None,
'___sec12'),
('Optimization Methods and Hyperparameters', 2, None, '___sec13'),
('Resampling', 2, None, '___sec14'),
('Other courses on Data science and Machine Learning at UiO',
2,
None,
'___sec6'),
('Additional courses of interest', 2, None, '___sec7'),
'___sec15'),
('Additional courses of interest', 2, None, '___sec16'),
('Best wishes to you all and thanks so much for your heroic '
'efforts this semester',
2,
None,
'___sec8')]}
'___sec17')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "AMS" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- ------------------- main content ---------------------- -->
@@ -89,7 +118,7 @@ end of tocinfo -->
<center>[2] <b>National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 29, 2018</h4></center> <!-- date -->
<center><h4>Nov 27, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -136,7 +165,6 @@ The following topics will be covered
<li> Central elements of Bayesian statistics and modeling;</li>
<li> Central elements from linear algebra</li>
<li> Gradient methods for data optimization</li>
<li> Monte Carlo methods, Markov chains, Metropolis-Hastings algorithm;</li>
<li> Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;</li>
<li> Practical optimization using Singular-value decomposition and least squares for parameterizing data.</li>
<li> Principal Component Analysis.</li>
@@ -151,9 +179,8 @@ The following topics will be covered
<ol>
<li> Linear methods for regression and classification;</li>
<li> Boltzmann machines;</li>
<li> Neural networks;</li>
<li> Decisions trees and nearest neighbor algorithms</li>
<li> Decisions trees, random forests, boosting and bagging</li>
<li> Support vector machines</li>
</ol>
@@ -175,16 +202,168 @@ ethical conduct is emphasized throughout the course.
<ul>
<li> Understand linear methods for regression and classification;</li>
<li> Learn about neural network;</li>
<li> Learn about basic data analysis, Bayesian statistics, Monte Carlo methods, data optimization and machine learning;</li>
<li> Learn about baggin, boosting and trees</li>
<li> Support vector machines</li>
<li> Learn about basic data analysis;</li>
<li> Be capable of extending the acquired knowledge to other systems and cases;</li>
<li> Have an understanding of central algorithms used in data analysis and machine learning;</li>
<li> Gain knowledge of central aspects of Monte Carlo methods, Markov chains, Metropolis and Gibbs samplers and their possible applications;</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and students are expected to know modern programming languages like Python or C++.</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++.</li>
</ul>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Other courses on Data science and Machine Learning at UiO </h2>
<h2 id="___sec6">Perspective on Machine Learning </h2>
<ol>
<li> Rapidly emerging application area</li>
<li> Experiment AND theory are evolving in many many fields. Still many low-hanging fruits.</li>
<li> Requires education/retraining for more widespread adoption</li>
<li> A lot of &#8220;word-of-mouth&#8221; development methods</li>
</ol>
Huge amounts of data sets require automation, classical analysis tools often inadequate.
High energy physics hit this wall in the 90&#8217;s.
In 2009 single top quark production was determined via <a href="https://arxiv.org/pdf/0903.0850.pdf" target="_blank">Boosted decision trees, Bayesian
Neural Networks, etc.</a>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">Machine Learning Research </h2>
<p>
Where to find recent results:
<ol>
<li> Conference proceedings, arXiv and blog posts!</li>
<li> <b>NIPS</b>: <a href="https://papers.nips.cc" target="_blank">Neural Information Processing Systems</a></li>
<li> <b>ICLR</b>: <a href="https://openreview.net/group?id=ICLR.cc/2018/Conference#accepted-oral-papers" target="_blank">International Conference on Learning Representations</a></li>
<li> <b>ICML</b>: International Conference on Machine Learning</li>
<li> <a href="http://www.jmlr.org/papers/v19/" target="_blank">Journal of Machine Learning Research</a></li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Hot Topics Now </h2>
<ol>
<li> Boosting techniques and complex neural networks</li>
<li> <a href="https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8" target="_blank">Adversarial examples</a></li>
<li> <a href="https://arxiv.org/pdf/1707.00600" target="_blank">Zero shot learning</a></li>
<li> Transfer learning</li>
<li> <a href="https://christophm.github.io/interpretable-mlbook/interpretability.html" target="_blank">Model interpretability</a></li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec9">Starting your Machine Learning Project </h2>
<ol>
<li> Identify problem type: classification, generation, regression</li>
<li> Consider your data carefully</li>
<li> Choose a simple model that fits 1. and 2.</li>
<li> Consider your data carefully again&#8230; data representation</li>
<li> Based on results, feedback loop to earliest possible point</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec10">Choose a Model and Algorithm </h2>
<ol>
<li> Supervised?</li>
<li> Start with the simplest model that fits your problem</li>
<li> Start with minimal processing of data</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec11">Preparing Your Data </h2>
<ol>
<li> Shuffle your data</li>
<li> Mean center your data</li>
<ul>
<li> Why?</li>
</ul>
<li> Normalize the variance</li>
<ul>
<li> Why?</li>
</ul>
<li> <b>Whitening</b></li>
<ul>
<li> Decorrelates data</li>
<li> Can be hit or miss</li>
</ul>
<li> When to do train/test split?</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec12">Which Activation and Weights to Choose in Neural Networks </h2>
<ol>
<li> RELU? ELU?</li>
<li> Sigmoid or Tanh?</li>
<li> Set all weights to 0?</li>
<ul>
<li> Terrible idea</li>
</ul>
<li> Set all weights to random values?</li>
<ul>
<li> Small random values</li>
</ul>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec13">Optimization Methods and Hyperparameters </h2>
<ol>
<li> Stochastic gradient descent
<ol type="a"></li>
<li> Stochastic gradient descent + momentum</li>
</ol>
<li> State-of-the-art approaches:</li>
<ul>
<li> RMSProp</li>
<li> Adam</li>
</ul>
</ol>
Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Resampling </h2>
<p>
When do we resample?
<ol>
<li> Bootstrap</li>
<li> Cross-validation</li>
<li> Jackknife and many other</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec15">Other courses on Data science and Machine Learning at UiO </h2>
<p>
The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" target="_blank"><tt>https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/</tt></a> gives an excellent overview of courses on Machine learning at UiO.
@@ -203,7 +382,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">Additional courses of interest </h2>
<h2 id="___sec16">Additional courses of interest </h2>
<ol>
<li> <a href="https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html" target="_blank">STK4051 Computational Statistics</a></li>
@@ -212,7 +391,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Best wishes to you all and thanks so much for your heroic efforts this semester </h2>
<h2 id="___sec17">Best wishes to you all and thanks so much for your heroic efforts this semester </h2>
<p>
<br /><br /><center><p><img src="figures/Nebbdyr2.png" align="bottom" width=500></p></center><br /><br />
@@ -223,7 +402,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen Email morten.hjorth-jensen@fys.uio.no. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen Email morten.hjorth-jensen@fys.uio.no. Released under CC Attribution-NonCommercial 4.0 license
</center>