lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update week 48

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-11-23 22:29:45 +01:00
parent 4c284d93d1
commit 6b9d1f8092
7 changed files with 271 additions and 256 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week48/html/week48-bs.html
+8 -8
View File
@@ -8,8 +8,8 @@ doconce format html week48.do.txt --html_style=bootstrap --pygments_html_style=d
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 48: Support Vector Machines and Summary of course">
<title>Week 48: Support Vector Machines and Summary of course</title>
<meta name="description" content="Week 48: Gradient boosting and summary of course">
<title>Week 48: Gradient boosting and summary of course</title>
<!-- Bootstrap style: bootstrap -->
<!-- doconce format html week48.do.txt --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=week48-bs --no_mako -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -143,7 +143,7 @@ doconce format html week48.do.txt --html_style=bootstrap --pygments_html_style=d
None,
'choose-a-model-and-algorithm'),
('Preparing Your Data', 2, None, 'preparing-your-data'),
('Which Activation and Weights to Choose in Neural Networks',
('Which activation and weights to choose in neural networks',
2,
None,
'which-activation-and-weights-to-choose-in-neural-networks'),
@@ -258,7 +258,7 @@ MathJax.Hub.Config({
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week48-bs.html">Week 48: Support Vector Machines and Summary of course</a>
<a class="navbar-brand" href="week48-bs.html">Week 48: Gradient boosting and summary of course</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
@@ -299,7 +299,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week48-bs032.html#starting-your-machine-learning-project" style="font-size: 80%;">Starting your Machine Learning Project</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs033.html#choose-a-model-and-algorithm" style="font-size: 80%;">Choose a Model and Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs034.html#preparing-your-data" style="font-size: 80%;">Preparing Your Data</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs035.html#which-activation-and-weights-to-choose-in-neural-networks" style="font-size: 80%;">Which Activation and Weights to Choose in Neural Networks</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs035.html#which-activation-and-weights-to-choose-in-neural-networks" style="font-size: 80%;">Which activation and weights to choose in neural networks</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs036.html#optimization-methods-and-hyperparameters" style="font-size: 80%;">Optimization Methods and Hyperparameters</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs037.html#resampling" style="font-size: 80%;">Resampling</a></li>
<!-- navigation toc: --> <li><a href="._week48-bs038.html#other-courses-on-data-science-and-machine-learning-at-uio" style="font-size: 80%;">Other courses on Data science and Machine Learning at UiO</a></li>
@@ -348,7 +348,7 @@ MathJax.Hub.Config({
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center>
<h1>Week 48: Support Vector Machines and Summary of course</h1>
<h1>Week 48: Gradient boosting and summary of course</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
@@ -357,11 +357,11 @@ MathJax.Hub.Config({
</center>
<!-- institution -->
<center>
<b>Department of Physics, University of Oslo, Norway</b>
<b>Department of Physics and Center for Computing in Science Education, University of Oslo, Norway</b>
</center>
<br>
<center>
<h4>Nov 21, 2024</h4>
<h4>Nov 23, 2024</h4>
</center> <!-- date -->
<br>
doc/pub/week48/html/week48-reveal.html
+27 -23
View File
@@ -9,8 +9,8 @@ doconce format html week48-reveal.html week48-reveal reveal --html_slide_theme=b
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 48: Support Vector Machines and Summary of course">
<title>Week 48: Support Vector Machines and Summary of course</title>
<meta name="description" content="Week 48: Gradient boosting and summary of course">
<title>Week 48: Gradient boosting and summary of course</title>
<!-- reveal.js: https://lab.hakim.se/reveal-js/ -->
@@ -168,7 +168,7 @@ MathJax.Hub.Config({
<section>
<!-- ------------------- main content ---------------------- -->
<center>
<h1 style="text-align: center;">Week 48: Support Vector Machines and Summary of course</h1>
<h1 style="text-align: center;">Week 48: Gradient boosting and summary of course</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
@@ -177,11 +177,11 @@ MathJax.Hub.Config({
</center>
<!-- institution -->
<center>
<b>Department of Physics, University of Oslo, Norway</b>
<b>Department of Physics and Center for Computing in Science Education, University of Oslo, Norway</b>
</center>
<br>
<center>
<h4>Nov 21, 2024</h4>
<h4>Nov 23, 2024</h4>
</center> <!-- date -->
<br>
@@ -205,7 +205,11 @@ MathJax.Hub.Config({
<p><li> Work and Discussion of project 3</li>
<p><li> Last weekly exercise,</li>
<p><li> Last weekly exercise</li>
<p><li> Lab sessions at usual times.</li>
<p><li> For the week of December 2-6, lab sessions atart at 10am and end 4pm, room F&#216;434, Tuesday and Wednesday</li>
</ul>
</div>
@@ -214,7 +218,7 @@ MathJax.Hub.Config({
<b>Plans for the lecture Monday 25 November, with video suggestions etc</b>
<p>
<ol>
<p><li> Bossting and gradient boosting and ensemble models</li>
<p><li> Boosting and gradient boosting and ensemble models</li>
<p><li> Summary of course</li>
<p><li> Readings and Videos:
<ol type="a"></li>
@@ -1249,7 +1253,7 @@ plt.show()
<p><li> Gradient methods for data optimization</li>
<p><li> Estimation of errors using cross-validation, bootstrapping and jackknife methods;</li>
<p><li> Practical optimization using Singular-value decomposition and least squares for parameterizing data.</li>
<p><li> Principal Component Analysis to reduce the number of features.</li>
<p><li> Not discussed: Principal Component Analysis to reduce the number of features.</li>
</ol>
</section>
@@ -1281,7 +1285,7 @@ plt.show()
<p><li> Boosting and gradient boosting</li>
</ol>
<p>
<p><li> Support vector machines
<p><li> Not discussed this year: Support vector machines
<ol type="a"></li>
<p><li> Binary classification and multiclass classification</li>
<p><li> Kernel methods</li>
@@ -1308,12 +1312,12 @@ ethical conduct is emphasized throughout the course.
<ul>
<p><li> Understand linear methods for regression and classification;</li>
<p><li> Learn about neural network;</li>
<p><li> Learn about bagging, boosting and trees</li>
<p><li> Support vector machines</li>
<p><li> Learn about bagging, boosting and trees
<!-- * Support vector machines --></li>
<p><li> Learn about basic data analysis;</li>
<p><li> Be capable of extending the acquired knowledge to other systems and cases;</li>
<p><li> Have an understanding of central algorithms used in data analysis and machine learning;</li>
<p><li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++.</li>
<p><li> Work on numerical projects to illustrate the theory. The projects play a central role.</li>
</ul>
</section>
@@ -1322,7 +1326,7 @@ ethical conduct is emphasized throughout the course.
<ol>
<p><li> Rapidly emerging application area</li>
<p><li> Experiment AND theory are evolving in many many fields. Still many low-hanging fruits.</li>
<p><li> Experiment AND theory are evolving in many many fields.</li>
<p><li> Requires education/retraining for more widespread adoption</li>
<p><li> A lot of &#8220;word-of-mouth&#8221; development methods</li>
</ol>
@@ -1354,7 +1358,7 @@ Neural Networks, etc.</a>
<ol>
<p><li> Identify problem type: classification, regression</li>
<p><li> Consider your data carefully</li>
<p><li> Choose a simple model that fits 1. and 2.</li>
<p><li> Choose a simple model that fits 1 and 2</li>
<p><li> Consider your data carefully again! Think of data representation more carefully.</li>
<p><li> Based on your results, feedback loop to earliest possible point</li>
</ol>
@@ -1400,10 +1404,10 @@ Neural Networks, etc.</a>
</section>
<section>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which Activation and Weights to Choose in Neural Networks </h2>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
<ol>
<p><li> RELU? ELU?</li>
<p><li> RELU? ELU? GELU? etc</li>
<p><li> Sigmoid or Tanh?</li>
<p><li> Set all weights to 0?</li>
<ul>
@@ -1461,17 +1465,17 @@ set of hyperparameters and regularization methods.
<section>
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<p>The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" target="_blank"><tt>https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/</tt></a> gives an excellent overview of courses on Machine learning at UiO.</p>
<ol>
<p><li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 &#8211; Advanced machine learning and data analysis for the physical sciences</a></li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI</li>
<p><li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 &#8211; Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Adaptive Methods for Data-Based Decision Making</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5400/" target="_blank">IN5400/INF5860 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Dyp l&#230;ring for autonome systemer</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Responsible Data Science</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html" target="_blank">IN4310 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5310/index.html" target="_blank">IN5310 &#8211; Advanced Deep Learning for Image Analysis</a></li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5490/index.html" target="_blank">IN5490 &#8211; Advanced Topics in Artificial Intelligence for Intelligent Systems</a></li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Deep learning for autonomous systems</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
</ol>
</section>
doc/pub/week48/html/week48-solarized.html
+26 -24
View File
@@ -8,8 +8,8 @@ doconce format html week48.do.txt --pygments_html_style=perldoc --html_style=sol
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 48: Support Vector Machines and Summary of course">
<title>Week 48: Support Vector Machines and Summary of course</title>
<meta name="description" content="Week 48: Gradient boosting and summary of course">
<title>Week 48: Gradient boosting and summary of course</title>
<link href="https://cdn.rawgit.com/doconce/doconce/master/bundled/html_styles/style_solarized_box/css/solarized_light_code.css" rel="stylesheet" type="text/css" title="light"/>
<script src="https://cdn.rawgit.com/doconce/doconce/master/bundled/html_styles/style_solarized_box/js/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
@@ -170,7 +170,7 @@ div.toc p,a {
None,
'choose-a-model-and-algorithm'),
('Preparing Your Data', 2, None, 'preparing-your-data'),
('Which Activation and Weights to Choose in Neural Networks',
('Which activation and weights to choose in neural networks',
2,
None,
'which-activation-and-weights-to-choose-in-neural-networks'),
@@ -279,7 +279,7 @@ MathJax.Hub.Config({
<!-- ------------------- main content ---------------------- -->
<center>
<h1>Week 48: Support Vector Machines and Summary of course</h1>
<h1>Week 48: Gradient boosting and summary of course</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
@@ -288,11 +288,11 @@ MathJax.Hub.Config({
</center>
<!-- institution -->
<center>
<b>Department of Physics, University of Oslo, Norway</b>
<b>Department of Physics and Center for Computing in Science Education, University of Oslo, Norway</b>
</center>
<br>
<center>
<h4>Nov 21, 2024</h4>
<h4>Nov 23, 2024</h4>
</center> <!-- date -->
<br>
@@ -307,7 +307,9 @@ MathJax.Hub.Config({
<p>
<ul>
<li> Work and Discussion of project 3</li>
<li> Last weekly exercise,</li>
<li> Last weekly exercise</li>
<li> Lab sessions at usual times.</li>
<li> For the week of December 2-6, lab sessions atart at 10am and end 4pm, room F&#216;434, Tuesday and Wednesday</li>
</ul>
</div>
@@ -316,7 +318,7 @@ MathJax.Hub.Config({
<b>Plans for the lecture Monday 25 November, with video suggestions etc</b>
<p>
<ol>
<li> Bossting and gradient boosting and ensemble models</li>
<li> Boosting and gradient boosting and ensemble models</li>
<li> Summary of course</li>
<li> Readings and Videos:
<ol type="a"></li>
@@ -1266,7 +1268,7 @@ plt.show()
<li> Gradient methods for data optimization</li>
<li> Estimation of errors using cross-validation, bootstrapping and jackknife methods;</li>
<li> Practical optimization using Singular-value decomposition and least squares for parameterizing data.</li>
<li> Principal Component Analysis to reduce the number of features.</li>
<li> Not discussed: Principal Component Analysis to reduce the number of features.</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="machine-learning">Machine learning </h2>
@@ -1293,7 +1295,7 @@ plt.show()
<li> Random forests</li>
<li> Boosting and gradient boosting</li>
</ol>
<li> Support vector machines
<li> Not discussed this year: Support vector machines
<ol type="a"></li>
<li> Binary classification and multiclass classification</li>
<li> Kernel methods</li>
@@ -1317,19 +1319,19 @@ ethical conduct is emphasized throughout the course.
<ul>
<li> Understand linear methods for regression and classification;</li>
<li> Learn about neural network;</li>
<li> Learn about bagging, boosting and trees</li>
<li> Support vector machines</li>
<li> Learn about bagging, boosting and trees
<!-- * Support vector machines --></li>
<li> Learn about basic data analysis;</li>
<li> Be capable of extending the acquired knowledge to other systems and cases;</li>
<li> Have an understanding of central algorithms used in data analysis and machine learning;</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++.</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role.</li>
</ul>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="perspective-on-machine-learning">Perspective on Machine Learning </h2>
<ol>
<li> Rapidly emerging application area</li>
<li> Experiment AND theory are evolving in many many fields. Still many low-hanging fruits.</li>
<li> Experiment AND theory are evolving in many many fields.</li>
<li> Requires education/retraining for more widespread adoption</li>
<li> A lot of &#8220;word-of-mouth&#8221; development methods</li>
</ol>
@@ -1357,7 +1359,7 @@ Neural Networks, etc.</a>
<ol>
<li> Identify problem type: classification, regression</li>
<li> Consider your data carefully</li>
<li> Choose a simple model that fits 1. and 2.</li>
<li> Choose a simple model that fits 1 and 2</li>
<li> Consider your data carefully again! Think of data representation more carefully.</li>
<li> Based on your results, feedback loop to earliest possible point</li>
</ol>
@@ -1390,10 +1392,10 @@ Neural Networks, etc.</a>
<li> When to do train/test split?</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which Activation and Weights to Choose in Neural Networks </h2>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
<ol>
<li> RELU? ELU?</li>
<li> RELU? ELU? GELU? etc</li>
<li> Sigmoid or Tanh?</li>
<li> Set all weights to 0?</li>
<ul>
@@ -1436,17 +1438,17 @@ set of hyperparameters and regularization methods.
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<p>The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" target="_blank"><tt>https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/</tt></a> gives an excellent overview of courses on Machine learning at UiO.</p>
<ol>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 &#8211; Advanced machine learning and data analysis for the physical sciences</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI</li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 &#8211; Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Adaptive Methods for Data-Based Decision Making</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5400/" target="_blank">IN5400/INF5860 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Dyp l&#230;ring for autonome systemer</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Responsible Data Science</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html" target="_blank">IN4310 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5310/index.html" target="_blank">IN5310 &#8211; Advanced Deep Learning for Image Analysis</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5490/index.html" target="_blank">IN5490 &#8211; Advanced Topics in Artificial Intelligence for Intelligent Systems</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Deep learning for autonomous systems</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="additional-courses-of-interest">Additional courses of interest </h2>
doc/pub/week48/html/week48.html
+26 -24
View File
@@ -8,8 +8,8 @@ doconce format html week48.do.txt --pygments_html_style=default --html_style=blo
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 48: Support Vector Machines and Summary of course">
<title>Week 48: Support Vector Machines and Summary of course</title>
<meta name="description" content="Week 48: Gradient boosting and summary of course">
<title>Week 48: Gradient boosting and summary of course</title>
<style type="text/css">
/* bloodish style */
body {
@@ -247,7 +247,7 @@ div.toc p,a {
None,
'choose-a-model-and-algorithm'),
('Preparing Your Data', 2, None, 'preparing-your-data'),
('Which Activation and Weights to Choose in Neural Networks',
('Which activation and weights to choose in neural networks',
2,
None,
'which-activation-and-weights-to-choose-in-neural-networks'),
@@ -356,7 +356,7 @@ MathJax.Hub.Config({
<!-- ------------------- main content ---------------------- -->
<center>
<h1>Week 48: Support Vector Machines and Summary of course</h1>
<h1>Week 48: Gradient boosting and summary of course</h1>
</center> <!-- document title -->
<!-- author(s): Morten Hjorth-Jensen -->
@@ -365,11 +365,11 @@ MathJax.Hub.Config({
</center>
<!-- institution -->
<center>
<b>Department of Physics, University of Oslo, Norway</b>
<b>Department of Physics and Center for Computing in Science Education, University of Oslo, Norway</b>
</center>
<br>
<center>
<h4>Nov 21, 2024</h4>
<h4>Nov 23, 2024</h4>
</center> <!-- date -->
<br>
@@ -384,7 +384,9 @@ MathJax.Hub.Config({
<p>
<ul>
<li> Work and Discussion of project 3</li>
<li> Last weekly exercise,</li>
<li> Last weekly exercise</li>
<li> Lab sessions at usual times.</li>
<li> For the week of December 2-6, lab sessions atart at 10am and end 4pm, room F&#216;434, Tuesday and Wednesday</li>
</ul>
</div>
@@ -393,7 +395,7 @@ MathJax.Hub.Config({
<b>Plans for the lecture Monday 25 November, with video suggestions etc</b>
<p>
<ol>
<li> Bossting and gradient boosting and ensemble models</li>
<li> Boosting and gradient boosting and ensemble models</li>
<li> Summary of course</li>
<li> Readings and Videos:
<ol type="a"></li>
@@ -1343,7 +1345,7 @@ plt<span style="color: #666666">.</span>show()
<li> Gradient methods for data optimization</li>
<li> Estimation of errors using cross-validation, bootstrapping and jackknife methods;</li>
<li> Practical optimization using Singular-value decomposition and least squares for parameterizing data.</li>
<li> Principal Component Analysis to reduce the number of features.</li>
<li> Not discussed: Principal Component Analysis to reduce the number of features.</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="machine-learning">Machine learning </h2>
@@ -1370,7 +1372,7 @@ plt<span style="color: #666666">.</span>show()
<li> Random forests</li>
<li> Boosting and gradient boosting</li>
</ol>
<li> Support vector machines
<li> Not discussed this year: Support vector machines
<ol type="a"></li>
<li> Binary classification and multiclass classification</li>
<li> Kernel methods</li>
@@ -1394,19 +1396,19 @@ ethical conduct is emphasized throughout the course.
<ul>
<li> Understand linear methods for regression and classification;</li>
<li> Learn about neural network;</li>
<li> Learn about bagging, boosting and trees</li>
<li> Support vector machines</li>
<li> Learn about bagging, boosting and trees
<!-- * Support vector machines --></li>
<li> Learn about basic data analysis;</li>
<li> Be capable of extending the acquired knowledge to other systems and cases;</li>
<li> Have an understanding of central algorithms used in data analysis and machine learning;</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++.</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role.</li>
</ul>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="perspective-on-machine-learning">Perspective on Machine Learning </h2>
<ol>
<li> Rapidly emerging application area</li>
<li> Experiment AND theory are evolving in many many fields. Still many low-hanging fruits.</li>
<li> Experiment AND theory are evolving in many many fields.</li>
<li> Requires education/retraining for more widespread adoption</li>
<li> A lot of &#8220;word-of-mouth&#8221; development methods</li>
</ol>
@@ -1434,7 +1436,7 @@ Neural Networks, etc.</a>
<ol>
<li> Identify problem type: classification, regression</li>
<li> Consider your data carefully</li>
<li> Choose a simple model that fits 1. and 2.</li>
<li> Choose a simple model that fits 1 and 2</li>
<li> Consider your data carefully again! Think of data representation more carefully.</li>
<li> Based on your results, feedback loop to earliest possible point</li>
</ol>
@@ -1467,10 +1469,10 @@ Neural Networks, etc.</a>
<li> When to do train/test split?</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which Activation and Weights to Choose in Neural Networks </h2>
<h2 id="which-activation-and-weights-to-choose-in-neural-networks">Which activation and weights to choose in neural networks </h2>
<ol>
<li> RELU? ELU?</li>
<li> RELU? ELU? GELU? etc</li>
<li> Sigmoid or Tanh?</li>
<li> Set all weights to 0?</li>
<ul>
@@ -1513,17 +1515,17 @@ set of hyperparameters and regularization methods.
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<p>The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" target="_blank"><tt>https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/</tt></a> gives an excellent overview of courses on Machine learning at UiO.</p>
<ol>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 &#8211; Advanced machine learning and data analysis for the physical sciences</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI</li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 &#8211; Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Adaptive Methods for Data-Based Decision Making</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5400/" target="_blank">IN5400/INF5860 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Dyp l&#230;ring for autonome systemer</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html" target="_blank">IN-STK5000 Responsible Data Science</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html" target="_blank">IN4310 &#8211; Machine Learning for Image Analysis</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5310/index.html" target="_blank">IN5310 &#8211; Advanced Deep Learning for Image Analysis</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN5490/index.html" target="_blank">IN5490 &#8211; Advanced Topics in Artificial Intelligence for Intelligent Systems</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/its/TEK5040/" target="_blank">TEK5040 &#8211; Deep learning for autonomous systems</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="additional-courses-of-interest">Additional courses of interest </h2>
doc/pub/week48/ipynb/ipynb-week48-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week48/ipynb/week48.ipynb
+174 -169
View File
File diff suppressed because it is too large Load Diff
doc/src/week48/week48.do.txt
+10 -8
View File
@@ -10,7 +10,9 @@ DATE: today
!bblock Lab sessions on Tuesday and Wednesday
* Work and Discussion of project 3
* Last weekly exercise,
* Last weekly exercise
* Lab sessions at usual times.
* For the week of December 2-6, lab sessions atart at 10am and end 4pm, room FØ434, Tuesday and Wednesday
!eblock
!bblock Plans for the lecture Monday 25 November, with video suggestions etc
@@ -999,16 +1001,16 @@ o Jackknife and many other
!split
===== Other courses on Data science and Machine Learning at UiO =====
The link here URL:"https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" gives an excellent overview of courses on Machine learning at UiO.
o "STK2100 Machine learning and statistical methods for prediction and classification":"http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html".
o "IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach.
o "FYS5429 – Advanced machine learning and data analysis for the physical sciences":"https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html"
o "IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI
o "STK-INF3000/4000 Selected Topics in Data Science":"http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html". The course provides insight into selected contemporary relevant topics within Data Science.
o "IN4080 Natural Language Processing":"https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html". Probabilistic and machine learning techniques applied to natural language processing.
o "STK-IN4300 – Statistical learning methods in Data Science":"https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html". An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.
o "IN-STK5000 Adaptive Methods for Data-Based Decision Making":"https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html". Methods for adaptive collection and processing of data based on machine learning techniques.
o "IN5400/INF5860 – Machine Learning for Image Analysis":"https://www.uio.no/studier/emner/matnat/ifi/IN5400/". An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
o "TEK5040 – Dyp læring for autonome systemer":"https://www.uio.no/studier/emner/matnat/its/TEK5040/". The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.
o "IN-STK5000 Responsible Data Science":"https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html". Methods for adaptive collection and processing of data based on machine learning techniques.
o "IN4310 – Machine Learning for Image Analysis":"https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html". An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
o "IN5310 – Advanced Deep Learning for Image Analysis":"https://www.uio.no/studier/emner/matnat/ifi/IN5310/index.html"
o "IN5490 – Advanced Topics in Artificial Intelligence for Intelligent Systems":"https://www.uio.no/studier/emner/matnat/ifi/IN5490/index.html"
o "TEK5040 – Deep learning for autonomous systems":"https://www.uio.no/studier/emner/matnat/its/TEK5040/". The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.
!split
===== Additional courses of interest =====