lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update lecture notes

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-08-20 21:16:55 +02:00
parent cbb47a8053
commit dd93950e7a
8 changed files with 15 additions and 637 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/.doctrees/environment.pickle
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/schedule.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/textbooks.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/_sources/schedule.md
-163
View File
@@ -22,166 +22,3 @@ For the reading assignments we use the following abbreviations:
- HTF: Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning
- AG: Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow
### Week 34 August 22-26
- Lab Wednesday: Introduction to software and repetition of Python Programming
- Lecture Thursday: Introduction to the course, what is Machine Learning and introduction to Linear Regression.
- Video of Lecture August 25, 2022 at https://youtu.be/KL0m3-yhd5w
- Lecture Friday: Basics of Linear Regression
- Video of Lecture August 26, 2022 at https://youtu.be/ne_xCL2ctM0
- Reading recommendations:
- Refresh linear algebra, GBC chapters 1 and 2.
- CMB sections 1.1 and 3.1.
- HTF chapters 2 and 3.
- See lecture notes for week 34 at https://compphysics.github.io/MachineLearning/doc/web/course.html
### Week 35 August 29-September 2
- Lab Wednesday: Work on exercises 1-5 for week 35
- Thursday: Review of ordinary Least Squares with applications, reminder on statistics and start discussion of Ridge Regression and Singular Value Decomposition
- Video of lecture Thursday at https://youtu.be/jYdg2xzKa5E
- Friday: Discussion of Ridge and Lasso Regression and links with Singular Value Decomposition
- Video of lecture Friday at https://youtu.be/07e-SUYRzM0
- Reading recommendations:
- See lecture notes for week 35 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For a review on statistics see jupyter-book https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/statistics.html, the most relevant parts are covered by sections 1.1.1-1.1.5
- HTF chapter 3. GBC chapters 1 and and sections 3.1-3.11 and 5.1
- CMB sections 1.1 and 3.1
- A good review on statistics is given by Murphy's text, chapter 2, see https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/MachineLearningMurphy.pdf
### Week 36 September 5-9
- Lab Wednesday: Exercises 1 and 2 from week 36
- Lecture Thursday: Summary from last week on SVD, more on Statistics, probability theory and linear regression
- Video of Lecture at https://youtu.be/qn_BAVhMD8U
- Friday: Linear Regression and more links with Statistics, Resampling methods and presentation of first project.
- Video of Lecture at https://youtu.be/_CPGg0JYH8M
- Reading recommendations:
- Lectures on Regression for week 36 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Bishop 1.1, 1.2, 2.1, 2.2, 2.3 and 3.1
- Hastie et al chapter 3
### Week 37 September 12-16
- Lab Wednesday: Work on Project 1
- Lecture Thursday: Resampling methods, cross-validation and Bootstrap
- Thursday September: Summary of Ridge and Lasso with examples and start resampling techniques
- Video of Lecture at https://youtu.be/YVQGvcsovpw
- Lecture Friday: More on Resampling methods and summary of linear regression
- Video of Lecture at https://youtu.be/rbaHRF-7bsQ
- Reading recommendations:
- Lectures on Resampling methods for week 37 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Bishop 1.3 (cross-validation) and 3.2 (bias-variance tradeoff)
- Hastie et al Chapter 7, here we recommend 7.1-7.5 and 7.10 (cross-validation) and 7.11 (bootstrap)
- Goodfellow et al discuss some of these topics in sections 5.2-5.5.
### Week 38 September 19-23
- Lab Wednesday: Work on Project 1
- Lecture Thursday: Classification problems and Logistic Regression, from binary cases to several categories and gradient optmization
- Video of Lecture at https://youtu.be/sdt_BFla8uA
- Lecture Friday: Logistic Regression and gradient optimization
- Video of Lecture at https://youtu.be/7OdqoLOphTA
- Reading recommendations:
- See lecture notes for week 38 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Bishop 4.1, 4.2 and 4.3. Not all the material is relevant or will be covered. Section 4.3 is the most relevant, but 4.1 and 4.2 give interesting background readings for logistic regression
- Hastie et al 4.1, 4.2 and 4.3 on logistic regression
- For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
### Week 39 September 26-30
- Lab Wednesday: Work on Project 1
- Lecture Thursday: * Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Examples on how to implement Logistic Regression and discussion of stochastic gradient descent
- Video of lecture at https://youtu.be/rDBj50Lv3Go
* Friday: Stochastic Gradient descent with examples and automatic differentiation
- Video of lecture at https://youtu.be/OH6I_oscwPc
* Reading recommendations:
- See lecture notes for week 39 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
### Week 40 October 3-7
- Lab Wednesday: Wrap up project 1
- Lecture Thursday: Stochastic gradient descent, automatic differentiation and start discussion of feed-forward Neural Network code for regression and classification
- Video of Lecture at https://youtu.be/sCHiGSoG2lE
- Lecture Friday: Deep Learning and Neural Networks: the back propagation algorithm
- Video of Lecture at https://youtu.be/hDTtA7PRRfI
- Reading recommendations:
- See lecture notes for week 40 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For neural networks we recommend Goodfellow et al chapters 6 and 7 and Bishop 5.1-5.4
- For stochastic gradient descent we recommend Goodfellow et al chapter 8
### Week 41 October 10-14
- Lab Wednesday: Work on project 2
- Lecture Thursday: Deep learning and Neural Networks, developing a code for Neural Networks
- Video of Lecture at https://youtu.be/yzbxJI6LgL0
- Lecture Friday: Developing a neural network code
- Video of Lecture at https://youtu.be/CPj4mh7M9no
- Reading recommendations:
- See lecture notes for week 41 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. chapter 11 and 12 on practicalities and applications
### Week 42 October 17-21
- Lab Wednesday: Work on project 2
- Lecture Thursday: Discussion of Neural Network calculations and tensorflow. Solving differential equations with neural networks
- Video of Lecture at https://youtu.be/MdYT6uwOkT0
- Lecture Friday: Convolutional Neural Networks and classification problems
- Video of Lecture at https://youtu.be/3bDkrB-E7cU
- Reading recommendations:
- See lecture notes for week 42 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. See also chapter 11 and 12 on practicalities and applications
### Week 43 October 24-28
- Lab Wednesday: Work on project 2
- Lecture Thursday: Recurrent Neural Networks
- Video of Lecture at https://youtu.be/Hm6Ay5DS6o0
- Lecture Friday: Recurrent Neural Networks and time series and principal component analysis (PCA)
- Video of Lecture at https://youtu.be/JHfgQ77fpqs
- Reading recommendations:
- See lecture notes for week 43 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- For RNNs, see Goodfellow et al chapter 10 and discussions in chapter 11 and 12 on practicalities and applications
- For PCA, see lecture notes chapter 11 and Geron's text chapter 8
### Week 44 October 31-November 4
- Lab Wednesday: Work on project 2
- Lecture Thursday: Decision trees, basic algorithms for classification and regression
- Video of Lecture at https://youtu.be/7jexGH5SOOE
- Lecture Friday: From trees to forests and ensemble methods
- Video of Lecture at https://youtu.be/9QcU8VcXxRU
- Reading recommendations:
- See lecture notes for week 44 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Hastie et al sections 9.1 and 9.2. Geron's text chapter 6 (Decision trees)
### Week 45 November 7-11
- Lab Wednesday: Work on project 2, project 3 available November 11.
- Lecture Thursday: Ensemble methods, bagging and random forests, boosting.
- Video of Lecture at https://youtu.be/mK48PfCxgYk
- Lecture Friday: Adaptive boosting and gradient boosting, summary of decision trees and ensemble methods
- Video of Lecture at https://youtu.be/v8eJBFeZKuI
- Reading recommendations:
- See lecture notes for week 45 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Hastie et al chapter 10 and Geron chapters 5 and 6
### Week 46 November 14-18
- Lab Wednesday: Work on project 3
- Lecture Thursday: Support Vector machines
- Video of Lecture at https://youtu.be/F3CkH-opbdY
- Lecture Friday: Workshop on project 3
- Video of Lecture at https://youtu.be/BbupEEvMXtg
- Reading recommendations:
- See lecture notes for week 46 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Hastie et al chapter 12
### Week 47 November 21-25
- Lab Wednesday: Work on project 3
- Lecture Thursday: Dimensionality reduction and unsupervised learning: Principal Component analysis (PCA) and clustering
- Video of Lecture https://youtu.be/VJIsEQM2lCI
- Lecture Friday: PCA and clustering and Summary of Course
- Video of Lecture https://youtu.be/olXksEL3P4A
- Reading recommendations:
- See lecture notes for week 47 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
- Geron's chapter 9 on PCA
- Hastie et al Chapter 13 (sections 13.1-13.2 are the most relevant ones)
- Excellent videos:
- We recommend highly the video on PCA by Brunton and Kutz at http://www.databookuw.com/page-2/page-4/, see in particular the video of section 1.5.
- And another good video on PCA is at https://www.youtube.com/watch?v=FgakZw6K1QQ
- k-means clustering video at https://www.youtube.com/watch?v=4b5d3muPQmA
doc/LectureNotes/_build/html/_sources/textbooks.md
+9 -7
View File
@@ -25,20 +25,22 @@ _General Machine Learning Books_:
## Links to relevant courses at the University of Oslo
The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.
- _STK2100 Machine learning and statistical methods for prediction and classification_ http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html.
- _IN3050 Introduction to Artificial Intelligence and Machine Learning_ https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html. Introductory course in machine learning and AI with an algorithmic approach.
- _STK-INF3000/4000 Selected Topics in Data Science_ http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html. The course provides insight into selected contemporary relevant topics within Data Science.
- _FYS5429 Advanced Machine Learning for the Physical Sciences_ https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html
- _FYS5419 Quantum Computing and Quantum Machine Learning_ https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html
- _STK2100 Machine learning and statistical methods for prediction and classification_ http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html.
- _IN3050/4050 Introduction to Artificial Intelligence and Machine Learning_ https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html. Introductory course in machine learning and AI with an algorithmic approach.
- _IN4080 Natural Language Processing_ https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html. Probabilistic and machine learning techniques applied to natural language processing.
- _IN5550 – Neural Methods in Natural Language Processing_ https://www.uio.no/studier/emner/matnat/ifi/IN5550/index.html. This course studies a selection of advanced techniques in Natural Language Processing (NLP), with particular emphasis on recent and current research literature. The focus will be on machine learning and specifically deep neural network approaches to the automated analysis of natural language text.
- _STK-IN4300 Statistical learning methods in Data Science_ https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.
- _INF4490 Biologically Inspired Computing_ http://www.uio.no/studier/emner/matnat/ifi/INF4490/. An introduction to self-adapting methods also called artificial intelligence or machine learning.
- _IN-STK5000 Adaptive Methods for Data-Based Decision Making_ https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html. Methods for adaptive collection and processing of data based on machine learning techniques.
- _IN5400/INF5860 Machine Learning for Image Analysis_ https://www.uio.no/studier/emner/matnat/ifi/IN5400/. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- _TEK5040 Deep learning for autonomous systems_ https://www.uio.no/studier/emner/matnat/its/TEK5040/. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.
- _IN4310 Deep Learning for Image Analysis_ https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- _STK4051 Computational Statistics_ https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html
- _STK4021 Applied Bayesian Analysis and Numerical Methods_ https://www.uio.no/studier/emner/matnat/math/STK4021/
doc/LectureNotes/_build/html/schedule.html
-460
View File
@@ -314,78 +314,6 @@ const thebe_selector_output = ".output, .cell_output"
<a class="reference internal nav-link" href="#weekly-schedule">
Weekly Schedule
</a>
<ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-34-august-22-26">
Week 34 August 22-26
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-35-august-29-september-2">
Week 35 August 29-September 2
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-36-september-5-9">
Week 36 September 5-9
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-37-september-12-16">
Week 37 September 12-16
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-38-september-19-23">
Week 38 September 19-23
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-39-september-26-30">
Week 39 September 26-30
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-40-october-3-7">
Week 40 October 3-7
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-41-october-10-14">
Week 41 October 10-14
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-42-october-17-21">
Week 42 October 17-21
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-43-october-24-28">
Week 43 October 24-28
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-44-october-31-november-4">
Week 44 October 31-November 4
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-45-november-7-11">
Week 45 November 7-11
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-46-november-14-18">
Week 46 November 14-18
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-47-november-21-25">
Week 47 November 21-25
</a>
</li>
</ul>
</li>
</ul>
@@ -411,78 +339,6 @@ const thebe_selector_output = ".output, .cell_output"
<a class="reference internal nav-link" href="#weekly-schedule">
Weekly Schedule
</a>
<ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-34-august-22-26">
Week 34 August 22-26
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-35-august-29-september-2">
Week 35 August 29-September 2
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-36-september-5-9">
Week 36 September 5-9
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-37-september-12-16">
Week 37 September 12-16
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-38-september-19-23">
Week 38 September 19-23
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-39-september-26-30">
Week 39 September 26-30
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-40-october-3-7">
Week 40 October 3-7
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-41-october-10-14">
Week 41 October 10-14
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-42-october-17-21">
Week 42 October 17-21
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-43-october-24-28">
Week 43 October 24-28
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-44-october-31-november-4">
Week 44 October 31-November 4
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-45-november-7-11">
Week 45 November 7-11
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-46-november-14-18">
Week 46 November 14-18
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#week-47-november-21-25">
Week 47 November 21-25
</a>
</li>
</ul>
</li>
</ul>
@@ -515,322 +371,6 @@ const thebe_selector_output = ".output, .cell_output"
<li><p>HTF: Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning</p></li>
<li><p>AG: Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow</p></li>
</ul>
<div class="section" id="week-34-august-22-26">
<h3>Week 34 August 22-26<a class="headerlink" href="#week-34-august-22-26" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Introduction to software and repetition of Python Programming</p></li>
<li><p>Lecture Thursday: Introduction to the course, what is Machine Learning and introduction to Linear Regression.</p></li>
<li><p>Video of Lecture August 25, 2022 at <a class="reference external" href="https://youtu.be/KL0m3-yhd5w">https://youtu.be/KL0m3-yhd5w</a></p></li>
<li><p>Lecture Friday: Basics of Linear Regression</p></li>
<li><p>Video of Lecture August 26, 2022 at <a class="reference external" href="https://youtu.be/ne_xCL2ctM0">https://youtu.be/ne_xCL2ctM0</a></p></li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>Refresh linear algebra, GBC chapters 1 and 2.</p></li>
<li><p>CMB sections 1.1 and 3.1.</p></li>
<li><p>HTF chapters 2 and 3.</p></li>
<li><p>See lecture notes for week 34 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a></p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-35-august-29-september-2">
<h3>Week 35 August 29-September 2<a class="headerlink" href="#week-35-august-29-september-2" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on exercises 1-5 for week 35</p></li>
<li><p>Thursday: Review of ordinary Least Squares with applications, reminder on statistics and start discussion of Ridge Regression and Singular Value Decomposition</p></li>
<li><p>Video of lecture Thursday at <a class="reference external" href="https://youtu.be/jYdg2xzKa5E">https://youtu.be/jYdg2xzKa5E</a></p></li>
<li><p>Friday: Discussion of Ridge and Lasso Regression and links with Singular Value Decomposition</p></li>
<li><p>Video of lecture Friday at <a class="reference external" href="https://youtu.be/07e-SUYRzM0">https://youtu.be/07e-SUYRzM0</a></p></li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 35 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For a review on statistics see jupyter-book <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/statistics.html">https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/statistics.html</a>, the most relevant parts are covered by sections 1.1.1-1.1.5</p></li>
<li><p>HTF chapter 3. GBC chapters 1 and and sections 3.1-3.11 and 5.1</p></li>
<li><p>CMB sections 1.1 and 3.1</p></li>
<li><p>A good review on statistics is given by Murphy’s text, chapter 2, see <a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/MachineLearningMurphy.pdf">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/MachineLearningMurphy.pdf</a></p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-36-september-5-9">
<h3>Week 36 September 5-9<a class="headerlink" href="#week-36-september-5-9" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Exercises 1 and 2 from week 36</p></li>
<li><p>Lecture Thursday: Summary from last week on SVD, more on Statistics, probability theory and linear regression</p></li>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/qn_BAVhMD8U">https://youtu.be/qn_BAVhMD8U</a></p></li>
<li><p>Friday: Linear Regression and more links with Statistics, Resampling methods and presentation of first project.</p></li>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/_CPGg0JYH8M">https://youtu.be/_CPGg0JYH8M</a></p></li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>Lectures on Regression for week 36 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Bishop 1.1, 1.2, 2.1, 2.2, 2.3 and 3.1</p></li>
<li><p>Hastie et al chapter 3</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-37-september-12-16">
<h3>Week 37 September 12-16<a class="headerlink" href="#week-37-september-12-16" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on Project 1</p></li>
<li><p>Lecture Thursday: Resampling methods, cross-validation and Bootstrap</p>
<ul>
<li><p>Thursday September: Summary of Ridge and Lasso with examples and start resampling techniques</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/YVQGvcsovpw">https://youtu.be/YVQGvcsovpw</a></p></li>
</ul>
</li>
</ul>
</li>
<li><p>Lecture Friday: More on Resampling methods and summary of linear regression</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/rbaHRF-7bsQ">https://youtu.be/rbaHRF-7bsQ</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>Lectures on Resampling methods for week 37 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Bishop 1.3 (cross-validation) and 3.2 (bias-variance tradeoff)</p></li>
<li><p>Hastie et al Chapter 7, here we recommend 7.1-7.5 and 7.10 (cross-validation) and 7.11 (bootstrap)</p></li>
<li><p>Goodfellow et al discuss some of these topics in sections 5.2-5.5.</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-38-september-19-23">
<h3>Week 38 September 19-23<a class="headerlink" href="#week-38-september-19-23" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on Project 1</p></li>
<li><p>Lecture Thursday: Classification problems and Logistic Regression, from binary cases to several categories and gradient optmization</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/sdt_BFla8uA">https://youtu.be/sdt_BFla8uA</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Logistic Regression and gradient optimization</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/7OdqoLOphTA">https://youtu.be/7OdqoLOphTA</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 38 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Bishop 4.1, 4.2 and 4.3. Not all the material is relevant or will be covered. Section 4.3 is the most relevant, but 4.1 and 4.2 give interesting background readings for logistic regression</p></li>
<li><p>Hastie et al 4.1, 4.2 and 4.3 on logistic regression</p></li>
<li><p>For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-39-september-26-30">
<h3>Week 39 September 26-30<a class="headerlink" href="#week-39-september-26-30" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on Project 1</p></li>
<li><p>Lecture Thursday: * Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Examples on how to implement Logistic Regression and discussion of stochastic gradient descent</p>
<ul>
<li><p>Video of lecture at <a class="reference external" href="https://youtu.be/rDBj50Lv3Go">https://youtu.be/rDBj50Lv3Go</a></p></li>
</ul>
</li>
</ul>
<ul class="simple">
<li><p>Friday: Stochastic Gradient descent with examples and automatic differentiation</p>
<ul>
<li><p>Video of lecture at <a class="reference external" href="https://youtu.be/OH6I_oscwPc">https://youtu.be/OH6I_oscwPc</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 39 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-40-october-3-7">
<h3>Week 40 October 3-7<a class="headerlink" href="#week-40-october-3-7" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Wrap up project 1</p></li>
<li><p>Lecture Thursday: Stochastic gradient descent, automatic differentiation and start discussion of feed-forward Neural Network code for regression and classification</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/sCHiGSoG2lE">https://youtu.be/sCHiGSoG2lE</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Deep Learning and Neural Networks: the back propagation algorithm</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/hDTtA7PRRfI">https://youtu.be/hDTtA7PRRfI</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 40 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For neural networks we recommend Goodfellow et al chapters 6 and 7 and Bishop 5.1-5.4</p></li>
<li><p>For stochastic gradient descent we recommend Goodfellow et al chapter 8</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-41-october-10-14">
<h3>Week 41 October 10-14<a class="headerlink" href="#week-41-october-10-14" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 2</p></li>
<li><p>Lecture Thursday: Deep learning and Neural Networks, developing a code for Neural Networks</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/yzbxJI6LgL0">https://youtu.be/yzbxJI6LgL0</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Developing a neural network code</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/CPj4mh7M9no">https://youtu.be/CPj4mh7M9no</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 41 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. chapter 11 and 12 on practicalities and applications</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-42-october-17-21">
<h3>Week 42 October 17-21<a class="headerlink" href="#week-42-october-17-21" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 2</p></li>
<li><p>Lecture Thursday: Discussion of Neural Network calculations and tensorflow. Solving differential equations with neural networks</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/MdYT6uwOkT0">https://youtu.be/MdYT6uwOkT0</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Convolutional Neural Networks and classification problems</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/3bDkrB-E7cU">https://youtu.be/3bDkrB-E7cU</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 42 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. See also chapter 11 and 12 on practicalities and applications</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-43-october-24-28">
<h3>Week 43 October 24-28<a class="headerlink" href="#week-43-october-24-28" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 2</p></li>
<li><p>Lecture Thursday: Recurrent Neural Networks</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/Hm6Ay5DS6o0">https://youtu.be/Hm6Ay5DS6o0</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Recurrent Neural Networks and time series and principal component analysis (PCA)</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/JHfgQ77fpqs">https://youtu.be/JHfgQ77fpqs</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 43 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>For RNNs, see Goodfellow et al chapter 10 and discussions in chapter 11 and 12 on practicalities and applications</p></li>
<li><p>For PCA, see lecture notes chapter 11 and Geron’s text chapter 8</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-44-october-31-november-4">
<h3>Week 44 October 31-November 4<a class="headerlink" href="#week-44-october-31-november-4" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 2</p></li>
<li><p>Lecture Thursday: Decision trees, basic algorithms for classification and regression</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/7jexGH5SOOE">https://youtu.be/7jexGH5SOOE</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: From trees to forests and ensemble methods</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/9QcU8VcXxRU">https://youtu.be/9QcU8VcXxRU</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 44 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Hastie et al sections 9.1 and 9.2. Geron’s text chapter 6 (Decision trees)</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-45-november-7-11">
<h3>Week 45 November 7-11<a class="headerlink" href="#week-45-november-7-11" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 2, project 3 available November 11.</p></li>
<li><p>Lecture Thursday: Ensemble methods, bagging and random forests, boosting.</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/mK48PfCxgYk">https://youtu.be/mK48PfCxgYk</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Adaptive boosting and gradient boosting, summary of decision trees and ensemble methods</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/v8eJBFeZKuI">https://youtu.be/v8eJBFeZKuI</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 45 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Hastie et al chapter 10 and Geron chapters 5 and 6</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-46-november-14-18">
<h3>Week 46 November 14-18<a class="headerlink" href="#week-46-november-14-18" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 3</p></li>
<li><p>Lecture Thursday: Support Vector machines</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/F3CkH-opbdY">https://youtu.be/F3CkH-opbdY</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: Workshop on project 3</p>
<ul>
<li><p>Video of Lecture at <a class="reference external" href="https://youtu.be/BbupEEvMXtg">https://youtu.be/BbupEEvMXtg</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 46 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Hastie et al chapter 12</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="week-47-november-21-25">
<h3>Week 47 November 21-25<a class="headerlink" href="#week-47-november-21-25" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Lab Wednesday: Work on project 3</p></li>
<li><p>Lecture Thursday: Dimensionality reduction and unsupervised learning: Principal Component analysis (PCA) and clustering</p>
<ul>
<li><p>Video of Lecture <a class="reference external" href="https://youtu.be/VJIsEQM2lCI">https://youtu.be/VJIsEQM2lCI</a></p></li>
</ul>
</li>
<li><p>Lecture Friday: PCA and clustering and Summary of Course</p>
<ul>
<li><p>Video of Lecture <a class="reference external" href="https://youtu.be/olXksEL3P4A">https://youtu.be/olXksEL3P4A</a></p></li>
</ul>
</li>
<li><p>Reading recommendations:</p>
<ul>
<li><p>See lecture notes for week 47 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
<li><p>Geron’s chapter 9 on PCA</p></li>
<li><p>Hastie et al Chapter 13 (sections 13.1-13.2 are the most relevant ones)</p></li>
</ul>
</li>
<li><p>Excellent videos:</p>
<ul>
<li><p>We recommend highly the video on PCA by Brunton and Kutz at <a class="reference external" href="http://www.databookuw.com/page-2/page-4/">http://www.databookuw.com/page-2/page-4/</a>, see in particular the video of section 1.5.</p></li>
<li><p>And another good video on PCA is at <a class="reference external" href="https://www.youtube.com/watch?v=FgakZw6K1QQ">https://www.youtube.com/watch?v=FgakZw6K1QQ</a></p></li>
<li><p>k-means clustering video at <a class="reference external" href="https://www.youtube.com/watch?v=4b5d3muPQmA">https://www.youtube.com/watch?v=4b5d3muPQmA</a></p></li>
</ul>
</li>
</ul>
</div>
</div>
</div>
doc/LectureNotes/_build/html/searchindex.js
+1 -1
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/html/textbooks.html
+5 -6
View File
@@ -385,17 +385,16 @@ The weekly plans will include reading suggestions from these two textbooks.
</div>
<div class="section" id="links-to-relevant-courses-at-the-university-of-oslo">
<h1>Links to relevant courses at the University of Oslo<a class="headerlink" href="#links-to-relevant-courses-at-the-university-of-oslo" title="Permalink to this headline"></a></h1>
<p>The link here <a class="reference external" href="https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/">https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/</a> gives an excellent overview of courses on Machine learning at UiO.</p>
<ul class="simple">
<li><p><em>FYS5429 Advanced Machine Learning for the Physical Sciences</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html">https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html</a></p></li>
<li><p><em>FYS5419 Quantum Computing and Quantum Machine Learning</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html">https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html</a></p></li>
<li><p><em>STK2100 Machine learning and statistical methods for prediction and classification</em> <a class="reference external" href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html">http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html</a>.</p></li>
<li><p><em>IN3050 Introduction to Artificial Intelligence and Machine Learning</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html">https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html</a>. Introductory course in machine learning and AI with an algorithmic approach.</p></li>
<li><p><em>STK-INF3000/4000 Selected Topics in Data Science</em> <a class="reference external" href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html">http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html</a>. The course provides insight into selected contemporary relevant topics within Data Science.</p></li>
<li><p><em>IN3050/4050 Introduction to Artificial Intelligence and Machine Learning</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html">https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html</a>. Introductory course in machine learning and AI with an algorithmic approach.</p></li>
<li><p><em>IN4080 Natural Language Processing</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html">https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html</a>. Probabilistic and machine learning techniques applied to natural language processing.</p></li>
<li><p><em>IN5550 – Neural Methods in Natural Language Processing</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN5550/index.html">https://www.uio.no/studier/emner/matnat/ifi/IN5550/index.html</a>. This course studies a selection of advanced techniques in Natural Language Processing (NLP), with particular emphasis on recent and current research literature. The focus will be on machine learning and specifically deep neural network approaches to the automated analysis of natural language text.</p></li>
<li><p><em>STK-IN4300 Statistical learning methods in Data Science</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html">https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</p></li>
<li><p><em>INF4490 Biologically Inspired Computing</em> <a class="reference external" href="http://www.uio.no/studier/emner/matnat/ifi/INF4490/">http://www.uio.no/studier/emner/matnat/ifi/INF4490/</a>. An introduction to self-adapting methods also called artificial intelligence or machine learning.</p></li>
<li><p><em>IN-STK5000 Adaptive Methods for Data-Based Decision Making</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html">https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html</a>. Methods for adaptive collection and processing of data based on machine learning techniques.</p></li>
<li><p><em>IN5400/INF5860 Machine Learning for Image Analysis</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN5400/">https://www.uio.no/studier/emner/matnat/ifi/IN5400/</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</p></li>
<li><p><em>TEK5040 Deep learning for autonomous systems</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/its/TEK5040/">https://www.uio.no/studier/emner/matnat/its/TEK5040/</a>. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.</p></li>
<li><p><em>IN4310 Deep Learning for Image Analysis</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html">https://www.uio.no/studier/emner/matnat/ifi/IN4310/index.html</a>. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.</p></li>
<li><p><em>STK4051 Computational Statistics</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html">https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html</a></p></li>
<li><p><em>STK4021 Applied Bayesian Analysis and Numerical Methods</em> <a class="reference external" href="https://www.uio.no/studier/emner/matnat/math/STK4021/">https://www.uio.no/studier/emner/matnat/math/STK4021/</a></p></li>
</ul>