Teaching schedule with links to material
Contents
Teaching schedule with links to material¶
This course will be delivered in a hybrid mode, with online lectures and on site or online laboratory sessions.
Four lectures per week, Fall semester, 10 ECTS. The lectures are in person but will be recorded and linked to this site and the official University of Oslo website for the course;
Two hours of laboratory sessions for work on computational projects and exercises for each group. There will also be fully digital laboratory sessions for those who cannot attend;
Three projects which are graded and count 1/3 each of the final grade. The deadlines for the projects are October 7 for project 1, November 11 for project 2 and December 9 for project 3.
A selected number of weekly assignments;
The course is part of the CS Master of Science program, but is open to other bachelor and Master of Science students at the University of Oslo;
The course is offered as a FYS-STK4155 (Master of Science level) and a FYS-STK3155 (senior undergraduate) course;
Videos of teaching material are available via the links at https://compphysics.github.io/MachineLearning/doc/web/course.html;
Weekly emails with summary of activities will be mailed to all participants;
Weekly Schedule¶
For the reading assignments we use the following abbreviations:
GBC: Goodfellow, Bengio, and Courville, Deep Learning
CMB: Christopher M. Bishop, Pattern Recognition and Machine Learning
HTF: Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning
AG: Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow
Week 34 August 22-26¶
Lab Wednesday: Introduction to software and repetition of Python Programming
Lecture Thursday: Introduction to the course, what is Machine Learning and introduction to Linear Regression.
Video of Lecture August 25, 2022 at https://youtu.be/KL0m3-yhd5w
Lecture Friday: Basics of Linear Regression
Video of Lecture August 26, 2022 at https://youtu.be/ne_xCL2ctM0
Reading recommendations:
Refresh linear algebra, GBC chapters 1 and 2.
CMB sections 1.1 and 3.1.
HTF chapters 2 and 3.
See lecture notes for week 34 at https://compphysics.github.io/MachineLearning/doc/web/course.html
Week 35 August 29-September 2¶
Lab Wednesday: Work on exercises 1-5 for week 35
Thursday: Review of ordinary Least Squares with applications, reminder on statistics and start discussion of Ridge Regression and Singular Value Decomposition
Video of lecture Thursday at https://youtu.be/jYdg2xzKa5E
Friday: Discussion of Ridge and Lasso Regression and links with Singular Value Decomposition
Video of lecture Friday at https://youtu.be/07e-SUYRzM0
Reading recommendations:
See lecture notes for week 35 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For a review on statistics see jupyter-book https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/statistics.html, the most relevant parts are covered by sections 1.1.1-1.1.5
HTF chapter 3. GBC chapters 1 and and sections 3.1-3.11 and 5.1
CMB sections 1.1 and 3.1
A good review on statistics is given by Murphy’s text, chapter 2, see https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/MachineLearningMurphy.pdf
Week 36 September 5-9¶
Lab Wednesday: Exercises 1 and 2 from week 36
Lecture Thursday: Summary from last week on SVD, more on Statistics, probability theory and linear regression
Video of Lecture at https://youtu.be/qn_BAVhMD8U
Friday: Linear Regression and more links with Statistics, Resampling methods and presentation of first project.
Video of Lecture at https://youtu.be/_CPGg0JYH8M
Reading recommendations:
Lectures on Regression for week 36 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Bishop 1.1, 1.2, 2.1, 2.2, 2.3 and 3.1
Hastie et al chapter 3
Week 37 September 12-16¶
Lab Wednesday: Work on Project 1
Lecture Thursday: Resampling methods, cross-validation and Bootstrap
Thursday September: Summary of Ridge and Lasso with examples and start resampling techniques
Video of Lecture at https://youtu.be/YVQGvcsovpw
Lecture Friday: More on Resampling methods and summary of linear regression
Video of Lecture at https://youtu.be/rbaHRF-7bsQ
Reading recommendations:
Lectures on Resampling methods for week 37 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Bishop 1.3 (cross-validation) and 3.2 (bias-variance tradeoff)
Hastie et al Chapter 7, here we recommend 7.1-7.5 and 7.10 (cross-validation) and 7.11 (bootstrap)
Goodfellow et al discuss some of these topics in sections 5.2-5.5.
Week 38 September 19-23¶
Lab Wednesday: Work on Project 1
Lecture Thursday: Classification problems and Logistic Regression, from binary cases to several categories and gradient optmization
Video of Lecture at https://youtu.be/sdt_BFla8uA
Lecture Friday: Logistic Regression and gradient optimization
Video of Lecture at https://youtu.be/7OdqoLOphTA
Reading recommendations:
See lecture notes for week 38 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Bishop 4.1, 4.2 and 4.3. Not all the material is relevant or will be covered. Section 4.3 is the most relevant, but 4.1 and 4.2 give interesting background readings for logistic regression
Hastie et al 4.1, 4.2 and 4.3 on logistic regression
For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
Week 39 September 26-30¶
Lab Wednesday: Work on Project 1
Lecture Thursday: * Thursday: Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Examples on how to implement Logistic Regression and discussion of stochastic gradient descent
Video of lecture at https://youtu.be/rDBj50Lv3Go
Friday: Stochastic Gradient descent with examples and automatic differentiation
Video of lecture at https://youtu.be/OH6I_oscwPc
Reading recommendations:
See lecture notes for week 39 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
Week 40 October 3-7¶
Lab Wednesday: Wrap up project 1
Lecture Thursday: Stochastic gradient descent, automatic differentiation and start discussion of feed-forward Neural Network code for regression and classification
Video of Lecture at https://youtu.be/sCHiGSoG2lE
Lecture Friday: Deep Learning and Neural Networks: the back propagation algorithm
Video of Lecture at https://youtu.be/hDTtA7PRRfI
Reading recommendations:
See lecture notes for week 40 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For neural networks we recommend Goodfellow et al chapters 6 and 7 and Bishop 5.1-5.4
For stochastic gradient descent we recommend Goodfellow et al chapter 8
Week 41 October 10-14¶
Lab Wednesday: Work on project 2
Lecture Thursday: Deep learning and Neural Networks, developing a code for Neural Networks
Video of Lecture at https://youtu.be/yzbxJI6LgL0
Lecture Friday: Developing a neural network code
Video of Lecture at https://youtu.be/CPj4mh7M9no
Reading recommendations:
See lecture notes for week 41 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. chapter 11 and 12 on practicalities and applications
Week 42 October 17-21¶
Lab Wednesday: Work on project 2
Lecture Thursday: Discussion of Neural Network calculations and tensorflow. Solving differential equations with neural networks
Video of Lecture at https://youtu.be/MdYT6uwOkT0
Lecture Friday: Convolutional Neural Networks and classification problems
Video of Lecture at https://youtu.be/3bDkrB-E7cU
Reading recommendations:
See lecture notes for week 42 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. See also chapter 11 and 12 on practicalities and applications
Week 43 October 24-28¶
Lab Wednesday: Work on project 2
Lecture Thursday: Recurrent Neural Networks
Video of Lecture at https://youtu.be/Hm6Ay5DS6o0
Lecture Friday: Recurrent Neural Networks and time series and principal component analysis (PCA)
Video of Lecture at https://youtu.be/JHfgQ77fpqs
Reading recommendations:
See lecture notes for week 43 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
For RNNs, see Goodfellow et al chapter 10 and discussions in chapter 11 and 12 on practicalities and applications
For PCA, see lecture notes chapter 11 and Geron’s text chapter 8
Week 44 October 31-November 4¶
Lab Wednesday: Work on project 2
Lecture Thursday: Decision trees, basic algorithms for classification and regression
Video of Lecture at https://youtu.be/7jexGH5SOOE
Lecture Friday: From trees to forests and ensemble methods
Video of Lecture at https://youtu.be/9QcU8VcXxRU
Reading recommendations:
See lecture notes for week 44 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Hastie et al sections 9.1 and 9.2. Geron’s text chapter 6 (Decision trees)
Week 45 November 7-11¶
Lab Wednesday: Work on project 2, project 3 available November 11.
Lecture Thursday: Ensemble methods, bagging and random forests, boosting.
Video of Lecture at https://youtu.be/mK48PfCxgYk
Lecture Friday: Adaptive boosting and gradient boosting, summary of decision trees and ensemble methods
Video of Lecture at https://youtu.be/v8eJBFeZKuI
Reading recommendations:
See lecture notes for week 45 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Hastie et al chapter 10 and Geron chapters 5 and 6
Week 46 November 14-18¶
Lab Wednesday: Work on project 3
Lecture Thursday: Support Vector machines
Video of Lecture at https://youtu.be/F3CkH-opbdY
Lecture Friday: Workshop on project 3
Video of Lecture at https://youtu.be/BbupEEvMXtg
Reading recommendations:
See lecture notes for week 46 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Hastie et al chapter 12
Week 47 November 21-25¶
Lab Wednesday: Work on project 3
Lecture Thursday: Dimensionality reduction and unsupervised learning: Principal Component analysis (PCA) and clustering
Video of Lecture https://youtu.be/VJIsEQM2lCI
Lecture Friday: PCA and clustering and Summary of Course
Video of Lecture https://youtu.be/olXksEL3P4A
Reading recommendations:
See lecture notes for week 47 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
Geron’s chapter 9 on PCA
Hastie et al Chapter 13 (sections 13.1-13.2 are the most relevant ones)
Excellent videos:
We recommend highly the video on PCA by Brunton and Kutz at http://www.databookuw.com/page-2/page-4/, see in particular the video of section 1.5.
And another good video on PCA is at https://www.youtube.com/watch?v=FgakZw6K1QQ
k-means clustering video at https://www.youtube.com/watch?v=4b5d3muPQmA