From 69450ca9cc19e3600f85fac7a75b9c18a54d06d0 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Sun, 22 Aug 2021 23:11:56 +0200 Subject: [PATCH] updating files --- README.md | 5 +-- doc/LectureNotes/_toc.yml | 74 ++++++++++++++++++----------------- doc/LectureNotes/intro.md | 42 +++++++++----------- doc/LectureNotes/teachers.md | 15 ++++++- doc/LectureNotes/textbooks.md | 18 ++++++--- 5 files changed, 85 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index 57a26112b..6d7f706ed 100644 --- a/README.md +++ b/README.md @@ -86,8 +86,6 @@ the theory and various algorithms discussed during the lectures. We recommend st ## Practicalities -It is not yet decided how the fall semester 2021 may be organized. We hope to be able to have in-person classes or a hybrid mode -with online and in-person lectures and on site or online laboratory sessions. Please follow updates on the covid-19 situation from Norwegian authorities and the University of Oslo. 1. Four lectures per week, Fall semester, 10 ECTS. The lectures will be recorded and linked to this site and the official University of Oslo website for the course; 2. Two hours of laboratory sessions for work on computational projects and exercises for each group. Due to social distancing, at most 15 participants can attend. There will also be fully digital laboratory sessions for those who cannot attend; @@ -221,7 +219,6 @@ The link here https://www.mn.uio.no/english/research/about/centre-focus/innovati -### Recommended prereading before the first lecture: Chapter 1-2 (linear algebra) and chapter 3 (statistics) of Goodfellow et al. and Bishop chapter 1 and chapter 2. These chapters give a relevant background to the basic mathematical and statistical foundations of the course. ## Weekly Schedule @@ -232,6 +229,8 @@ For the reading assignments we use the following abbreviations: - HTF: Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning - AG: Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow +Recommended prereading: Chapters 1-2 (linear algebra) and chapter 3 (statistics) of Goodfellow et al. and Bishop chapter 1 and chapter 2. These chapters give a relevant background to the basic mathematical and statistical foundations of the course. Parts of these chapters will be covered during the lectures the first three weeks. + ### Week 35 August 23-27 - Lab Wednesday: Introduction to software and repetition of Python Programming - Lecture Thursday: Introduction to the course, what is Machine Learning and introduction to Linear Regression diff --git a/doc/LectureNotes/_toc.yml b/doc/LectureNotes/_toc.yml index 823ac3ef8..55a74e737 100644 --- a/doc/LectureNotes/_toc.yml +++ b/doc/LectureNotes/_toc.yml @@ -1,36 +1,38 @@ -- file: intro -- part: About the course - chapters: - - file: schedule - - file: teachers - - file: textbooks -- part: Review of Statistics with Resampling Techniques and Linear Algebra - numbered: true - chapters: - - file: statistics.ipynb - - file: linalg.ipynb -- part: From Regression to Support Vector Machines - numbered: true - chapters: - - file: chapter1.ipynb - - file: chapter2.ipynb - - file: chapter3.ipynb - - file: chapter4.ipynb - - file: chapter5.ipynb -- part: Decision Trees, Ensemble Methods and Boosting - numbered: true - chapters: - - file: chapter6.ipynb - - file: chapter7.ipynb -- part: Dimensionality Reduction - numbered: true - chapters: - - file: chapter8.ipynb - - file: Clustering.ipynb -- part: Deep Learning Methods - numbered: true - chapters: - - file: chapter9.ipynb - - file: chapter10.ipynb - - file: chapter11.ipynb - - file: chapter12.ipynb +format: jb-book +root: intro +parts: + - caption: About the course + chapters: + - file: schedule + - file: teachers + - file: textbooks + - caption: Review of Statistics with Resampling Techniques and Linear Algebra + numbered: true + chapters: + - file: statistics.ipynb + - file: linalg.ipynb + - caption: From Regression to Support Vector Machines + numbered: true + chapters: + - file: chapter1.ipynb + - file: chapter2.ipynb + - file: chapter3.ipynb + - file: chapter4.ipynb + - file: chapter5.ipynb + - caption: Decision Trees, Ensemble Methods and Boosting + numbered: true + chapters: + - file: chapter6.ipynb + - file: chapter7.ipynb + - caption: Dimensionality Reduction + numbered: true + chapters: + - file: chapter8.ipynb + - file: Clustering.ipynb + - caption: Deep Learning Methods + numbered: true + chapters: + - file: chapter9.ipynb + - file: chapter10.ipynb + - file: chapter11.ipynb + - file: chapter12.ipynb diff --git a/doc/LectureNotes/intro.md b/doc/LectureNotes/intro.md index 811e85085..95698c010 100644 --- a/doc/LectureNotes/intro.md +++ b/doc/LectureNotes/intro.md @@ -1,10 +1,9 @@ -# Applied Data Analysis and Machine Learning - +# Applied Data Analysis and Machine Learning, FYS-STK3155/4155 at the University of Oslo, Norway ## Introduction -Probability theory and statistical methods play a central role in science. Nowadays we are -surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one +Probability theory and statistical methods play a central role in Science. Nowadays we are +surrounded by huge amounts of data. For example, there are more than one trillion web pages; more than one hour of video is uploaded to YouTube every second, amounting to years of content every day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have been sequenced by various labs and so on. This deluge of data calls for automated methods of data analysis, @@ -12,38 +11,29 @@ which is exactly what machine learning aims at providing. ## Learning outcomes -This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will +This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects and weekly exercises you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach with an emphasis on predictions and correaltions. However, we will try, where appropriate, to link our machine learning models with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will - Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning; - Be capable of extending the acquired knowledge to other systems and cases; - Have an understanding of central algorithms used in data analysis and machine learning; -- Understand linear methods for regression and classification, from ordinary least squares, via Lasso and Ridge to Logistic regression; +- Understand linear methods for regression and classification, from ordinary least squares, via Lasso and Ridge to Logistic regression and Kernel regression; - Learn about neural networks and deep learning methods for supervised and unsupervised learning. Emphasis on feed forward neural networks, convolutional and recurrent neural networks; - Learn about about decision trees, random forests, bagging and boosting methods; - Learn about support vector machines and kernel transformations; -- Reduction of data sets, from PCA to clustering; +- Reduction of data sets and unsupervised learning, from PCA to clustering; - Autoencoders and Reinforcement Learning; - Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later). -## Prerequisites - -Basic knowledge in programming and mathematics, with an emphasis on -linear algebra. Knowledge of Python or/and C++ as programming -languages is strongly recommended and experience with Jupiter notebook -is recommended. Required courses are the equivalents to the University -of Oslo mathematics courses MAT1100, MAT1110, MAT1120 and at least one -of the corresponding computing and programming courses INF1000/INF1110 -or MAT-INF1100/MAT-INF1100L/BIOS1100/KJM-INF1100. Most universities -offer nowadays a basic programming course (often compulsory) where -Python is the recurring programming language. +## Prerequisites and background +Basic knowledge in programming and mathematics, with an emphasis on linear algebra. Knowledge of Python or/and C++ as programming languages is strongly recommended and experience with Jupyter notebooks is recommended. Required courses are the equivalents to the University of Oslo mathematics courses MAT1100, MAT1110, MAT1120 and at least one of the corresponding computing and programming courses INF1000/INF1110 or MAT-INF1100/MAT-INF1100L/BIOS1100/KJM-INF1100. Most universities offer nowadays a basic programming course (often compulsory) where Python is the recurring programming language. +We recommend also refreshing your knowledge on Statistics and Probability theory. The lecture notes at https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html offer a review of Statistics and Probability theory. ## The course has two central parts 1. Statistical analysis and optimization of data 2. Machine learning -These topics will be scattered thorughout the course and may not necessarily be taught separately. Rather, we will often take an approach (during the lectures and project/exercise sessions) where say elements from statistical data analysis are mixed with specific Machine Learning algorithms. ### Statistical analysis and optimization of data @@ -65,24 +55,27 @@ The following topics will be covered: - Support vector machines - Bayesian linear and logistic regression - Boltzmann Machines -- Unsupervised learning Dimensionality reduction, from PCA to cluster models +- Unsupervised learning Dimensionality reduction, PCA, k-means and clustering +- Autoenconders Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics. Computational aspects play a central role and you are expected to work on numerical examples and projects which illustrate -the theory and varous algorithms discussed during the lectures. We recommend strongly to form small project groups of 2-3 participants, if possible. +the theory and various algorithms discussed during the lectures. We recommend strongly to form small project groups of 2-3 participants, if possible. + + ## Required Technologies -Course participants are expected to have their own laptops/PCs. We use _Git_ as version control software and the usage of providers like _GitHub_, _GitLab_ or similar are strongly recommended. +Course participants are expected to have their own laptops/PCs. We use _Git_ as version control software and the usage of providers like _GitHub_, _GitLab_ or similar are strongly recommended. If you are not familiar with Git as version control software, the following video may be of interest, see https://www.youtube.com/watch?v=RGOj5yH7evk&ab_channel=freeCodeCamp.org We will make extensive use of Python as programming language and its myriad of available libraries. You will find Jupyter notebooks invaluable in your work. You can run _R_ codes in the Jupyter/IPython notebooks, with the immediate benefit of visualizing your data. You can also use compiled languages like C++, -Rust, Julia, Fortran etc if you prefer. The focus in these lectures will be mainly +Rust, Julia, Fortran etc if you prefer. The focus in these lectures will be on Python. @@ -143,3 +136,6 @@ Here we list several useful Python libraries we strongly recommend (if you use a * _Keras_:https://keras.io/ is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano * And many more such as _pytorch_:https://pytorch.org/, _Theano_:https://pypi.org/project/Theano/ etc + + + diff --git a/doc/LectureNotes/teachers.md b/doc/LectureNotes/teachers.md index 4b934a68b..a1bb4f199 100644 --- a/doc/LectureNotes/teachers.md +++ b/doc/LectureNotes/teachers.md @@ -6,7 +6,16 @@ * _Email_: morten.hjorth-jensen@fys.uio.no * _Phone_: +47-48257387 * _Office_: Department of Physics, University of Oslo, Eastern wing, room FØ470 -* _Office hours_: *Anytime*! In Fall Semester 2020 (FS20), as a rule of thumb office hours are planned via computer or telephone. Individual or group office hours will be performed via zoom. Feel free to send an email for planning. In person meetings may also be possible if allowed by the University of Oslo's COVID-19 instructions (see below for links). +* _Office hours_: *Anytime*! Feel free to send an email for planning. Both in person meetings or digital meetings are possible. + +## Teaching Assistants Fall semester 2021 +* Øyvind Sigmundson Schøyen, oyvinssc@student.matnat.uio.no +* Stian Bilek, stian.bilek@fys.uio.no +* Linus Ekstrøm, linueks@gmail.com, linus.ekstrom@fys.uio.no +* Nicholas Karlsen, nicholaskarlsen1102@gmail.com, nicholas.karlsen@fys.uio.no +* Bendik Steinsvåg Dalen, b.s.dalen@fys.uio.no +* Philip Karim Sørli Niane, p.k.s.niane@fys.uio.no + ## Grading @@ -21,3 +30,7 @@ The final number of points is based on the average of all projects (including ev * 40-45 points: E * 0-39 points: F-failed + + + + diff --git a/doc/LectureNotes/textbooks.md b/doc/LectureNotes/textbooks.md index 92d980b58..cabdaf8ea 100644 --- a/doc/LectureNotes/textbooks.md +++ b/doc/LectureNotes/textbooks.md @@ -1,12 +1,15 @@ -# Textbooks +## Textbooks _Recommended textbooks_: -- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, https://www.springer.com/gp/book/9780387310732. This is the main textbook and this course covers chapters 1-7, 11 and 12. -- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer, https://www.springer.com/gp/book/9780387848570. This is a well-known text and serves as additional text. -- Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly, https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/. This text is very useful since it contains many code examples. +The lecture notes are collected as a jupyter-book at https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html. In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. +- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, https://www.springer.com/gp/book/9780387310732. This is the main textbook and this course covers chapters 1-7, 11 and 12. You can download for free the textbook in PDF format at https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf +- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. The different chapters are available for free at https://www.deeplearningbook.org/. Chapters 2-14 are highly recommended. The lectures follow to a larg extent this text. +The weekly plans will include reading suggestions from these two textbooks. +_Additional textbooks_: +- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer, https://www.springer.com/gp/book/9780387848570. This is a well-known text and serves as additional literature. +- Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly, https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/. This text is very useful since it contains many code examples and hands-on applications of all algorithms discussed in this course. -The books by Bishop and Hastie et al. can be downloaded for free if you access the university library via an IP number of your home university. @@ -16,10 +19,11 @@ _General learning book on statistical analysis_: _General Machine Learning Books_: - Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press -- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer - David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press - David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press + + ## Links to relevant courses at the University of Oslo The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO. @@ -36,3 +40,5 @@ The link here https://www.mn.uio.no/english/research/about/centre-focus/innovati - _STK4021 Applied Bayesian Analysis and Numerical Methods_ https://www.uio.no/studier/emner/matnat/math/STK4021/ + +