This site contains all material relevant for the course on Applied Data Analysis and Machine Learning FYS-STK3155/4155 at the University of Oslo, Norway.

Introduction

Probability theory and statistical methods play a central role in science. Nowadays we are surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one hour of video is uploaded to YouTube every second, amounting to years of content every day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have been sequenced by various labs and so on. This deluge of data calls for automated methods of data analysis, which is exactly what machine learning aims at providing.

Learning outcomes

This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will

Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;
Be capable of extending the acquired knowledge to other systems and cases;
Have an understanding of central algorithms used in data analysis and machine learning;
Understand linear methods for regression and classification, from ordinary least squares, via Lasso and Ridge to Logistic regression;
Learn about neural networks and deep learning methods for supervised and unsupervised learning. Emphasis on feed forward neural networks, convolutional and recurrent neural networks;
Learn about about decision trees, random forests, bagging and boosting methods;
Learn about support vector machines and kernel transformations;
Reduction of data sets, from PCA to clustering;
Autoencoders and Reinforcement Learning;
Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).

Prerequisites

Basic knowledge in programming and mathematics, with an emphasis on linear algebra. Knowledge of Python or/and C++ as programming languages is strongly recommended and experience with Jupiter notebook is recommended. Required courses are the equivalents to the University of Oslo mathematics courses MAT1100, MAT1110, MAT1120 and at least one of the corresponding computing and programming courses INF1000/INF1110 or MAT-INF1100/MAT-INF1100L/BIOS1100/KJM-INF1100. Most universities offer nowadays a basic programming course (often compulsory) where Python is the recurring programming language.

The course has two central parts

Statistical analysis and optimization of data
Machine learning

These topics will be scattered thorughout the course and may not necessarily be taught separately. Rather, we will often take an approach (during the lectures and project/exercise sessions) where say elements from statistical data analysis are mixed with specific Machine Learning algorithms.

Statistical analysis and optimization of data

The following topics will be covered

Basic concepts, expectation values, variance, covariance, correlation functions and errors;
Simpler models, binomial distribution, the Poisson distribution, simple and multivariate normal distributions;
Central elements of Bayesian statistics and modeling;
Gradient methods for data optimization,
Monte Carlo methods, Markov chains, Gibbs sampling and Metropolis-Hastings sampling;
Estimation of errors and resampling techniques such as the cross-validation, blocking, bootstrapping and jackknife methods;
Principal Component Analysis (PCA) and its mathematical foundation

Machine learning

The following topics will be covered

Linear Regression and Logistic Regression;
Neural networks and deep learning, including convolutional and recurrent neural networks
Decisions trees, Random Forests, Bagging and Boosting
Support vector machines
Bayesian linear and logistic regression
Boltzmann Machines
Unsupervised learning Dimensionality reduction, from PCA to cluster models

Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.

Computational aspects play a central role and you are expected to work on numerical examples and projects which illustrate the theory and varous algorithms discussed during the lectures. We recommend strongly to form small project groups of 2-3 participants.

Instructor information

***Name: *** Office: *** Office hours: In FS20, all office hours must be via computer or telephone. No matter what the modality of the course, instructors are required to hold the number of office hours specified by their college. Indicate the platform office hours will be held on and how appointments can be made outside office hours. Be explicit about communication channels and expectations, e.g. if you'll need 24 hours to reply to student emails. We suggest Zoom meetings and the use of a Zoom waiting room. ***

***Phone: (Specify if the office phone will be forwarded to the instructor's personal phone or if some other means of communication is to be used) ***E-mail:

Teaching Assistants FS20

Practicalities

Four lectures per week, Fall semester, 10 ECTS;
Four hours of laboratory sessions for work on computational projects;
Three projects which are graded and count 1/3 each of the final grade;
A selected number of weekly assignments;
The course is part of the CS Master of Science program, but is open to other bachelor and Master of Science students at the University of Oslo;
Grading scale: Grades are awarded on a scale from A to F, where A is the best grade and F is a fail;
The course is offered as a FYS-MAT4155 (Master of Science level) and a FYS-MAT3155 (senior undergraduate) course;
We use Piazza for course communication, a special link on how to register to Piazza can be found at the official University of Oslo page for the course. Slack is also used for course communication. The Slack link is machinelearninguio.slack.com ;

Required Technologies:

Specify any additional materials, including software, and hardware, and Internet connection, that students must purchase (or have access to) for the course. State if any hardware or software is required for assessment (e.g. a webcam). In selecting/preparing materials abide by the MSU Accessibility Policies. • If you use any special technology e.g. clickers, Tophat etc, please add instructions here. If there is a cost, this must be indicated in the Schedule of Courses. • Consider including pointers for which browsers and internet speeds are best for working with D2L: o Browser/mobile support for D2L:https://documentation.brightspace.com/EN/brightspace/requirements/all/browser_support.htm • Consider including a guide for internet speed: https://broadbandnow.com/guides/how-much-internet-speed-do-i-need. For most courses, 25 Mbps should work. If there is no mandatory video component, then students may be able to interact with the course with a slower connection. Most courses have a generic statement that says the course requires access to "high speed" internet. • Consider including information about how students can access internet connections, including those companies offering students free or reduced cost internet: https://remote.msu.edu/learning/internet.html • Remember that not all students have access to reliable internet. Specify your expectations if students encounter difficulties--including outages--during synchronous sessions, exams, etc.

Personal Hygiene:

Washing hands Personal Hygiene. All students must maintain proper hygiene and health practices, including: • frequently with soap and water or, if soap is unavailable, using hand sanitizer with at least 60% alcohol • Routinely cleaning and sanitizing living spaces and/or workspace • Using the bend of the elbow or shoulder to shield a cough or sneeze • Refraining from shaking hands

Adherence to Signage and Instructions. Students will (a) look for instructional signs posted by MSU or public health authorities, (b) observe instructions from MSU or public health authorities that are emailed to my “msu.edu” account, and (c) follow those instructions. Self-Monitoring. Students will self-monitor for flu-like symptoms (for example, cough, shortness of breath, difficulty breathing, fever, sore throat or loss of taste or smell). If a student experiences any flu-like symptoms, they will stay home and contact a health care provider to determine what steps should be taken. Exposure to COVID-19. If a student is exposed to someone who is ill or has tested positive for the COVID-19 virus, they will stay home, contact a health care provider and follow all public health recommendations. Compliance and reporting. Those who come to MSU facilities must commit to the personal responsibility necessary for us to remain as safe as possible, including following the specific guidelines outlined in this syllabus and provided by MSU more broadly (see below). There may be times when action will be necessary to reinforce expectations. If you do not wear appropriate face coverings (see MSU’s guidelines), do not wear your face covering appropriately (i.e., over your mouth and nose), or do not adhere to physical distancing guidelines (i.e., six feet apart), you will be asked to correct the situation or leave the facility. In addition, MSU will utilize the processes already in place to respond to any issues of noncompliance with standards established for the health and safety of our community. For classroom disruptions or issues, the responses and processes that have been used previously remain the first line of action. If necessary, the student conduct system will be the avenue used to adjudicate student disciplinary situations.

Additional information. See https://msu.edu/together-we-will/keeping-spartans-safe/ for details about these new policies and procedures and see https://ehs.msu.edu/_assets/docs/fact-sheets/cloth-face-covering-fact-sheet.pdf for more details about the cloth face coverings guidelines.***

Possible textbooks

Recommended textbooks:

Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer
Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly

General learning book on statistical analysis:

Christian Robert and George Casella, Monte Carlo Statistical Methods, Springer
Peter Hoff, A first course in Bayesian statistical models, Springer

General Machine Learning Books:

Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press
Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer
David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press

Links to relevant courses at the University of Oslo

The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.

STK2100 Machine learning and statistical methods for prediction and classification http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html.
IN3050 Introduction to Artificial Intelligence and Machine Learning https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html. Introductory course in machine learning and AI with an algorithmic approach.
STK-INF3000/4000 Selected Topics in Data Science http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html. The course provides insight into selected contemporary relevant topics within Data Science.
IN4080 Natural Language Processing https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html. Probabilistic and machine learning techniques applied to natural language processing.
STK-IN4300 Statistical learning methods in Data Science https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.
INF4490 Biologically Inspired Computing http://www.uio.no/studier/emner/matnat/ifi/INF4490/. An introduction to self-adapting methods also called artificial intelligence or machine learning.
IN-STK5000 Adaptive Methods for Data-Based Decision Making https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html. Methods for adaptive collection and processing of data based on machine learning techniques.
IN5400/INF5860 Machine Learning for Image Analysis https://www.uio.no/studier/emner/matnat/ifi/IN5400/. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
TEK5040 Deep learning for autonomous systems https://www.uio.no/studier/emner/matnat/its/TEK5040/. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.
STK4051 Computational Statistics https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html
STK4021 Applied Bayesian Analysis and Numerical Methods https://www.uio.no/studier/emner/matnat/math/STK4021/i

Course platforms/Structure: Include information about the online course management system you will use. For example: This course will be delivered online through the course management system and you will need your MSU NetID to login to the course from the D2L homepage (http://d2l.msu.edu).

In D2L, you will access online lessons, course materials, and additional resources. Activities may consist of readings, discussion forums, email, journaling, wikis, and other online activities. Add and remove listed activities as appropriate to your course. Include information regarding the synchronous or asynchronous elements of your online course.

This course will be delivered in a hybrid mode. The online components of the class will be [list activities here] and will be accessible here [add information such as the materials and lectures will be on D2L]. The in-person components of the class include [list activities here] and will be held at [include relevant location, time, dates and activities information here].

This course will be delivered face to face until Thanksgiving (November 26, 2020), after which all instruction will be fully online. When transitioning to online after Thanksgiving we will be doing [list activities here] using [list platform information here].

Technical Assistance: Information helpful for the use of required software, including D2L: Include information on how to navigate necessary software, expectations, instructions on how to access course material, and where to find technical and online course development assistance. For example: If you need technical assistance at any time during the course or to report a problem you can:

• Visit the MSU Help site at http://help.msu.edu • Visit the Desire2Learn Help Site at http://help.d2l.msu.edu • Call the MSU IT Service Desk at (517)432-6200, (844)678-6200, or e-mail at ithelp@msu.edu • Request assistance navigating and requesting instructional design help: https://tech.msu.edu/service-catalog/teaching/instructional-design-development/ Consider including pointers for which browsers and internet speeds are best for working with D2L: Browser/mobile support for D2L: https://documentation.brightspace.com/EN/brightspace/requirements/all/browser_support.htm Guide for internet speed: https://broadbandnow.com/guides/how-much-internet-speed-do-i-need. For most courses, 25 Mbps should work. If there is no mandatory video component, then students may be able to interact with the course with a slower connection. Most courses have a generic statement that says the course requires access to "high speed" internet.

README.md Unescape Escape

FYS-STK3155/4155 Applied Data Analysis and Machine Learning, http://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/index-eng.html