+The first exercise here is of a mere technical art. We want you to have
+
+
+
git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo GitHub facilities.
+
Install various Python packages
+
+
+We will make extensive use of Python as programming language and its
+myriad of available libraries. You will find
+IPython/Jupyter notebooks invaluable in your work. You can run R
+codes in the Jupyter/IPython notebooks, with the immediate benefit of
+visualizing your data. You can also use compiled languages like C++,
+Rust, Fortran etc if you prefer. The focus in these lectures will be
+on Python.
+
+
+If you have Python installed (we recommend Python3) and you feel
+pretty familiar with installing different packages, we recommend that
+you install the following Python packages via pip as
+
+
+For OSX users we recommend, after having installed Xcode, to
+install brew. Brew allows for a seamless installation of additional
+software via for example
+
+
+
brew install python3
+
+
+For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
+you can use pip as well and simply install Python as
+
+
+
sudo apt-get install python3 (or python for pyhton2.7)
+
+
+If you don't want to perform these operations separately and venture
+into the hassle of exploring how to set up dependencies and paths, we
+recommend two widely used distrubutions which set up all relevant
+dependencies for Python, namely
+
+
+
+which is an open source
+distribution of the Python and R programming languages for large-scale
+data processing, predictive analytics, and scientific computing, that
+aims to simplify package management and deployment. Package versions
+are managed by the package management system conda.
+
+
+
+is a Python
+distribution for scientific and analytic computing distribution and
+analysis environment, available for free and under a commercial
+license.
+
+
+We recommend using Anaconda.
+
+
Exercise 2
+
+
+We will generate our own dataset for a function \( y(x) \) where \( x \in [0,1] \) and defined by random numbers computed with the uniform distribution. The function \( y \) is a quadratic polynomial in \( x \) with added stochastic noise according to the normal distribution \( \cal {N}(0,1) \).
+The following simple Python instructions define our \( x \) and \( y \) values (with 100 data points).
+
+
+
+
x = np.random.rand(100,1)
+y =5*x*x+0.1*np.random.randn(100,1)
+
+
+
Write your own code (following the examples under the regression slides) for computing the parametrization of the data set fitting a second-order polynomial.
+
Use thereafter scikit-learn (see again the examples in the regression slides) and compare with your own code.
+
Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
+
+
+$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
+\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
+$$
+
+and the \( R^2 \) score function.
+If \( \tilde{\hat{y}}_i \) is the predicted value of the \( i-th \) sample and \( y_i \) is the corresponding true value, then the score \( R^2 \) is defined as
+$$
+R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
+$$
+
+where we have defined the mean value of \( \hat{y} \) as
+$$
+\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
+$$
+
+You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
+Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
+
+
Exercise 3, mean values and variances in linear regression
+The assumption we have made is
+that there exists a function \( f(\boldsymbol{x}) \) and a normal distributed error \( \boldsymbol{\varepsilon}\sim \mathcal{N}(0, \sigma^2) \)
+which describes our data
+$$
+\boldsymbol{y} = f(\boldsymbol{x})+\boldsymbol{\varepsilon}
+$$
+
+
+We then approximate this function with our model from the solution of the linear regression equations (ordinary least squares OLS), that is our
+function \( f \) is approximated by \( \boldsymbol{\tilde{y}} \) where we minimized \( (\boldsymbol{y}-\boldsymbol{\tilde{y}})^2 \), with
+$$
+\boldsymbol{\tilde{y}} = \boldsymbol{X}\boldsymbol{\beta}.
+$$
+
+The matrix \( \boldsymbol{X} \) is the so-called design matrix.
+
+
+Show that the expectation value of \( \boldsymbol{y} \) for a given element \( i \)
+$$
+\begin{align*}
+\mathbb{E}(y_i) & =\mathbf{X}_{i, \ast} \, \beta,
+\end{align*}
+$$
+
+and that
+its variance is
+$$
+\begin{align*} \mbox{Var}(y_i) & = \sigma^2.
+\end{align*}
+$$
+
+Hence, \( y_i \sim \mathcal{N}( \mathbf{X}_{i, \ast} \, \boldsymbol{\beta}, \sigma^2) \), that is \( \boldsymbol{y} \) follows a normal distribution with
+mean value \( \boldsymbol{X}\boldsymbol{\beta} \) and variance \( \sigma^2 \).
+
+
+With the OLS expressions for the parameters \( \boldsymbol{\beta} \) show that
+$$
+\mathbb{E}(\boldsymbol{\beta}) = \boldsymbol{\beta}.
+$$
+
+This means that the estimator of the regression parameters is unbiased.
+
+
+Show finally that the variance of \( \boldsymbol{\beta} \) is
+$$
+\begin{eqnarray*}
+\mbox{Var}(\boldsymbol{\beta}) & = & \sigma^2 \, (\mathbf{X}^{T} \mathbf{X})^{-1}.
+\end{eqnarray*}
+$$
+
+
+The first exercise here is of a mere technical art. We want you to have
+
+
+
git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo GitHub facilities.
+
Install various Python packages
+
+
+We will make extensive use of Python as programming language and its
+myriad of available libraries. You will find
+IPython/Jupyter notebooks invaluable in your work. You can run R
+codes in the Jupyter/IPython notebooks, with the immediate benefit of
+visualizing your data. You can also use compiled languages like C++,
+Rust, Fortran etc if you prefer. The focus in these lectures will be
+on Python.
+
+
+If you have Python installed (we recommend Python3) and you feel
+pretty familiar with installing different packages, we recommend that
+you install the following Python packages via pip as
+
+
+For OSX users we recommend, after having installed Xcode, to
+install brew. Brew allows for a seamless installation of additional
+software via for example
+
+
+
brew install python3
+
+
+For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
+you can use pip as well and simply install Python as
+
+
+
sudo apt-get install python3 (or python for pyhton2.7)
+
+
+If you don't want to perform these operations separately and venture
+into the hassle of exploring how to set up dependencies and paths, we
+recommend two widely used distrubutions which set up all relevant
+dependencies for Python, namely
+
+
+
+which is an open source
+distribution of the Python and R programming languages for large-scale
+data processing, predictive analytics, and scientific computing, that
+aims to simplify package management and deployment. Package versions
+are managed by the package management system conda.
+
+
+
+is a Python
+distribution for scientific and analytic computing distribution and
+analysis environment, available for free and under a commercial
+license.
+
+
+We recommend using Anaconda.
+
+
Exercise 2
+
+
+We will generate our own dataset for a function \( y(x) \) where \( x \in [0,1] \) and defined by random numbers computed with the uniform distribution. The function \( y \) is a quadratic polynomial in \( x \) with added stochastic noise according to the normal distribution \( \cal {N}(0,1) \).
+The following simple Python instructions define our \( x \) and \( y \) values (with 100 data points).
+
+
+
+
x = np.random.rand(100,1)
+y =5*x*x+0.1*np.random.randn(100,1)
+
+
+
Write your own code (following the examples under the regression slides) for computing the parametrization of the data set fitting a second-order polynomial.
+
Use thereafter scikit-learn (see again the examples in the regression slides) and compare with your own code.
+
Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
+
+
+$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
+\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
+$$
+
+and the \( R^2 \) score function.
+If \( \tilde{\hat{y}}_i \) is the predicted value of the \( i-th \) sample and \( y_i \) is the corresponding true value, then the score \( R^2 \) is defined as
+$$
+R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
+$$
+
+where we have defined the mean value of \( \hat{y} \) as
+$$
+\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
+$$
+
+You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
+Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
+
+
Exercise 3, mean values and variances in linear regression
+The assumption we have made is
+that there exists a function \( f(\boldsymbol{x}) \) and a normal distributed error \( \boldsymbol{\varepsilon}\sim \mathcal{N}(0, \sigma^2) \)
+which describes our data
+$$
+\boldsymbol{y} = f(\boldsymbol{x})+\boldsymbol{\varepsilon}
+$$
+
+
+We then approximate this function with our model from the solution of the linear regression equations (ordinary least squares OLS), that is our
+function \( f \) is approximated by \( \boldsymbol{\tilde{y}} \) where we minimized \( (\boldsymbol{y}-\boldsymbol{\tilde{y}})^2 \), with
+$$
+\boldsymbol{\tilde{y}} = \boldsymbol{X}\boldsymbol{\beta}.
+$$
+
+The matrix \( \boldsymbol{X} \) is the so-called design matrix.
+
+
+Show that the expectation value of \( \boldsymbol{y} \) for a given element \( i \)
+$$
+\begin{align*}
+\mathbb{E}(y_i) & =\mathbf{X}_{i, \ast} \, \beta,
+\end{align*}
+$$
+
+and that
+its variance is
+$$
+\begin{align*} \mbox{Var}(y_i) & = \sigma^2.
+\end{align*}
+$$
+
+Hence, \( y_i \sim \mathcal{N}( \mathbf{X}_{i, \ast} \, \boldsymbol{\beta}, \sigma^2) \), that is \( \boldsymbol{y} \) follows a normal distribution with
+mean value \( \boldsymbol{X}\boldsymbol{\beta} \) and variance \( \sigma^2 \).
+
+
+With the OLS expressions for the parameters \( \boldsymbol{\beta} \) show that
+$$
+\mathbb{E}(\boldsymbol{\beta}) = \boldsymbol{\beta}.
+$$
+
+This means that the estimator of the regression parameters is unbiased.
+
+
+Show finally that the variance of \( \boldsymbol{\beta} \) is
+$$
+\begin{eqnarray*}
+\mbox{Var}(\boldsymbol{\beta}) & = & \sigma^2 \, (\mathbf{X}^{T} \mathbf{X})^{-1}.
+\end{eqnarray*}
+$$
+
+
+The first exercise here is of a mere technical art. We want you to have
+
+
+
git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo GitHub facilities.
+
Install various Python packages
+
+
+We will make extensive use of Python as programming language and its
+myriad of available libraries. You will find
+IPython/Jupyter notebooks invaluable in your work. You can run R
+codes in the Jupyter/IPython notebooks, with the immediate benefit of
+visualizing your data. You can also use compiled languages like C++,
+Rust, Fortran etc if you prefer. The focus in these lectures will be
+on Python.
+
+
+If you have Python installed (we recommend Python3) and you feel
+pretty familiar with installing different packages, we recommend that
+you install the following Python packages via pip as
+
+
+For OSX users we recommend, after having installed Xcode, to
+install brew. Brew allows for a seamless installation of additional
+software via for example
+
+
+
brew install python3
+
+
+For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
+you can use pip as well and simply install Python as
+
+
+
sudo apt-get install python3 (or python for pyhton2.7)
+
+
+If you don't want to perform these operations separately and venture
+into the hassle of exploring how to set up dependencies and paths, we
+recommend two widely used distrubutions which set up all relevant
+dependencies for Python, namely
+
+
+
+which is an open source
+distribution of the Python and R programming languages for large-scale
+data processing, predictive analytics, and scientific computing, that
+aims to simplify package management and deployment. Package versions
+are managed by the package management system conda.
+
+
+
+is a Python
+distribution for scientific and analytic computing distribution and
+analysis environment, available for free and under a commercial
+license.
+
+
+We recommend using Anaconda.
+
+
Exercise 2
+
+
+We will generate our own dataset for a function \( y(x) \) where \( x \in [0,1] \) and defined by random numbers computed with the uniform distribution. The function \( y \) is a quadratic polynomial in \( x \) with added stochastic noise according to the normal distribution \( \cal {N}(0,1) \).
+The following simple Python instructions define our \( x \) and \( y \) values (with 100 data points).
+
+
+
+
x = np.random.rand(100,1)
+y =5*x*x+0.1*np.random.randn(100,1)
+
+
+
Write your own code (following the examples under the regression slides) for computing the parametrization of the data set fitting a second-order polynomial.
+
Use thereafter scikit-learn (see again the examples in the regression slides) and compare with your own code.
+
Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
+
+
+$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
+\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
+$$
+
+and the \( R^2 \) score function.
+If \( \tilde{\hat{y}}_i \) is the predicted value of the \( i-th \) sample and \( y_i \) is the corresponding true value, then the score \( R^2 \) is defined as
+$$
+R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
+$$
+
+where we have defined the mean value of \( \hat{y} \) as
+$$
+\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
+$$
+
+You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
+Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
+
+
Exercise 3, mean values and variances in linear regression
+The assumption we have made is
+that there exists a function \( f(\boldsymbol{x}) \) and a normal distributed error \( \boldsymbol{\varepsilon}\sim \mathcal{N}(0, \sigma^2) \)
+which describes our data
+$$
+\boldsymbol{y} = f(\boldsymbol{x})+\boldsymbol{\varepsilon}
+$$
+
+
+We then approximate this function with our model from the solution of the linear regression equations (ordinary least squares OLS), that is our
+function \( f \) is approximated by \( \boldsymbol{\tilde{y}} \) where we minimized \( (\boldsymbol{y}-\boldsymbol{\tilde{y}})^2 \), with
+$$
+\boldsymbol{\tilde{y}} = \boldsymbol{X}\boldsymbol{\beta}.
+$$
+
+The matrix \( \boldsymbol{X} \) is the so-called design matrix.
+
+
+Show that the expectation value of \( \boldsymbol{y} \) for a given element \( i \)
+$$
+\begin{align*}
+\mathbb{E}(y_i) & =\mathbf{X}_{i, \ast} \, \beta,
+\end{align*}
+$$
+
+and that
+its variance is
+$$
+\begin{align*} \mbox{Var}(y_i) & = \sigma^2.
+\end{align*}
+$$
+
+Hence, \( y_i \sim \mathcal{N}( \mathbf{X}_{i, \ast} \, \boldsymbol{\beta}, \sigma^2) \), that is \( \boldsymbol{y} \) follows a normal distribution with
+mean value \( \boldsymbol{X}\boldsymbol{\beta} \) and variance \( \sigma^2 \).
+
+
+With the OLS expressions for the parameters \( \boldsymbol{\beta} \) show that
+$$
+\mathbb{E}(\boldsymbol{\beta}) = \boldsymbol{\beta}.
+$$
+
+This means that the estimator of the regression parameters is unbiased.
+
+
+Show finally that the variance of \( \boldsymbol{\beta} \) is
+$$
+\begin{eqnarray*}
+\mbox{Var}(\boldsymbol{\beta}) & = & \sigma^2 \, (\mathbf{X}^{T} \mathbf{X})^{-1}.
+\end{eqnarray*}
+$$
+
+
+
+
+
+
+
+
diff --git a/doc/Projects/2019/hw1/ipynb/ipynb-hw1-src.tar.gz b/doc/Projects/2019/hw1/ipynb/ipynb-hw1-src.tar.gz
new file mode 100644
index 000000000..1f556cb3d
Binary files /dev/null and b/doc/Projects/2019/hw1/ipynb/ipynb-hw1-src.tar.gz differ
diff --git a/doc/Projects/2019/hw1/pdf/hw1.pdf b/doc/Projects/2019/hw1/pdf/hw1.pdf
new file mode 100644
index 000000000..275a618ae
Binary files /dev/null and b/doc/Projects/2019/hw1/pdf/hw1.pdf differ
diff --git a/doc/Projects/2019/hw1/pdf/hw1.tex b/doc/Projects/2019/hw1/pdf/hw1.tex
new file mode 100644
index 000000000..321c8139c
--- /dev/null
+++ b/doc/Projects/2019/hw1/pdf/hw1.tex
@@ -0,0 +1,298 @@
+%%
+%% Automatically generated file from DocOnce source
+%% (https://github.com/hplgit/doconce/)
+%%
+%%
+
+
+%-------------------- begin preamble ----------------------
+
+\documentclass[%
+oneside, % oneside: electronic viewing, twoside: printing
+final, % draft: marks overfull hboxes, figures with paths
+10pt]{article}
+
+\listfiles % print all files needed to compile this document
+
+\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
+\usepackage[table]{xcolor}
+\usepackage{bm,ltablex,microtype}
+
+\usepackage[pdftex]{graphicx}
+
+\usepackage{fancyvrb} % packages needed for verbatim environments
+
+\usepackage[T1]{fontenc}
+%\usepackage[latin1]{inputenc}
+\usepackage{ucs}
+\usepackage[utf8x]{inputenc}
+
+\usepackage{lmodern} % Latin Modern fonts derived from Computer Modern
+
+% Hyperlinks in PDF:
+\definecolor{linkcolor}{rgb}{0,0,0.4}
+\usepackage{hyperref}
+\hypersetup{
+ breaklinks=true,
+ colorlinks=true,
+ linkcolor=linkcolor,
+ urlcolor=linkcolor,
+ citecolor=black,
+ filecolor=black,
+ %filecolor=blue,
+ pdfmenubar=true,
+ pdftoolbar=true,
+ bookmarksdepth=3 % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
+ }
+%\hyperbaseurl{} % hyperlinks are relative to this root
+
+\setcounter{tocdepth}{2} % levels in table of contents
+
+% --- fancyhdr package for fancy headers ---
+\usepackage{fancyhdr}
+\fancyhf{} % sets both header and footer to nothing
+\renewcommand{\headrulewidth}{0pt}
+\fancyfoot[LE,RO]{\thepage}
+% Ensure copyright on titlepage (article style) and chapter pages (book style)
+\fancypagestyle{plain}{
+ \fancyhf{}
+ \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
+% \renewcommand{\footrulewidth}{0mm}
+ \renewcommand{\headrulewidth}{0mm}
+}
+% Ensure copyright on titlepages with \thispagestyle{empty}
+\fancypagestyle{empty}{
+ \fancyhf{}
+ \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
+ \renewcommand{\footrulewidth}{0mm}
+ \renewcommand{\headrulewidth}{0mm}
+}
+
+\pagestyle{fancy}
+
+
+% prevent orhpans and widows
+\clubpenalty = 10000
+\widowpenalty = 10000
+
+% --- end of standard preamble for documents ---
+
+
+% insert custom LaTeX commands...
+
+\raggedbottom
+\makeindex
+\usepackage[totoc]{idxlayout} % for index in the toc
+\usepackage[nottoc]{tocbibind} % for references/bibliography in the toc
+
+%-------------------- end preamble ----------------------
+
+\begin{document}
+
+% matching end for #ifdef PREAMBLE
+
+\newcommand{\exercisesection}[1]{\subsection*{#1}}
+
+
+% ------------------- main content ----------------------
+
+
+
+% ----------------- title -------------------------
+
+\thispagestyle{empty}
+
+\begin{center}
+{\LARGE\bf
+\begin{spacing}{1.25}
+Homework 1 Fall Semester 2019
+\end{spacing}
+}
+\end{center}
+
+% ----------------- author(s) -------------------------
+
+\begin{center}
+{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html}}{Data Analysis and Machine Learning FYS-STK3155/FYS4155}}
+\end{center}
+
+ \begin{center}
+% List of all institutions:
+\centerline{{\small Department of Physics, University of Oslo, Norway}}
+\end{center}
+
+% ----------------- end author(s) -------------------------
+
+% --- begin date ---
+\begin{center}
+Aug 20, 2019
+\end{center}
+% --- end date ---
+
+\vspace{1cm}
+
+
+\subsection*{Exercise 1}
+
+The first exercise here is of a mere technical art. We want you to have
+\begin{itemize}
+\item git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo \href{{https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html}}{GitHub facilities}.
+
+\item Install various Python packages
+\end{itemize}
+
+\noindent
+We will make extensive use of Python as programming language and its
+myriad of available libraries. You will find
+IPython/Jupyter notebooks invaluable in your work. You can run \textbf{R}
+codes in the Jupyter/IPython notebooks, with the immediate benefit of
+visualizing your data. You can also use compiled languages like C++,
+Rust, Fortran etc if you prefer. The focus in these lectures will be
+on Python.
+
+If you have Python installed (we recommend Python3) and you feel
+pretty familiar with installing different packages, we recommend that
+you install the following Python packages via \textbf{pip} as
+
+\begin{enumerate}
+\item pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow
+\end{enumerate}
+
+\noindent
+For \textbf{Tensorflow}, we recommend following the instructions in the text of
+\href{{http://shop.oreilly.com/product/0636920052289.do}}{Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly}
+
+We will come back to \textbf{tensorflow} later.
+
+For Python3, replace \textbf{pip} with \textbf{pip3}.
+
+For OSX users we recommend, after having installed Xcode, to
+install \textbf{brew}. Brew allows for a seamless installation of additional
+software via for example
+
+\begin{enumerate}
+\item brew install python3
+\end{enumerate}
+
+\noindent
+For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
+you can use \textbf{pip} as well and simply install Python as
+
+\begin{enumerate}
+\item sudo apt-get install python3 (or python for pyhton2.7)
+\end{enumerate}
+
+\noindent
+If you don't want to perform these operations separately and venture
+into the hassle of exploring how to set up dependencies and paths, we
+recommend two widely used distrubutions which set up all relevant
+dependencies for Python, namely
+
+\begin{itemize}
+\item \href{{https://docs.anaconda.com/}}{Anaconda},
+\end{itemize}
+
+\noindent
+which is an open source
+distribution of the Python and R programming languages for large-scale
+data processing, predictive analytics, and scientific computing, that
+aims to simplify package management and deployment. Package versions
+are managed by the package management system \textbf{conda}.
+
+\begin{itemize}
+\item \href{{https://www.enthought.com/product/canopy/}}{Enthought canopy}
+\end{itemize}
+
+\noindent
+is a Python
+distribution for scientific and analytic computing distribution and
+analysis environment, available for free and under a commercial
+license.
+
+We recommend using \textbf{Anaconda}.
+
+\subsection*{Exercise 2}
+
+We will generate our own dataset for a function $y(x)$ where $x \in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\cal {N}(0,1)$.
+The following simple Python instructions define our $x$ and $y$ values (with 100 data points).
+\begin{verbatim}
+x = np.random.rand(100,1)
+y = 5*x*x+0.1*np.random.randn(100,1)
+\end{verbatim}
+
+\begin{enumerate}
+\item Write your own code (following the examples under the \href{{https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html}}{regression slides}) for computing the parametrization of the data set fitting a second-order polynomial.
+
+\item Use thereafter \textbf{scikit-learn} (see again the examples in the regression slides) and compare with your own code.
+
+\item Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
+\end{enumerate}
+
+\noindent
+\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
+\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
+\]
+and the $R^2$ score function.
+If $\tilde{\hat{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as
+\[
+R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
+\]
+where we have defined the mean value of $\hat{y}$ as
+\[
+\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
+\]
+You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
+Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
+
+
+
+
+\subsection*{Exercise 3, mean values and variances in linear regression}
+
+
+This exercise deals with various mean values ad variances in linear regression method (here it may be useful to look up chapter 3, equation (3.8) of \href{{https://www.springer.com/gp/book/9780387848570}}{Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer}).
+
+The assumption we have made is
+that there exists a function $f(\bm{x})$ and a normal distributed error $\bm{\varepsilon}\sim \mathcal{N}(0, \sigma^2)$
+which describes our data
+\[
+\bm{y} = f(\bm{x})+\bm{\varepsilon}
+\]
+
+We then approximate this function with our model from the solution of the linear regression equations (ordinary least squares OLS), that is our
+function $f$ is approximated by $\bm{\tilde{y}}$ where we minimized $(\bm{y}-\bm{\tilde{y}})^2$, with
+\[
+\bm{\tilde{y}} = \bm{X}\bm{\beta}.
+\]
+The matrix $\bm{X}$ is the so-called design matrix.
+
+
+Show that the expectation value of $\bm{y}$ for a given element $i$
+\begin{align*}
+\mathbb{E}(y_i) & =\mathbf{X}_{i, \ast} \, \beta,
+\end{align*}
+and that
+its variance is
+\begin{align*} \mbox{Var}(y_i) & = \sigma^2.
+\end{align*}
+Hence, $y_i \sim \mathcal{N}( \mathbf{X}_{i, \ast} \, \bm{\beta}, \sigma^2)$, that is $\bm{y}$ follows a normal distribution with
+mean value $\bm{X}\bm{\beta}$ and variance $\sigma^2$.
+
+
+With the OLS expressions for the parameters $\bm{\beta}$ show that
+\[
+\mathbb{E}(\bm{\beta}) = \bm{\beta}.
+\]
+This means that the estimator of the regression parameters is unbiased.
+
+Show finally that the variance of $\bm{\beta}$ is
+\begin{eqnarray*}
+\mbox{Var}(\bm{\beta}) & = & \sigma^2 \, (\mathbf{X}^{T} \mathbf{X})^{-1}.
+\end{eqnarray*}
+
+
+
+% ------------------- end of main content ---------------
+
+\end{document}
+
diff --git a/doc/Projects/2019/hw1/pdf/hw1.tex~ b/doc/Projects/2019/hw1/pdf/hw1.tex~
new file mode 100644
index 000000000..8839a47df
--- /dev/null
+++ b/doc/Projects/2019/hw1/pdf/hw1.tex~
@@ -0,0 +1,298 @@
+%%
+%% Automatically generated file from DocOnce source
+%% (https://github.com/hplgit/doconce/)
+%%
+%%
+
+
+%-------------------- begin preamble ----------------------
+
+\documentclass[%
+oneside, % oneside: electronic viewing, twoside: printing
+final, % draft: marks overfull hboxes, figures with paths
+10pt]{article}
+
+\listfiles % print all files needed to compile this document
+
+\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
+\usepackage[table]{xcolor}
+\usepackage{bm,ltablex,microtype}
+
+\usepackage[pdftex]{graphicx}
+
+\usepackage{fancyvrb} % packages needed for verbatim environments
+
+\usepackage[T1]{fontenc}
+%\usepackage[latin1]{inputenc}
+\usepackage{ucs}
+\usepackage[utf8x]{inputenc}
+
+\usepackage{lmodern} % Latin Modern fonts derived from Computer Modern
+
+% Hyperlinks in PDF:
+\definecolor{linkcolor}{rgb}{0,0,0.4}
+\usepackage{hyperref}
+\hypersetup{
+ breaklinks=true,
+ colorlinks=true,
+ linkcolor=linkcolor,
+ urlcolor=linkcolor,
+ citecolor=black,
+ filecolor=black,
+ %filecolor=blue,
+ pdfmenubar=true,
+ pdftoolbar=true,
+ bookmarksdepth=3 % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
+ }
+%\hyperbaseurl{} % hyperlinks are relative to this root
+
+\setcounter{tocdepth}{2} % levels in table of contents
+
+% --- fancyhdr package for fancy headers ---
+\usepackage{fancyhdr}
+\fancyhf{} % sets both header and footer to nothing
+\renewcommand{\headrulewidth}{0pt}
+\fancyfoot[LE,RO]{\thepage}
+% Ensure copyright on titlepage (article style) and chapter pages (book style)
+\fancypagestyle{plain}{
+ \fancyhf{}
+ \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
+% \renewcommand{\footrulewidth}{0mm}
+ \renewcommand{\headrulewidth}{0mm}
+}
+% Ensure copyright on titlepages with \thispagestyle{empty}
+\fancypagestyle{empty}{
+ \fancyhf{}
+ \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
+ \renewcommand{\footrulewidth}{0mm}
+ \renewcommand{\headrulewidth}{0mm}
+}
+
+\pagestyle{fancy}
+
+
+% prevent orhpans and widows
+\clubpenalty = 10000
+\widowpenalty = 10000
+
+% --- end of standard preamble for documents ---
+
+
+% insert custom LaTeX commands...
+
+\raggedbottom
+\makeindex
+\usepackage[totoc]{idxlayout} % for index in the toc
+\usepackage[nottoc]{tocbibind} % for references/bibliography in the toc
+
+%-------------------- end preamble ----------------------
+
+\begin{document}
+
+% matching end for #ifdef PREAMBLE
+
+\newcommand{\exercisesection}[1]{\subsection*{#1}}
+
+
+% ------------------- main content ----------------------
+
+
+
+% ----------------- title -------------------------
+
+\thispagestyle{empty}
+
+\begin{center}
+{\LARGE\bf
+\begin{spacing}{1.25}
+Homework 1 Fall Semester 2019
+\end{spacing}
+}
+\end{center}
+
+% ----------------- author(s) -------------------------
+
+\begin{center}
+{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html}}{Data Analysis and Machine Learning FYS-STK3155/FYS4155}}
+\end{center}
+
+ \begin{center}
+% List of all institutions:
+\centerline{{\small Department of Physics, University of Oslo, Norway}}
+\end{center}
+
+% ----------------- end author(s) -------------------------
+
+% --- begin date ---
+\begin{center}
+Aug 20, 2019
+\end{center}
+% --- end date ---
+
+\vspace{1cm}
+
+
+\subsection*{Exercise 1}
+
+The first exercise here is of a mere technical art. We want you to have
+\begin{itemize}
+\item git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo \href{{https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html}}{GitHub facilities}.
+
+\item Install various Python packages
+\end{itemize}
+
+\noindent
+We will make extensive use of Python as programming language and its
+myriad of available libraries. You will find
+IPython/Jupyter notebooks invaluable in your work. You can run \textbf{R}
+codes in the Jupyter/IPython notebooks, with the immediate benefit of
+visualizing your data. You can also use compiled languages like C++,
+Rust, Fortran etc if you prefer. The focus in these lectures will be
+on Python.
+
+If you have Python installed (we recommend Python3) and you feel
+pretty familiar with installing different packages, we recommend that
+you install the following Python packages via \textbf{pip} as
+
+\begin{enumerate}
+\item pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow
+\end{enumerate}
+
+\noindent
+For \textbf{Tensorflow}, we recommend following the instructions in the text of
+\href{{http://shop.oreilly.com/product/0636920052289.do}}{Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly}
+
+We will come back to \textbf{tensorflow} later.
+
+For Python3, replace \textbf{pip} with \textbf{pip3}.
+
+For OSX users we recommend, after having installed Xcode, to
+install \textbf{brew}. Brew allows for a seamless installation of additional
+software via for example
+
+\begin{enumerate}
+\item brew install python3
+\end{enumerate}
+
+\noindent
+For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
+you can use \textbf{pip} as well and simply install Python as
+
+\begin{enumerate}
+\item sudo apt-get install python3 (or python for pyhton2.7)
+\end{enumerate}
+
+\noindent
+If you don't want to perform these operations separately and venture
+into the hassle of exploring how to set up dependencies and paths, we
+recommend two widely used distrubutions which set up all relevant
+dependencies for Python, namely
+
+\begin{itemize}
+\item \href{{https://docs.anaconda.com/}}{Anaconda},
+\end{itemize}
+
+\noindent
+which is an open source
+distribution of the Python and R programming languages for large-scale
+data processing, predictive analytics, and scientific computing, that
+aims to simplify package management and deployment. Package versions
+are managed by the package management system \textbf{conda}.
+
+\begin{itemize}
+\item \href{{https://www.enthought.com/product/canopy/}}{Enthought canopy}
+\end{itemize}
+
+\noindent
+is a Python
+distribution for scientific and analytic computing distribution and
+analysis environment, available for free and under a commercial
+license.
+
+We recommend using \textbf{Anaconda}.
+
+\subsection*{Exercise 2}
+
+We will generate our own dataset for a function $y(x)$ where $x \in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\cal {N}(0,1)$.
+The following simple Python instructions define our $x$ and $y$ values (with 100 data points).
+\begin{print}
+x = np.random.rand(100,1)
+y = 5*x*x+0.1*np.random.randn(100,1)
+\end{print}
+
+\begin{enumerate}
+\item Write your own code (following the examples under the \href{{https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html}}{regression slides}) for computing the parametrization of the data set fitting a second-order polynomial.
+
+\item Use thereafter \textbf{scikit-learn} (see again the examples in the regression slides) and compare with your own code.
+
+\item Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
+\end{enumerate}
+
+\noindent
+\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
+\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
+\]
+and the $R^2$ score function.
+If $\tilde{\hat{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as
+\[
+R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
+\]
+where we have defined the mean value of $\hat{y}$ as
+\[
+\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
+\]
+You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
+Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
+
+
+
+
+\subsection*{Exercise 3, mean values and variances in linear regression}
+
+
+This exercise deals with various mean values ad variances in linear regression method (here it may be useful to look up chapter 3, equation (3.8) of \href{{https://www.springer.com/gp/book/9780387848570}}{Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer}).
+
+The assumption we have made is
+that there exists a function $f(\bm{x})$ and a normal distributed error $\bm{\varepsilon}\sim \mathcal{N}(0, \sigma^2)$
+which describes our data
+\[
+\bm{y} = f(\bm{x})+\bm{\varepsilon}
+\]
+
+We then approximate this function with our model from the solution of the linear regression equations (ordinary least squares OLS), that is our
+function $f$ is approximated by $\bm{\tilde{y}}$ where we minimized $(\bm{y}-\bm{\tilde{y}})^2$, with
+\[
+\bm{\tilde{y}} = \bm{X}\bm{\beta}.
+\]
+The matrix $\bm{X}$ is the so-called design matrix.
+
+
+Show that the expectation value of $\bm{y}$ for a given element $i$
+\begin{align*}
+\mathbb{E}(y_i) & =\mathbf{X}_{i, \ast} \, \beta,
+\end{align*}
+and that
+its variance is
+\begin{align*} \mbox{Var}(y_i) & = \sigma^2.
+\end{align*}
+Hence, $y_i \sim \mathcal{N}( \mathbf{X}_{i, \ast} \, \bm{\beta}, \sigma^2)$, that is $\bm{y}$ follows a normal distribution with
+mean value $\bm{X}\bm{\beta}$ and variance $\sigma^2$.
+
+
+With the OLS expressions for the parameters $\bm{\beta}$ show that
+\[
+\mathbb{E}(\bm{\beta}) = \bm{\beta}.
+\]
+This means that the estimator of the regression parameters is unbiased.
+
+Show finally that the variance of $\bm{\beta}$ is
+\begin{eqnarray*}
+\mbox{Var}(\bm{\beta}) & = & \sigma^2 \, (\mathbf{X}^{T} \mathbf{X})^{-1}.
+\end{eqnarray*}
+
+
+
+% ------------------- end of main content ---------------
+
+\end{document}
+