diff --git a/doc/pub/LogReg/html/._LogReg-bs000.html b/doc/pub/LogReg/html/._LogReg-bs000.html new file mode 100644 index 000000000..bbb583e11 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs000.html @@ -0,0 +1,170 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + + + +
+

Data Analysis and Machine Learning: Logistic Regression

+ +

+ + +

+Morten Hjorth-Jensen [1, 2] +
+ +

+ + +

[1] Department of Physics, University of Oslo
+
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
+
+

+

Sep 17, 2018

+
+

+

+ +

+ + +
+ + + + + + + +
+ © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs001.html b/doc/pub/LogReg/html/._LogReg-bs001.html new file mode 100644 index 000000000..5bfcf585c --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs001.html @@ -0,0 +1,176 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ +

 

 

 

+ + + + + +
+ +

Logistic Regression

+ +

+So far we have focused on learning from datasets for which there is a +continuous output. In linear regression we have been +concerned with learning the coefficients of a polynomial to predict +the response of a continuous variable \( y_i \) on unseen data based on +its independent variables \( {\bf x}_i \). + +

+Classification problems, +however, are concerned with outcomes taking the form of discrete +variables (i.e. categories). For example, we may want to detect if +there's a cat or a dog in an image. Or given a specific system, +we'd like to identify its state, say whether it is an ordered or disordered system (typical situation in solid state physics). +(e.g. ordered/disordered). + +

+Logistic regression deals with binary, dichotomous outcomes (e.g. True or +False, Success or Failure, etc.). It is worth noting that logistic +regression is also commonly used in modern supervised Deep Learning +models, as we will see later. + +

+

+ +

+ + + + + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs002.html b/doc/pub/LogReg/html/._LogReg-bs002.html new file mode 100644 index 000000000..c7761cde7 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs002.html @@ -0,0 +1,164 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

Basics

+ +

+We consider the case where the dependent variables \( y_i\in\mathbb{Z} \) +are discrete and only take values from \( m=0,\dots,M-1 \) (i.e. \( M \) +classes). + +

+The goal is to predict the +output classes from the design matrix \( X\in\mathbb{R}^{n\times p} \) +made of \( n \) samples, each of which bears \( p \) features. The +primary goal is to identify the classes to which new unseen samples +belong. + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs003.html b/doc/pub/LogReg/html/._LogReg-bs003.html new file mode 100644 index 000000000..a0d1dc1b2 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs003.html @@ -0,0 +1,164 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

Linear classifier

+ +

+Let us start by considering a slightly simpler classifier: a linear classifier that categorizes examples using a weighted linear-combination of the features and an additive offset +$$ +\begin{equation} +s_i = \boldsymbol{x}_i^T\boldsymbol{w} + b_0 \equiv \mathbf{x}_i^T\mathbf{w}, +\tag{1} +\end{equation} +$$ + +where we use the short-hand notation +\( \mathbf{x}_i = (1,\boldsymbol{x}_i) \) and \( \mathbf{w}_i = (b_0,\boldsymbol{w}_i) \). + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs004.html b/doc/pub/LogReg/html/._LogReg-bs004.html new file mode 100644 index 000000000..6c222f591 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs004.html @@ -0,0 +1,174 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

Some selected properties

+ +

+This function takes values on the entire real axis. In the case of +logistic regression, however, the labels \( y_i \) are discrete +variables. One simple way to get a discrete output is to have sign +functions that map the output of a linear regressor to \( \{0,1\} \), +\( f(s_i)=sign(s_i)=1 \) if \( s_i\ge 0 \) and 0 if otherwise. Indeed, +this is commonly known as the "perceptron" in the machine learning +literature. This model is extremely simple, and it is favorable in +many cases (e.g. noisy data) to have a ``soft" classifier that outputs +the probability of a given category. For example, given +\( \mathbf{x}_i \), the classifier outputs the probability of being in +category \( m \). One such function is the logistic (or sigmoid) function: + +$$ +\begin{equation} +f(s) = \frac{1}{1+\mathrm e^{-s}}. +\tag{2} +\end{equation} +$$ + +Note that \( 1-f(s)= f(-s) \), which will be useful shortly. + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs005.html b/doc/pub/LogReg/html/._LogReg-bs005.html new file mode 100644 index 000000000..07482a3a8 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs005.html @@ -0,0 +1,170 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

The cross-entropy as a cost function for logistic regression

+ +

+The perceptron is an example of a ``hard classification": each datapoint is deterministically assigned to a category (i.e \( y_i=0 \) or \( y_i=1 \)). In many cases, it is favorable to have a "soft" classifier that outputs the probability of a given category rather than a single value. For example, given \( \mathbf{x}_i \), the classifier outputs the probability of being in category \( m \). +Logistic regression is the most canonical example of a soft classifier. In logistic regression, the probability that a data point \( \boldsymbol{x}_i \) belongs to a category \( y_i=\{0,1\} \) is is given by +$$ +\begin{eqnarray} +P(y_i=1|\boldsymbol{x}_i,\boldsymbol{\theta)} &=& \frac{1}{1+\mathrm{e}^{-\mathbf{x}^T_i\mathbf{w}}},\nonumber\\ +P(y_i=0|\boldsymbol{x}_i,\boldsymbol{\theta)} &=& 1 - P(y_i=1|\boldsymbol{x}_i,\boldsymbol{\theta)}, +\end{eqnarray} +$$ + +where \( \boldsymbol{\theta}=\mathbf{w} \) are the weights we wish to learn from the data. + +

+Notice that in terms of the logistic function, we can write +$$ +P(y_i=1) =f(\mathbf{x}_i^T\mathbf{w})=1-P(y_i=0). +$$ + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs006.html b/doc/pub/LogReg/html/._LogReg-bs006.html new file mode 100644 index 000000000..bed88a490 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs006.html @@ -0,0 +1,174 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

Maximum likelihood

+ +

+We now define the cost function for logistic regression using Maximum +Likelihood Estimation (MLE). Recall, that in MLE we choose parameters +to maximize the probability of seeing the observed data. Consider a +dataset \( \mathcal{D}=\{(y_i,\boldsymbol{x}_i)\} \) with binary labels +\( y_i\in\{0,1\} \) where the data points are drawn independently. The +likelihood of the seeing the data under our model is just: +$$ +\begin{align} +P(\mathcal{D}|\mathbf{w})& = \prod_{i=1}^n \left[f(\mathbf{x}_i^T\mathbf{w})\right]^{y_i}\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]^{1-y_i}\nonumber \\ +\tag{3} +\end{align} +$$ + +from which we can readily compute the log-likelihood: +$$ +\begin{equation} +l(\mathbf{w}) = \sum_{i=1}^n y_i\log f(\mathbf{x}_i^T\mathbf{w}) + (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]. +\tag{4} +\end{equation} +$$ + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs007.html b/doc/pub/LogReg/html/._LogReg-bs007.html new file mode 100644 index 000000000..cd38b5b73 --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs007.html @@ -0,0 +1,165 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + +The maximum likelihood estimator is defined as the set of parameters that maximize the log-likelihood where we maximize with respect to \( \theta \) +$$ +\hat{\mathbf{w}} = \sum_{i=1}^n y_i\log f(\mathbf{x}_i^T\mathbf{w}) + (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]. +$$ + +Since the cost (error) function is just the negative log-likelihood, for logistic regression we have that +$$ +\begin{eqnarray} +\mathcal{C}(\mathbf{w}) &=& - l(\mathbf{w}) \\ +&=& \sum_{i=1}^n -y_i\log f(\mathbf{x}_i^T\mathbf{w}) - (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right].\nonumber +\end{eqnarray} +$$ + +This equation is known in statistics as the \emph{cross entropy}. Finally, we note that just as in linear regression, +in practice we usually supplement the cross-entropy with additional regularization terms, usually \( L_1 \) and \( L_2 \) regularization as we did for Ridge and Lasso regression. + +

+

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/pub/LogReg/html/._LogReg-bs008.html b/doc/pub/LogReg/html/._LogReg-bs008.html new file mode 100644 index 000000000..16143ecbd --- /dev/null +++ b/doc/pub/LogReg/html/._LogReg-bs008.html @@ -0,0 +1,169 @@ + + + + + + + +Data Analysis and Machine Learning: Logistic Regression + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +

 

 

 

+ + + + +

Minimizing the cross entropy

+ +

+The cross entropy is a convex function of the weights \( \mathbf{w} \) and, +therefore, any local minimizer is a global minimizer. Minimizing this +cost function leads to the following equation + +$$ +\begin{equation} +\boldsymbol{0}=\boldsymbol{\nabla} \mathcal{C}(\mathbf{w}) = \sum_{i=1}^n\left[f(\mathbf{x}_i^T\mathbf{w})-y_i\right]\mathbf{x}_i, +\tag{5} +\end{equation} +$$ + +

+where we made use of the logistic function identity \( \partial_z f(z) = +f(z)[1-f(z)] \). This equation defines a transcendental equation for +\( \mathbf{w} \), the solution of which, unlike linear regression, cannot +be written in a closed form. +Here we need gradient descent methods! + +

+ +

+ + +
+ + + + + + + +
+ +
+ + + + + + diff --git a/doc/src/LogisticRegression/LogReg.do.txt b/doc/src/LogisticRegression/LogReg.do.txt new file mode 100644 index 000000000..3c60670e4 --- /dev/null +++ b/doc/src/LogisticRegression/LogReg.do.txt @@ -0,0 +1,153 @@ +TITLE: Data Analysis and Machine Learning: Logistic Regression +AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University +DATE: today + + +!split +===== Logistic Regression ===== + +So far we have focused on learning from datasets for which there is a +_continuous_ output. In linear regression we have been +concerned with learning the coefficients of a polynomial to predict +the response of a continuous variable $y_i$ on unseen data based on +its independent variables ${\bf x}_i$. + +Classification problems, +however, are concerned with outcomes taking the form of discrete +variables (i.e. categories). For example, we may want to detect if +there's a cat or a dog in an image. Or given a specific system, +we'd like to identify its state, say whether it is an ordered or disordered system (typical situation in solid state physics). +(e.g. ordered/disordered). + +_Logistic regression deals with binary, dichotomous outcomes (e.g. True or +False, Success or Failure, etc.). It is worth noting that logistic +regression is also commonly used in modern supervised Deep Learning +models_, as we will see later. + + +!split +===== Basics ===== + +We consider the case where the dependent variables $y_i\in\mathbb{Z}$ +are discrete and only take values from $m=0,\dots,M-1$ (i.e. $M$ +classes). + +The goal is to predict the +output classes from the design matrix $X\in\mathbb{R}^{n\times p}$ +made of $n$ samples, each of which bears $p$ features. The +primary goal is to identify the classes to which new unseen samples +belong. + + +!split +===== Linear classifier ===== + +Let us start by considering a slightly simpler classifier: a linear classifier that categorizes examples using a weighted linear-combination of the features and an additive offset +!bt +\begin{equation} +s_i = \boldsymbol{x}_i^T\boldsymbol{w} + b_0 \equiv \mathbf{x}_i^T\mathbf{w}, +\end{equation} +!et +where we use the short-hand notation +$\mathbf{x}_i = (1,\boldsymbol{x}_i)$ and $\mathbf{w}_i = (b_0,\boldsymbol{w}_i)$. + +!split +===== Some selected properties ===== + +This function takes values on the entire real axis. In the case of +logistic regression, however, the labels $y_i$ are discrete +variables. One simple way to get a discrete output is to have sign +functions that map the output of a linear regressor to $\{0,1\}$, +$f(s_i)=sign(s_i)=1$ if $s_i\ge 0$ and 0 if otherwise. Indeed, +this is commonly known as the ``perceptron" in the machine learning +literature. This model is extremely simple, and it is favorable in +many cases (e.g. noisy data) to have a ``soft" classifier that outputs +the probability of a given category. For example, given +$\mathbf{x}_i$, the classifier outputs the probability of being in +category $m$. One such function is the logistic (or sigmoid) function: + +!bt +\begin{equation} +f(s) = \frac{1}{1+\mathrm e^{-s}}. +\label{eq:log_fun} +\end{equation} +!et +Note that $1-f(s)= f(-s)$, which will be useful shortly. + +!split +===== The cross-entropy as a cost function for logistic regression ===== + +The perceptron is an example of a ``hard classification'': each datapoint is deterministically assigned to a category (i.e $y_i=0$ or $y_i=1$). In many cases, it is favorable to have a ``soft'' classifier that outputs the probability of a given category rather than a single value. For example, given $\mathbf{x}_i$, the classifier outputs the probability of being in category $m$. +Logistic regression is the most canonical example of a soft classifier. In logistic regression, the probability that a data point $\boldsymbol{x}_i$ belongs to a category $y_i=\{0,1\}$ is is given by +!bt +\begin{eqnarray} +P(y_i=1|\boldsymbol{x}_i,\boldsymbol{\theta)} &=& \frac{1}{1+\mathrm{e}^{-\mathbf{x}^T_i\mathbf{w}}},\nonumber\\ +P(y_i=0|\boldsymbol{x}_i,\boldsymbol{\theta)} &=& 1 - P(y_i=1|\boldsymbol{x}_i,\boldsymbol{\theta)}, +\end{eqnarray} +!et +where $\boldsymbol{\theta}=\mathbf{w}$ are the weights we wish to learn from the data. + + +Notice that in terms of the logistic function, we can write +!bt +\[ +P(y_i=1) =f(\mathbf{x}_i^T\mathbf{w})=1-P(y_i=0). +\] +!et + +!split +===== Maximum likelihood ===== + +We now define the cost function for logistic regression using Maximum +Likelihood Estimation (MLE). Recall, that in MLE we choose parameters +to maximize the probability of seeing the observed data. Consider a +dataset $\mathcal{D}=\{(y_i,\boldsymbol{x}_i)\}$ with binary labels +$y_i\in\{0,1\}$ where the data points are drawn independently. The +likelihood of the seeing the data under our model is just: +!bt +\begin{align} +P(\mathcal{D}|\mathbf{w})& = \prod_{i=1}^n \left[f(\mathbf{x}_i^T\mathbf{w})\right]^{y_i}\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]^{1-y_i}\nonumber \\ +\end{align} +!et +from which we can readily compute the log-likelihood: +!bt +\begin{equation} +l(\mathbf{w}) = \sum_{i=1}^n y_i\log f(\mathbf{x}_i^T\mathbf{w}) + (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]. +\end{equation} +!et + +!split +The maximum likelihood estimator is defined as the set of parameters that maximize the log-likelihood where we maximize with respect to $\theta$ +!bt +\[ +\hat{\mathbf{w}} = \sum_{i=1}^n y_i\log f(\mathbf{x}_i^T\mathbf{w}) + (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right]. +\] +!et +Since the cost (error) function is just the negative log-likelihood, for logistic regression we have that +!bt +\begin{eqnarray} +\mathcal{C}(\mathbf{w}) &=& - l(\mathbf{w}) \\ +&=& \sum_{i=1}^n -y_i\log f(\mathbf{x}_i^T\mathbf{w}) - (1-y_i)\log\left[1-f(\mathbf{x}_i^T\mathbf{w})\right].\nonumber +\end{eqnarray} +!et +This equation is known in statistics as the \emph{cross entropy}. Finally, we note that just as in linear regression, +in practice we usually supplement the cross-entropy with additional regularization terms, usually $L_1$ and $L_2$ regularization as we did for Ridge and Lasso regression. + +!split +===== Minimizing the cross entropy ===== + +The cross entropy is a convex function of the weights $\mathbf{w}$ and, +therefore, any local minimizer is a global minimizer. Minimizing this +cost function leads to the following equation + +!bt +\begin{equation} +\boldsymbol{0}=\boldsymbol{\nabla} \mathcal{C}(\mathbf{w}) = \sum_{i=1}^n\left[f(\mathbf{x}_i^T\mathbf{w})-y_i\right]\mathbf{x}_i, +\end{equation} +!et + +where we made use of the logistic function identity $\partial_z f(z) = +f(z)[1-f(z)]$. This equation defines a transcendental equation for +$\mathbf{w}$, the solution of which, unlike linear regression, cannot +be written in a closed form. +Here we need gradient descent methods! diff --git a/doc/src/LogisticRegression/beamerthemered_plain.sty b/doc/src/LogisticRegression/beamerthemered_plain.sty new file mode 100644 index 000000000..a6b2d33bd --- /dev/null +++ b/doc/src/LogisticRegression/beamerthemered_plain.sty @@ -0,0 +1,12 @@ +\mode +\usecolortheme[rgb={0.8, 0.2, 0}]{structure} +\usefonttheme[onlysmall]{structurebold} + +\setbeamertemplate{navigation symbols}{} +%\setbeamertemplate{footline}[frame number] + +\usepackage{tikz} +\usetikzlibrary{arrows,shapes,backgrounds,decorations,mindmap} + +\mode + diff --git a/doc/src/LogisticRegression/beamerthemered_shadow.sty b/doc/src/LogisticRegression/beamerthemered_shadow.sty new file mode 100644 index 000000000..eff5ac479 --- /dev/null +++ b/doc/src/LogisticRegression/beamerthemered_shadow.sty @@ -0,0 +1,15 @@ +\mode + +\useoutertheme{smoothbars} +\useinnertheme[shadow=true]{rounded} +\usecolortheme{orchid} +\usecolortheme{whale} +\usecolortheme[rgb={0.7, 0.2, 0}]{structure} % (darker red) +\useoutertheme{shadow} +\usefonttheme[onlysmall]{structurebold} + +\setbeamercolor{title}{use=structure,fg=white,bg=structure.fg} +\setbeamerfont{block title}{size={}} + +\mode + diff --git a/doc/src/LogisticRegression/clean.sh b/doc/src/LogisticRegression/clean.sh new file mode 100755 index 000000000..2e5da2c72 --- /dev/null +++ b/doc/src/LogisticRegression/clean.sh @@ -0,0 +1,3 @@ +#!/bin/sh +doconce clean +rm -rf *.pdf *.tex ipynb*.tar.gz *.html ._*.html *~ reveal.js Trash README.txt diff --git a/doc/src/LogisticRegression/make.sh b/doc/src/LogisticRegression/make.sh new file mode 100755 index 000000000..edfed7c87 --- /dev/null +++ b/doc/src/LogisticRegression/make.sh @@ -0,0 +1,118 @@ +#!/bin/sh +set -x + +function system { + "$@" + if [ $? -ne 0 ]; then + echo "make.sh: unsuccessful command $@" + echo "abort!" + exit 1 + fi +} + +if [ $# -eq 0 ]; then +echo 'bash make.sh slides1|slides2' +exit 1 +fi + +name=$1 +rm -f *.tar.gz + +opt="--encoding=utf-8" +# Note: Makefile examples contain constructions like ${PROG} which +# looks like Mako constructions, but they are not. Use --no_mako +# to turn off Mako processing. +opt="--no_mako" + +rm -f *.aux + + +html=${name}-reveal +system doconce format html $name --pygments_html_style=perldoc --keep_pygments_html_bg --html_links_in_new_window --html_output=$html $opt +system doconce slides_html $html reveal --html_slide_theme=beige + +# Plain HTML documents + +html=${name}-solarized +system doconce format html $name --pygments_html_style=perldoc --html_style=solarized3 --html_links_in_new_window --html_output=$html $opt +system doconce split_html $html.html --method=space10 + +html=${name} +system doconce format html $name --pygments_html_style=default --html_style=bloodish --html_links_in_new_window --html_output=$html $opt +system doconce split_html $html.html --method=space10 + +# Bootstrap style +html=${name}-bs +system doconce format html $name --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=$html $opt +system doconce split_html $html.html --method=split --pagination --nav_button=bottom + +# IPython notebook +system doconce format ipynb $name $opt + +# LaTeX Beamer slides +beamertheme=red_plain +system doconce format pdflatex $name --latex_title_layout=beamer --latex_table_format=footnotesize $opt +system doconce ptex2tex $name envir=minted +# Add special packages +doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex +system doconce slides_beamer $name --beamer_slide_theme=$beamertheme +system pdflatex -shell-escape ${name} +system pdflatex -shell-escape ${name} +cp $name.pdf ${name}-beamer.pdf +cp $name.tex ${name}-beamer.tex + +# Handouts +system doconce format pdflatex $name --latex_title_layout=beamer --latex_table_format=footnotesize $opt +system doconce ptex2tex $name envir=minted +# Add special packages +doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex +system doconce slides_beamer $name --beamer_slide_theme=red_shadow --handout +system pdflatex -shell-escape $name +pdflatex -shell-escape $name +pdflatex -shell-escape $name +pdfnup --nup 2x3 --frame true --delta "1cm 1cm" --scale 0.9 --outfile ${name}-beamer-handouts2x3.pdf ${name}.pdf +rm -f ${name}.pdf + +# Ordinary plain LaTeX document +rm -f *.aux # important after beamer +system doconce format pdflatex $name --minted_latex_style=trac --latex_admon=paragraph $opt +system doconce ptex2tex $name envir=minted +# Add special packages +doconce subst "% Add user's preamble" "\g<1>\n\\usepackage{simplewick}" $name.tex +doconce replace 'section{' 'section*{' $name.tex +pdflatex -shell-escape $name +pdflatex -shell-escape $name +mv -f $name.pdf ${name}-minted.pdf +cp $name.tex ${name}-plain-minted.tex + + + +# Publish +dest=../../pub +if [ ! -d $dest/$name ]; then +mkdir $dest/$name +mkdir $dest/$name/pdf +mkdir $dest/$name/html +mkdir $dest/$name/ipynb +fi +cp ${name}*.pdf $dest/$name/pdf +cp -r ${name}*.html ._${name}*.html reveal.js $dest/$name/html + +# Figures: cannot just copy link, need to physically copy the files +if [ -d fig-${name} ]; then +if [ ! -d $dest/$name/html/fig-$name ]; then +mkdir $dest/$name/html/fig-$name +fi +cp -r fig-${name}/* $dest/$name/html/fig-$name +fi + +cp ${name}.ipynb $dest/$name/ipynb +ipynb_tarfile=ipynb-${name}-src.tar.gz +if [ ! -f ${ipynb_tarfile} ]; then +cat > README.txt < @@ -384,7 +385,36 @@ formulas in HTML or ipython notebook files. -

Neural Networks

+

Logistic Regression

+ + + +

Neural Networks

  • LaTeX PDF:
  • @@ -413,7 +443,7 @@ formulas in HTML or ipython notebook files.
-

Elements of Bayesian theory

+

Elements of Bayesian theory

  • LaTeX PDF:
  • @@ -442,7 +472,7 @@ formulas in HTML or ipython notebook files.
-

Decision trees, from simple to random ones

+

Decision trees, from simple to random ones

  • LaTeX PDF:
  • @@ -471,7 +501,7 @@ formulas in HTML or ipython notebook files.
-

Support Vector Machines

+

Support Vector Machines

  • LaTeX PDF:
  • @@ -500,7 +530,7 @@ formulas in HTML or ipython notebook files.
-

Unsupervised Learning, Boltzmann Machines

+

Unsupervised Learning, Boltzmann Machines

  • LaTeX PDF:
  • @@ -531,7 +561,7 @@ formulas in HTML or ipython notebook files. -

    Python and Scikit Learn, a short guide

    +

    Python and Scikit Learn, a short guide

    • HTML format only:
    • @@ -544,7 +574,7 @@ formulas in HTML or ipython notebook files. -

      Teach yourself C++

      +

      Teach yourself C++

      • HTML format only:
      • @@ -557,9 +587,9 @@ formulas in HTML or ipython notebook files. -

        Projects and Exercises Fall 2018

        +

        Projects and Exercises Fall 2018

        -

        First homework set, week 35

        +

        First homework set, week 35

        • LaTeX and PDF:
        • @@ -578,7 +608,7 @@ formulas in HTML or ipython notebook files.
        -

        Second homework set, week 36

        +

        Second homework set, week 36

        • LaTeX and PDF:
        • @@ -597,7 +627,7 @@ formulas in HTML or ipython notebook files.
        -

        Project 1, Deadline October 1

        +

        Project 1, Deadline October 1

        • LaTeX and PDF:
        • @@ -616,7 +646,7 @@ formulas in HTML or ipython notebook files.
        -

        Project 2, Deadline November 5

        +

        Project 2, Deadline November 5

        • LaTeX and PDF:
        • @@ -635,7 +665,7 @@ formulas in HTML or ipython notebook files.
        -

        Project 3, Deadline November 30

        +

        Project 3, Deadline November 30

        • LaTeX and PDF:
        • @@ -654,7 +684,7 @@ formulas in HTML or ipython notebook files.
        -

        Course content

        +

        Course content

        Probability theory and statistical methods play a central role in science. Nowadays we are @@ -673,7 +703,7 @@ tools of probability theory, the aim of this course is to expose you to central This course covers thus topics like Monte Carlo methods and Markov chains, Bayesian statistics, error estimates, various linear methods, optimization of data and error analysis and central algorithms in machine learning. The course has several numerical projects and numerical exercises that are meant to illustrate the theory. -

        Learning outcomes

        +

        Learning outcomes

        The course introduces a variety of central algorithms and methods @@ -690,19 +720,19 @@ essential for studies of data analysis and machine learning. The course is proje

      • Work on numerical projects to illustrate the theory. The projects play a central role and students are expected to know modern programming languages like Python or C++.
      -

      Prerequisites

      +

      Prerequisites

      Basic knowledge in programming and numerics. Required courses are the equivalents to the University of Oslo mathematics courses MAT1100, MAT1110, MAT1120 and at least one of the corresponding computing and programming courses INF1000/INF1110 or MAT-INF1100/MAT-INF1100L/BIOS1100/KJM-INF1xxx. -

      The course has two central parts

      +

      The course has two central parts

      1. Statistical analysis and optimization of data
      2. Machine learning
      -

      Statistical analysis and optimization of data

      +

      Statistical analysis and optimization of data

      The following topics will be covered @@ -719,7 +749,7 @@ The following topics will be covered

    • Practical optimization using Singular-value decomposition and least squares for parameterizing data.
    -

    Machine learning

    +

    Machine learning

    The following topics will be covered @@ -735,14 +765,14 @@ The following topics will be covered All the above topics will be supported by examples, hands-on exercises and project work. -

    Recommended textbooks

    +

    Recommended textbooks

    -

    Other textbooks

    +

    Other textbooks

    General learning book on statistical analysis: @@ -764,7 +794,7 @@ All the above topics will be supported by examples, hands-on exercises and proje -

    Teaching schedule Fall 2018

    +

    Teaching schedule Fall 2018

    Acronyms for textbooks and references to chapter