diff --git a/doc/src/LectureNotes/_build/.doctrees/logistic.doctree b/doc/src/LectureNotes/_build/.doctrees/logistic.doctree
new file mode 100644
index 000000000..813f302b2
Binary files /dev/null and b/doc/src/LectureNotes/_build/.doctrees/logistic.doctree differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/gettingstarted_118_12.png b/doc/src/LectureNotes/_build/jupyter_execute/gettingstarted_118_12.png
new file mode 100644
index 000000000..d3b03a64d
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/gettingstarted_118_12.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/logistic.ipynb
new file mode 100644
index 000000000..f251ed680
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/logistic.ipynb
@@ -0,0 +1,891 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<!-- dom:TITLE: Data Analysis and Machine Learning: Logistic Regression -->\n",
+    "# Data Analysis and Machine Learning: Logistic Regression\n",
+    "<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->\n",
+    "<!-- Author: -->  \n",
+    "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
+    "\n",
+    "Date: **Oct 17, 2019**\n",
+    "\n",
+    "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "<!-- !split  -->\n",
+    "## Logistic Regression\n",
+    "\n",
+    "In linear regression our main interest was centered on learning the\n",
+    "coefficients of a functional fit (say a polynomial) in order to be\n",
+    "able to predict the response of a continuous variable on some unseen\n",
+    "data. The fit to the continuous variable $y_i$ is based on some\n",
+    "independent variables $\\hat{x}_i$. Linear regression resulted in\n",
+    "analytical expressions for standard ordinary Least Squares or Ridge\n",
+    "regression (in terms of matrices to invert) for several quantities,\n",
+    "ranging from the variance and thereby the confidence intervals of the\n",
+    "parameters $\\hat{\\beta}$ to the mean squared error. If we can invert\n",
+    "the product of the design matrices, linear regression gives then a\n",
+    "simple recipe for fitting our data.\n",
+    "\n",
+    "\n",
+    "Classification problems, however, are concerned with outcomes taking\n",
+    "the form of discrete variables (i.e. categories). We may for example,\n",
+    "on the basis of DNA sequencing for a number of patients, like to find\n",
+    "out which mutations are important for a certain disease; or based on\n",
+    "scans of various patients' brains, figure out if there is a tumor or\n",
+    "not; or given a specific physical system, we'd like to identify its\n",
+    "state, say whether it is an ordered or disordered system (typical\n",
+    "situation in solid state physics); or classify the status of a\n",
+    "patient, whether she/he has a stroke or not and many other similar\n",
+    "situations.\n",
+    "\n",
+    "The most common situation we encounter when we apply logistic\n",
+    "regression is that of two possible outcomes, normally denoted as a\n",
+    "binary outcome, true or false, positive or negative, success or\n",
+    "failure etc.\n",
+    "\n",
+    "## Optimization and Deep learning\n",
+    "\n",
+    "Logistic regression will also serve as our stepping stone towards\n",
+    "neural network algorithms and supervised deep learning. For logistic\n",
+    "learning, the minimization of the cost function leads to a non-linear\n",
+    "equation in the parameters $\\hat{\\beta}$. The optimization of the\n",
+    "problem calls therefore for minimization algorithms. This forms the\n",
+    "bottle neck of all machine learning algorithms, namely how to find\n",
+    "reliable minima of a multi-variable function. This leads us to the\n",
+    "family of gradient descent methods. The latter are the working horses\n",
+    "of basically all modern machine learning algorithms.\n",
+    "\n",
+    "We note also that many of the topics discussed here on logistic \n",
+    "regression are also commonly used in modern supervised Deep Learning\n",
+    "models, as we will see later.\n",
+    "\n",
+    "\n",
+    "<!-- !split  -->\n",
+    "## Basics\n",
+    "\n",
+    "We consider the case where the dependent variables, also called the\n",
+    "responses or the outcomes, $y_i$ are discrete and only take values\n",
+    "from $k=0,\\dots,K-1$ (i.e. $K$ classes).\n",
+    "\n",
+    "The goal is to predict the\n",
+    "output classes from the design matrix $\\hat{X}\\in\\mathbb{R}^{n\\times p}$\n",
+    "made of $n$ samples, each of which carries $p$ features or predictors. The\n",
+    "primary goal is to identify the classes to which new unseen samples\n",
+    "belong.\n",
+    "\n",
+    "Let us specialize to the case of two classes only, with outputs\n",
+    "$y_i=0$ and $y_i=1$. Our outcomes could represent the status of a\n",
+    "credit card user that could default or not on her/his credit card\n",
+    "debt. That is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "y_i = \\begin{bmatrix} 0 & \\mathrm{no}\\\\  1 & \\mathrm{yes} \\end{bmatrix}.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Linear classifier\n",
+    "\n",
+    "Before moving to the logistic model, let us try to use our linear\n",
+    "regression model to classify these two outcomes. We could for example\n",
+    "fit a linear model to the default case if $y_i > 0.5$ and the no\n",
+    "default case $y_i \\leq 0.5$.\n",
+    "\n",
+    "We would then have our \n",
+    "weighted linear combination, namely"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<!-- Equation labels as ordinary links -->\n",
+    "<div id=\"_auto1\"></div>\n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\hat{y} = \\hat{X}^T\\hat{\\beta} +  \\hat{\\epsilon},\n",
+    "\\label{_auto1} \\tag{1}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $\\hat{y}$ is a vector representing the possible outcomes, $\\hat{X}$ is our\n",
+    "$n\\times p$ design matrix and $\\hat{\\beta}$ represents our estimators/predictors.\n",
+    "\n",
+    "## Some selected properties\n",
+    "\n",
+    "The main problem with our function is that it takes values on the\n",
+    "entire real axis. In the case of logistic regression, however, the\n",
+    "labels $y_i$ are discrete variables. A typical example is the credit\n",
+    "card data discussed below here, where we can set the state of\n",
+    "defaulting the debt to $y_i=1$ and not to $y_i=0$ for one the persons\n",
+    "in the data set (see the full example below).\n",
+    "\n",
+    "One simple way to get a discrete output is to have sign\n",
+    "functions that map the output of a linear regressor to values $\\{0,1\\}$,\n",
+    "$f(s_i)=sign(s_i)=1$ if $s_i\\ge 0$ and 0 if otherwise. \n",
+    "We will encounter this model in our first demonstration of neural networks. Historically it is called the \"perceptron\" model in the machine learning\n",
+    "literature. This model is extremely simple. However, in many cases it is more\n",
+    "favorable to use a ``soft\" classifier that outputs\n",
+    "the probability of a given category. This leads us to the logistic function.\n",
+    "\n",
+    "\n",
+    "## The logistic function\n",
+    "\n",
+    "The perceptron is an example of a ``hard classification\" model. We\n",
+    "will encounter this model when we discuss neural networks as\n",
+    "well. Each datapoint is deterministically assigned to a category (i.e\n",
+    "$y_i=0$ or $y_i=1$). In many cases, it is favorable to have a \"soft\"\n",
+    "classifier that outputs the probability of a given category rather\n",
+    "than a single value. For example, given $x_i$, the classifier\n",
+    "outputs the probability of being in a category $k$.  Logistic regression\n",
+    "is the most common example of a so-called soft classifier. In logistic\n",
+    "regression, the probability that a data point $x_i$\n",
+    "belongs to a category $y_i=\\{0,1\\}$ is given by the so-called logit function (or Sigmoid) which is meant to represent the likelihood for a given event,"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "p(t) = \\frac{1}{1+\\mathrm \\exp{-t}}=\\frac{\\exp{t}}{1+\\mathrm \\exp{t}}.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that $1-p(t)= p(-t)$.\n",
+    "\n",
+    "## Examples of likelihood functions used in logistic regression and nueral networks\n",
+    "\n",
+    "\n",
+    "The following code plots the logistic function, the step function and other functions we will encounter from here and on."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXiU5bnH8e9NdkggQCBAwqayuAACEdyNW4t7XU7dW60eSis92mrrUluPbY/aY0+rba1Kq9XWBaXSVit1q01xqcoOIoJhTQiBsITs+33+mNEEDBBgkncy+X2uay7yzvvMvHceJr88ed7N3B0REen8ugVdgIiIRIYCXUQkRijQRURihAJdRCRGKNBFRGKEAl1EJEYo0KVDmNkdZva7aNuuma0zszP2sC7FzF4ys51mNqv9qmx128vNLLcjtymdX3zQBUjX4O73dMLtXgJkAn3dvSFCJX2OmT0BFLr7nZ8+5+5Httf2JHZphC6yZ0OBVe0Z5iKRpECXiDKzW81so5mVm9lKMzs9/Px/m9lTLdp9xczWm9k2M/tBy6mPcNtZZvZU+H2WmdlIM7vdzLaYWYGZfaHFew0ysxfNbLuZ5ZvZf7ZYt/t2r26x3e/v5fu4G/ghcKmZVZjZda281zAzczOLDy/nmdmPzeydcN2vmVlGi/Ynmtm7ZlYa/h6uMbOpwJXA98LbeSnctmV/JJnZA2ZWFH48YGZJ4XW5ZlZoZjeH+2aTmV17oP9/0rkp0CVizGwUMB04xt3TgC8C61ppdwTwG0JBNhDoBWTt1uw84I9Ab2AR8Cqhz2sW8CPg0RZtnwUKgUGEpknu+fQXSSvbfRi4Oty2L5Dd2vfi7ncB9wDPuXuquz+2zw4IuQK4FugPJAK3hLc9BPg78CugH3A0sNjdZwBPA/8b3s55rbzn94Fjw68ZB0wC7myxfgDNfXgd8JCZ9W5jvRJDFOgSSY1AEnCEmSW4+zp3X91Ku0uAl9z9bXevIzQS3v2iQm+5+6vh6Y5ZhELwPnevB2YCw8ws3cwGAycCt7p7jbsvBn5HKLRb2+7f3H2uu9cCPwCaDv7b3sXv3X2Vu1cDzxMKYQj98nrD3Z9193p33xautS2uBH7k7lvcvQS4m12/v/rw+np3nwNUAKMi8+1IZ6JAl4hx93zgJuC/gS1mNtPMBrXSdBBQ0OJ1VcC23dpsbvF1NbDV3RtbLAOkht9ru7uXt2i/ns+P+FvbbmUr2z1YxS2+rgrXCDAYaO2XW1sMIvQ9fWp9+LlPbdttnr/ldqULUaBLRLn7M+5+IqEdig78tJVmm2gx1WFmKYSmPw5EEdDHzNJaPDcE2LiH7Q5usd3u+7ndSqB7i+UB+/HaAuDQPazb1yVPiwj156eGhJ8T2YUCXSLGzEaZ2WnhHXY1hEbSja00/RNwnpkdb2aJhKYQ7EC26e4FwLvAvWaWbGZjCc0jP72H7Z4b3jmZSGgufn9+BhYDJ5vZEDPrBdy+H699GjjDzL5sZvFm1tfMPp2O2QwcspfXPgvcaWb9wjtZfwg8tZf20kUp0CWSkoD7gK2Eph76A3fs3sjdlwPfIjQXvgkoB7YAtQe43cuBYYRGrX8G7nL31/ew3RuAZ8Lb3UFoZ2qbhN/zOWApsAD42368dgNwNnAzsJ3QL4dx4dWPEdrvUGpmf2nl5T8B5oe3uwxYGH5OZBemG1xI0MwsFSgFRrj72qDrEemsNEKXQJjZeWbW3cx6AD8jNPJcF2xVIp2bAl2CcgGhKZIiYARwmevPRZGDoikXEZEYoRG6iEiMCOxqixkZGT5s2LCgNv+ZyspKevToEXQZUUF90Ux90Ux90Swa+mLBggVb3b1fa+sCC/Rhw4Yxf/78oDb/mby8PHJzc4MuIyqoL5qpL5qpL5pFQ1+Y2fo9rdOUi4hIjFCgi4jECAW6iEiMUKCLiMQIBbqISIxQoIuIxAgFuohIjFCgi4jECAW6iEiMUKCLiMQIBbqISIxQoIuIxAgFuohIjNhnoJvZ42a2xcw+3MN6M7Nfmlm+mS01swmRL1NERPalLSP0J4Ape1l/FqFbiI0ApgIPH3xZIiKyv/YZ6O4+F9i+lyYXAH/wkPeAdDMbGKkCRUSkbSJxg4ssoKDFcmH4uU27NzSzqYRG8WRmZpKXlxeBzR+cioqKqKgjGqgvmqkvmqkvmkV7X0Qi0K2V51q987S7zwBmAOTk5HjQd/6A6LgDSbRQXzRTXzRTXzTbn75wdyrrGimrrqe8poGymnrKquupqG2gvCb0qKxtoKI29G9VXSOVdQ1U1TZSVR9arq5rpKqukTvPOZz/yBm8z21GItALgZZbygaKIvC+IiJRo76xiR01TSwr3MnWilq2VtSyvbLus8eOqjp2VNWzo6qOnVX17Kyup6Gp1bHtZ7oZ9EiKJzUpnh5J8fRIjCMlMY7+acmkJMbRPSG0PCyjbfcxjUSgvwhMN7OZwGRgp7t/brpFRCRa1dQ3srG0mo07qtm0s5qi0hqKd9ZQXFbD5rIaSspr2VZZF2qc9/Yur02M70af7on07pFInx4JHD6gJ726J5CekkCv8KNnSgI9kxNITY4nLfxITYonJSEOs9YmOQ7MPgPdzJ4FcoEMMysE7gISANz9EWAOcDaQD1QB10asOhGRCNlRWcearZWs3VrJ+m2VbNhexYbtVRRsr2ZrRe0ubc0gIzWJAT2Tye7dnQlDe9M/LYkdRes5YeIYMtKSyOiRRJ/URHokRjaUD8Y+A93dL9/HegduiFhFIiIHoaS8lpXF5XxcXMbqkgo+2VxBfkkFpVX1n7XpZjAoPYUhfbpz+uj+ZPdOIat3ClnpKQxKTyGzZzKJ8Z8/CDAvr4jcIwd05LezXyIx5SIi0uHcnfXbqviwaCcfbixjedFOPioqa54aAfr0SOSw/qmcPWYgh2T0YHj4kd27e6uB3dkp0EWkUyivqWfRhlLmr9/BkoJSlhSWfjbqTogzRmamcdro/owe2JPDB6QxakAafVOTAq66YynQRSQq7ayq5/2123h39TY+WLudj4vLaPLQdMnIzDSmHDmAcYPTGZPVi5GZaTE54t5fCnQRiQr1jU0s2lDK3FUlzP2khGUbd+IOyQndmDCkN986bQTHDOvD0UPSSU1SdLVGvSIigSmtqiNvZQmvr9jM3JUllNc2ENfNGD84nRtPH8Fxh/Tl6CHpJMXHBV1qp6BAF5EOta2illeWF/Py0k28v3Y7jU1Ov7Qkzhk7kNxR/Tju0Ax6pSQEXWanpEAXkXZXUdvA35dt4i+LN/Lv1dtocjgkowdfP/kQzjwik3HZ6XTrFh3HcndmCnQRaRdNTc47q7cya34hr31UTE19E0P7dueGUw/j7DEDGT0gLWpOyIkVCnQRiagt5TXMml/IzHkbKNheTa+UBC6ZmM2F47OZMCRdId6OFOgiEhGLC0p54p21vLxsE/WNznGH9OWWL4zii0cOIDlBOzU7ggJdRA5YY5Pz2vJiHp27hsUFpaQlxXPVsUO5+tihHNIvNejyuhwFuojst9qGRv68cCMz5q5hzdZKhvbtzt3nH8nFE7N1jHiA1PMi0mZ1DU3MWlDAQ2/mU7SzhqOyevLQFROYctQA4nSUSuAU6CKyT41NzgsLCnnwH5+wsbSa8UPSuffisZw8IkM7OaOIAl1E9sjdWbylgXsenMuqzRWMy+7FTy48ityR/RTkUUiBLiKt+qiojB/9bTnvrallWN84Hr4yNLWiII9eCnQR2UVpVR3/99oqnn5/Pb1SErjq8ETuuuoUEuJ0NcNop0AXESA0vTJrQSH3zlnBzup6rj52KN85cxSLPnhHYd5JKNBFhLVbK7l99lLeW7OdY4b15u7zj+KIQT2DLkv2kwJdpAtraGxixltreOCNT0iK78a9F43h0pzBulBWJ6VAF+miVpdUcPPzS1hcUMpZRw3g7vOPpH/P5KDLkoOgQBfpYpqanCfeXcdPX/mYlMQ4fnX5eM4bNyjosiQCFOgiXUhJeS03z1rC3FUlnDa6P/ddNEaj8hiiQBfpIt7+ZCs3PbeYspp6fvylo7hq8hAdUx5jFOgiMa6xyXngjVX8+p/5HNovlaeun8ToATqCJRYp0EVi2I7KOv5r5iLe+mQrX87J5u7zjyIlUdcmj1UKdJEYtaxwJ9OeWkBJeS33XTSGyyYNCbokaWcKdJEY9NfFG/nun5bSLzWJWdOOY9zg9KBLkg7QpvN5zWyKma00s3wzu62V9b3M7CUzW2Jmy83s2siXKiL70tTk/N9rK7lx5mKOzk7nxeknKMy7kH2O0M0sDngIOBMoBOaZ2Yvu/lGLZjcAH7n7eWbWD1hpZk+7e127VC0in1Nd18i3n1vMK8uL+XJONj/50hgS43UNlq6kLVMuk4B8d18DYGYzgQuAloHuQJqFjoFKBbYDDRGuVUT2YFtFLV97cj5LC0u585zDue7E4ToksQsyd997A7NLgCnufn14+WpgsrtPb9EmDXgRGA2kAZe6+8utvNdUYCpAZmbmxJkzZ0bq+zhgFRUVpKbqZragvmipM/XF5somfr6ghu01zrRxSUzMjOyusc7UF+0tGvri1FNPXeDuOa2ta8v/fGu/5nf/LfBFYDFwGnAo8LqZveXuZbu8yH0GMAMgJyfHc3Nz27D59pWXl0c01BEN1BfNOktfLC4o5eYn5tFEPDO/fgwTh/aO+DY6S190hGjvi7ZMsBUCg1ssZwNFu7W5FpjtIfnAWkKjdRFpJ+/mb+WK375H96Q4XvjG8e0S5tK5tCXQ5wEjzGy4mSUClxGaXmlpA3A6gJllAqOANZEsVESavba8mGuemEd27xRemHY8h/TTlIi0YcrF3RvMbDrwKhAHPO7uy81sWnj9I8CPgSfMbBmhKZpb3X1rO9Yt0mX9ZdFGbp61hKOyevHENcfQu0di0CVJlGjT3hN3nwPM2e25R1p8XQR8IbKlicjunp9XwK2zl3Ls8L789qs5pCbp3EBppk+DSCcx84MN3DZ7GSeNyOC3X8khOUHXZJFd6awDkU7g6ffXc9vsZeSO6qcwlz3SCF0kyj37wQa+/+cPOW10fx6+agJJ8QpzaZ1G6CJRbPbCQu748zJOHdVPYS77pEAXiVIvL93ELbOWcPyhfXn4qokKc9knBbpIFPrHis3cOHMRE4f21py5tJkCXSTKvL9mG998eiFHDurJ49ccQ/dE7eqStlGgi0SR5UU7uf7J+WT3TuH3104iLTkh6JKkE1Ggi0SJ9dsq+erj80hLjueP102mj84Alf2kQBeJAiXltVz92Ac0NjXxh+smMyg9JeiSpBPS5JxIwKrqGrjuyXmUlNfy7NRjOay/LrQlB0YjdJEANTQ2Mf2ZRXy4cSe/vmI8R+v+n3IQNEIXCYi784O/fsibH2/hfy48itMPzwy6JOnkNEIXCcjD/1rNsx8U8M3cQ7ly8tCgy5EYoEAXCcDfl23if19ZyXnjBvHdL44KuhyJEQp0kQ62tLCUbz+/mAlD0rn/krGYtXbbXpH9p0AX6UBFpdVc9+R8MlKTmKFT+iXCtFNUpINU1TVw/ZPzqalr5OnrJ5ORmhR0SRJjFOgiHcDduWXWEj4uLuOxa45hZGZa0CVJDNKUi0gH+NWb+cxZVsztZx3OqaP6B12OxCgFukg7e+XDYn7++ioumpDF9ScND7ociWEKdJF2tLK4nO88v5ijB6dzz4VjdESLtCsFukg72VlVz9Q/zqdHUjyPXj1RR7RIu1Ogi7SDxibnxucWUVRazSNXTSCzZ3LQJUkXoEAXaQe/eH0VeStLuOu8I5k4tE/Q5UgXoUAXibBXPizm1//M59KcwVw5eUjQ5UgXokAXiaA1JRXcMmsJY7N7cfcFR2onqHSoNgW6mU0xs5Vmlm9mt+2hTa6ZLTaz5Wb2r8iWKRL9quoa+MZTC0mIMx6+SjtBpePt80xRM4sDHgLOBAqBeWb2ort/1KJNOvAbYIq7bzAznTkhXYq7c8fsZazaUs6T104iS7eQkwC0ZYQ+Cch39zXuXgfMBC7Yrc0VwGx33wDg7lsiW6ZIdPvje+v5y+IivnPGSE4e2S/ocqSLasu1XLKAghbLhcDk3dqMBBLMLA9IAx509z/s/kZmNhWYCpCZmUleXt4BlBxZFRUVUVFHNFBfNNufvlhT2sj/vF/DuH5xHNmtkLy8je1bXAfT56JZtPdFWwK9tb063sr7TAROB1KAf5vZe+6+apcXuc8AZgDk5OR4bm7ufhccaXl5eURDHdFAfdGsrX1RWlXH93/5NgN6pfDkN04kvXti+xfXwfS5aBbtfdGWQC8EBrdYzgaKWmmz1d0rgUozmwuMA1YhEqOampzvPL+EkvJaZk07LibDXDqXtsyhzwNGmNlwM0sELgNe3K3NX4GTzCzezLoTmpJZEdlSRaLLw/9azZsfb+HOcw9n3OD0oMsR2fcI3d0bzGw68CoQBzzu7svNbFp4/SPuvsLMXgGWAk3A79z9w/YsXCRI76/Zxv+9Fron6NXH6gbPEh3adIMLd58DzNntuUd2W74fuD9ypYlEp60VtXzr2UUM7duDey/SFRQleuhMUZH90NTkfPu5xZRW1/PQFRNITdJNvyR6KNBF9sNv8vJ565Ot/Pd5R3LEoJ5BlyOyCwW6SBu9t2YbP399FeePG8Tlkwbv+wUiHUyBLtIG2ypquXFmaN78Hs2bS5RSoIvsw6fHm++oqufXV4zXvLlELQW6yD48OncN/1pVwg/OPYIjB/UKuhyRPVKgi+zFgvXb+dlrKzlnzECu0s0qJMop0EX2oLSqjm89s4hB6cnce7HmzSX6aTJQpBXuzi2zllJSUcufph1Pz+SEoEsS2SeN0EVa8cb6Bt5YsZlbp4zWdVqk01Cgi+xmWeFOnltZx+mj+3PdicODLkekzRToIi2U19Qz/dmF9EwyfvYf4zRvLp2KAl0kzN35/p8/pGB7FV8fm0TvHrq+uXQuCnSRsOfnF/DikiK+fcZIRvWJC7ockf2mQBcBVm0u564Xl3PCYX355qmHBV2OyAFRoEuXV13XyA1PLyQ1KZ5fXHo0cd00by6dk45Dly7v7peW88mWCv7wtUn0T0sOuhyRA6YRunRpf128kZnzCvhm7qGcPLJf0OWIHBQFunRZa0oquGP2MnKG9uY7Z44MuhyRg6ZAly6ppr6R6c8sIiG+G7+8fDzxcfpRkM5Pc+jSJd0zZwUfbSrj8WtyGJSeEnQ5IhGhYYl0OS8v3cQf/r2e/zxpOKeNzgy6HJGIUaBLl7JuayW3vrCU8UPS+d6U0UGXIxJRCnTpMmrqG7nhmYXEdTN+fcUEEjRvLjFGc+jSZfzPyytYXlTG776SQ5bmzSUGaYgiXcJLS4r443vrmXryIZxxhObNJTYp0CXmrS6p4LYXlpIztDff/eKooMsRaTdtCnQzm2JmK80s38xu20u7Y8ys0cwuiVyJIgeuuq6Rbz61kKSEOH51xXjNm0tM2+en28zigIeAs4AjgMvN7Ig9tPsp8GqkixQ5EO7OnX/5kFVbynnwsqMZ2Evz5hLb2jJcmQTku/sad68DZgIXtNLuW8ALwJYI1idywJ6fX8ALCwv5r9NGcNIIXadFYl9bjnLJAgpaLBcCk1s2MLMs4ELgNOCYPb2RmU0FpgJkZmaSl5e3n+VGXkVFRVTUEQ1iqS/W7WzkJ+/XcGTfboyL30heXtF+vT6W+uJgqS+aRXtftCXQW7s4tO+2/ABwq7s37u0ejO4+A5gBkJOT47m5uW0ss/3k5eURDXVEg1jpi9KqOu781dv0T0vmj988iT4HcCu5WOmLSFBfNIv2vmhLoBcCg1ssZwO7D3dygJnhMM8AzjazBnf/S0SqFGmjpibnpucWs6WsluenHXdAYS7SWbUl0OcBI8xsOLARuAy4omUDdx/+6ddm9gTwN4W5BOGXb35C3soSfvKlozh6cHrQ5Yh0qH0Gurs3mNl0QkevxAGPu/tyM5sWXv9IO9co0ib/WLGZB//xCRdNyOLKyUOCLkekw7Xp1H93nwPM2e25VoPc3a85+LJE9s+akgpumrmYIwb25J4Lx7C3fTkisUpnWUinV1HbwNf/uID4OOPRqyeSnBAXdEkigdDFuaRTc3e+O2sJq0sqeOq6yWT37h50SSKB0QhdOrWH/pnP3z8s5vazDuf4wzKCLkckUAp06bReW17Mz15bxYXjs7j+pOH7foFIjFOgS6f0cXEZ335uMeMGp3PvRdoJKgIKdOmEtlfW8Z9/mE+PpHhmaCeoyGe0U1Q6lbqGJqY9tYDNZbU8//XjyOyZHHRJIlFDI3TpNNyd22cv44O127n/krE6E1RkNwp06TR+k7eaFxYWctMZI7jg6KygyxGJOgp06RTmLNvE/a+u5IKjB3Hj6SOCLkckKinQJeotWL+Dbz+3mAlD0vnpxWN1RIvIHijQJaqtKang+ifnMbBXMr/9So6OaBHZCwW6RK2tFbVc8/t5mBlPXDuJvqlJQZckEtUU6BKVquoauO6JeWwpr+Gxr+YwLKNH0CWJRD0FukSduoYmvvHUQpZt3MkvLxvP+CG9gy5JpFPQiUUSVZqanFtmLeFfq0q476IxfOHIAUGXJNJpaIQuUcPduful5by4pIjvTRnFZZN01yGR/aFAl6jx4D8+4cl/r+f6E4fzjVMODbockU5HgS5R4dF/reaBNz7hkonZ3HH24TrWXOQAKNAlcE++u457//4x544dyE8vHku3bgpzkQOhQJdAPTdvA3e9uJwvHJHJLy49mjiFucgBU6BLYJ6fV8Bts5dxysh+/OqK8STE6eMocjD0EySBePaDDXzvhaWcNKIfj149kaR4ndIvcrAU6NLhnnpvPbfPXkbuqH6645BIBOnEIulQv39nLXe/9BGnje7Pw1dN0MhcJIIU6NIh3J1f/iOfX7yxii8emckvLx+vMBeJMAW6tDt35ycvr+Cxt9dy8YRsfnrxGOK1A1Qk4tr0U2VmU8xspZnlm9ltray/0syWhh/vmtm4yJcqnVF9YxPf/dNSHnt7LdccP4z7LxmrMBdpJ/scoZtZHPAQcCZQCMwzsxfd/aMWzdYCp7j7DjM7C5gBTG6PgqXzqKht4JtPL2TuqhJuPH0EN50xQmeAirSjtky5TALy3X0NgJnNBC4APgt0d3+3Rfv3gOxIFimdz5ayGq59Yh4fF5fz04vHcOkxutCWSHszd997A7NLgCnufn14+WpgsrtP30P7W4DRn7bfbd1UYCpAZmbmxJkzZx5k+QevoqKC1NTUoMuICpHqi4LyJh5YUENFvXPD0UmM7df5dtXoc9FMfdEsGvri1FNPXeDuOa2ta8tPWmt/I7f6W8DMTgWuA05sbb27zyA0HUNOTo7n5ua2YfPtKy8vj2ioIxpEoi/e+Ggz9725iB5Jifzp+mMYk90rMsV1MH0umqkvmkV7X7Ql0AuBwS2Ws4Gi3RuZ2Vjgd8BZ7r4tMuVJZ+HuzJi7hvte+ZijBvXit1/JYUCv5KDLEulS2hLo84ARZjYc2AhcBlzRsoGZDQFmA1e7+6qIVylRraqugdtnL+Ovi4s4Z8xAfvYf40hJ1DHmIh1tn4Hu7g1mNh14FYgDHnf35WY2Lbz+EeCHQF/gN+GjGBr2NMcjsWXd1kqmPbWAlZvLufnMkdxw6mG6/K1IQNq0t8rd5wBzdnvukRZfXw98bieoxLbXlhdz86wlxHUznrx2EieP7Bd0SSJdWuc7/EACV1PfyH1//5gn3l3HmKxePHzVBLJ7dw+6LJEuT4Eu+2V1SQXTn1nEik1lfO2E4dx61ihdk0UkSijQpU3cnafeW889cz4mOaEbj301h9MPzwy6LBFpQYEu+1RUWs2tLyzlrU+2ctKIDO6/ZJwOSRSJQgp02SN3Z9b8Qn788kc0Njk/+dJRXDl5iK7HIhKlFOjSqrVbK7l99lLeW7OdScP7cP8lYxnat0fQZYnIXijQZRc19Y38du4afvXPfJLiu3HvRWO4NGewji0X6QQU6AKEplcWbWngh7+Yy4btVZwzZiB3nXcE/Xtqrlyks1CgC6s2l3PPnBXkrazlsP6pPH39ZE44LCPoskRkPynQu7DinTX84vVVzFpQQI/EeC4blciPv3ISCbqjkEinpEDvgrZX1vHo3NU8+e46Gpuca44fzvTTDmPpvHcV5iKdmAK9CymtquO3b63hiXfWUVXfyPnjBnHzmaMY0len7YvEAgV6F7BpZzWPvbWWZz7YQFVdI+eOHciNp49gRGZa0KWJSAQp0GPYR0VlPP7OWv66eCNNDueNHci03EMZPaBn0KWJSDtQoMeYhsYm3lixmd+/s473124nOaEbV04eynUnDmdwH02tiMQyBXqMKNxRxfPzCnhufgGby2rJSk/hjrNHc2nOEHp1Twi6PBHpAAr0TqyqroHXlm/mhYWFvJ2/FYBTRvbjRxcM4YzDM4nT2Z0iXYoCvZOpa2jinfytvLS0iFc+LKaqrpGs9BS+ddoIvpyTrRtNiHRhCvROoLqukXfyt/LK8mJeW15MWU0DacnxnD9uEBeOz+KYYX10rRURUaBHq6LSauauKuGNFVt4O7+Emvom0pLj+cIRAzhn7ABOOCxDdwoSkV0o0KNERW0D89Zu5+38rcxdVcInWyoAyEpP4bJjQnPik4b3ITFeZ3KKSOsU6AEprapjwfodzFu3g/fXbmNp4U4am5zEuG5MGt6HL+cM5pRR/RjRP1U3lBCRNlGgd4C6hiZWbS5nUUEpSwpKWVxQSn54BJ4QZ4zJ6sU3TjmU4w7ty8ShvUlO0FSKiOw/BXqE7ayq5+PiMj4uLmfFpjI+LNrJyuJy6hsdgL49Ehk3OJ0Lx2eRM7Q34wanK8BFJCIU6AegqcnZXF7D2pJK1mytJH9LxWeP4rKaz9qld09gTFYvrjvxEI7K6sm47HSye6doCkVE2oUCfQ/Kauop3F7NxtJqNu6oomBHNRu2V7FhWxXrt1dSU9/0WdseiXEc2j+V4w/ty6gBaYwakMboAT3J7Jmk8BaRDtPlAr2qroGS8lq2VtRSUl7H2+vrmf/qSjaX1VBcVsOmnTUU76yhorZhl9elJMQxpE93BvfpzokjMhie0eOzx8BeyQpuEc2tcsAAAAYJSURBVAlcpwx0d6emvonymnrKaurZWd0Q+reqnp3Vocf2yjpKq+rYXlXPjso6tlfWsa2ydpeR9afiVq6mX2oSmT2TOKxfKicelsHAXslk9+5OVu8UstJTyEhNVGiLSFRrU6Cb2RTgQSAO+J2737fbeguvPxuoAq5x94V7e8/ymnr+trSI6rpGauobqa5vpLquiar6BqrrGqmsbaSqroGK2gYqaxuoqmukoja0XFHTQEOT77XmtKR40nsk0Kd7In1TExmRmUrfHon06ZFEv7QkMlITyUhNYvWHCzn3zFxd90REOr19BrqZxQEPAWcChcA8M3vR3T9q0ewsYET4MRl4OPzvHq3bVsX0ZxZ97vnE+G70SIyje2I83RPj6J4UT2pSHBmpSaQmxZOWHE9qcjypSQn0TIknLTmBnsnx9ExJoFdKAukpCfRMSWjzrdS2fmIKcxGJCW0ZoU8C8t19DYCZzQQuAFoG+gXAH9zdgffMLN3MBrr7pj296WH9Uvnzt08mJSGOpIRudE+MJyUhTuEqInKA2hLoWUBBi+VCPj/6bq1NFrBLoJvZVGAqQGZmJkUrFuxvvRFXUVFBXl5e0GVEBfVFM/VFM/VFs2jvi7YEemtD5t0nsNvSBnefAcwAyMnJ8dzc3DZsvn3l5eURDXVEA/VFM/VFM/VFs2jvi7ZMNBcCg1ssZwNFB9BGRETaUVsCfR4wwsyGm1kicBnw4m5tXgS+YiHHAjv3Nn8uIiKRt88pF3dvMLPpwKuEDlt83N2Xm9m08PpHgDmEDlnMJ3TY4rXtV7KIiLSmTcehu/scQqHd8rlHWnztwA2RLU1ERPaH7pYgIhIjFOgiIjFCgS4iEiMU6CIiMUKBLiISIxToIiIxQoEuIhIjFOgiIjFCgS4iEiMU6CIiMUKBLiISIxToIiIxwkLX1Qpgw2YlwPpANr6rDGBr0EVECfVFM/VFM/VFs2joi6Hu3q+1FYEFerQws/nunhN0HdFAfdFMfdFMfdEs2vtCUy4iIjFCgS4iEiMU6OGbVgugvmhJfdFMfdEsqvuiy8+hi4jECo3QRURihAJdRCRGKNDDzOwWM3Mzywi6lqCY2f1m9rGZLTWzP5tZetA1dTQzm2JmK80s38xuC7qeoJjZYDP7p5mtMLPlZnZj0DUFzczizGyRmf0t6Fr2RIFO6MMLnAlsCLqWgL0OHOXuY4FVwO0B19OhzCwOeAg4CzgCuNzMjgi2qsA0ADe7++HAscANXbgvPnUjsCLoIvZGgR7yC+B7QJfeQ+zur7l7Q3jxPSA7yHoCMAnId/c17l4HzAQuCLimQLj7JndfGP66nFCQZQVbVXDMLBs4B/hd0LXsTZcPdDM7H9jo7kuCriXKfA34e9BFdLAsoKDFciFdOMQ+ZWbDgPHA+8FWEqgHCA36moIuZG/igy6gI5jZG8CAVlZ9H7gD+ELHVhScvfWFu/813Ob7hP7kfroja4sC1spzXfqvNjNLBV4AbnL3sqDrCYKZnQtscfcFZpYbdD170yUC3d3PaO15MxsDDAeWmBmEphgWmtkkdy/uwBI7zJ764lNm9lXgXOB073onKRQCg1ssZwNFAdUSODNLIBTmT7v77KDrCdAJwPlmdjaQDPQ0s6fc/aqA6/ocnVjUgpmtA3LcPeirqQXCzKYAPwdOcfeSoOvpaGYWT2hn8OnARmAecIW7Lw+0sABYaITzJLDd3W8Kup5oER6h3+Lu5wZdS2u6/By67OLXQBrwupktNrNHgi6oI4V3CE8HXiW0E/D5rhjmYScAVwOnhT8Li8MjVIliGqGLiMQIjdBFRGKEAl1EJEYo0EVEYoQCXUQkRijQRURihAJdRCRGKNBFRGKEAl0kzMymtTiJZq2Z/TPomkT2h04sEtlN+BombwL/6+4vBV2PSFtphC7yeQ8CbyrMpbPpEldbFGkrM7sGGEromi4inYqmXETCzGwioSsMnuTuO4KuR2R/acpFpNl0oA/wz/CO0ai+3ZjI7jRCFxGJERqhi4jECAW6iEiMUKCLiMQIBbqISIxQoIuIxAgFuohIjFCgi4jEiP8HMZTZOG7U54AAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_0.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAZwElEQVR4nO3dfZBddX3H8fcnS6JoohTTLOQBQiV1pJag2QYdaNlIQwNFUzu2hbaAD3SHjmm1LbW0dpSZPlFt1VqwcYdmBCvE2pKatpEQhVtqKZqAISSBhG1Es2w0JURhfWgM++0f52zudXN3czf3lz33eD6vmZ2955zfueeb32zu557feVJEYGZm1TWt6ALMzKxYDgIzs4pzEJiZVZyDwMys4hwEZmYV5yAwM6s4B4HZJEh6haQvS3pO0m9P4XbPkDQsqWuqtmnV4SCwHyqSbpT0DydwE+8GahExKyI+cqI2IulJST87Oh0RX4uImRHx/InaplWXg8Bscs4EdhRdhFlKDgIrJUl/IOmpfIhml6SLJa0A/gj4lXwY5ZG87Usl/b2kffk6fzo6xCLpLZL+S9LfSvqWpMclXTzONu8FlgE35+//45Jqkq5taPMWSV9omA5J10l6QtJBSbdIUsPy35D0WP7v2CnpNZI+AZwB/Gu+nXdLWpi/10n5enMlrZf0jKQBSb/R8J43SvpHSbfn77tDUk/K/rcfLg4CKx1JrwBWAT8VEbOAnwOejIi7gT8HPpUPoyzOV7kNOAycDbwauAS4tuEtzwf2ALOB9wF3STp17HYj4vXAfwKr8vff3WLJlwM/BSwGfjmvF0m/BNwIXA28BHgjcCAirgK+Brwh3877m7znncAgMBd4M/DnYwLsjcBa4BRgPXBzi7VaBTkIrIyeB14AnCNpekQ8GRH/06yhpG7gUuBdEfHtiNgPfAi4oqHZfuDDEfH9iPgUsAv4+YT13hQR34yIrwH3Aefl868F3h8RmyMzEBFfPdabSVoAXAj8QUR8LyK2ArcCVzU0+0JEbMiPKXyCLITMmnIQWOlExADwLrJv0/slrZU0d5zmZwLTgX2Svinpm8DHgDkNbZ6KH7z74lfJvmmn8vWG198BZuavFwBNA+wY5gLPRMRzDfO+CsybYJsvHB1WMhvLQWClFBF3RMSFZB/0Afzl6KIxTfcC/wfMjohT8p+XRMRPNLSZ1zhuTzY+P9RiKd8GXtQwfVrL/4istpePs2yi2wIPAadKmtUw7wzgqUls2+wIB4GVTn4u/+slvQD4HvBdsuEigG8ACyVNA4iIfcA9wF9LeomkaZJeLumihrecA/y2pOn5uP0rgQ0tlrMV+EVJL5J0NvD2SfxTbgWul7REmbMlndnw7/ixZitFxF7gAeAvJL1Q0rn5dj85iW2bHeEgsDJ6AXAT8DTZEMgcsrOFAD6d/z4g6eH89dXADGAncBD4J+D0hvf7IrAof78/A94cEQdarOVDwCGyD+7bmMSHcUR8Ot/eHcBzwL8Aowep/wL443w46/omq18JLCTbO1gHvC8iNrW6bbNG8oNprMokvQW4Nh9mMqsk7xGYmVVckiCQtEbSfknbx1nem1+sszX/eW/DshX5BUEDkm5IUY+ZmbUuydCQpJ8BhoHbI+JVTZb3AtdHxOVj5ncBu4HlZBfHbAaujIidbRdlZmYtSbJHEBH3A88cx6pLgYGI2BMRh8iuhFyZoiYzM2vNVF5g8rr83i9DZHsHO8gugNnb0GaQ7HL/o0jqA/oATj755CULFiw4weVObGRkhGnTfIgF3BeN3Bd17ou6TumL3bt3Px0RPzp2/lQFwcPAmRExLOkystPkFgFq0rbpWFVE9AP9AD09PbFly5YTVWtLarUavb29hdbQKdwXde6LOvdFXaf0haSmtzCZkoiKiGcjYjh/vQGYLmk22R5A41f7+bR+RaeZmSUwJUEg6bTRS/glLc23e4Ds4PAiSWdJmkF2I7D1U1GTmZllkgwNSboT6AVmSxoku5XvdICIWE12m9zflHSY7HYAV+Q3+TosaRWwEegC1uTHDszMbIokCYKIuPIYy29mnPuh50NFrd7XxczMEiv+MLaZmRXKQWBmVnEOAjOzinMQmJlVnIPAzKziHARmZhXnIDAzqzgHgZlZxTkIzMwqzkFgZlZxDgIzs4pzEJiZVZyDwMys4hwEZmYV5yAwM6s4B4GZWcU5CMzMKs5BYGZWcUmCQNIaSfslbR9n+a9J2pb/PCBpccOyJyU9KmmrpC0p6jEzs9al2iP4OLBiguVfAS6KiHOBPwH6xyxfFhHnRURPonrMzKxFqR5ef7+khRMsf6Bh8kFgfortmplZ+4o4RvB24LMN0wHcI+khSX0F1GNmVmmKiDRvlO0R/FtEvGqCNsuAjwIXRsSBfN7ciBiSNAfYBPxWRNzfZN0+oA+gu7t7ydq1a5PUfbyGh4eZOXNmoTV0CvdFnfuizn1R1yl9sWzZsoeaDcEnGRpqhaRzgVuBS0dDACAihvLf+yWtA5YCRwVBRPSTH1vo6emJ3t7eqSh7XLVajaJr6BTuizr3RZ37oq7T+2JKhoYknQHcBVwVEbsb5r9Y0qzR18AlQNMzj8zM7MRIskcg6U6gF5gtaRB4HzAdICJWA+8FXgZ8VBLA4Xz3pBtYl887CbgjIu5OUZOZmbUm1VlDVx5j+bXAtU3m7wEWH72GmZlNFV9ZbGZWcQ4CM7OKcxCYmVWcg8DMrOIcBGZmFecgMDOrOAeBmVnFOQjMzCrOQWBmVnEOAjOzinMQmJlVnIPAzKziHARmZhXnIDAzqzgHgZlZxTkIzMwqzkFgZlZxDgIzs4pzEJiZVVySIJC0RtJ+SdvHWS5JH5E0IGmbpNc0LFshaVe+7IYU9ZiZWeuSPLwe+DhwM3D7OMsvBRblP+cDfwecL6kLuAVYDgwCmyWtj4idieoym1IRwfe+/3zRZXSEQ8+7L0Z1Ql9I4y9LEgQRcb+khRM0WQncHhEBPCjpFEmnAwuBgYjYkxWqtXlbB4GV0u07D/HWjXcXXUbn2OS+OKLgvlg0Z+a4y1LtERzLPGBvw/RgPq/Z/PObvYGkPqAPoLu7m1qtdkIKbdXw8HDhNXQK90XdU89+n5e9cBrLzpiq/1qd69D/HWLGC2YUXUZH6IS+mDX9EJ8bZ9lU/bU22ymJCeYfPTOiH+gH6Onpid7e3mTFHY9arUbRNXQK90XdTV/6LC8/7RT+6m2vK7qUwvnvoq5T+uLGq5vPn6ogGAQWNEzPB4aAGePMNyulGO/rjVkHm6rTR9cDV+dnD70W+FZE7AM2A4sknSVpBnBF3taslAKY5iCwkkmyRyDpTqAXmC1pEHgfMB0gIlYDG4DLgAHgO8Bb82WHJa0CNgJdwJqI2JGiJrOiTJvo9AyzDpTqrKErj7E8gHeMs2wDWVCYld5ITHyanlkn8pXFZol5j8DKxkFgltBI03PezDqbg8AsoexgsfcIrFwcBGYphc8asvJxEJglNALIewRWMg4Cs8S8R2Bl4yAwSyg7WOwksHJxEJgl5j0CKxsHgVlCEeELyqx0HARmCY3g00etfBwEZimFg8DKx0FgltAI+FixlY6DwCwl7xFYCTkIzBLyc2msjBwEZgn5wTRWRg4Cs4QifIsJKx8HgVlCfjCNlZGDwCwxHyy2skkSBJJWSNolaUDSDU2W/76krfnPdknPSzo1X/akpEfzZVtS1GNWFB8stjJq+5nFkrqAW4DlwCCwWdL6iNg52iYiPgB8IG//BuB3IuKZhrdZFhFPt1uLWdHCp49aCaXYI1gKDETEnog4BKwFVk7Q/krgzgTbNes4gY8RWPm0vUcAzAP2NkwPAuc3ayjpRcAKYFXD7ADukRTAxyKif5x1+4A+gO7ubmq1WvuVt2F4eLjwGjqF+6JuZGSEr+/bR632zLEb/5Dz30Vdp/dFiiBo9v1nvEd4vwH4rzHDQhdExJCkOcAmSY9HxP1HvWEWEP0APT090dvb22bZ7anVahRdQ6dwXzS499+ZO28uvb0/WXQlhfPfRV2n90WKoaFBYEHD9HxgaJy2VzBmWCgihvLf+4F1ZENNZqUUfmaxlVCKINgMLJJ0lqQZZB/268c2kvRS4CLgMw3zXixp1uhr4BJge4KazAqRXVnsJLByaXtoKCIOS1oFbAS6gDURsUPSdfny1XnTNwH3RMS3G1bvBtblV2KeBNwREXe3W5NZUXz6qJVRimMERMQGYMOYeavHTH8c+PiYeXuAxSlqMOsEvsWElZGvLDZLyKePWhk5CMwS8gVlVkYOArOEfIzAyshBYJZQBEzz+aNWMg4Cs4R8jMDKyEFgllAEyINDVjIOArOE/KhKKyMHgVlCHhqyMnIQmCXk00etjBwEZgn59FErIweBWSIR2d3XfYsJKxsHgVkieQ54aMhKx0FglsjIkT2CggsxmyQHgVkio4/l8+mjVjYOArNERnyMwErKQWCWyOgxAueAlY2DwCyRI0HgE0itZBwEZomMDg35GIGVTZIgkLRC0i5JA5JuaLK8V9K3JG3Nf97b6rpmZVE/WOwksHJp+5nFkrqAW4DlwCCwWdL6iNg5pul/RsTlx7muWcfz6aNWVin2CJYCAxGxJyIOAWuBlVOwrllHqR8sdhJYubS9RwDMA/Y2TA8C5zdp9zpJjwBDwPURsWMS6yKpD+gD6O7uplartV95G4aHhwuvoVO4LzLDh7Ik+J+BAWqHv1pwNcXz30Vdp/dFiiBo9vUnxkw/DJwZEcOSLgP+BVjU4rrZzIh+oB+gp6cnent7j7vgFGq1GkXX0CncF5lnvn0I7t3Ejy86m94Lziq6nML576Ku0/sixdDQILCgYXo+2bf+IyLi2YgYzl9vAKZLmt3KumZlMXrTOT+z2MomRRBsBhZJOkvSDOAKYH1jA0mnKR84lbQ03+6BVtY1K4uRI9cRmJVL20NDEXFY0ipgI9AFrImIHZKuy5evBt4M/Kakw8B3gSsi+/rUdN12azIrQuBbTFg5pThGMDrcs2HMvNUNr28Gbm51XbMy8i0mrKx8ZbFZIn4egZWVg8AskSMXlBVch9lkOQjMEqnfa8hRYOXiIDBLxMcIrKwcBGaJ+BYTVlYOArNERk8f9fVkVjYOArNERjw0ZCXlIDBLJHyw2ErKQWCWyEjT2yWadT4HgVki3iOwsnIQmCXiR1VaWTkIzBLxoyqtrBwEZonU7zVUbB1mk+UgMEtkdI/AdxuysnEQmCXiPQIrKweBWSK+xYSVlYPALJH63UcLLsRskhwEZon49FErqyRBIGmFpF2SBiTd0GT5r0nalv88IGlxw7InJT0qaaukLSnqMSvCkYPFzgErmbafWSypC7gFWA4MApslrY+InQ3NvgJcFBEHJV0K9APnNyxfFhFPt1uLWZH8qEorqxR7BEuBgYjYExGHgLXAysYGEfFARBzMJx8E5ifYrllHCT+q0kqq7T0CYB6wt2F6kB/8tj/W24HPNkwHcI+kAD4WEf3NVpLUB/QBdHd3U6vV2qm5bcPDw4XX0CncF5ndB58H4NFt2xgZ6iq4muL576Ku0/siRRA0+wLU9D6MkpaRBcGFDbMviIghSXOATZIej4j7j3rDLCD6AXp6eqK3t7ftwttRq9UouoZO4b7InLznAHzxQc47bzEXnD276HIK57+Luk7vixRDQ4PAgobp+cDQ2EaSzgVuBVZGxIHR+RExlP/eD6wjG2oyKx0/mMbKKkUQbAYWSTpL0gzgCmB9YwNJZwB3AVdFxO6G+S+WNGv0NXAJsD1BTWZTrv6oSieBlUvbQ0MRcVjSKmAj0AWsiYgdkq7Ll68G3gu8DPhoftXl4YjoAbqBdfm8k4A7IuLudmsyK4LPHrWySnGMgIjYAGwYM291w+trgWubrLcHWDx2vlkZHTl91JcWW8n4ymKzREZ8+qiVlIPALJEjN6H2MQIrGQeBWSJ+QpmVlYPALBE/vN7KykFglogfTGNl5SAwS+TIBWU+XGwl4yAwSyR8jMBKykFglohvMWFl5SAwS8QHi62sHARmidSvIyi0DLNJcxCYJTLiPQIrKQeBWSI+fdTKykFglsiRh9f79FErGQeBWWLeI7CycRCYJVK/15CTwMrFQWCWyMhI9tt7BFY2DgKzREaPEPisISsbB4FZIvWDxWblkiQIJK2QtEvSgKQbmiyXpI/ky7dJek2r65qVhh9VaSXVdhBI6gJuAS4FzgGulHTOmGaXAovynz7g7yaxrlkp+FGVVlYpHl6/FBjIH0SPpLXASmBnQ5uVwO2R3YzlQUmnSDodWNjCukd59rvf554dX09Q+vHb/o3DHCq4hk7hvsjsGHoW8DECK58UQTAP2NswPQic30KbeS2uC4CkPrK9CWacdjZ9n3iovapT+HIH1NAp3BcATFPw8Jf+mxdNdxgMDw9Tq9WKLqMjdHpfpAiCZn/xY4+ajdemlXWzmRH9QD/ATyx+dXzqty6cTI3JPfTQFpYs6Sm0hk7hvqjbte0hLlu+rOgyOkKtVqO3t7foMjpCp/dFiiAYBBY0TM8HhlpsM6OFdY9y8vQuXjXvpcdVbCpPP1F8DZ3CfVH39BM+Ec/KJ8Vf7WZgkaSzJM0ArgDWj2mzHrg6P3votcC3ImJfi+uamdkJ1PYeQUQclrQK2Ah0AWsiYoek6/Llq4ENwGXAAPAd4K0TrdtuTWZm1roUQ0NExAayD/vGeasbXgfwjlbXNTOzqeMBTTOzinMQmJlVnIPAzKziHARmZhXnIDAzqzgHgZlZxTkIzMwqzkFgZlZxDgIzs4pzEJiZVZyDwMys4hwEZmYV5yAwM6s4B4GZWcU5CMzMKs5BYGZWcQ4CM7OKcxCYmVVcW0Eg6VRJmyQ9kf/+kSZtFki6T9JjknZIemfDshslPSVpa/5zWTv1mJnZ5LW7R3AD8PmIWAR8Pp8e6zDwexHxSuC1wDskndOw/EMRcV7+42cXm5lNsXaDYCVwW/76NuAXxjaIiH0R8XD++jngMWBem9s1M7NE2g2C7ojYB9kHPjBnosaSFgKvBr7YMHuVpG2S1jQbWjIzsxNLETFxA+lzwGlNFr0HuC0iTmloezAimn6YS5oJ/AfwZxFxVz6vG3gaCOBPgNMj4m3jrN8H9AF0d3cvWbt27TH+aSfW8PAwM2fOLLSGTuG+qHNf1Lkv6jqlL5YtW/ZQRPQctSAijvsH2EX24Q1wOrBrnHbTgY3A707wXguB7a1sd8mSJVG0++67r+gSOob7os59Uee+qOuUvgC2RJPP1HaHhtYD1+SvrwE+M7aBJAF/DzwWER8cs+z0hsk3AdvbrMfMzCap3SC4CVgu6QlgeT6NpLmSRs8AugC4Cnh9k9NE3y/pUUnbgGXA77RZj5mZTdJJ7awcEQeAi5vMHwIuy19/AdA461/VzvbNzKx9vrLYzKziHARmZhXnIDAzqzgHgZlZxTkIzMwqzkFgZlZxDgIzs4pzEJiZVZyDwMys4hwEZmYV5yAwM6s4B4GZWcU5CMzMKs5BYGZWcQ4CM7OKcxCYmVWcg8DMrOIcBGZmFecgMDOruLaCQNKpkjZJeiL//SPjtHsyf0j9VklbJru+mZmdOO3uEdwAfD4iFgGfz6fHsywizouInuNc38zMToB2g2AlcFv++jbgF6Z4fTMza5Mi4vhXlr4ZEac0TB+MiKOGdyR9BTgIBPCxiOifzPr5sj6gL598BbDruAtPYzbwdME1dAr3RZ37os59UdcpfXFmRPzo2JknHWstSZ8DTmuy6D2T2PgFETEkaQ6wSdLjEXH/JNYnD4/+yaxzIknaMmaYq7LcF3Xuizr3RV2n98UxgyAifna8ZZK+Ien0iNgn6XRg/zjvMZT/3i9pHbAUuB9oaX0zMztx2j1GsB64Jn99DfCZsQ0kvVjSrNHXwCXA9lbXNzOzE6vdILgJWC7pCWB5Po2kuZI25G26gS9IegT4EvDvEXH3ROuXRMcMU3UA90Wd+6LOfVHX0X3R1sFiMzMrP19ZbGZWcQ4CM7OKcxAkIOl6SSFpdtG1FEXSByQ9LmmbpHWSTjn2Wj9cJK2QtEvSgKTKXiUvaYGk+yQ9JmmHpHcWXVPRJHVJ+rKkfyu6lmYcBG2StIDsQPfXiq6lYJuAV0XEucBu4A8LrmdKSeoCbgEuBc4BrpR0TrFVFeYw8HsR8UrgtcA7KtwXo94JPFZ0EeNxELTvQ8C7ya6arqyIuCciDueTDwLzi6ynAEuBgYjYExGHgLVkt1CpnIjYFxEP56+fI/sAnFdsVcWRNB/4eeDWomsZj4OgDZLeCDwVEY8UXUuHeRvw2aKLmGLzgL0N04NU+MNvlKSFwKuBLxZbSaE+TPZlcaToQsZzzCuLq+4Yt9j4I7IL5Cphor6IiM/kbd5DNjTwyamsrQOoybxK7yVKmgn8M/CuiHi26HqKIOlyYH9EPCSpt+h6xuMgOIbxbrEh6SeBs4BHJEE2FPKwpKUR8fUpLHHKTHS7EQBJ1wCXAxdH9S5QGQQWNEzPB4YKqqVwkqaThcAnI+Kuousp0AXAGyVdBrwQeImkf4iIXy+4rh/gC8oSkfQk0BMRnXCHwSknaQXwQeCiiPjfouuZapJOIjtIfjHwFLAZ+NWI2FFoYQVQ9s3oNuCZiHhX0fV0inyP4PqIuLzoWsbyMQJL5WZgFtndZbdKWl10QVMpP1C+CthIdnD0H6sYArkLgKuA1+d/C1vzb8TWobxHYGZWcd4jMDOrOAeBmVnFOQjMzCrOQWBmVnEOAjOzinMQmJlVnIPAzKziHARmCUi6ruHiqa9Iuq/omsxa5QvKzBLK77FzL/D+iPjXousxa4X3CMzS+hvgXoeAlYnvPmqWiKS3AGeS3XPIrDQ8NGSWgKQlZHfc/OmIOFh0PWaT4aEhszRWAacC9+UHjDv2sYRmY3mPwMys4rxHYGZWcQ4CM7OKcxCYmVWcg8DMrOIcBGZmFecgMDOrOAeBmVnF/T/SYBgXLr/wKAAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_1.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3xV1Z338c8vd0i4BTBcBdRA8QYaBLHVkiqOUltsp52qrcV2fBj7qn1qZzpTbZ2OM/N0Hqd9pjPtU6fWKq2dsaWdemNaKlULUq2AgCAgBkIQCLdACCThktv5zR9nY2M4uZBzkn0u3/frdV5n773WOvu3CDm/7LX3XtvcHRERyVxZYQcgIiLhUiIQEclwSgQiIhlOiUBEJMMpEYiIZDglAhGRDKdEIBnPzNzMLuhhXTOzH5lZnZmt6evYOuz7N2a2oD/3KZkhJ+wARM6Wmb0N3OnuL4Sw+/cBc4Fx7n68r3ZiZg8AF7j7p05vc/cb+2p/ktl0RCBydiYAb/dlEhDpb0oEklLM7D+Ac4H/NrNGM/ubYPt/mdkBMztmZivN7KJ2bX5sZg+Z2a/NrMHMVpvZ+R0++joz2x4M+TxkZhZj338OPArMDvb992Z2h5m93KHeO0NN3e3bzC4ys+fN7IiZHTSzr5rZDcBXgU8E+9kY1F1hZncGy1lmdr+Z7TKzGjP7iZkNCcomBjEsMLPdZnbYzL4W9z++pC0lAkkp7n47sBv4kLsXufs3g6LfAKXAOcB64IkOTW8F/h4YBlQC3+hQfhNwBTAN+DPgT2Ls+zHgLuDVYN9/18OwY+7bzAYBLwDPAWOAC4AX3f054J+Anwf7mRbjM+8IXuXAeUAR8L0Odd4HTAGuBb5uZlN7GK9kGCUCSQvuvsjdG9y9CXgAmHb6L+TAU+6+xt1biSaJ6R0+4kF3P+ruu4HlMcrj0dm+bwIOuPu/uPupIP7VPfzMTwLfdvcqd28E7gNuMbP25/3+3t1PuvtGYCPRJCdyBiUCSXlmlm1mD5rZDjOrB94Oika0q3ag3fIJon9Bcxbl8ejss8cDO3r5mWOAXe3WdxG9+KOkB/sVeRclAklFHafMvQ2YD1wHDAEmBtvPGOfvA8eBgadXzGzUWbTdA3Q8V3Fad9MC7yN64vq0c4FW4OBZ7F8EUCKQ1HSQ6Lj4aYOAJqCW6JfyP/VjLBuBi8xsupkVEB2W6qlfAaPM7B4zyzezQWY2Kyg7CEw0s85+R38GfMnMJplZEX88p9Day35IBlMikFT0f4H7zeyomX0Z+AnRoZG9wJvAqv4KxN23Af9A9KTvduDlrlu8q20D0XsSPkR0GGc70ZO/AP8VvNea2foYzRcB/wGsBHYCp4Av9KILIpgeTCMiktl0RCAikuESkgjMbFFwU8vmTsrNzL5rZpVm9oaZXd6u7AYzqwjK7k1EPCIi0nOJOiL4MXBDF+U3Er3ZpxRYCHwfopf9AQ8F5RcCt5rZhQmKSUREeiAhicDdVwJHuqgyH/iJR60ChprZaGAmUBncFNMMLA7qiohIP+mv2UfHEr1m+rTqYFus7bOIwcwWEj2aYMCAAWXjx4+PuaNIJEJWVvqd+kjHfqlPqaO3/WpzaItAmzsRj65Hzng5TnTZPXoDxTvvie5IhirIhlGFWWzbtu2wu4/sWN5fiSDWjT3exfYzN7o/AjwCMGPGDF+7dm3MHa1YsYI5c+b0Lsoklo79Up9SR2f9Ot7UyvaaRnbUNLKr9jhv155g79GT7D96koMNTbRFor/OWcHr9BfOoPwcBhXkMKggl8L8bArzcxiQm83AvGwG5GWTn5NNfm5W9D0ni7zsLPJyssjJNnKzs8jNNrKzssjNMrI7vszIyjKyzMjOAjPDgOwswzDMIMuMdevWcsUVM9ptA4Ll6NIf2wLttv/xa+vMqQljbyP4rK7Ec/fj6Y9+9dVXmT179hnledlZDC/Kx8x2nVFI/yWCaqK30582juidkXmdbBeRJNPU2saG3UfZsOcoG6uPsmnvMfYcOflOeZbBmKEDGD9sILPPH8HoIQWUDM5n5KB8hhflU1yYx7CBeQwuyCEnO/yjppptWbxn1OCww0io4oIsRg8ZcNbt+isRLAHuNrPFRId+jrn7fjM7BJSa2SSiNwPdQnS6ABFJApU1jTz/5kGWrDlJ1Qu/pak1AsD44gFcOnYof1Y2nsmjBnHBOUWMHzaQvJzwv+Dl7CUkEZjZz4A5wAgzqwb+DsgFcPeHgaXAPKJT8J4APhOUtZrZ3cAyIBtY5O5bEhGTiPTO7toTPLm+mv/euI+qw9Hn74wflMUnZ01g9vnDKZswjOLCvJCjlERKSCJw91u7KXfg852ULSWaKEQkJJGI8+JbNSx6eSevVtViBrPPG84d753IdVNL2LZhNXPm6MrudKVnFotksLaI89T6ar6/YgdVh48zdugAvnz9ZD5y+TjGDv3jWPO2EGOUvqdEIJKB3J0VFYd48DdvUXGwgYvHDua7t17GvItHJcWJXOlfSgQiGeZg/Snuf2Yzz795kInDB/LQbZcz75JR3V7eKOlLiUAkQ7g7v1xXzT/86k2aWyPcd+N7+Oz7JpGrI4CMp0QgkgFOtbRx/zOb+eW6amZOLOafP3Ypk0YUhh2WJAklApE0t+fICT73xDo2763nf19byj3XlpKVpWEg+SMlApE0VnGggU89tpqmljYeWzCDa6eWdN9IMo4SgUiaen13HXf86DUKcrN48nNXUVoyKOyQJEkpEYikobVvH+HTi9YwclA+//nnsxhfPDDskCSJKRGIpJntBxv488fXUjK4gJ8vvJJzBheEHZIkOV03JpJG9h87yYJFa8jLyeInn52pJCA9okQgkiZONLfymR+9Rv2pVn78mSs0HCQ9pqEhkTTg7tz/9GYqDjbw+GdmctGYIWGHJClERwQiaeDnr+3hqdf3cs+1k7lm8hlPIhTpkhKBSIrbsu8YX1+yhatLR/CFD1wQdjiSgpQIRFJYU2sb9yzewLCBufzbJ6brjmHpFZ0jEElh/758B9trGvnRHVcwvCg/7HAkRemIQCRFVRxo4N9XVHLz9DGUv+ecsMORFJaQRGBmN5hZhZlVmtm9Mcr/2sw2BK/NZtZmZsVB2dtmtikoW5uIeETSXVvE+Zsn32BQQS5f/9BFYYcjKS7uoSEzywYeAuYC1cBrZrbE3d88XcfdvwV8K6j/IeBL7n6k3ceUu/vheGMRyRQ/W7ObjXuO8p1bputB8hK3RBwRzAQq3b3K3ZuBxcD8LurfCvwsAfsVyUgNp1r4txe2MXNSMR+eNibscCQNJCIRjAX2tFuvDradwcwGAjcAT7bb7MBvzWydmS1MQDwiae0HL1VxuLGZr82bqsdLSkKYu8f3AWYfB/7E3e8M1m8HZrr7F2LU/QTwKXf/ULttY9x9n5mdAzwPfMHdV8ZouxBYCFBSUlK2ePHimPE0NjZSVFQUV5+SUTr2S306e0dORfjKypOUlWRz17T+m0dIP6vU0F2fysvL17n7jDMK3D2uFzAbWNZu/T7gvk7qPg3c1sVnPQB8ubt9lpWVeWeWL1/eaVkqS8d+qU9n7y9/vsFLv7bU9xw53qf76Ug/q9TQXZ+AtR7jOzURQ0OvAaVmNsnM8oBbgCUdK5nZEOD9wLPtthWa2aDTy8D1wOYExCSSdiprGnnq9Wo+c9VExg3ThHKSOHFfNeTurWZ2N7AMyAYWufsWM7srKH84qPoR4Lfufrxd8xLg6WCcMwf4qbs/F29MIuno+yt2kJ+TxcJrzgs7FEkzCbmz2N2XAks7bHu4w/qPgR932FYFTEtEDCLpbM+REzyzYS+fnj1BdxBLwunOYpEU8IOVO8gydDQgfUKJQCTJ1dSf4hdrq/nTy8cxesiAsMORNKREIJLkHnt5J61tEe56//lhhyJpSolAJImdbG7jZ2t2c8PFo5g4ojDscCRNKRGIJLElG/dSf6qVBbMnhh2KpDElApEk5e48/oddTCkZxMxJxWGHI2lMiUAkSa3fXceb++v59FUTNKeQ9CklApEk9ZNXdzEoP4ebp8ecw1EkYZQIRJLQoYYmlm7az8dmjKMwX0+Ulb6lRCCShJ5aX01Lm/OpKyeEHYpkACUCkSTj7jy5vprLzx3K+SPTa5pkSU5KBCJJZtPeY2w72Miflo0LOxTJEEoEIknmyXXV5OVkcdOlegyl9A8lApEk0twaYcnGfcy9sIQhA3LDDkcyhBKBSBL53Vs11J1o4WOXa1hI+o8SgUgSeXJ9NSMH5XN16YiwQ5EMokQgkiSOnWhhRUUN86eNISdbv5rSf/S/TSRJLHvzAC1tzk3TdJJY+pcSgUiSWLppP2OHDmDauCFhhyIZJiGJwMxuMLMKM6s0s3tjlM8xs2NmtiF4fb2nbUUywbETLbxSeZh5l4zSBHPS7+KexMTMsoGHgLlANfCamS1x9zc7VP29u9/Uy7Yiae23wbDQB3XvgIQgEUcEM4FKd69y92ZgMTC/H9qKpA0NC0mYEjGt4VhgT7v1amBWjHqzzWwjsA/4srtvOYu2mNlCYCFASUkJK1asiBlMY2Njp2WpLB37pT5FHW9xVm47wdwJObz00kt9E1ic9LNKDb3tUyISQawBTe+wvh6Y4O6NZjYPeAYo7WHb6Eb3R4BHAGbMmOFz5syJGcyKFSvorCyVpWO/1KeoX66rps038hfzZnLZucP6JrA46WeVGnrbp0QMDVUD49utjyP6V/873L3e3RuD5aVArpmN6ElbkXT33OYDjBlSwPTxQ8MORTJUIhLBa0CpmU0yszzgFmBJ+wpmNsqCSyHMbGaw39qetBVJZ6da2ni58hBzLyzR1UISmriHhty91czuBpYB2cAid99iZncF5Q8DHwM+Z2atwEngFnd3IGbbeGMSSRWvVB7mVEuEa6eWhB2KZLCEPAMvGO5Z2mHbw+2Wvwd8r6dtRTLFC1sPUpSfw6zzisMORTKY7iwWCUkk4ry4tYZrJo8gPyc77HAkgykRiIRk095j1DQ0cZ2GhSRkSgQiIXlx60GyDMqnnBN2KJLhlAhEQvL81hpmTChmWGFe2KFIhlMiEAnB3qMn2bq/nmun6mhAwqdEIBKC5W/VAOiyUUkKSgQiIXhp2yHGDh3A+SMLww5FRIlApL81t0Z4dUct758yUncTS1JQIhDpZ+t319HY1Mo1pSPDDkUEUCIQ6Xcrtx0iJ8u46oLhYYciAigRiPS7l7Yd4vJzhzG4IDfsUEQAJQKRfnWooYkt++p5/xQNC0nyUCIQ6Ue/334IgPdPViKQ5KFEINKPXtp2iOGFeVw4enDYoYi8Q4lApJ9EIs7L2w9zdekIsrJ02agkDyUCkX6y9UA9tcebuVqXjUqSUSIQ6Sd/qKwF4L0XjAg5EpF3UyIQ6ScvVx7m/JGFjBpSEHYoIu+SkERgZjeYWYWZVZrZvTHKP2lmbwSvP5jZtHZlb5vZJjPbYGZrExGPSLJpbo2wZucRHQ1IUor7mcVmlg08BMwFqoHXzGyJu7/ZrtpO4P3uXmdmNwKPALPalZe7++F4YxFJVq/vruNkSxtXna9EIMknEUcEM4FKd69y92ZgMTC/fQV3/4O71wWrq4BxCdivSMp4ZUctWQazz9O0EpJ8zN3j+wCzjwE3uPudwfrtwCx3v7uT+l8G3tOu/k6gDnDgB+7+SCftFgILAUpKSsoWL14cM57GxkaKiori6lMySsd+ZVKfvrHqJK0Ofzd7QAhRxS+TflaprLs+lZeXr3P3GWcUuHtcL+DjwKPt1m8H/n8ndcuBrcDwdtvGBO/nABuBa7rbZ1lZmXdm+fLlnZalsnTsV6b0qeFUi59/36/9n3+ztf8DSpBM+Vmluu76BKz1GN+piRgaqgbGt1sfB+zrWMnMLgUeBea7e227RLQveK8BniY61CSSNtbsrKU14jpRLEkrEYngNaDUzCaZWR5wC7CkfQUzOxd4Crjd3be1215oZoNOLwPXA5sTEJNI0nilspb8nCzKJgwLOxSRmOK+asjdW83sbmAZkA0scvctZnZXUP4w8HVgOPDvwROZWj06TlUCPB1sywF+6u7PxRuTSDJ5dUctl587jILc7LBDEYkp7kQA4O5LgaUdtj3cbvlO4M4Y7aqAaR23i6SLoyea2Xqgni9dNznsUEQ6pTuLRfrQ6p1HcIfZ5+uyUUleSgQifejVHbUU5GZx6bghYYci0iklApE+tKqqlhkTisnP0fkBSV5KBCJ95MjxZt460MCV5xWHHYpIl5QIRPrImp3R22V0fkCSnRKBSB95dUctA3KzuWTs0LBDEemSEoFIH1lVdYQZE4eRl6NfM0lu+h8q0gdqG5uoONjAlZptVFKAEoFIH1i98wiAEoGkBCUCkT6wuqqWgXnZun9AUoISgUgfWL3zCGUThpGbrV8xSX76XyqSYHXB/QOzJun+AUkNSgQiCbbm7ej5gVk6PyApQolAJMFWVUWfP6DzA5IqlAhEEmx11REuP3eY5heSlKFEIJJAx1ucrQfqmaX5hSSFKBGIJNC2ujbcYdYknR+Q1KFEIJJAFUfayMvO4rJzNb+QpI6EJAIzu8HMKsys0szujVFuZvbdoPwNM7u8p21FUknFkQjTxw/V84klpcSdCMwsG3gIuBG4ELjVzC7sUO1GoDR4LQS+fxZtRVJCw6kWdjVEdH5AUk4ijghmApXuXuXuzcBiYH6HOvOBn3jUKmComY3uYVuRlLB2Vx0RnR+QFJSTgM8YC+xpt14NzOpBnbE9bAuAmS0kejRBSUkJK1asiBlMY2Njp2WpLB37lW59+mVFM1nmHN+1iRV7LexwEirdflagPrWXiEQQ63+897BOT9pGN7o/AjwCMGPGDJ8zZ07MYFasWEFnZaksHfuVbn367puvcN6QNv7kuvKwQ0m4dPtZgfrUXiKGhqqB8e3WxwH7elinJ21Fkt6J5lbeqD7GlGE6SSypJxGJ4DWg1MwmmVkecAuwpEOdJcCng6uHrgSOufv+HrYVSXrrdx2lNeJMKdYV2ZJ64h4acvdWM7sbWAZkA4vcfYuZ3RWUPwwsBeYBlcAJ4DNdtY03JpH+tnpnLVkGpToikBSUiHMEuPtSol/27bc93G7Zgc/3tK1IqllddYSLxw5hQE5r2KGInDUdx4rE6VRLGxv2HNXzByRlKRGIxOn13Udpbovo/gFJWUoEInFas/MIZnCFjggkRSkRiMRp9c5apo4azJABuWGHItIrSgQicWhqbWPdrjqu1GMpJYUpEYjEYeOeYzS1RrhSE81JClMiEInDqqpazGCmzg9IClMiEInDqqro+YGhA/PCDkWk15QIRHpJ5wckXSgRiPSSzg9IulAiEOklnR+QdKFEINJLOj8g6UKJQKQXdH5A0okSgUgvnD4/oAfVSzpQIhDphT/sOIwZXKmJ5iQNKBGI9MIfdtRy8ZghDBmo+YUk9SkRiJylk81tvL67jqsu0NGApAclApGztHbXEVranKvOHxF2KCIJEVciMLNiM3vezLYH78Ni1BlvZsvNbKuZbTGzL7Yre8DM9prZhuA1L554RPrDK5W15GQZV0w847+7SEqK94jgXuBFdy8FXgzWO2oF/srdpwJXAp83swvblf+ru08PXnp2sSS9V3cc5rJzhzIwLyGP/BYJXbyJYD7weLD8OHBzxwruvt/d1wfLDcBWYGyc+xUJxbGTLWzae4zZGhaSNGLu3vvGZkfdfWi79Tp37/R42cwmAiuBi9293sweAO4A6oG1RI8c6jppuxBYCFBSUlK2ePHimPtobGykqKioN91JaunYr1Ts0+s1rXxnfRP3zSxgSnH2GeWp2KeeSMd+ZWKfysvL17n7jDMK3L3LF/ACsDnGaz5wtEPdui4+pwhYB3y03bYSIJvokck3gEXdxePulJWVeWeWL1/eaVkqS8d+pWKfHliy2afcv9RPtbTGLE/FPvVEOvYrE/sErPUY36ndDnK6+3WdlZnZQTMb7e77zWw0UNNJvVzgSeAJd3+q3WcfbFfnh8CvuotHJEx/qKzlionF5OeceTQgkqriPUewBFgQLC8Anu1YwcwMeAzY6u7f7lA2ut3qR4geaYgkpZr6U1QcbGD2+bp/QNJLvIngQWCumW0H5gbrmNkYMzt9BdB7gduBD8S4TPSbZrbJzN4AyoEvxRmPSJ95ufIwANeUjgw5EpHEiuv6N3evBa6NsX0fMC9YfhmwTtrfHs/+RfrT77cfZnhhHheOHhx2KCIJpTuLRXogEnF+v/0w7ysdQVZWzL9rRFKWEoFID7x1oIHDjU1crWEhSUNKBCI9sHL7IQCuLtWNZJJ+lAhEeuD32w8xpWQQJYMLwg5FJOGUCES6cbK5jdd21uloQNKWEoFIN1bvrKW5LcLVk3V+QNKTEoFIN1ZuO0xeThYzJ+r5xJKelAhEurGiooZZk4oZkKdpJSQ9KRGIdOHtw8epOnyca99zTtihiPQZJQKRLvzureg8ih94T0nIkYj0HSUCkS4sr6jh/JGFnDt8YNihiPQZJQKRTjQ2tbKqqpYPaFhI0pwSgUgnXt5+mJY217CQpD0lApFOLH+rhkEFOcyY2OnTV0XSghKBSAzuzvKKGq4pHUlutn5NJL3pf7hIDJv31lPT0ES5zg9IBlAiEIlh2ZYDZBmUT9G0EpL+lAhEYnhuywFmTRrO8KL8sEMR6XNxJQIzKzaz581se/Ae86yamb0dPJt4g5mtPdv2Iv2psqaByppGbrxkVNihiPSLeI8I7gVedPdS4MVgvTPl7j7d3Wf0sr1Iv3hu8wEArr9QiUAyQ7yJYD7weLD8OHBzP7cXSbjnthzgsnOHMmqIHkIjmcHcvfeNzY66+9B263XufsbwjpntBOoAB37g7o+cTfugbCGwEKCkpKRs8eLFMWNqbGykqKio131KVunYr2Ts06ETEf565Uk+MSWPGyflnnX7ZOxTIqRjvzKxT+Xl5es6jMpEuXuXL+AFYHOM13zgaIe6dZ18xpjg/RxgI3BNsN6j9h1fZWVl3pnly5d3WpbK0rFfydinH67c4RO+8it/+3Bjr9onY58SIR37lYl9AtZ6jO/UnO4yjLtf11mZmR00s9Huvt/MRgM1nXzGvuC9xsyeBmYCK4EetRfpL7/ZfICpowczYXhh2KGI9Jt4zxEsARYEywuAZztWMLNCMxt0ehm4nugRRY/ai/SXvUdPsm5XHfMu1kliySzxJoIHgblmth2YG6xjZmPMbGlQpwR42cw2AmuAX7v7c121FwnDsxv2AjB/+tiQIxHpX90ODXXF3WuBa2Ns3wfMC5argGln016kv7k7z7y+l7IJw/TsAck4urNYBNi6v4FtBxu5efqYsEMR6XdKBCJEh4VysowPXqpEIJlHiUAyXlvEeXbDPt4/eSTFhXlhhyPS75QIJOOt3lnLgfpT3HyZThJLZlIikIz31Pq9FOZlc91UPZJSMpMSgWS0Yydb+NUb+/jw9LEMyMsOOxyRUCgRSEZ75vW9nGqJ8MlZ54YdikholAgkY7k7P129m0vHDeHisUPCDkckNEoEkrHW766j4mADt83U0YBkNiUCyVhPrN5NUX4OH5qmewcksykRSEY6dqKFX7+xn5svG0NhflwzrYikPCUCyUg/XbObptYIt82cEHYoIqFTIpCM09Taxo9e2cn7LhjBhWMGhx2OSOiUCCTjPLthHzUNTSy85rywQxFJCkoEklEiEeeHK6uYOnowV5eOCDsckaSgRCAZZXlFDdtrGvmLa87DzMIORyQpKBFIRvnByirGDCngg5eODjsUkaShRCAZ4+Xth1mz8wj/65rzyM3Wf32R0+L6bTCzYjN73sy2B+/DYtSZYmYb2r3qzeyeoOwBM9vbrmxePPGIdMbd+daytxg7dAC3aV4hkXeJ98+ie4EX3b0UeDFYfxd3r3D36e4+HSgDTgBPt6vyr6fL3X1px/YiibBsywE2Vh/ji9eVkp+jWUZF2os3EcwHHg+WHwdu7qb+tcAOd98V535Feqy1LcK3llVwwTlFfFQPnxE5Q7yJoMTd9wME7+d0U/8W4Gcdtt1tZm+Y2aJYQ0si8Xpq/V52HDrOl6+fTI7ODYicwdy96wpmLwCjYhR9DXjc3Ye2q1vn7jG/zM0sD9gHXOTuB4NtJcBhwIF/BEa7+2c7ab8QWAhQUlJStnjx4pjxNjY2UlRU1GWfUlE69qs/+nS8xbnv9ycYMSCLv72yoM8vGU3HnxOkZ78ysU/l5eXr3H3GGQXu3usXUEH0yxtgNFDRRd35wG+7KJ8IbO7JfsvKyrwzy5cv77QslaVjv/qjT197+g2fdO+vfFP10T7fl3t6/pzc07NfmdgnYK3H+E6N9zh5CbAgWF4APNtF3VvpMCxkZu0v5v4IsDnOeETesWHPUZ5YvZtPz56oB8+IdCHeRPAgMNfMtgNzg3XMbIyZvXMFkJkNDMqf6tD+m2a2yczeAMqBL8UZjwgAbRHn/mc2MbIon7+6fnLY4YgktbgmYnf3WqJXAnXcvg+Y1279BDA8Rr3b49m/SGcefmkHm/fW873bLmNQQW7Y4YgkNV1CIWnn9d11fPv5bdx06Wg+eImmkhDpjhKBpJWGUy18cfEGRg0u4BsfuUQTy4n0gJ7RJ2nD3fnbZzZTXXeCX/zFbIYM0JCQSE/oiEDSxiMrq3hmwz7uuW4yMyYWhx2OSMpQIpC0sGzLAR587i0+eOlo7i6/IOxwRFKKEoGkvE3Vx7hn8QamjRvKv3x8GllZOi8gcjaUCCSlbd57jNsXraa4MI8ffnoGBbmaWVTkbCkRSMraVH2MTz66msK8HBYvvJKRg/LDDkkkJSkRSEpas/MIn3x0FUX50SQwvnhg2CGJpCwlAkk5v3htD598dBUjBuUrCYgkgO4jkJTR1NrGP/+mgkWv7OTq0hF877bLda+ASAIoEUhKeOtAPfcs3sBbBxq446qJ3P/BqXrIjEiCKBFIUjvV0sajv6/iuy9WMnhADo8tmMG1U0vCDkskrSgRSFJyd5ZtOcD/+fVWqutOMu+SUfzD/IsZUaQrg0QSTYlAkkok4jy35QAPLa9ky756ppQM4ok7Z/HeC0aEHZpI2lIikKRQ29jEk+ur+dmaPaPCLbgAAAcaSURBVOw8fJxJIwr55scu5aOXjdW5AJE+pkQgoTnR4jz9ejVLNx1gRUUNLW3OjAnD+Mu5k5l3yWiyNVWESL9QIpB+09waYfO+Y7y6o5aV2w6x9u0TtPlGRg0u4PYrJ3LLzPFMLhkUdpgiGUeJQPrE8aZWqg4dZ+v+et7cX8/mvcd4Y+8xmlsjAFw4ejDXT8zlzhuu4LLxQzVRnEiI4koEZvZx4AFgKjDT3dd2Uu8G4DtANvCou59+yH0x8HNgIvA28GfuXhdPTNL3mlsj1J1o5lBDE4camjhYf4r9x05RXXeS6roTvF17nIP1Te/UH5iXzdTRg1kwewKXnzuMKyYVM6IonxUrVlA2YViIPRERiP+IYDPwUeAHnVUws2zgIWAuUA28ZmZL3P1N4F7gRXd/0MzuDda/EmdMKc3dcQc/vQxE3Gluc042t+E4EY9u80j0PeJOmzuRCMG70xpx2iJOayRCa1t0vbUtQkub09IWobk1QnPw3tTaxqmWCKda2jgZvI43tXKiqY2GplYaT7VSf6qFYydbOHaihYam1jPiNoNRgwsYO3QA77tgJOeNLOT8kYVMGTWYCcUD9Re/SBKLKxG4+1agu+fCzgQq3b0qqLsYmA+8GbzPCeo9DqygB4lgy756Lvr6c++OJXhva2sj+3fPndmoh9y7KSd2hdPt3lXq7Rf9jHqnv+h7sl8Anu99v85Gfk4Whfk5DMjNZlBBDoMKcigZXMCUkkEMHpBLcWEew4vyGF6YxzmDCygZXMDIonzycnR1j0gq6o9zBGOBPe3Wq4FZwXKJu+8HcPf9ZnZOZx9iZguBhcFq45v/eGNFJ1VHAIfjCzkppWO/1KfUkY79ysQ+TYi1sdtEYGYvAKNiFH3N3Z/tQWCxDhd68vfvuxu4PwI80u3OzNa6+4yz/fxkl479Up9SRzr2S336o24Tgbtf17uQ3lENjG+3Pg7YFywfNLPRwdHAaKAmzn2JiMhZ6o9B3deAUjObZGZ5wC3AkqBsCbAgWF4A9OQIQ0REEiiuRGBmHzGzamA28GszWxZsH2NmSwHcvRW4G1gGbAV+4e5bgo94EJhrZtuJXlX0YDzxBLodPkpR6dgv9Sl1pGO/1KeAeY8uVxERkXSl6/1ERDKcEoGISIZL20RgZl8wswoz22Jm3ww7nkQysy+bmZtZyk/Sb2bfMrO3zOwNM3vazIaGHVNvmdkNwf+5yuBO+ZRmZuPNbLmZbQ1+j74YdkyJYmbZZva6mf0q7FgSxcyGmtkvg9+nrWY2u6dt0zIRmFk50buWL3X3i4D/F3JICWNm44meWN8ddiwJ8jxwsbtfCmwD7gs5nl5pN5XKjcCFwK1mdmG4UcWtFfgrd58KXAl8Pg36dNoXiV68kk6+Azzn7u8BpnEW/UvLRAB8DnjQ3ZsA3D2d7k/4V+Bv6MVNecnI3X8bXFkGsIrofSap6J2pVNy9GTg9lUrKcvf97r4+WG4g+sUyNtyo4mdm44APAo+GHUuimNlg4BrgMQB3b3b3oz1tn66JYDJwtZmtNrOXzOyKsANKBDP7MLDX3TeGHUsf+Szwm7CD6KVYU6mk/JfmaWY2EbgMWB1uJAnxb0T/mIqEHUgCnQccAn4UDHk9amaFPW2css8j6GrqC6L9Gkb0cPYK4Bdmdp6nwLWy3fTrq8D1/RtR/HoyTYmZfY3oUMQT/RlbAiVkKpVkZGZFwJPAPe5eH3Y88TCzm4Aad19nZnPCjieBcoDLgS+4+2oz+w7R2Zz/tqeNU1JXU1+Y2eeAp4Iv/jVmFiE6GdOh/oqvtzrrl5ldAkwCNgazvY4D1pvZTHc/0I8hnrXupikxswXATcC1qZCsO9HVVCopy8xyiSaBJ9z9qbDjSYD3Ah82s3lAATDYzP7T3T8Vclzxqgaq3f30EdsviSaCHknXoaFngA8AmNlkII8Un2XQ3Te5+znuPtHdJxL9wV+e7EmgO8FDi74CfNjdT4QdTxy6mkolJVn0L47HgK3u/u2w40kEd7/P3ccFv0O3AL9LgyRA8D2wx8ymBJuuJTrVf4+k7BFBNxYBi8xsM9AMLEjhvzTT3feAfOD54EhnlbvfFW5IZ8/dW83s9FQq2cCidlOppKr3ArcDm8xsQ7Dtq+6+NMSYpHNfAJ4I/hCpAj7T04aaYkJEJMOl69CQiIj0kBKBiEiGUyIQEclwSgQiIhlOiUBEJMMpEYiIZDglAhGRDKdEIJIAZnaXmW0IXjvNbHnYMYn0lG4oE0mgYG6e3wHfdPf/DjsekZ7QEYFIYn2H6Pw1SgKSMtJ1riGRfmdmdwATgLtDDkXkrGhoSCQBzKwMeBy42t3rwo5H5GxoaEgkMe4GioHlwQnjtHkMoqQ/HRGIiGQ4HRGIiGQ4JQIRkQynRCAikuGUCEREMpwSgYhIhlMiEBHJcEoEIiIZ7n8AYVdtrDoTOagAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_2.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "%matplotlib inline\n",
+    "\n",
+    "\"\"\"The sigmoid function (or the logistic curve) is a\n",
+    "function that takes any real number, z, and outputs a number (0,1).\n",
+    "It is useful in neural networks for assigning weights on a relative scale.\n",
+    "The value z is the weighted sum of parameters involved in the learning algorithm.\"\"\"\n",
+    "\n",
+    "import numpy\n",
+    "import matplotlib.pyplot as plt\n",
+    "import math as mt\n",
+    "\n",
+    "z = numpy.arange(-5, 5, .1)\n",
+    "sigma_fn = numpy.vectorize(lambda z: 1/(1+numpy.exp(-z)))\n",
+    "sigma = sigma_fn(z)\n",
+    "\n",
+    "fig = plt.figure()\n",
+    "ax = fig.add_subplot(111)\n",
+    "ax.plot(z, sigma)\n",
+    "ax.set_ylim([-0.1, 1.1])\n",
+    "ax.set_xlim([-5,5])\n",
+    "ax.grid(True)\n",
+    "ax.set_xlabel('z')\n",
+    "ax.set_title('sigmoid function')\n",
+    "\n",
+    "plt.show()\n",
+    "\n",
+    "\"\"\"Step Function\"\"\"\n",
+    "z = numpy.arange(-5, 5, .02)\n",
+    "step_fn = numpy.vectorize(lambda z: 1.0 if z >= 0.0 else 0.0)\n",
+    "step = step_fn(z)\n",
+    "\n",
+    "fig = plt.figure()\n",
+    "ax = fig.add_subplot(111)\n",
+    "ax.plot(z, step)\n",
+    "ax.set_ylim([-0.5, 1.5])\n",
+    "ax.set_xlim([-5,5])\n",
+    "ax.grid(True)\n",
+    "ax.set_xlabel('z')\n",
+    "ax.set_title('step function')\n",
+    "\n",
+    "plt.show()\n",
+    "\n",
+    "\"\"\"tanh Function\"\"\"\n",
+    "z = numpy.arange(-2*mt.pi, 2*mt.pi, 0.1)\n",
+    "t = numpy.tanh(z)\n",
+    "\n",
+    "fig = plt.figure()\n",
+    "ax = fig.add_subplot(111)\n",
+    "ax.plot(z, t)\n",
+    "ax.set_ylim([-1.0, 1.0])\n",
+    "ax.set_xlim([-2*mt.pi,2*mt.pi])\n",
+    "ax.grid(True)\n",
+    "ax.set_xlabel('z')\n",
+    "ax.set_title('tanh function')\n",
+    "\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Two parameters\n",
+    "\n",
+    "We assume now that we have two classes with $y_i$ either $0$ or $1$. Furthermore we assume also that we have only two parameters $\\beta$ in our fitting of the Sigmoid function, that is we define probabilities"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "p(y_i=1|x_i,\\hat{\\beta}) &= \\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}},\\nonumber\\\\\n",
+    "p(y_i=0|x_i,\\hat{\\beta}) &= 1 - p(y_i=1|x_i,\\hat{\\beta}),\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $\\hat{\\beta}$ are the weights we wish to extract from data, in our case $\\beta_0$ and $\\beta_1$. \n",
+    "\n",
+    "Note that we used"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "p(y_i=0\\vert x_i, \\hat{\\beta}) = 1-p(y_i=1\\vert x_i, \\hat{\\beta}).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<!-- !split  -->\n",
+    "## Maximum likelihood\n",
+    "\n",
+    "In order to define the total likelihood for all possible outcomes from a  \n",
+    "dataset $\\mathcal{D}=\\{(y_i,x_i)\\}$, with the binary labels\n",
+    "$y_i\\in\\{0,1\\}$ and where the data points are drawn independently, we use the so-called [Maximum Likelihood Estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) principle. \n",
+    "We aim thus at maximizing \n",
+    "the probability of seeing the observed data. We can then approximate the \n",
+    "likelihood in terms of the product of the individual probabilities of a specific outcome $y_i$, that is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "P(\\mathcal{D}|\\hat{\\beta})& = \\prod_{i=1}^n \\left[p(y_i=1|x_i,\\hat{\\beta})\\right]^{y_i}\\left[1-p(y_i=1|x_i,\\hat{\\beta}))\\right]^{1-y_i}\\nonumber \\\\\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "from which we obtain the log-likelihood and our **cost/loss** function"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\mathcal{C}(\\hat{\\beta}) = \\sum_{i=1}^n \\left( y_i\\log{p(y_i=1|x_i,\\hat{\\beta})} + (1-y_i)\\log\\left[1-p(y_i=1|x_i,\\hat{\\beta}))\\right]\\right).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The cost function rewritten\n",
+    "\n",
+    "Reordering the logarithms, we can rewrite the **cost/loss** function as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\mathcal{C}(\\hat{\\beta}) = \\sum_{i=1}^n  \\left(y_i(\\beta_0+\\beta_1x_i) -\\log{(1+\\exp{(\\beta_0+\\beta_1x_i)})}\\right).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The maximum likelihood estimator is defined as the set of parameters that maximize the log-likelihood where we maximize with respect to $\\beta$.\n",
+    "Since the cost (error) function is just the negative log-likelihood, for logistic regression we have that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\mathcal{C}(\\hat{\\beta})=-\\sum_{i=1}^n  \\left(y_i(\\beta_0+\\beta_1x_i) -\\log{(1+\\exp{(\\beta_0+\\beta_1x_i)})}\\right).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This equation is known in statistics as the **cross entropy**. Finally, we note that just as in linear regression, \n",
+    "in practice we often supplement the cross-entropy with additional regularization terms, usually $L_1$ and $L_2$ regularization as we did for Ridge and Lasso regression.\n",
+    "\n",
+    "## Minimizing the cross entropy\n",
+    "\n",
+    "The cross entropy is a convex function of the weights $\\hat{\\beta}$ and,\n",
+    "therefore, any local minimizer is a global minimizer. \n",
+    "\n",
+    "\n",
+    "Minimizing this\n",
+    "cost function with respect to the two parameters $\\beta_0$ and $\\beta_1$ we obtain"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\beta_0} = -\\sum_{i=1}^n  \\left(y_i -\\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}}\\right),\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\beta_1} = -\\sum_{i=1}^n  \\left(y_ix_i -x_i\\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}}\\right).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A more compact expression\n",
+    "\n",
+    "Let us now define a vector $\\hat{y}$ with $n$ elements $y_i$, an\n",
+    "$n\\times p$ matrix $\\hat{X}$ which contains the $x_i$ values and a\n",
+    "vector $\\hat{p}$ of fitted probabilities $p(y_i\\vert x_i,\\hat{\\beta})$. We can rewrite in a more compact form the first\n",
+    "derivative of cost function as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\hat{\\beta}} = -\\hat{X}^T\\left(\\hat{y}-\\hat{p}\\right).\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If we in addition define a diagonal matrix $\\hat{W}$ with elements \n",
+    "$p(y_i\\vert x_i,\\hat{\\beta})(1-p(y_i\\vert x_i,\\hat{\\beta})$, we can obtain a compact expression of the second derivative as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial^2 \\mathcal{C}(\\hat{\\beta})}{\\partial \\hat{\\beta}\\partial \\hat{\\beta}^T} = \\hat{X}^T\\hat{W}\\hat{X}.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Extending to more predictors\n",
+    "\n",
+    "Within a binary classification problem, we can easily expand our model to include multiple predictors. Our ratio between likelihoods is then with $p$ predictors"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\log{ \\frac{p(\\hat{\\beta}\\hat{x})}{1-p(\\hat{\\beta}\\hat{x})}} = \\beta_0+\\beta_1x_1+\\beta_2x_2+\\dots+\\beta_px_p.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here we defined $\\hat{x}=[1,x_1,x_2,\\dots,x_p]$ and $\\hat{\\beta}=[\\beta_0, \\beta_1, \\dots, \\beta_p]$ leading to"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "p(\\hat{\\beta}\\hat{x})=\\frac{ \\exp{(\\beta_0+\\beta_1x_1+\\beta_2x_2+\\dots+\\beta_px_p)}}{1+\\exp{(\\beta_0+\\beta_1x_1+\\beta_2x_2+\\dots+\\beta_px_p)}}.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Including more classes\n",
+    "\n",
+    "Till now we have mainly focused on two classes, the so-called binary\n",
+    "system. Suppose we wish to extend to $K$ classes.  Let us for the sake\n",
+    "of simplicity assume we have only two predictors. We have then\n",
+    "following model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1\n",
+    "5\n",
+    " \n",
+    "<\n",
+    "<\n",
+    "<\n",
+    "!\n",
+    "!\n",
+    "M\n",
+    "A\n",
+    "T\n",
+    "H\n",
+    "_\n",
+    "B\n",
+    "L\n",
+    "O\n",
+    "C\n",
+    "K"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\log{\\frac{p(C=2\\vert x)}{p(K\\vert x)}} = \\beta_{20}+\\beta_{21}x_1,\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and so on till the class $C=K-1$ class"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\log{\\frac{p(C=K-1\\vert x)}{p(K\\vert x)}} = \\beta_{(K-1)0}+\\beta_{(K-1)1}x_1,\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and the model is specified in term of $K-1$ so-called log-odds or\n",
+    "**logit** transformations.\n",
+    "\n",
+    "\n",
+    "## More classes\n",
+    "\n",
+    "In our discussion of neural networks we will encounter the above again\n",
+    "in terms of a slightly modified function, the so-called **Softmax** function.\n",
+    "\n",
+    "The softmax function is used in various multiclass classification\n",
+    "methods, such as multinomial logistic regression (also known as\n",
+    "softmax regression), multiclass linear discriminant analysis, naive\n",
+    "Bayes classifiers, and artificial neural networks.  Specifically, in\n",
+    "multinomial logistic regression and linear discriminant analysis, the\n",
+    "input to the function is the result of $K$ distinct linear functions,\n",
+    "and the predicted probability for the $k$-th class given a sample\n",
+    "vector $\\hat{x}$ and a weighting vector $\\hat{\\beta}$ is (with two\n",
+    "predictors):"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "p(C=k\\vert \\mathbf {x} )=\\frac{\\exp{(\\beta_{k0}+\\beta_{k1}x_1)}}{1+\\sum_{l=1}^{K-1}\\exp{(\\beta_{l0}+\\beta_{l1}x_1)}}.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It is easy to extend to more predictors. The final class is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "p(C=K\\vert \\mathbf {x} )=\\frac{1}{1+\\sum_{l=1}^{K-1}\\exp{(\\beta_{l0}+\\beta_{l1}x_1)}},\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and they sum to one. Our earlier discussions were all specialized to\n",
+    "the case with two classes only. It is easy to see from the above that\n",
+    "what we derived earlier is compatible with these equations.\n",
+    "\n",
+    "To find the optimal parameters we would typically use a gradient\n",
+    "descent method.  Newton's method and gradient descent methods are\n",
+    "discussed in the material on [optimization\n",
+    "methods](https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html).\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## A simple classification problem"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9aZB0aXXn93vumvtW+7tvTTeIXhDQTYMEzciINYRmkS1pwjOeCBtrxgp7tg+yHJ4J+4NjHCGHQ7ZmJGEFHivC1hJCo0ESSDMwAo1AQAOi6W6g6eXdqt7aq7Jyz7s8jz/czKxcbmZlVWWt7/1FNLxVeevem9v/nnuec/5HKKWIiIiIiDj/aCd9AhERERERx0Mk+BEREREPCZHgR0RERDwkRIIfERER8ZAQCX5ERETEQ4Jx0icwipxhqXkzcdKnEREREXFmeKWxs6GUmgl77FQL/ryZ4FO3fuSkTyMiIiLizPCel/747rDHopRORERExENCJPgRERERDwmR4EdEREQ8JESCHxEREfGQEAl+RERExENCJPgRERERDwmR4EdEREQ8JESCHxEREfGQEAl+RERExENCJPgRERERDwmR4EdEREQ8JESCHxEREfGQEAl+RERExENCJPgRERERDwmR4EdEREQ8JESCHxEREfGQEAl+RERExENCJPgRERERDwkTEXwhxKeEEGtCiJeGPP6cEGJHCPHt1n//bBLHjYiIiIgYn0nNtP3XwK8Avzlim/+olPrYhI4XEREREbFPJhLhK6X+HNiaxL4iIiIiIo6G48zhPyuEeEEI8TkhxA8N20gI8QkhxDeEEN8o+s4xnl5ERETE+ea4BP9bwFWl1JPA/wn8wbANlVKfVEq9Qyn1jpxuHdPpRURERJx/jkXwlVIlpVSl9e/PAqYQYvo4jh0REREREXAsgi+EmBdCiNa/n24dd/M4jh0RERERETCRKh0hxG8BzwHTQohF4J8DJoBS6teAvwX8fSGEB9SBn1ZKqUkcOyIiIiJiPCYi+Eqpn9nj8V8hKNuMiIiIiDghok7biCNFSoXrSKSMbugiIk6aSTVeRUT0oJRifdWluOUHvxAwNW1QmDZoLedEREQcM5HgRxwJG2sexS2fzkqNgs11D12HXME8tvPwfUWp6OE4ilhcI53R0bToghPxcBIJfsTEUUqxveXRvyyvFGxu+Mcm+M2m5N4bTZQKji2Ez+aax5UbNoYRiX7Ew0eUw4+YOEqBkuGPea6iVvVx3aPP6a8sOUhJ58KjFLiuYmPVPfJjR0ScRqIIP2LiCAGGKfCGiPrSPQelIJXWmL9ooWkCpRSepxBCTCT6lr6iUQ8/frnsM3/oI0REnD0iwY+YOEIIZucMlpfcgbQOgGxF/5WyZGPNJZ3RWV508bxgYzsmuHDJwrQOcQM64poRJXMiHlailE7EkZDOGly6ahFPaOhDwgqloLjlc/+ug+uqTq69UVfcu+NwmN48TRMkkoMfbyEgk9MPvN+IiLNMJPgRR0YiqXPlus2tR+NDtxmW75e+olYdshAwJgsXLQxTILRA6IUI7h6mZ4+vSigi4jQRpXQijoV4QqNeGxRwTQfpD26vFEPXAMbFMAU3HrGpViSuo7BjgnhCi/oAIh5aogg/4liYnTcRfZ82ISCf1xmmv7H44T+eQghSaZ38lEEiqUdiH/FQE0X4EcdCLK5x7abN1oZHsyGxYxqFKQPDFJRKMijTbAX0QkAypWHHongkImKSRIIfcWxYlsb8hcGhNldv2Gyuu5RLEiEglw8i8uPEdSXlnaAzOJnWiUUXm4hzSCT4pwQpFeWSj9OU2LZG6pgtAGpVn/VVF6epMK1gYTOVPp5qFl0XzM5bzJ5QcfxO0WP1gRvcYLQsIHIFndn5o5u4Vq9LGjWJYQpSKQ0R2T1EHAOR4J8CXFdx740Gvr9rAaCvuly9EcMwj14IqhW/0wwF0GwoHtx3mL9oksme74+I76lA7LvWh9vloumMTzwx2YueUoqle06nAqldPXTluo1lR3cVEUdL9Ak7Baw+cPC8XgsAz4PVleMZ4r6+OtggpRSsr5x/C4JKJaREiOD5l4rhjx2G7U2PWlV2eg6kBN+HB4vH815HPNxEgn/CKKWoVsLrzavlw9Whj4vTDC9/9DxQ59zHXnT+Z9iDk2Vn2w/tPnaa6lj8hSIebs73/fpZQdCpUDkJdCPc90bTmIjoKaWoVSSNhkQ3gpy95zLRunilFPWa7ETlmZw+1r6TaR0ehN/J2DENz1U9abVa1Wdz3cNzFfGkxtS0sS8LiFHNw0FncZTLjzg6IsE/YYQQpNM65dJg+iCVOZ5F0+kZg9Xl3rSOEExkWImUinu3mziO6u2obXe+2oLL1+xDL1Cvrbg90XNpxyeb05kLqQrqRtcF8xdMVlqi3/0arK+4rCmXZEpj4ZJFueT35Psdx6e843P1po01puinszrbm4PW0YYhMI9hvSbi4SZK6ZwCZhdMTEt0GpOEBqYlmJs/HguAbN5gZs5Aa1kQaBoUZoLpVIdlcz2o/BmwT2hZKjQbio21w60VNBtyIFWiFOwUfRr1vdNimZzBjUdizMyZpNKi0wjWtlauViSrDxzWlgfXOqSEzTVv7HOdmjawut9rEbzfC5esqCks4siJIvxTgGEIrt8KLACcpsSyNZKp47UAyE+Z5AoGvg+6zsSOXSqG56zbtBdHD1OSWamEHyMQa39ox670FZ6vMAyBYQryU8bQwS3lkhz6PPbj+aPpgqs3bSplSa3qY5qCTM6IBrJEHAuR4J8S2hYAHFPt+7BzME7gE3EIU0wANBFE5f37CaLnQSFVSnVSQMGGu/N2fS/8ZEado7HPGzEhBOmMTvqYUnYREW2ilM45xmlK1lcdVpcdalX/UHbDByWTHe6V0+awaxXp7PC/DxPV9dXdfH/brXNz3aNU9Iknwr8ShgHJrnRPm2CtI3LfjDgbRIJ/Tiluudx5vcnWhk9xy2fxrtMaSHK8oj81Y2LZYmTxSb5wuNsKwxDMXzQ76w/ttYj5i+bAQqhSqne4euf3sLnhMRNi8gZBiWoubxBP9O6vMBNF6hFnh4kIvhDiU0KINSHES0MeF0KI/0MI8ZoQ4jtCiB+exHEjwvE9xdqKN7CIWSn71IbU/I9CSsXOtsfyksPmursv22JNF1y9YROLDVF8Aa5z+ItQJmtw8002qYwWiL4e5Na9vhRN94zbfnxPYdsa2SEDUlaWXBoN1YnyhYDS9uAxIiJOK5OK8P818KERj38YeKT13yeAX53QcSNCqFb90IhaSSiFlH+OwvcVd15vsrrsUioGNehvvNYI9bYfhhBi6MKpEAydiLVf1lY9yjsSzwPfC5qc7rzewPd3BVnTGLpO0TZMG3ZR9P3Au/+gQ9FdV+E48kRSaxERMCHBV0r9ObA1YpOPA7+pAr4K5IQQC5M4dsQgoyps9lt8s7nmdsYPwm7Oe3lpfyMIc3kj9Ni6ztC8+X5wnV23y26kDzvbu2WTQoggbROSi59plcHu9zUK66HoxnEkd15vcPvVBndea/LGq03qtcnbNkRE7MVx5fAvAve7fl5s/W4AIcQnhBDfEEJ8o+hH/iIHIZkaHk1nc/sLp8slGdoF7LkKf/zyc+yYxvxFsyfHbllB09UkSkC7Uy3dKDVYNpnJGly8EszbNQxBMqVx5YbduQvJjhjKEsqIbZVS3L/dpNnYndnruYp7tx02Vg83tzciYr8cVxFe2Fci9JOulPok8EmAx+K56NtwADRNcPGyxdI9p8e2oTBt7DuaDlvAhJaQ1nwSSX3sGvJM1iCd1mk0FJoeCP6k6v1NUwzNzZvW4DGSKZ1kKjxXnysY1Gpy18uotRhsWYJ6rfcgQkB2RJVQtSKRQ7Jfmxs+TUdx8bI99O8jIibJcQn+InC56+dLwINjOvZDSTKlc/PRGNWyj5RB1L8fz5c2ubzOxtpgMxLQshlwyeb1YIThGOItNDFQ6TIJ7JjAsgTN5qAgh1UB+Z5ia9OlWpbohqAwZQS+OgRpn4uXbRqNlme9IUimNTwP7t1u4vtB57BoXQRGDUX3utJhYVTLkmZrBkJExFFzXIL/GeDnhRC/DTwD7Cillo/p2A8tuh50cY6L5ylcJxiA0o7a81NBtNteyOwWr3bkurPtY9uCXMEMBA6O3RdGKTD6BF/XA8uCfp9531OtxdzW82kq6jWHqVmDqa6a+lhM65l8ZZoEQ9HLEsdV2LYgkRzdEb3nXF4BjXok+BHHw0QEXwjxW8BzwLQQYhH454AJoJT6NeCzwEeA14Aa8PcmcdyHDceR+F4QzU5yGpZSqlOF0+5YTaV1Fi6aCE1w6UoQ7darPmsrg4l7pWBtxWNzw8Nzg6jaNAULl61jGxX44L4zkKv3fUJfp61Nb1fsWygVeOLk8waaPmrRW+yrUSwWD2wyKiOsriPTtIjjYiKCr5T6mT0eV8B/M4ljPYx4nmLpXrDw1xbkmTmDTM5gZzsYqGHZgnxhf1a9bbY2vI7nTVsEK2Wf9TU6Y/5iMQ1DF6yvhqd3gsXI3X87TrBYeeNNMfQRAjoJPFd1hor0s7XhcvFKb468NsJ757VXGggNsjmd6Vkz9IKhVHA831PEE3unyi5ctthc99hcH7xYGoaYSJVSRMQ4RF46Z4Cle00a9UCh2kK1vup1cuuBSVgwlu/SVYtEMjwCbTYlxa3Ayz2Z1slkg7m5YXa97TF/M3Oqk7IIvOyDrtNxUArKOz65Q3bS7oXr7l4I+3FCmroMU0BjuGeO8oPn3qwrLl/vvVg4Tcm9O82O+6dS7LmGIUSQ50+mNJaXdhvXYnGNCy2XTNeVFLd8XEcRTwiyudF3GhERYTz1YQ9C218DIsE/5TiOpBkiTt3RePfvVh64XL81mFcu73gta4Xg52pFsr3pcfW6PbSKZNCMTDB3weLBfSdUXMP+3nEkxW2P0nbQDJbN6y1/ncmJmWUPr9AJi57zUwbVyujnoFRr0HhddvLwSikW7zkD5ag720G10l4WC/GEzvVbGr7XbjgLXoN6zef+HaczRL1SDtJOV2/EIhfNiJG8+8V/wrc2bgPwj3+pbTn7p0O3jwT/lNMWh3HLtT1X4fu93aRKKVZCBnW7jmJ7yyMW10I7Zy17sGwylda5csNm+X4TZ482CaEFF5Zu75pGXVIp+QNplsOg6y1r4747FU0LXDD7SSR1ZucN1lY9BAy94CGCu6K24DtNFWoroVQwq3YcTx0hRI+7plKK5cXB98Zzg6a3vQa4RDw8PPXh3UjjI9p/G/zjF+rA+N7ikeCfcuzY8Og1DAX0p53D7hCg7fPuM3/B6klTQHCRmVsILzeMxTQuXrG583pz+LmJ4KLTPy+3PVCkXpfE96pg2QfTswamBdsbPr6vSCQ1pmfNofn1XMHsrIGELUQHJ0vPJCsph198DzL7t1rx2Wh1ModRKfvM7XuvEeeN2J/9DQA+8kuHGBrRIhL8U46mCWbmDda7zdBEIOphkWkyqQ3kfjVt+B2CpgU+N9du2GxteDTqEisWzGq1R1TYWLYWjAZcdju9XYJ2F60gndVRSrK9OXiSSkG96k9U8IUQ5PImufx4VsVSKppNycaIaVWWLYjFd1/LYQZwQoy2aA5je9MdugC+u1/BTtFDSUimNUwzWtw977Sj+E4ED/BLk9t/JPjHTLXss7Hu4joKO6YxM2fuWaudL5hYlsb2pofnKZIpjXzBYG3Fo1L2O920li2Yv9ibAlBK4Tjhi5rtpiQpg0Hd/X+7F5mcQSqj06jLlkFabwpoe9NDiMHqGSE48sqdUWyuu52KmaEXQp0B2wehBa/R8uJu/l+I4HUfd2Fa+orNDZetjdFeOkIEi9Gr7QHrKzA1YzA1E3nvnzfaETxMJoofRST4x8hO0esZgl2rSu7dbnL5mr1naV6YFcCFy1awqFsPmqXsWK/gSqlYvNOk0T9TtmVPn8pobG+5PFgMTiiR0li4YAVVLGOiaWJoVVA6q7Me4iQpRPhgkuOgtBOUR+6VJrNtLfSilM7o2DdtittBz0EyrZHO6GP1RSipuHu7OZYddLdZXZvNdY9kSt+7mSvi1HLUEfxeRIJ/TCilWF8dHIKtVDCB6cr1gy1iWpaGNSQwL255NBrhrf1ByWRvuqVWkdy93eTGI5MxNDMMwaWru1U9iiDlc/GyfWIlh+OIPQw3oIMgndXuT9gP5VJQdnlQv7RgMLtHLB4t5J4ljjOC34tI8I8JKRnqLtlshJeJeJ5ie7PVWGUFlSj7ie6GDhAfITjSV1QrMpivOwESycDTp9FQCBi4Czluhs2s7adS9ieePhnWHLYfInPN0007go//1A/z/k//SPDLY4zg9yIS/GOibQkc9oUNq7V2XcXd1xv4ElDQqAcR4oXL1vhifABdlbJVWZPe/98OPQ0hiMdPTuS7iSc1KqW9h7e0G90miWGKHvfS/XKSqbCI0cT+7G/w//wgxi9+Jhf84tMnez7DiAT/mBBCkC/obPfNUxUCpmYH34bNNRe/b10vaKxyuPmm2FhRcjav91b3jHOeWqsUtP3z+H96qpBSIWXQGdz9Ws3MmtQqzeG19y20I0iTZ/MGWxvegfS+LfaJZJS/P0lCI3g4VVH8KCLBP0am50wUQds+BF/i6RmDTHbwbahWwqs4pB80V4V5vPeTyxtUK4HTpVJ7N3AJAYat85Ub7+FO6jIKuFp9wI9sfJOk3xjnKZ44UgZGcOWd4PXTdJidNzuvsWVrXL1ps7nuUa9JlFR4Pj1RtxCBLfSkMc3Wmsaig2y9vaYlyOZ1NkJKNIUG+YKOlIHYxxOjnTlPmppu87XCE9xJXkJTkkfLt3nH9ssY6uxP9zorEfxeRIJ/jAghmJ23mJ7d7YYd9gXWdAFD8s3jLngKEThd1uuSelWi6bC+4oZGt5oO6azBF9/+MUqxNKo1+eRu8gLrsQI/c/eP0dn/APTjZvGu09M17HvB8HHDFCQSgYhblsZCqwRVScXykkul3OsUOsrj/jAkkjo33xTrlMpaloZSikZNUinvXpgBLl0Z7ot02nCFzu9f/AA1I4YSwTm/lH2E1dgUP/Hgz079naIrdHQl0VA89WHvzEbwexEJ/gmgaWLPlEF+SmdteTDqSyTDywVHEY9rnSanRFJjZcntiGJ3KebtxEVqdrLzhQVQQsPRTG6nLnGrcm9fxz1uyiU/1CIisD52sS9rncawNkITXLhs4boSp6mwbO3I7YqFENi26Pl54ZJFo66oVX00XZDJ6B2vnbPA66krNHWr57PjawYbdp41u8Bcc9TI65PjQWyGP595B6VYGk2HYjLGFkn49Nl57fdDJPinlGzOwGkqilu7kWcsHgjDXnieol6TaBoDAzosS+PKdRvZsgLorh8vWmk8MRhRuprJtjnBVdwjYnNtsOa/Ta2qeO37DQxTMDtvkM70fvRNU8OcUFDfaEg2Vl0aDYlptp0yR0fqQgSTwM6qVfKaXcDTBl9AhWDTyp0qwW9H8D/+/76L+bs7aAqQ4EtI7jTRfMnGxcxJn+aREAn+KcJ1FdWyj9CCtMLsvEVhWtFsCUf/5KYwNjdcNte84CJBYMFw6ao9UM4Z1iiUd0oYysftG2RrSpe8UzrUc5sk0ldsrLuUWnn6TFZnesYc6knTjecGZmX6td0UzyRp1INmuvadme8plu45zF0w9z1Afi+kVGysuuzs+B37hdl580QsGHJuCV16+Frvc9RQZLzqsZ9Pm+XYNF8vPM5qfhYtBw+MNA3Ngk/D1FYZ0feR0RTEKy66K/HPoZVFJPinhLZQt1nFZeGSSTpjYOwRHbap13w2uzzyAXxg8W6Tm4/uXdlzpbZMwmtQNjVkK9IXSmL7Ta5Xlw70vCaNUop7d5o4zd0GpuKWT60qMUwxYNYWvo9gulXi2uQFf1Rz3aRtoRfvOtTrsrPgXClJ6tUm1x85+qEz/bypfJdv5t+K3x72Cwjlk/DqXKyvHuu5tCtp/unb/nPKvyuDCL4B/grMiDIbCynqGRuz6YevLQiB4fqR4EccDY2G7Ah1N8uLLolH9bG/vMWt4ZOcalW5Z1pBQ/Hxpc/zlem3cTsZzJy/Vl3k3Zt/dWoWbKsVidPXrdqesJUvGGw7va9j29Stn3HsDQ5Cox7+OkmfAdvqwx6n0SX2nePIoBu3MHW8njsx6fDxpS/wpdmnWbfzCOBSbYX3rT9/LAu2z37qCf6h+1ZeaFfRAPN/UMQOieDzazXqGRsnbmCFib5SuNbZWCzfL5HgnwJKxSG18iLo+Bw3FeCPsOjdq+68TVw6/Nja14CvjfcHx0C3aDfqstcXqL2NDGrn5xZM1lddfAlS00FKRMiLGzuiRrBRdxn6BAPGZjP8DVXqaJrGxqHglvjrS5/HFToChRH2Rk2AdgT/3//k39kV+JAyScsJLwc1PAlKUSrESZaaIHc/X1JAJWsjjfMX3UMk+CeC7ykqlXbeVR9aG69kUFK4+sAlldGZmzdHVm5kMnqn5r5nP4oz2bDzSuoaz089TtVIkPBqvHPzRRaKryE0BkRfaIHYZnMGmZzOtp7i96/8OJde/S5XXn8JveVroQA07chcJ6dmDFaWetM6ojXpS0xw8LxlhXftCkFPBdBJYB5B3f27X/wn/HdfWe6J4PnM6L/xdA3TG7zoyNb74Fk6K1ez5Fdr2HUXqWuU8jHKhdgkT/1UEQn+MdMeNdhhxSWb00Y2RbVnwzYbkms3hxubpbM6xW2PRr035ZHOaJR3fBIprWegxyg8TwXma/XDDUg/KD9IXeUvZt6O11oErBkJvjzzdt6jFMbK9+mXFK3LdkAIwVa8gCYEdx99kmY8wdVXX8R0GpRy0zSuX+ax2otHct6ZrIHnqh6TtmwumHl7EBxHsrbsUq3K4MKR1ZmZDyy1LUsMDLcJLi5n/2v91Ie93gj+F+pAbuTf9LMzHaewWg1y+C2kgFIh1ml2cG2DtSvnsyInjLP/yThD+J7qmSvbZqcoSaY0qiHReTeuq0bm4oUQXL5mUy75lHd8FIEDZrksKbf8Y/YauA2ByNx9I5iA1RmQvu1z5dpgtc9R8Xzh8Y7Yt/E0g29MPc5/ev02y0tOJ3URiwsWLlo9lUcZrxoEv0KwcvVNrFx9EwCa9Hly5/tQO7pzL0yb5AsGnqfQDbGndbLryo6/TyqtdS6svqe4+0az05UbuGX6NJuKK9dtLl+zWX3gUi4FG8TigvkL1pmcg9uO4IFdkd8jgt+LatZG8xW5zXonmioV4pSm4ofb8RkmEvx9IqXC94KBIfutuKgMsUtQKrhFz1+zqJR8alWfZoiTgZLQbCqSqeHHEEKQyRqkMzqvv9IIPuddF5GdbZ9USic5woBtfcXtiEz3sVceOFy7eTy3uxUjEfr7qpHAtDWu3ojh+8ETC1vUnmlukXPKbFnZIJffQkfylp3Xj+akuxCaGMv+orjl9oxYXF+FmVmD/LTZmXbVTZCj3x2ufuGyhWp5Tw9LGbW3Ny0x0JdxUgz4wh8ggt8TIShPxSkXYmi+Qupit415GN2tzueQSPDHRCnF2orLznaXD86sQX4f1RCj1rAUkEjoJBI6O9seqytuaJ563PxsvRZ+t6AUFLe9kYJfrYafaLOhkFJ1IlanKdlsj0W0BFMze0/vGpe0V6VsDl7Z0l61s8A2qnpJAB9d/hJfmnkn95ILgCDrlHhu/XlSfv1Q5yalCipkRNDFfFABdV3JWoi53fpa8P70p+Y69A1XF0KEliJJGfQAdLqPReDMeuWava8hN5MgNII/LoRA7nHXEy81ya/XMFyJrwt2puNUcrFzJ/4TEXwhxIeAXwZ04DeUUv+i7/HngH8L3G796veVUv/zJI59XKyvBmLfPYlofdVDN0So+VkYybQOK3tPgEpnddbXgkqTbkxTjL342l2LH/bYKDTBQI68fZ7tz3+zEQxLaV+UnKaiWmly8Yo1NOX0IDbDV6eeZMvKkvAbvH3rZR6t3And9pnNF/ji7DM9aR1Dejy9+Z3RJ99FTDp8cPXLeELHFxq2HN6JOy6Vss/yotP5WQi4eGXviWWh+xpi09weLm/HBJVyyPulGK8Jr20Q1/57FZSjLi85XL52sIE743IsEfxBUAq77mE1PDxTp54yiVdcppcrnVy/4SvyazVQUCmcr/TPoQVfCKED/xL4ALAIPC+E+IxS6rt9m/5HpdTHDnu8k0BJFVrjrlTwpRpX8IM2e4ONrpp7ISCT04l3dX1qmuDqjRhryw6VcrBY167SGTeajMXFUGFPpgROM2hUCssvZ/MG25u9kWf7otQ+/trq4B2IUrD6wOX6I4NR74o9xWcX3tvpxCxrKf5i5u00NZMnSq8OnMPN6iKswdcLT1A2k6S9Ku/cfJFb1ftjPf9uDOVPxLHRdWRnelc37ca2ccYcdrNX8WTHTrnvfbBjYuhA9W6GlfvWqrLnTm1S/M6v/yxwAhH8uEjF7GIJu+51an2lLlBC9CzsQlCvn9usU8mfryh/EhH+08BrSqk3AIQQvw18HOgX/DNLf6TdjTdGO383henAV6W0E3wZA9vbwYjYNAUXrxw8CqtVwweXA6yt+AjNBwX5KYPpWaNHoKdnDJympFqRPT4+swu76aswkzIIFpbbPvTdfH3q8YG2e08z+Gbhrby19BpaiPzdrC4Gwn9K2Bk2QYwg8h/3wt8mldbYCGtCbV3gDUNw9UawMFurBe9FOrv3onubEW0Z3L/TpNkIFpUL0zq5vLGv1FTobNZDLrIeNZmtOnbd2xV3BcLrW+TqQvMVQoE6P3o/EcG/CHSHXYvAMyHbPSuEeAF4APxTpdTLYTsTQnwC+ATAnHk6bqd0PWjq6R9IAhwoZ23HNGZiRzuXtNkYXfHTjs63Nz10PbgQtRFacLFxmpJmU2FZAjvW+zx1XeCFKIoQ4cNDtq3wqM8XGnXdPhN+++1F4n6UCv9s7IVlaeQLOlub/SvkUK/62LaGZWtcvm4HC7MMt9MOI53W2SmGn1i7wslzFesrHr7HnpbQpz6C34PUTnMgkh81gCyI/o/6rI6XSQh+2EvS/xp+C7iqlKoIIT4C/AHwSNjOlFKfBD4J8Fg8dzItg30IIZiZN1l9MNhQMzN3dC3stWPyIHsAACAASURBVKrPxpqH40hsW2Nq1hjL8EtKRaU8ngIpBVubXo/gtzEtQbXis7bs4XlBKmF23iSR1ClM6ayvDqYbMrlwv5i0W6GhD96xCCDmOwO/P42kUoGAhi2+Jw/Y2OYNmXO8vuqRzRmdypuDLAxPz5lUqz6+H1zgh93xKQVbGx6FaaOT5gn1hD/lEfxehHVct5GCgXr94nT8XKVzYDKCvwhc7vr5EkEU30EpVer692eFEP9KCDGtlNqYwPGPhWzOwNAFG+surqOIxTWmZydTleJ5LR90TZBMaggtENqle7v54ponqd9xRi6KttlYc8cyEWszbLj65rrXk0NuNhSLd4MFv1zBwHUV25u9g0OGNRi9c+sl/t38ewYWYR8vvnJqfHr2IpEK5gp0L4S2L3LjLKKGMSw1pgjSY9YBumaVUmxteGxvBZG7ZQssWxCPa2xteuHvt4DP/A8fpXktiN5/8TO5MzvVqR8hFUpANWOT3m70CLsCXEunOJMgv1YNqnQMjeJ0nGru/HXcTkLwnwceEUJcB5aAnwZ+tnsDIcQ8sKqUUkKIpwEN2JzAsY+VZHp0/Xo/tarP+mogvqYV+KL3DyDfXHfZXPegXVkn4PJVm7XlcNfFtRWX67dGn0N3NdE42CELgFKqgQXD9jlsrLvML5iBWLVsmE1T9ESI/Vyur/Dc2tf46tRTVI04hvR4svgKP1w8O0s9QgQjCks7PqWi3+lqTaUPftHXTRFu66xGl52OYuVBMOKx/d45TYXnKmbnzGDkZYjdgKvp/NXzC7jfOdrqnePErrkUVqqYjg8CqikLz9AwvMBBUwpACDYupHBjBvX00aZZTwOHFnyllCeE+HngTwnKMj+llHpZCPFzrcd/DfhbwN8XQnhAHfhppfYjSWeP/gi92VA8uO8wf3F3vmq95u+24KvdPNj9O8OHbI8TuY9rlAZBhBoWlXtDxitCsD5w73azJx3huor7dwJr3mGdnjeri9yoLuIJHUMNsaY95QgR+PVMytt+atoYqPwRIvC2P8jEK89VPWLfRkr46lM32flHN7j5jz6H1txN+bmGwetvfQuufX7E3nB8Zu+XehZoExWHZtxgZyaBXffwTI1q1kZO0tXulDORT61S6rPAZ/t+92td//4V4FcmcayzwvrKMF90j3RGp16TPX4r/WhauHD3V7+EEU9ooamCeEJjZi4oC3WaCjsmhqalRrXn67oIrU4KWv89pkLWA9oIjsZc66ySSuvMzBvBEHMABcnU7szdvZC+otGQ6HqwsN5syqG5er68zVeuP8rdj5k88/kvkCyV8Q2DV556km+970cn+bROnPRWPXS4iV332Jo3qGXOz8VtP0SdtkeEM8Rv3XODUXuK4Z23iqBkr1KSA5FfYXrvt2xuweTe7d67BE2D2QWTWEzj8hiDPzRNkC/obPf1HwgR5LJ3tgZFW6mj85k/z+QLwTQs11UYuuhE9p6ncB2JaWmhF+DtDZf19nQzFeTqv/PLH+WRn/8sWl+0IIVge2YagKWbN/j9G9cxXA/f0FF7DVg+g4T63AOqNdzEO6d+93sRCf4RoRvhUTCMkXJRQZrFsny2NnbzJoVpg/zU3m+ZHdO4dstme9Oj2Qgi+YO4XU7PmQhNsL3pIWVQtTM7b2IYIlTwhQaJMzqT9aTRtN3B5kopVpYCU7S2mGeyOnMXduvvqxWfzR3Z01Fdbyhu/oPPgVQDg198Q+e773jH7i+EwLOOd0jKcdKMG1h1j/5Po3aOh5uMQyT4R8TUjDGw8DrKArl7m5l5A8PQmJ7VKEwb+GO6LnZjmhqz84dbhBIiSPlMzRidn9skkhq1am+1imkIUpmz/WVqaiavpa5S1WPMNze4XFs59rWG9dVA7LvFvLTjY5iCL/ybvwvA1N/9My413ug5Nw0Qfm9kq4CdfJ6vfPiDlKYKx/QMTp5yPk6q2ERJ1TPcpJa28M2z/Rk9DJHgHxG5vIHyFRutPL0QQQ69WgkP7y0L4smg47E7p65pAm0M18VhKKVoNhSeF5SSHsQ6N6wG/OIVi61NL6gIkpDOBkNFJt2u341EUDKTxHyHmJx87f6aXeCPLjyHQrQWlj2mmkU+tvzFI5ve1I9Sw208lipmp+npI7Va6IWo/3dS01i8dZP1SxeP5HxPK76psXItS36tSqzmIjVBOR+jdM68cfZLJPhHSH7aJDdl4PvBYqvTVNSqzYEvsxAwu7B3ff1+8VzF4t0mjrNrs5Av6EGqpkvElVJUK5JmI7DQTaf3ns4khGBq2hy5QDtJXk1d4cvTP4wvNCQal+or/LW1r03EEA2CSPjfz70bV9t9Pp4w2bDzvJR5hKd2XpnIcfbid3/1Z3jyuf87VMzNZhO7XidWq7F0/Rr5tXWMPVp8dSlJlkojtzmveJbO+qWHZ7jJOESCvwe+r9hccymVglvlTE7fVyQrhOgMrrZjgkxWp9RVNidEkB45ihGES/ebNFtlnO3jbW/52HGtUxrq+4p7t5u4rgq6MTVY11yu3IhhHrOFbhsFrNlT3E1cwFAeGbfMn8+8s6dpazE+z+fnnuWjy38+kWPumKnQTmBfM/hB+tqRCP6zn3oCgL+6fot//EvzwS//GK5Mz1BYXx/Y3rUsfupf/Tq+riEUeIaB7o8ub3VNgwfXr0383E8UpYhXnKCJSiqqaYtKPo46wrvL80Ik+CNQqiWGzq4v+damT60quXJ9+KjBUcxdMEmmA897pYLxd+lsuB3BYXBdNTD+DgLh397cdfhsN4Z1HpfgtYadXL56/KVrCvjz6XfwWvoqntDRlEQJgeqTNanpLMdmqejxQ/vbA4FJ1rDH9vS13EUp1enEjSe0gcCgx49mSCfr7UffRL4l+N1eL6bjoEuJ3orqha4jhUAfsjDkGQbVTJbbb35s7PM/C+TWaqSLux2zZrNOquSwfDUbeHuHoRSxmotdc/ENnVrGeqjq79tEgj+CSlkGkW/390kFU6fqNUkiuf8UjBCCdEbv8b8/CqSvhjpDdWcByjvhKYFaRQYLXkcYNTU1k68VnuD11BUAblbucaW2HIh9K5qXYvjrpCmfmjEZwc94FZJenZKZ6vFPMaTHY6U3xtpHoy5ZvLubstNsjff8jzf45fe+b/yxfUrx2Ldf6Lm8td9Gva+8y/B95JBAwTUMXnzX03zv7W/HN89PNY7u+mSKjZ4ae00FjVbJUpNqLobR9Mhu1LFbnvc7hRi5zTpWw+u4X+bXq2zMp7CaPrrr48YMKhkLZZzvBd1I8EfQqIcbZSkZfLkPIvjHhWWLIG/f/4CA9CGsACaFRPAHF3+MkpHqjCB8JX2d11KX8cJEPmT0nBQaOWcy+WkB/Pjql/nDC+9HCg1faGhKslDf4C2lvUciSqkGOqRlXfKFf3aHN/7e+yA/3nlYjQax2uDA3WGX3TBDME/X+c6zz/DSs+8a76BnCLvuDZScQiD68aqLEzOYv7uDaMU7piuJ1VwUdEo0RauzfeZBZXcHJYf8Wo3iVIzSTPI4nsqJEAn+CExTQ4jBiglNCx47zQghmFswWekami4EA1bI6Uy4hW6iZeJ2VNxLLFA1Ej3zZqWmI0f50bYXGQBderx9+7tYaojz2wGYcnb423f/kNvJS9SMGPP1DeaamyNz5M9+6gn+oftWtn9pnfe8/idYTm/1kJCSmy+9xLd/9EeG7KEXz7KCRqgx/DEUUEsmsZwmphu8Dp6u00gk+MHbnhrreGeNYWkYBfiGRm691hH7NoLwC2bY77KbDdyYeW59dSLBH0F71GC/E4DQOJRh1nGRyRpYtkZx08N1FYmURi5v9JhyzcyZ1GoSz1MdC11Nh/kLR5sG2LJyuKHpmnB51ZWP5TvUjXgrpy4w5OTEvo2pfN5UuTv08bbAQ28e/lbjPiLkdlCXklht/JST1HVee/yHuPXiyxhdZkWSQeESgN1o8OWPfIhHv/0Cdq3O4q0bvPz0O3Fi58/pEaCRMJC6hvBkb7+BgHIuFkT3h9i/RjAoJRL8hxBdF1y5brO86ATVLgriccH8JetIo99JEotpzI/wZdENwfVbNpVyUJZpWUHz1CTr6V2hcz+xgKOZXKyvkvZqZLwKmlKBY2E3QoCSrXy1QqBQCHJOiW0rC60FXF/A16eeIOuVuVJb2fMcXktd5lv5H6Kqx5ltbvHM5gtMO8U9/64t8J0c/JCF1pUrl0PTK65psnjzxp7H6eb5v/Z+dM/jxne/j69raFLhWBaJkFSP1HXKuRx/+jP/2b6OcWYRgtUrGWbvl9A92bGY3ZhP4tk6vqGhOyG2H/ReLMPSQm20USPuzjiR4O+BbWtcuxnrTDs6qGXtaeYoF5JX7Ck+t/DeTqWNQvD4ziu8fftlmHk69G9M6fPM5gtITcOQPnONdX7/0gd70j8QjEh8IfvYnoL/ncwjPD/1RGcheDE+x8rFH+Mnlz7PlLPTs+2wCH4vKrkcrzz1FG964TuYbtAb4JoGGwvzLN24Pt5OWkhd5ysf/hDfeP9zxCtVKtkM7/u3f0j8jduDuWvfp57azTlb9TpXXnsdzfdZvHGdWiaDXatx4c5dfF3nwfXrZ95SwbN0HtzIYTo+QiqcmNFZ3ykVYhRWqwPDTHxdoLdHFjJc7BVQT57P6B4iwR+b8yj0R42P4E8WfhRH7/0CvZR9Exfrqzxavs33MjdDFmMF882NjhhvWlk0JD6DF6SqMbpz0kfjG1OP99TwIwQeGs8X3sr/9L+UAXYnOx1i6Mc33v8+lq9e4ZEXXsTwXG6/5c288ebHDmxO5sRindTMy888zfy9+5hdaR5P11m5cplaOg3A5R+8ynv/6LOoVpfdO7/wZ9y/cZ3Lb9xu5b4FQin+7K//BMvXrh38iZ4GhMC1B+WrmrXRPUl2czeNVkvbbM4lsByfWM3DNzR8XTC7VO5UNQiCtJnUBaWp89uNGwl+xJGxEp9BDthXgSd0vp++wTNb3+HV9FU80RVxKoUvdL6XucmzG99GR5JzyqH715TPpT2i+6oRH6jhB0BovJ69zPs/3ecvIxVW00NqGp69zzseIVi6eYOlfaZwxmH18iX+8oMf4Jkv/Ac0X6JJyYPr1/iLj34YCKp73vtHn+3J+wNce/W14Nl31eK+/998ht/9Bz+HZ5+OSFZIheYHk6YOPVJQCErTCcqFOLrr4xsaqrXQ68Q1nPjuZ+3+rTyZrTrxihPMEU7blAuxc12fHwl+xJHhC53QRgAh8DSDtFfjYw++yBdmn6VsJoMve+sL//30dTyh89z68+hI3rXxbf5y+m2dSF1TPpZ0eVvxeyPP4blfucXv/e8ahKzven0mWomdBlOr1eAHFTy+dimNf0rcFW//0Fu48+bHSBeLNONxmvHdSPTSa68Hkf0YKODy669z+y1vPpoTVQq75mF4Pk7MCI3E29sVVqskd5rBj0JQnElQyR9+wVlpAm/Ycdvb6Bo7M0l2znEZZj+R4B+C9tCuvbpkO92XEuLJwe7L88pCfR0lQoarSJdb5aASZq65RdqrULZSPdv4msFrqas8u/ltbOnylvIbZLwKL+Qeo2okuFRb4cni90n6jYH993Sz/iEUUhWSO83evC6wM7UrLGbDY2qlN/drOj5z90s8uJE7NcOslaZRKgy6Xuq+v7cVawsBGO5kPIgGzsOTzN3dQfdl51pfT5pszSWxmj6+qXUuAG2x351KpcivVfEN7VRUyeiexK65KE1QT5qn5jNwGCLBPwCuq1h94HScL5MpjfkLFkaI90y9Jlm6t9t9qVRgrzCpEXmnGVN5vHf9eb40806kECihY0iXhfoGN6qLne1KZjr07zUkdT3WMUi7VF/jUn2tZ5u2H00nBw8D3axbc0mUgFSx2enQFMDMgyrFgk95Kk56uzEwIUkQfOmthteTCjiNLN68wdOf/w9jbSuk3PdC8rhMLZUx3N6SyUTFJVEpIrVgDcG1DdYvpgYuwhA0UGU3aycu+JmNGrnNenDNEkG92NqVTLBAfIY522d/AkipuPdGo2eea7UiuXu7wY1HYj3RvpSBW2V/D83qA5d4XMOyz2+usM0jlXvMNrd4JXWdhm5xrfaAy7XlHkGYbW4Gufa+uwGFIOnVWI5NsxybISYdblbu8Qe/+lOdbX7x0zlQilSxTnq7iSYVtbTFzlQcabT2JwTbcymatsFUVwWHkIrcZh2EQO+r6+4gQB8x3/e0UE+l+OZz7+XtX/qPaL6PUArP0Kml0iSqVUzXRQqB1HVeePe7Ogu9k0TzJbG6N/A6tn/WZfA6Wg2Pqe4u1z4M92TLIu2aS3azvtvA1Zo9OXu/xOKt/JmO9CPB3yeVsk9Yma7vB9473aWN1YoMtd1qz36dmTv529bjIOtWeHr7xaGPv2PrZe4lFoI0e0v0Denxtu3v8q0PfpDv3UnScDWUgC/NvoO1307STOxG3FPLFRJlpyPk6e0GibLDg+s5VFd1VW6zPiSirFOcihGruQOPo4LpSWeB77/9h1m+dpXrL38P3fe596ZHWL+wwIU7d7n2/VfwDZ3X3vpWNhfmj+T4ol3vuNeQHyBW9wJ3y740lOLkX+9UcfBuD4LnZ9c8msnTfbc3irPxST5FOE011F/HaUroKh2U/nALxj1szB8q8m6Jn1z6Al8rPMFqbBpvxmLFyvK5hXdReK2G1oq02l/CmaVyJ9IyHL9H7CHYVvMlyZ0Gla6BF7oXHjlqUlHNxsjsNMGVnX1JAaVC153CGWBnaopvv7fXxuHB9WvHYpHs6wLP0DDHiNCVgGLL1Kz9eqv276cTR3uie6DJ0Lqu4LEx10lOK5Hg7xPb1hDa4AByTWMgRTPM414ISKWHV340mxKnqbBtcWRpH9eRlEsShSKd1k8svdTOwQP8b59+vOexubs7gxE3gWGY1fBx4gZWI9xeQVMQq7k9gu9aOnZz8EoblO4Jlq/mSG/XSZQdpK5RLsSopx6Ou7CJIAQbF1LM3SuBCmwK2m/fwCQuXaM8FcezDbKbNQxX0owbFKcTuCecJ6+lbWLVsLs9ReOM3O0N42yf/QmQymgYqwJX9n4adEMM+OuYlka+oLO91TvwJJ7QSKYGBVZKxdI9h3pNdiZUJZIaFy5bE63sKW67rC0HQqkUbK55FKYNpmeP71b1d379Z/fVyToMf0j0rQg6MrspziaYWSwPdGFuz8QDywYdStMJSiccYY5Cb1XXnFbLYydu8uBGntROA8PxcS2dzFaQD9fUbhS/OR+U4dbT1rEu0Gq+JFlsYjU9HNugmrMH6u6rGYvUTmsIetc5b80lOzX9EFg1ZzfqxGouvq5Rmoqf+GLzXkSCv0+EEFy5YbO24lIpBdFiOqMzO2+GlmfOzFskUj7FbR8lFZns8IEnaytuZ3hG+wJRq0o2Vl1mFybzQfJcxdqy15M6VQq2NjxSGZ1YbLKRfncEP6qSJoxK1sZqeAORlhICJxaIeTNu4JkapjNoplXJ9dZzN5IW65cy5NarmE0fz9RwbIP0dpNExaFUiJ/aapzkzg7v+dyfMrcYVDetXrrElz/8IarZQ4zwU4pb33mRx7/2PLFajfULF/jmc+9le3bmUOfqmxo7XRfNSi5GqtggVnXxLJ1yITa8Nv8IMRw/MFeTCk2BFA7ZzTor17J4lo6QwSQtoRTrCylidY94Jbjbq2TtnjsP3fVZuL3TSf+YrsR6UKY4naB8ijt1hZpATkoI8SHglwkS2L+hlPoXfY+L1uMfAWrAf6GU+tZe+30snlOfujWerex54AffrYeWUmsaPPLmyXyIilseaytu6HEK0/rEFpJ/8aP/4PA7UYqZpTKxqtsZXAGwdjnTs2ire5LppTJ2I/BKl7rGxkJq5OKa2XCZu1fufGG7o7hq7nQ5TWqex9/89d8gVqt1cshSCBqJBJ/+r/9LpHEw8XzyL77CDz3/fMdaWQGeafJHf+dvU5qamtTpnxpm75eCz1LX7xTQSJiUpmLMLJY7i84C2J4JOnbDyK9USBebg6kqAYuPFE503OKX/tePflMp9Y6wxw59mRVC6MC/BD4ALALPCyE+o5T6btdmHwYeaf33DPCrrf+PaKGUGto3M4Y1+okSWgs/CYRg/WIau+5h11ykoVFNWz231RCkdVavZtE8iVCqt0W/O5fWIllsMLUSdNS2f9teFC6sVlECdF/RSJgnm09WitmlJW68/D3MZrNnwVBTCsNxuPLqa9w5wAhDw3F469ef77FiCPoOPJ74y6/yFx/76CSewamiX+yhVTHUGn2oQU+RRW69NvQzEKsN7qu9Q9PxT229/iTO6mngNaXUGwBCiN8GPg50C/7Hgd9Uwe3EV4UQOSHEglJqeQLHPxcIIYgnNOq1QXWf5IDzVFpnbWWwy1IISGfH/zj0RPCHzMOHIaQit1YltRNEUfWEOZBD7ae7mkb4MujkLAc+KU5MZ3M+hdQ1CqvVEROkYGq52rkS1FMWGxd6xx6Owmx6ZDbrGI5PM2FSzsfwzf1bM9j1Oj/+W79LemcHzffRQq76huuSLu5t8RxGamcHGWLqpinF9PLedtOnBavukVuvYTU9XFNnZyZOY4jbpRKEllsCwQpz30ssFCR3mhRDxNs3dXDCyvWGryudBiYh+BeB+10/LzIYvYdtcxGIBL+LuQWTe7ebyNYItra1zOzC5PLKhimYnTdYW/Ha/SQIAYVpY2j+vh3B/9X1W/zjXzqaGu4elGL2XgmruZu/j1ddFu7usHQjN1L028wuljszTAGshs/8vR12CnsvyHZHevGKQ3KnOVaaJ1Z1mFksdxp27IZPqtjs5Ij3w7N/8u/Ibm0NzLHtxjdNtmcOlm+vpVKdYejdKKCUH3Me4wlj1V3m7pU6r7fue1iLZTYXUtQy9sD21YxNstQcWLR3YqOqvcKvEDuFeHBX0LevRsI894IfFvr0v0rjbBNsKMQngE8AzJmnd/HjKLBjGtdvxdjecmk2FHZMI18wQi0bDkOuYJJM6ZRLQfVQKqNj95VlHnUEPwqr4fWIPbRSLlKR2mkOzau2MRvewGKvICiltevDPWSGzUpNFccQfKUo9HnxCIKa7tx6jY2L43e2ap7HpdffGC32uk41kzmwRYITj3P7sUe59soPetI6vmHw4rNnI9uaX6uFNtLl16rU0tbAXdn2bODnYzZ3n69r62wspLh4e/BOSQmC/YTQbPkD5ddqCBWsAzWSJhsLqdDtTwuTEPxF4HLXz5eABwfYBgCl1CeBT0KwaDuB8ztTGKY4lg5c09IoTAcif+wR/B6YIbXyEHyZ25FYrOqSW6tiOT6eobEzFe+Isun4oR2fGuFDvxnctAexV+sooPkKI6SxS7TOdT9oUo08TycW4/abH+VbP/ojB/baB/jLD/04rmXxyIsvIaSknkzytQ/8GOsXLx54n8fJsKhc9xRCqp4uawClC1auZrAaHmbTx7X1zvCU7ZlEZx4utMQ+ZdFIDL+7ruZiVLM2hiORujgTDXqTEPzngUeEENeBJeCngZ/t2+YzwM+38vvPADtR/v5kefZTT0xk6MdR4A7xoZcCHNvArrrMLJY60Z3pBvl6IRWVQhzH1kMVvH3LXUvb5NeqnWoMBVSyFsmKi+6rgb+phqQH+hlVlSFHDM8xGx6xliNje0Has0y2ZmeZWl3tueOQQnD7zY/xFx/7yJ7nM4pYtcrNl14mWSqzevkS33jfj6JLiWvbZ8onxjc0tJCuXiVGvB9C4MTNgfLbciFOI2G2DN0CP6ZGYgyHTCH2PzfhBDm04CulPCHEzwN/SlCW+Sml1MtCiJ9rPf5rwGcJSjJfIyjL/HuHPW7E+IRG8KdM5LtxYgaupWM2/c74lKBsUlDN2czcL4feyuc26lTyMTzboJEwe7xx2n9fycWQhkY9ZZIoOwgVRHKerVOruczeL3X2187vlsfI37cFu9/mQQoohfm7K0XhQZl0sY7SdJQmyK9WWbuUoZk0+cqHP8iH/r/fRvN9DN/HNQxc2+ab73vvvl/PbmaWHvCB3/09hJQYvs+tl16mnMvxub/900PFTfgKpXEqLgZW3SO/VsVqBF48EnpG7LTtMIadq+H4xCsOSghqaasnKndjRugC7XliInX4R8XDVoc/SXoi+DNIf5VNI2GwNZ/Cs3Qu/2ALTQ5+bpWAxZv54EssFbmNWssSWdFImmzPJvdcPNW8wINH9xTNpLkvH3QhFTOL5WCdoFURUs7abM8le/YhpOTpf/9FHnnpRTTp04gnefXxZ9iau4yvCRYfCXyCYtUat77zIrnNTdYXFnjjrW8JovCDohR/89f/L1Kl3glinq7z0rue5oX3vLvn93bNpbBS7aTIqmmLrfnUgWrMe17XhEk9tX9/ebPhMd9nt9Gqb+gsvpRzMYqziYF9665Pbq1Gsux0+jkg6PitZU9X38VhOdI6/IjTwbOfeuLMRPDjoHSNzQtpNkPq6F1TC/XEUaIrfaIJirNJirP7m2YkDY3y1MGsFZQWeKYbjo/uSlxbD83rPvP5L3Dzpe+i+0EOOl6r8EPPf5EX3v1BitOzxGoejaRJI5ngpQkuoKaLO8Rq9YHfG77P9e9+v0fwDcdn9v5u2gwFibKD7pVYu5Ld13Ht/junYgPX1lm9kt3XxSO3UQudWaAEPLiWDUolQ/aXW6uS2Wp0tu/ex9RKlUbSOhP590kQCf4ZZSCCP+MCP5SQKLA4k2BmadATZ2fErfxx4ln60DsJs9nk5osvY/SVRGrS59or3+bb0z/Onv7CByTwjAnft9R7zzfd8r/pOUcFdt3DaPrD89ZSEa866J6ikTDwLJ3pB73vlaaChfnUdn1fF1er4Q/vnxAiVOxjFScYbjNiv4mKM2DDcV6JBP+M8O4X/wkAz/1CK0I7rwI/Bo2UxeZCivxarWN53G6ScWIGjVPscJmoVALh7RN8AcSrQRTcOCI/n2omQymfJ7e+0ZP3dg2DV558omdbszlEXIXAcMMFiUe1tgAAGYJJREFU32x4zN0vBRVGLYGvJ0w0f/AioylIlZx9Cb5raaGVUAD+kN6MdLER6rj6sBIJ/illIIL/hcFb8YeZWsZGyFbte+t3liuZWSqzfjF9akW/ksmErj9IoJydYuNCOjRS3Q9mI1jYjNU9pCYo5WOUpoK7ny9+/Cf40G/9DobrokkfEDy4fo0fvO3Jnn004wZ23WNARqUis1VnZqmM0gSVrE1xJhDtmaUymt/rJR8PsTNos18d3plOYHenmQju7CpZe6AEs40Iea37qQ/pzD2PRIJ/Sogi+H2igoamfkHSFOTXayyfUsH3TZOXnn5nj2kZgDQMvv6fvPfQ/vsdR8hO96kiu1nHcCVbCynKhTy/93P/FZfeuE2iUmH9wgJbc3MD+ykX4qSLTVTXMBBJ23umNcbQV6S3G5iOT3E6ETomUiMQ5XYJbOf5Cqjk9rcA3UwEjU2F1p2dElDOxzoXnDBqGTu4cPU7rhLk/rdnE/jmw5G/h0jwT5SOJzxEEfw+EYqBmvk2hnP048SEVCSLDeI1F8/QKLfKQcfhhfc8Sy2V5PGvfp1Yvcbm/DzfeO59bC7MHvq82rn3bnEN0idNijMJpKGhdJ37j9wauR9paKxcy5JbqxKvuUhNBANk+sRTU0FjmZEd/pp7pobuq55ou540D5Q3r2dsltIWQjJWqWgla3f877XdTBO1lElxjKqt80Yk+MfEs596AvHOD+xG8DCWJ3xEOEqA1ERnMHY3R+1lInzJwp0ddE92RCS102TjQnq8ARhC8OpTT/LqU0/uve0+sUOGiEPQtGU6Ps19vDaepbNxaddvf6YvndKh5RAZZkwmCfzwy7kYiYqD7kmaCfNwbpKtYTXjbrt6NUOi7BCvOPi6RiUXO1PNUpMkEvwjpCeC/zTw6SiKnxhCsDMdD9I6fTndUbf4kyCz1eiIPeyW+k2tVFhM5U+0Usi1dayQBVehFN4BXDv79y2r7mBeX0G8Em4fIQh6EdBEqKHZgVFq/NdZBMee6PHPKJHgT4gogj9+yq0O1uxmHc1X+IagOJPAtXVyq1V0X1JPWaFGWt1YdZdkyQGC8XZ7Tb3q76ZtI6QKPFoOEb3qrk9mq4Fdd3EtnVIhvq/9labinQ7iNlIENs+HzVWX8zHSxUaPjXC7G9lwwqt6lABdKvxJBNStdZt0sYGQgeX11lwK54zPmT1OolfqEEQR/AkjBOVCPBD+ltVlcqfJ/J3dRctE2SFdNFi9nAkV/dxaNajTbglkqtgIFgJHNGzJIVU07QEq1bRFNRfbd0eq4fjM39kdm2c1fBJlh/VL6aEe7/24tsHa5QyF1WCMo2otjm7P7K8BLQzf1Fm5kmVqpYLVCLpva2mLrbkUc/dLGH64mVn/zNiDMrVc6bnY2g2fuXs7LF/PPXS5+IMSCf6YtCN46KqkiSL404EQgZWBVBRWqwOLilbdI1lyqGZ7b+nNpkd6u7dOWyhIbzeoZO2hi7DlQgxruTLQ4g8QqwfWzOlig5Vruf11kq5VO2IPXVO4Vqo8uDFoRaB7Es2TeJbec5xmwmT5ei502tdhcWMGK9dyIFtX1Na+i9Px0Ga4Ss7e+zVQKlgM9hXNuBHa9aq7kmTfnQsEr09ms87WKbclPi1Egj+CKII/W9g1FyUE/bMiNRVUr9TSVo/4xCtu6EKjUJCouJSGCH4tbWHXbVLFZmf7/qoYw5XhnaRKofkKpYkBIeyUO/ZhuJLcWhXDkzTjJtWUxfRqNXDZbD3f7ZkElf45AUe5ltB37o3/v717+W3svg44/j338l6Kb0mjkTQvj8eJUcQo2gQIjARZ9BUEjhdNE8BtsiiyCGCgaNAW6KIB8g901V0WNdqiXTTNA60RownqNEFbb4oi6WNhd+Jm4tieGUkjjV58k/fx6+KSHFK8pChREknxfADDkkjx/sSRzr339zu/c7Iuu+sZlrerWKHB0FXXZohEI2DtfhErDDFE793BlRTFld7vS3hBbMcqAdxG/J2F6qcBv0Wv4GeYMWQOG9ECblxRNcBtBNy8t8/2zVynAbqR+KYnRugpsNVHhP21LMXlFJmDBoUBZQjSJY9SVy/wVLER3YG0ruLL+SR7a5lO8Azt+KwjgNx+A4voJLW4XY2GwZP6/ks7VQLXHjuPfxzVwgLVfBIrNNG013EnHGNYvV/sy98v7NZophzqXU3ofTe+5LUB/IR1skXcOTb3Ab/T2Umv4GdW4XGN/F5t4Bb6zhRJaFh9UOL+B5eirJGcy+JONTaQDOp01C1wbGo5l8Je/O9Ndx38ZNVj5cg0UNRuz0S7a4nKKC8dyTpqn5Dar9ROA+3b4NSa2phkwAdAZGj9/25uPcAO+jdriYHsfq0n4AcJi2pM+WmIdvOubJRP1Ht4Xs1VwO/bzapmngQmNti3P437809VPGo5l8Cx2VvLsPyo0vP47npm5MbjXtLGdyycZti3k7S0/GRjUX530F1AE8sPCRMW5aUFHC8gd9AgFJAwfvyDQpo9oM7MJFlBiOUbfNfqC8Zxd2PQag0Zs6ludz1DohmQ7Cqi1l7nSJWbpEtNTb08xqUP+D29WTXQXzqOF9/OcOB1nultH1hZXKCWdUlVmoBQyzonyyoRYedmvjM1gQhiDIdXUj2ZNW4jfn7eiGC3An57qujwShqnGRBYwvV3D0cahoGeK+JJk9BwZbNEuux16tVXsi5uM0DC6A6qsWAP3KwVd4eVKXmxewyg3Xu4rgH/GJcq4OsV/PzxE1Zs0Bh2hX+0T2mYsKiM0QTDd202nlmMGqcHhmYq0XPSSJUa2P6A+SZj+lIKw4TV2RFbyzhDC5BB9LOGlnB4ZXBzd8sLsMLWsS5g2uPKZqmzKC6tQWaLzc7Pkdit9UxVtRmiqbC4pvHDpu3UaGY+4M/aFbxbr3Pn7k9Il0ps37jBxp2nx2pEPe/ChEU165IqN/vmvpupRBSETVexrFY9mTPX6pXaxxiWtqvxV/dEG6WGpS3uXoty3NuN3eMCZDXrsL8WPw1lN3zW3y/21B3aW43J6DlDVhCSjsmA6slkIr5apkBsBhNEXbMGCWGsk/a8mKmA376C/6/HP3/S2WmGXNnc4lPf/DYShji+j+c47F9d4fu/8xKBMz2347Pm8bUsy48qZItRmmRgW+ytZahlo6vjdLFBaFuUC8mxdsGeSitFc8BDQ6/KITqhbT5dYPV+kVS1P/3QWFDNJePXHIzhetcmtLbl7SqeY9HInc/0h+Wb2IXlowY2Mxkwt1/POGS67hLa2tNZlfx0VkidJlMf8D/8aZ8XrT+IPulcwc9esMcYfuW1f8RtNjtfcjyP5e0dPvSf/82bH3t+goObcZawdy3L3lomSgm0n6QE1rLu+WWuhAanGRAmrMEF2wakfp6ICLWsG1vmFwPeQvwCc+aw0Rfs265ulnlwTgE/WqDl2IL3g96XYECWz8HVNKmy10lrbb/8wUqqU+9fDTfVcwkPF68+CfYzLndwQKpS6ft6wvf5wFtvTWBEl5AlncXP85bdr3Hrp3usv1/kxs/2WX2/iAQxV/ISpX/Gxj6JNluNoto6aXW/TijReoQ3YIOYM6BMtABWSN8GtTMjEk2ddf0zGEZreGJg4Ak6cGw27yxSXFqgkbSp5ly2bheiTVoa7Ecy9Vf4l4UZ8gtpZKrPu+qIhXKTpe1WvnwraCarHlcfltl+Kt/3fN+N//cVA27dPz67xhjWHhR7mogYoivhnRuDSwrUM06neXfs8UNGLzN8QuWlFL5jd5qvGAyJ1lQPRCerMGGR8HrTWY0Vfe8ggWNxsDZ+XaAnBzS4jQArCGksJDBnVPdnWmnAvyDlQoFyPkdhb7/nF9xPJPi/X/rFiY1LnVxctogFJGseth/2Te/4jh1bFsBI1BzkOOlSE9sLe27HhagBjNsIaKbiX6OedgZmK/k2A9sCnpV61qWRdrj+zgEJv7dzlu/abN/KsbxVId0qrewlbXavZS+sA1WiGXSl0wKG+BIVl4gG/Isiwr/91m/ywte/iYQBth8Q2jbbN67z9kc+POnRqRMYtAiLCFZMwK/mkyxtVzHG9FyhG0s6UzXDxM7d06r5U2wOLucswtZTOa69X+rMl7enVvau99+JHCs0LD6ukj2IdgjX0w57a8O7RqVLDawju2ktoukmpxFGDVbCaG/EhV5dt8o6dO4wWu/v0k4VbyHRKb9x2WjAv0AHKyt8+/de5qmf3iNdLrN94zo716/r/OOMqWUcEgeN/gVHY/Bigp+xhK3beVY2yrjNAExUy33UhuW+YxEKfUFfgNxBncOr6YGpnV7aZeOZRfJ7NZI1H8+1ObwypMZ+q3JlutgAESr5ZKfe/NWHJRaqXmccCxWP9XcP2XhmcWCqq1sPBubOuw2fRsYBSzDjLWufmNMIYnvwSqvQngb8GCKyDHwTeBp4F/htY8x+zPPeBUpAAPjGmI+Oc9xZFjgOP3/uQ5MehhpD8UqKTLHZU8o4lKgh9qAA7icTbN1Z7OSSn2QvQKWQZHGnGp/V0iorMGyHqe/a7K2PUD7YROWlM10ns9x+nXo6wf7VTE+whyfF27L7dYoDuox5bvzJCiH25HhR7MAM3KE9qFfyZTDuPdRXgB8aY54Fftj6fJBfM8Z8eJ6DvbocomyRAqXFBZquRS3tsHMzF7vYKEFIYafK9Xf2Wf/5Aelyc+TiYm2hHRUOi/0uia87cxpu3SdzGFXllK7/Fqo+q/cPY3c0WwaS9cGZRpV81Aim+1ujBWdroqUgGguJ2LShUEYrnDerxp3S+Qzwq62P/wb4V+BPxnxNpaZe4Njsr2eAIRkjoWH9vUMS3pP+t86jCsmqx26rQuaoKvnkwNaKR0tFnNbR1ohtAtgDli1CosXWQYwtbN0usLxV7qSgVrNOdMcxwalMY0epo0s71c5ehbC1iF6OKetwWYwb8NeMMZsAxphNEVkd8DwDfF9EDPDnxphXBr2giLwMvAyQzF8dc3hKTU6m2OgJ9hBdEWeKTYpLHt4xvXO71TMOjVSiZwE3lGi6x28HXGOw/RAjcqryEcNKPAwqDYH1pLfwIL5rs/1U4Vw6cI2j3OoXnNuvY/khtZxL+RStKWfJsQFfRH5A/NbWr57gOJ8wxmy0Tgj/LCI/Mca8EffE1sngFYDctWcv72SauvSOznl3W31Y4uEHlkYPfiJs38qTOWyQKTYwIpQXk51NSm4tqgnfXohsJm12buRGLvMM0V1E/nHtRMunW08VRj/GlAT6bo20c2kXaOMcG/CNMZ8c9JiIPBKRa62r+2vA9oDX2Gj9f1tEXgWeB2IDvlLHsfyQhBfiu9aZNcg+D37Cil1oFaJ6M8lqK0tlVBJVkTxaSdLyQ9beL/acXNx6wPp7RR5+YHHkQOu7Nntraa48etJRaxhjycXXJlJjGfev5TXgi62Pvwh85+gTRCQjIrn2x8CngDfHPK6aR8awvFnm5s/2Wbt/yI17+yxvlc+vRMCYysdMdQwqfXBS2YP+3bRRE5GQhYp3oteqLKXYvJ0nSAhGommjuLIIoUBpUWvPz5pxT89/CnxLRL4EvA+8BCAi14G/MMa8CKwBr0p0lZEAvm6M+acxj6vm0OJOlUyxVRCsFYEyhw38hNXX9HoaBI5NcTFJPi5nXxi6Yekkjq4T9Dx2ii5YXsrh4QeXSTQCrDCk6dqsbJZJVaKm6WIMtYzDwYBUTDW9xgr4xphd4Ddivr4BvNj6+B3gl8c5jlIYQ+6g3l/SwEB+vz6VAR/gcDVDttjA6mpX2C4tUE+fzXRII+20+uPGPDbGlEu0GBydlB7fzGN7AU4zwHPtE60NjMwYcvv1aBE1fHJSOZdjzSmdgFMzQwZcrJ5VHvp5MJaweWeR5a0KqYoHrTzv/bXMmS1iVnIuhV0Luq70Q4kye046xy5BSGG3RrrUBIFSYSHqzStC4JxToG9Z3qr0nLgyxSapsjd0J686GQ34ajaI4CVt3Eb/vPc4V7EXIXBsdm7lo7WG88hUsYTN2wUKuzUyxSbGgtLiwrHpkn2MYf29IgnvSTmExcdVFmoeOzdPUXvnBGwvINuarmtr7+TN7UflI9T4pvsvRakue2sZVu8XOxtlOm0Lz7Jc7nk6x7REY1scrGY4WD39e5EuNXuCPURTZgsVD6fun2tGjtMIOusD3SwDyWoT0IB/FjTgq5nRSDtsta5knUZAcyEqBOYPaAAyFUJD9qBOtpM7vxC14juv4D/GXURyyL6BZO18A77vWLHZVgZYqAUUdqocrmhXq3FN8V+KUv28hQSPb5ysLMGFCQ35vRrZVmvBSt4lWfFwm0+umt16mYWKe+LSCkMZQ26vTmG3hhUafMdi/2qa2pCCanGCAVU5Ec69Rr2fTNBM2iTrQU9GU/vj/F4N37WpFDQVdBy6EqLUWTCG1QfF6O7DC0n4Ifm9OslG/xRJutTEaYzW2nAU+d0ai4+r2K3qnY4XsrJZZqHSPPZ7u5UL/XP+BggtoXYBhc4G9gWmlY21Vxv4uBqNBnylzkCy5vc1KmlXm4x9/oi9bI9lDIW4Dlwm2rdwEmHC4tFTebzWlb6RqG7/o6cK5z6VYgUh6Yo3dHevdYo9BaqXTukodQbcuj9al24A4cQlkgexAjPwuInmyQNkM+Ww8cziWEXYTsPyTWwbyDYDEy2nfFlowFfqDPiOFd0vH4mxR2vpRGUKRmttOIrQlujqO2bB89QNRlo59xdpWG/f9rTS4ZRurpslOqWj1BmoZV1C6W/0YQQCC0Ir2gwVtKZNRmltOBIRDq6kCI+8XChwsDpDAdIS9lfSPT9Hu4ZPJe+yeadwZqUo5ple4St1FkR4dLvAykYJtxFgiK5aH1/PdTaMGVrNQs54Pry0HNVwL+zWsP0Qz7XZv5qaubK/5eUUgWO3fo6ARioqraCB/uxowFfqjPiuzdbTUd9aMb2pjM3z3A0sQnlpgUohyfJWmUypyerDMr5bZXc9O1OBv5ZzqV3iFoOTplM6Sp2xMGGde956nKsPSp02hQI4zZDV+0USMeUo1HzSgK/UJZBoBiRr/TtlxUB+X/PXVUQDvlKXQMKLatEcJUR1apQCDfhKXQqem+grPAZRlmgjpUt1KqIBX6lLIHAsKvlkf1qjJZSWUhMbl5oueupX6pLYW8/guTb5dseodIKD1cxEFpDVdNKAr9RlIULpSorSFb2iV/E04CulxuI0fJYeVUnWPEJLKC0vUFzW2vXTSAO+UurUbC9g/b1DpNWk3QoMhcc1Es2QvWvZSQ9PHaGTe0qpU8vv1jvBvs0ykC02tJzxFNKAr5Q6tWQ9voa9Ec3/n0Ya8JVSp9ZMJuLL8RvwXQ0v02asfxEReUlE3hKRUEQ+OuR5L4jI2yJyT0S+Ms4xlVLTo7i8gIkpzVzPOBdeU18db9xT8JvA54A3Bj1BRGzga8CngeeAL4jIc2MeVyk1Bfxkgu1beZquHTUqEagUkjw+yybt6syMlaVjjLkLIMPTr54H7hlj3mk99xvAZ4D/HefYSqnp0Eg7bD6zCGGrTKemY06ti5hkuwHc7/r8QetrsUTkZRH5sYj82KsenvvglFJnxBIN9lPu2Ct8EfkBsB7z0FeNMd8Z4Rixi/iDnmyMeQV4BSB37dlR20IrpZQ6xrEB3xjzyTGP8QC41fX5TWBjzNdUSil1QhcxpfMj4FkRuSMiLvB54LULOK5SSqku46ZlflZEHgAfB74rIq+3vn5dRL4HYIzxgS8DrwN3gW8ZY94ab9hKKaVOatwsnVeBV2O+vgG82PX594DvjXMspZRS49GtcEopNSc04Cul1JzQgK+UUnNCA75SSs0JDfhKKTUnNOArpdSc0ICvlFJzQgO+UkrNCQ34Sik1JzTgK6XUnNCAr5RSc0IDvlJKzQkN+EopNSc04Cul1JzQgK+UUnNCA75SSs0JDfhKKTUnNOArpdSc0ICvlFJzQgO+UkrNCQ34Sik1JzTgK6XUnNCAr5RSc0KMMZMew0AisgO8N+lxTMAK8HjSg5hy+h4Np+/P8S7re3TbGHM17oGpDvjzSkR+bIz56KTHMc30PRpO35/jzeN7pFM6Sik1JzTgK6XUnNCAP51emfQAZoC+R8Pp+3O8uXuPdA5fKaXmhF7hK6XUnNCAr5RSc0ID/pQSkZdE5C0RCUVkrlLHhhGRF0TkbRG5JyJfmfR4po2I/JWIbIvIm5Mey7QSkVsi8i8icrf1N/aHkx7TRdGAP73eBD4HvDHpgUwLEbGBrwGfBp4DviAiz012VFPnr4EXJj2IKecDf2yM+RDwMeD35+X3SAP+lDLG3DXGvD3pcUyZ54F7xph3jDFN4BvAZyY8pqlijHkD2Jv0OKaZMWbTGPNfrY9LwF3gxmRHdTE04KtZcgO43/X5A+bkD1WdDxF5GvgI8B+THcnFSEx6APNMRH4ArMc89FVjzHcuejwzQGK+pnnF6lREJAv8PfBHxpjipMdzETTgT5Ax5pOTHsOMeQDc6vr8JrAxobGoGSYiDlGw/1tjzD9MejwXRad01Cz5EfCsiNwRERf4PPDahMekZoyICPCXwF1jzJ9NejwXSQP+lBKRz4rIA+DjwHdF5PVJj2nSjDE+8GXgdaKFtm8ZY96a7Kimi4j8HfDvwC+IyAMR+dKkxzSFPgH8LvDrIvI/rf9enPSgLoKWVlBKqTmhV/hKKTUnNOArpdSc0ICvlFJzQgO+UkrNCQ34Sik1JzTgK6XUnNCAr5RSc+L/AZT924qpnsquAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/src/LectureNotes/_build/jupyter_execute/logistic_42_0.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "from sklearn import datasets, linear_model\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "\n",
+    "def generate_data():\n",
+    "    np.random.seed(0)\n",
+    "    X, y = datasets.make_moons(200, noise=0.20)\n",
+    "    return X, y\n",
+    "\n",
+    "\n",
+    "def visualize(X, y, clf):\n",
+    "    plot_decision_boundary(lambda x: clf.predict(x), X, y)\n",
+    "\n",
+    "def plot_decision_boundary(pred_func, X, y):\n",
+    "    # Set min and max values and give it some padding\n",
+    "    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
+    "    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
+    "    h = 0.01\n",
+    "    # Generate a grid of points with distance h between them\n",
+    "    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n",
+    "    # Predict the function value for the whole gid\n",
+    "    Z = pred_func(np.c_[xx.ravel(), yy.ravel()])\n",
+    "    Z = Z.reshape(xx.shape)\n",
+    "    # Plot the contour and training examples\n",
+    "    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)\n",
+    "    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Spectral)\n",
+    "    plt.show()\n",
+    "\n",
+    "\n",
+    "def classify(X, y):\n",
+    "    clf = linear_model.LogisticRegressionCV()\n",
+    "    clf.fit(X, y)\n",
+    "    return clf\n",
+    "\n",
+    "\n",
+    "def main():\n",
+    "    X, y = generate_data()\n",
+    "    # visualize(X, y)\n",
+    "    clf = classify(X, y)\n",
+    "    visualize(X, y, clf)\n",
+    "\n",
+    "if __name__ == \"__main__\":\n",
+    "    main()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The Credit Card example\n",
+    "Here we use the the [credit card data](https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients). \n",
+    "The data are from an extensive database from Taiwan and include more than ten predictors.\n",
+    "\n",
+    "For categorical data -Scikit-Learn- provides a so-called **one-hot encoder**.\n",
+    "This is called one-hot\n",
+    "encoding, because only one attribute will be equal to 1 (hot), while the others will be 0 (cold).\n",
+    "**Scikit-Learn** provides a OneHotEncoder encoder to convert integer categorical values into one-hot"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sklearn.preprocessing import OneHotEncoder\n",
+    "encoder = OneHotEncoder()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## How to read the Credit Card data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ImportError",
+     "evalue": "Missing optional dependency 'xlrd'. Install xlrd >= 1.0.0 for Excel support Use pip or conda to install xlrd.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)",
+      "\u001b[0;32m<ipython-input-4-4edad39a9543>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     19\u001b[0m \u001b[0mfilename\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcwd\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'/default of credit card clients.xls'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0mnanDict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 21\u001b[0;31m \u001b[0mdf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_excel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheader\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mskiprows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex_col\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mna_values\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnanDict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     22\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     23\u001b[0m \u001b[0mdf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrename\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m\"default payment next month\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m\"defaultPaymentNextMonth\"\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minplace\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_base.py\u001b[0m in \u001b[0;36mread_excel\u001b[0;34m(io, sheet_name, header, names, index_col, usecols, squeeze, dtype, engine, converters, true_values, false_values, skiprows, nrows, na_values, keep_default_na, verbose, parse_dates, date_parser, thousands, comment, skipfooter, convert_float, mangle_dupe_cols, **kwds)\u001b[0m\n\u001b[1;32m    302\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    303\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mio\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mExcelFile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 304\u001b[0;31m         \u001b[0mio\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExcelFile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mio\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mengine\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    305\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    306\u001b[0m         raise ValueError(\n",
+      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_base.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, io, engine)\u001b[0m\n\u001b[1;32m    822\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_io\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstringify_path\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mio\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    823\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 824\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engines\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_io\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    825\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    826\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__fspath__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_xlrd.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, filepath_or_buffer)\u001b[0m\n\u001b[1;32m     18\u001b[0m         \"\"\"\n\u001b[1;32m     19\u001b[0m         \u001b[0merr_msg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Install xlrd >= 1.0.0 for Excel support\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m         \u001b[0mimport_optional_dependency\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"xlrd\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextra\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merr_msg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     21\u001b[0m         \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     22\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/compat/_optional.py\u001b[0m in \u001b[0;36mimport_optional_dependency\u001b[0;34m(name, extra, raise_on_missing, on_version)\u001b[0m\n\u001b[1;32m     90\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mImportError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     91\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mraise_on_missing\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 92\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mImportError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     93\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     94\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;31mImportError\u001b[0m: Missing optional dependency 'xlrd'. Install xlrd >= 1.0.0 for Excel support Use pip or conda to install xlrd."
+     ]
+    }
+   ],
+   "source": [
+    "import pandas as pd\n",
+    "import os\n",
+    "import numpy as np\n",
+    "\n",
+    "\n",
+    "from sklearn.model_selection import train_test_split\n",
+    "from sklearn.preprocessing import OneHotEncoder\n",
+    "from sklearn.compose import ColumnTransformer\n",
+    "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
+    "from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score\n",
+    "\n",
+    "# Trying to set the seed\n",
+    "np.random.seed(0)\n",
+    "import random\n",
+    "random.seed(0)\n",
+    "\n",
+    "# Reading file into data frame\n",
+    "cwd = os.getcwd()\n",
+    "filename = cwd + '/default of credit card clients.xls'\n",
+    "nanDict = {}\n",
+    "df = pd.read_excel(filename, header=1, skiprows=0, index_col=0, na_values=nanDict)\n",
+    "\n",
+    "df.rename(index=str, columns={\"default payment next month\": \"defaultPaymentNextMonth\"}, inplace=True)\n",
+    "\n",
+    "# Features and targets \n",
+    "X = df.loc[:, df.columns != 'defaultPaymentNextMonth'].values\n",
+    "y = df.loc[:, df.columns == 'defaultPaymentNextMonth'].values\n",
+    "\n",
+    "# Categorical variables to one-hot's\n",
+    "onehotencoder = OneHotEncoder(categories=\"auto\")\n",
+    "\n",
+    "X = ColumnTransformer(\n",
+    "    [(\"\", onehotencoder, [3]),],\n",
+    "    remainder=\"passthrough\"\n",
+    ").fit_transform(X)\n",
+    "\n",
+    "y.shape\n",
+    "\n",
+    "# Train-test split\n",
+    "trainingShare = 0.5 \n",
+    "seed  = 1\n",
+    "XTrain, XTest, yTrain, yTest=train_test_split(X, y, train_size=trainingShare, \\\n",
+    "                                              test_size = 1-trainingShare,\n",
+    "                                             random_state=seed)\n",
+    "\n",
+    "# Input Scaling\n",
+    "sc = StandardScaler()\n",
+    "XTrain = sc.fit_transform(XTrain)\n",
+    "XTest = sc.transform(XTest)\n",
+    "\n",
+    "# One-hot's of the target vector\n",
+    "Y_train_onehot, Y_test_onehot = onehotencoder.fit_transform(yTrain), onehotencoder.fit_transform(yTest)\n",
+    "\n",
+    "# Remove instances with zeros only for past bill statements or paid amounts\n",
+    "'''\n",
+    "df = df.drop(df[(df.BILL_AMT1 == 0) &\n",
+    "                (df.BILL_AMT2 == 0) &\n",
+    "                (df.BILL_AMT3 == 0) &\n",
+    "                (df.BILL_AMT4 == 0) &\n",
+    "                (df.BILL_AMT5 == 0) &\n",
+    "                (df.BILL_AMT6 == 0) &\n",
+    "                (df.PAY_AMT1 == 0) &\n",
+    "                (df.PAY_AMT2 == 0) &\n",
+    "                (df.PAY_AMT3 == 0) &\n",
+    "                (df.PAY_AMT4 == 0) &\n",
+    "                (df.PAY_AMT5 == 0) &\n",
+    "                (df.PAY_AMT6 == 0)].index)\n",
+    "'''\n",
+    "df = df.drop(df[(df.BILL_AMT1 == 0) &\n",
+    "                (df.BILL_AMT2 == 0) &\n",
+    "                (df.BILL_AMT3 == 0) &\n",
+    "                (df.BILL_AMT4 == 0) &\n",
+    "                (df.BILL_AMT5 == 0) &\n",
+    "                (df.BILL_AMT6 == 0)].index)\n",
+    "\n",
+    "df = df.drop(df[(df.PAY_AMT1 == 0) &\n",
+    "                (df.PAY_AMT2 == 0) &\n",
+    "                (df.PAY_AMT3 == 0) &\n",
+    "                (df.PAY_AMT4 == 0) &\n",
+    "                (df.PAY_AMT5 == 0) &\n",
+    "                (df.PAY_AMT6 == 0)].index)\n",
+    "\n",
+    "from sklearn.linear_model import LogisticRegression\n",
+    "from sklearn.model_selection import GridSearchCV\n",
+    "\n",
+    "lambdas=np.logspace(-5,7,13)\n",
+    "parameters = [{'C': 1./lambdas, \"solver\":[\"lbfgs\"]}]#*len(parameters)}]\n",
+    "scoring = ['accuracy', 'roc_auc']\n",
+    "logReg = LogisticRegression()\n",
+    "gridSearch = GridSearchCV(logReg, parameters, cv=5, scoring=scoring, refit='roc_auc')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic.py b/doc/src/LectureNotes/_build/jupyter_execute/logistic.py
new file mode 100644
index 000000000..39168e60a
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/logistic.py
@@ -0,0 +1,544 @@
+<!-- dom:TITLE: Data Analysis and Machine Learning: Logistic Regression -->
+# Data Analysis and Machine Learning: Logistic Regression
+<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->
+<!-- Author: -->  
+**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
+
+Date: **Oct 17, 2019**
+
+Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+
+
+
+
+<!-- !split  -->
+## Logistic Regression
+
+In linear regression our main interest was centered on learning the
+coefficients of a functional fit (say a polynomial) in order to be
+able to predict the response of a continuous variable on some unseen
+data. The fit to the continuous variable $y_i$ is based on some
+independent variables $\hat{x}_i$. Linear regression resulted in
+analytical expressions for standard ordinary Least Squares or Ridge
+regression (in terms of matrices to invert) for several quantities,
+ranging from the variance and thereby the confidence intervals of the
+parameters $\hat{\beta}$ to the mean squared error. If we can invert
+the product of the design matrices, linear regression gives then a
+simple recipe for fitting our data.
+
+
+Classification problems, however, are concerned with outcomes taking
+the form of discrete variables (i.e. categories). We may for example,
+on the basis of DNA sequencing for a number of patients, like to find
+out which mutations are important for a certain disease; or based on
+scans of various patients' brains, figure out if there is a tumor or
+not; or given a specific physical system, we'd like to identify its
+state, say whether it is an ordered or disordered system (typical
+situation in solid state physics); or classify the status of a
+patient, whether she/he has a stroke or not and many other similar
+situations.
+
+The most common situation we encounter when we apply logistic
+regression is that of two possible outcomes, normally denoted as a
+binary outcome, true or false, positive or negative, success or
+failure etc.
+
+## Optimization and Deep learning
+
+Logistic regression will also serve as our stepping stone towards
+neural network algorithms and supervised deep learning. For logistic
+learning, the minimization of the cost function leads to a non-linear
+equation in the parameters $\hat{\beta}$. The optimization of the
+problem calls therefore for minimization algorithms. This forms the
+bottle neck of all machine learning algorithms, namely how to find
+reliable minima of a multi-variable function. This leads us to the
+family of gradient descent methods. The latter are the working horses
+of basically all modern machine learning algorithms.
+
+We note also that many of the topics discussed here on logistic 
+regression are also commonly used in modern supervised Deep Learning
+models, as we will see later.
+
+
+<!-- !split  -->
+## Basics
+
+We consider the case where the dependent variables, also called the
+responses or the outcomes, $y_i$ are discrete and only take values
+from $k=0,\dots,K-1$ (i.e. $K$ classes).
+
+The goal is to predict the
+output classes from the design matrix $\hat{X}\in\mathbb{R}^{n\times p}$
+made of $n$ samples, each of which carries $p$ features or predictors. The
+primary goal is to identify the classes to which new unseen samples
+belong.
+
+Let us specialize to the case of two classes only, with outputs
+$y_i=0$ and $y_i=1$. Our outcomes could represent the status of a
+credit card user that could default or not on her/his credit card
+debt. That is
+
+$$
+y_i = \begin{bmatrix} 0 & \mathrm{no}\\  1 & \mathrm{yes} \end{bmatrix}.
+$$
+
+## Linear classifier
+
+Before moving to the logistic model, let us try to use our linear
+regression model to classify these two outcomes. We could for example
+fit a linear model to the default case if $y_i > 0.5$ and the no
+default case $y_i \leq 0.5$.
+
+We would then have our 
+weighted linear combination, namely
+
+<!-- Equation labels as ordinary links -->
+<div id="_auto1"></div>
+
+$$
+\begin{equation}
+\hat{y} = \hat{X}^T\hat{\beta} +  \hat{\epsilon},
+\label{_auto1} \tag{1}
+\end{equation}
+$$
+
+where $\hat{y}$ is a vector representing the possible outcomes, $\hat{X}$ is our
+$n\times p$ design matrix and $\hat{\beta}$ represents our estimators/predictors.
+
+## Some selected properties
+
+The main problem with our function is that it takes values on the
+entire real axis. In the case of logistic regression, however, the
+labels $y_i$ are discrete variables. A typical example is the credit
+card data discussed below here, where we can set the state of
+defaulting the debt to $y_i=1$ and not to $y_i=0$ for one the persons
+in the data set (see the full example below).
+
+One simple way to get a discrete output is to have sign
+functions that map the output of a linear regressor to values $\{0,1\}$,
+$f(s_i)=sign(s_i)=1$ if $s_i\ge 0$ and 0 if otherwise. 
+We will encounter this model in our first demonstration of neural networks. Historically it is called the "perceptron" model in the machine learning
+literature. This model is extremely simple. However, in many cases it is more
+favorable to use a ``soft" classifier that outputs
+the probability of a given category. This leads us to the logistic function.
+
+
+## The logistic function
+
+The perceptron is an example of a ``hard classification" model. We
+will encounter this model when we discuss neural networks as
+well. Each datapoint is deterministically assigned to a category (i.e
+$y_i=0$ or $y_i=1$). In many cases, it is favorable to have a "soft"
+classifier that outputs the probability of a given category rather
+than a single value. For example, given $x_i$, the classifier
+outputs the probability of being in a category $k$.  Logistic regression
+is the most common example of a so-called soft classifier. In logistic
+regression, the probability that a data point $x_i$
+belongs to a category $y_i=\{0,1\}$ is given by the so-called logit function (or Sigmoid) which is meant to represent the likelihood for a given event,
+
+$$
+p(t) = \frac{1}{1+\mathrm \exp{-t}}=\frac{\exp{t}}{1+\mathrm \exp{t}}.
+$$
+
+Note that $1-p(t)= p(-t)$.
+
+## Examples of likelihood functions used in logistic regression and nueral networks
+
+
+The following code plots the logistic function, the step function and other functions we will encounter from here and on.
+
+%matplotlib inline
+
+"""The sigmoid function (or the logistic curve) is a
+function that takes any real number, z, and outputs a number (0,1).
+It is useful in neural networks for assigning weights on a relative scale.
+The value z is the weighted sum of parameters involved in the learning algorithm."""
+
+import numpy
+import matplotlib.pyplot as plt
+import math as mt
+
+z = numpy.arange(-5, 5, .1)
+sigma_fn = numpy.vectorize(lambda z: 1/(1+numpy.exp(-z)))
+sigma = sigma_fn(z)
+
+fig = plt.figure()
+ax = fig.add_subplot(111)
+ax.plot(z, sigma)
+ax.set_ylim([-0.1, 1.1])
+ax.set_xlim([-5,5])
+ax.grid(True)
+ax.set_xlabel('z')
+ax.set_title('sigmoid function')
+
+plt.show()
+
+"""Step Function"""
+z = numpy.arange(-5, 5, .02)
+step_fn = numpy.vectorize(lambda z: 1.0 if z >= 0.0 else 0.0)
+step = step_fn(z)
+
+fig = plt.figure()
+ax = fig.add_subplot(111)
+ax.plot(z, step)
+ax.set_ylim([-0.5, 1.5])
+ax.set_xlim([-5,5])
+ax.grid(True)
+ax.set_xlabel('z')
+ax.set_title('step function')
+
+plt.show()
+
+"""tanh Function"""
+z = numpy.arange(-2*mt.pi, 2*mt.pi, 0.1)
+t = numpy.tanh(z)
+
+fig = plt.figure()
+ax = fig.add_subplot(111)
+ax.plot(z, t)
+ax.set_ylim([-1.0, 1.0])
+ax.set_xlim([-2*mt.pi,2*mt.pi])
+ax.grid(True)
+ax.set_xlabel('z')
+ax.set_title('tanh function')
+
+plt.show()
+
+## Two parameters
+
+We assume now that we have two classes with $y_i$ either $0$ or $1$. Furthermore we assume also that we have only two parameters $\beta$ in our fitting of the Sigmoid function, that is we define probabilities
+
+$$
+\begin{align*}
+p(y_i=1|x_i,\hat{\beta}) &= \frac{\exp{(\beta_0+\beta_1x_i)}}{1+\exp{(\beta_0+\beta_1x_i)}},\nonumber\\
+p(y_i=0|x_i,\hat{\beta}) &= 1 - p(y_i=1|x_i,\hat{\beta}),
+\end{align*}
+$$
+
+where $\hat{\beta}$ are the weights we wish to extract from data, in our case $\beta_0$ and $\beta_1$. 
+
+Note that we used
+
+$$
+p(y_i=0\vert x_i, \hat{\beta}) = 1-p(y_i=1\vert x_i, \hat{\beta}).
+$$
+
+<!-- !split  -->
+## Maximum likelihood
+
+In order to define the total likelihood for all possible outcomes from a  
+dataset $\mathcal{D}=\{(y_i,x_i)\}$, with the binary labels
+$y_i\in\{0,1\}$ and where the data points are drawn independently, we use the so-called [Maximum Likelihood Estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) principle. 
+We aim thus at maximizing 
+the probability of seeing the observed data. We can then approximate the 
+likelihood in terms of the product of the individual probabilities of a specific outcome $y_i$, that is
+
+$$
+\begin{align*}
+P(\mathcal{D}|\hat{\beta})& = \prod_{i=1}^n \left[p(y_i=1|x_i,\hat{\beta})\right]^{y_i}\left[1-p(y_i=1|x_i,\hat{\beta}))\right]^{1-y_i}\nonumber \\
+\end{align*}
+$$
+
+from which we obtain the log-likelihood and our **cost/loss** function
+
+$$
+\mathcal{C}(\hat{\beta}) = \sum_{i=1}^n \left( y_i\log{p(y_i=1|x_i,\hat{\beta})} + (1-y_i)\log\left[1-p(y_i=1|x_i,\hat{\beta}))\right]\right).
+$$
+
+## The cost function rewritten
+
+Reordering the logarithms, we can rewrite the **cost/loss** function as
+
+$$
+\mathcal{C}(\hat{\beta}) = \sum_{i=1}^n  \left(y_i(\beta_0+\beta_1x_i) -\log{(1+\exp{(\beta_0+\beta_1x_i)})}\right).
+$$
+
+The maximum likelihood estimator is defined as the set of parameters that maximize the log-likelihood where we maximize with respect to $\beta$.
+Since the cost (error) function is just the negative log-likelihood, for logistic regression we have that
+
+$$
+\mathcal{C}(\hat{\beta})=-\sum_{i=1}^n  \left(y_i(\beta_0+\beta_1x_i) -\log{(1+\exp{(\beta_0+\beta_1x_i)})}\right).
+$$
+
+This equation is known in statistics as the **cross entropy**. Finally, we note that just as in linear regression, 
+in practice we often supplement the cross-entropy with additional regularization terms, usually $L_1$ and $L_2$ regularization as we did for Ridge and Lasso regression.
+
+## Minimizing the cross entropy
+
+The cross entropy is a convex function of the weights $\hat{\beta}$ and,
+therefore, any local minimizer is a global minimizer. 
+
+
+Minimizing this
+cost function with respect to the two parameters $\beta_0$ and $\beta_1$ we obtain
+
+$$
+\frac{\partial \mathcal{C}(\hat{\beta})}{\partial \beta_0} = -\sum_{i=1}^n  \left(y_i -\frac{\exp{(\beta_0+\beta_1x_i)}}{1+\exp{(\beta_0+\beta_1x_i)}}\right),
+$$
+
+and
+
+$$
+\frac{\partial \mathcal{C}(\hat{\beta})}{\partial \beta_1} = -\sum_{i=1}^n  \left(y_ix_i -x_i\frac{\exp{(\beta_0+\beta_1x_i)}}{1+\exp{(\beta_0+\beta_1x_i)}}\right).
+$$
+
+## A more compact expression
+
+Let us now define a vector $\hat{y}$ with $n$ elements $y_i$, an
+$n\times p$ matrix $\hat{X}$ which contains the $x_i$ values and a
+vector $\hat{p}$ of fitted probabilities $p(y_i\vert x_i,\hat{\beta})$. We can rewrite in a more compact form the first
+derivative of cost function as
+
+$$
+\frac{\partial \mathcal{C}(\hat{\beta})}{\partial \hat{\beta}} = -\hat{X}^T\left(\hat{y}-\hat{p}\right).
+$$
+
+If we in addition define a diagonal matrix $\hat{W}$ with elements 
+$p(y_i\vert x_i,\hat{\beta})(1-p(y_i\vert x_i,\hat{\beta})$, we can obtain a compact expression of the second derivative as
+
+$$
+\frac{\partial^2 \mathcal{C}(\hat{\beta})}{\partial \hat{\beta}\partial \hat{\beta}^T} = \hat{X}^T\hat{W}\hat{X}.
+$$
+
+## Extending to more predictors
+
+Within a binary classification problem, we can easily expand our model to include multiple predictors. Our ratio between likelihoods is then with $p$ predictors
+
+$$
+\log{ \frac{p(\hat{\beta}\hat{x})}{1-p(\hat{\beta}\hat{x})}} = \beta_0+\beta_1x_1+\beta_2x_2+\dots+\beta_px_p.
+$$
+
+Here we defined $\hat{x}=[1,x_1,x_2,\dots,x_p]$ and $\hat{\beta}=[\beta_0, \beta_1, \dots, \beta_p]$ leading to
+
+$$
+p(\hat{\beta}\hat{x})=\frac{ \exp{(\beta_0+\beta_1x_1+\beta_2x_2+\dots+\beta_px_p)}}{1+\exp{(\beta_0+\beta_1x_1+\beta_2x_2+\dots+\beta_px_p)}}.
+$$
+
+## Including more classes
+
+Till now we have mainly focused on two classes, the so-called binary
+system. Suppose we wish to extend to $K$ classes.  Let us for the sake
+of simplicity assume we have only two predictors. We have then
+following model
+
+1
+5
+ 
+<
+<
+<
+!
+!
+M
+A
+T
+H
+_
+B
+L
+O
+C
+K
+
+$$
+\log{\frac{p(C=2\vert x)}{p(K\vert x)}} = \beta_{20}+\beta_{21}x_1,
+$$
+
+and so on till the class $C=K-1$ class
+
+$$
+\log{\frac{p(C=K-1\vert x)}{p(K\vert x)}} = \beta_{(K-1)0}+\beta_{(K-1)1}x_1,
+$$
+
+and the model is specified in term of $K-1$ so-called log-odds or
+**logit** transformations.
+
+
+## More classes
+
+In our discussion of neural networks we will encounter the above again
+in terms of a slightly modified function, the so-called **Softmax** function.
+
+The softmax function is used in various multiclass classification
+methods, such as multinomial logistic regression (also known as
+softmax regression), multiclass linear discriminant analysis, naive
+Bayes classifiers, and artificial neural networks.  Specifically, in
+multinomial logistic regression and linear discriminant analysis, the
+input to the function is the result of $K$ distinct linear functions,
+and the predicted probability for the $k$-th class given a sample
+vector $\hat{x}$ and a weighting vector $\hat{\beta}$ is (with two
+predictors):
+
+$$
+p(C=k\vert \mathbf {x} )=\frac{\exp{(\beta_{k0}+\beta_{k1}x_1)}}{1+\sum_{l=1}^{K-1}\exp{(\beta_{l0}+\beta_{l1}x_1)}}.
+$$
+
+It is easy to extend to more predictors. The final class is
+
+$$
+p(C=K\vert \mathbf {x} )=\frac{1}{1+\sum_{l=1}^{K-1}\exp{(\beta_{l0}+\beta_{l1}x_1)}},
+$$
+
+and they sum to one. Our earlier discussions were all specialized to
+the case with two classes only. It is easy to see from the above that
+what we derived earlier is compatible with these equations.
+
+To find the optimal parameters we would typically use a gradient
+descent method.  Newton's method and gradient descent methods are
+discussed in the material on [optimization
+methods](https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html).
+
+
+
+
+## A simple classification problem
+
+import numpy as np
+from sklearn import datasets, linear_model
+import matplotlib.pyplot as plt
+
+
+def generate_data():
+    np.random.seed(0)
+    X, y = datasets.make_moons(200, noise=0.20)
+    return X, y
+
+
+def visualize(X, y, clf):
+    plot_decision_boundary(lambda x: clf.predict(x), X, y)
+
+def plot_decision_boundary(pred_func, X, y):
+    # Set min and max values and give it some padding
+    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
+    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
+    h = 0.01
+    # Generate a grid of points with distance h between them
+    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
+    # Predict the function value for the whole gid
+    Z = pred_func(np.c_[xx.ravel(), yy.ravel()])
+    Z = Z.reshape(xx.shape)
+    # Plot the contour and training examples
+    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)
+    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Spectral)
+    plt.show()
+
+
+def classify(X, y):
+    clf = linear_model.LogisticRegressionCV()
+    clf.fit(X, y)
+    return clf
+
+
+def main():
+    X, y = generate_data()
+    # visualize(X, y)
+    clf = classify(X, y)
+    visualize(X, y, clf)
+
+if __name__ == "__main__":
+    main()
+
+## The Credit Card example
+Here we use the the [credit card data](https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients). 
+The data are from an extensive database from Taiwan and include more than ten predictors.
+
+For categorical data -Scikit-Learn- provides a so-called **one-hot encoder**.
+This is called one-hot
+encoding, because only one attribute will be equal to 1 (hot), while the others will be 0 (cold).
+**Scikit-Learn** provides a OneHotEncoder encoder to convert integer categorical values into one-hot
+
+from sklearn.preprocessing import OneHotEncoder
+encoder = OneHotEncoder()
+
+## How to read the Credit Card data
+
+import pandas as pd
+import os
+import numpy as np
+
+
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.compose import ColumnTransformer
+from sklearn.preprocessing import StandardScaler, OneHotEncoder
+from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score
+
+# Trying to set the seed
+np.random.seed(0)
+import random
+random.seed(0)
+
+# Reading file into data frame
+cwd = os.getcwd()
+filename = cwd + '/default of credit card clients.xls'
+nanDict = {}
+df = pd.read_excel(filename, header=1, skiprows=0, index_col=0, na_values=nanDict)
+
+df.rename(index=str, columns={"default payment next month": "defaultPaymentNextMonth"}, inplace=True)
+
+# Features and targets 
+X = df.loc[:, df.columns != 'defaultPaymentNextMonth'].values
+y = df.loc[:, df.columns == 'defaultPaymentNextMonth'].values
+
+# Categorical variables to one-hot's
+onehotencoder = OneHotEncoder(categories="auto")
+
+X = ColumnTransformer(
+    [("", onehotencoder, [3]),],
+    remainder="passthrough"
+).fit_transform(X)
+
+y.shape
+
+# Train-test split
+trainingShare = 0.5 
+seed  = 1
+XTrain, XTest, yTrain, yTest=train_test_split(X, y, train_size=trainingShare, \
+                                              test_size = 1-trainingShare,
+                                             random_state=seed)
+
+# Input Scaling
+sc = StandardScaler()
+XTrain = sc.fit_transform(XTrain)
+XTest = sc.transform(XTest)
+
+# One-hot's of the target vector
+Y_train_onehot, Y_test_onehot = onehotencoder.fit_transform(yTrain), onehotencoder.fit_transform(yTest)
+
+# Remove instances with zeros only for past bill statements or paid amounts
+'''
+df = df.drop(df[(df.BILL_AMT1 == 0) &
+                (df.BILL_AMT2 == 0) &
+                (df.BILL_AMT3 == 0) &
+                (df.BILL_AMT4 == 0) &
+                (df.BILL_AMT5 == 0) &
+                (df.BILL_AMT6 == 0) &
+                (df.PAY_AMT1 == 0) &
+                (df.PAY_AMT2 == 0) &
+                (df.PAY_AMT3 == 0) &
+                (df.PAY_AMT4 == 0) &
+                (df.PAY_AMT5 == 0) &
+                (df.PAY_AMT6 == 0)].index)
+'''
+df = df.drop(df[(df.BILL_AMT1 == 0) &
+                (df.BILL_AMT2 == 0) &
+                (df.BILL_AMT3 == 0) &
+                (df.BILL_AMT4 == 0) &
+                (df.BILL_AMT5 == 0) &
+                (df.BILL_AMT6 == 0)].index)
+
+df = df.drop(df[(df.PAY_AMT1 == 0) &
+                (df.PAY_AMT2 == 0) &
+                (df.PAY_AMT3 == 0) &
+                (df.PAY_AMT4 == 0) &
+                (df.PAY_AMT5 == 0) &
+                (df.PAY_AMT6 == 0)].index)
+
+from sklearn.linear_model import LogisticRegression
+from sklearn.model_selection import GridSearchCV
+
+lambdas=np.logspace(-5,7,13)
+parameters = [{'C': 1./lambdas, "solver":["lbfgs"]}]#*len(parameters)}]
+scoring = ['accuracy', 'roc_auc']
+logReg = LogisticRegression()
+gridSearch = GridSearchCV(logReg, parameters, cv=5, scoring=scoring, refit='roc_auc')
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic_42_0.png b/doc/src/LectureNotes/_build/jupyter_execute/logistic_42_0.png
new file mode 100644
index 000000000..65260025b
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/logistic_42_0.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_0.png b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_0.png
new file mode 100644
index 000000000..cf1468b14
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_0.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_1.png b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_1.png
new file mode 100644
index 000000000..d777fd705
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_1.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_2.png b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_2.png
new file mode 100644
index 000000000..0d02b02c6
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/logistic_7_2.png differ
diff --git a/doc/src/LectureNotes/_build/latex/reports/logistic.log b/doc/src/LectureNotes/_build/latex/reports/logistic.log
new file mode 100644
index 000000000..b718efedf
--- /dev/null
+++ b/doc/src/LectureNotes/_build/latex/reports/logistic.log
@@ -0,0 +1,151 @@
+Traceback (most recent call last):
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/jupyter_cache/executors/utils.py", line 56, in single_nb_execution
+    record_timing=False,
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/client.py", line 1082, in execute
+    return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute()
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/util.py", line 74, in wrapped
+    return just_run(coro(*args, **kwargs))
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/util.py", line 53, in just_run
+    return loop.run_until_complete(coro)
+  File "/Users/MortenImac/anaconda3/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
+    return future.result()
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/client.py", line 536, in async_execute
+    cell, index, execution_count=self.code_cells_executed + 1
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/client.py", line 827, in async_execute_cell
+    self._check_raise_for_error(cell, exec_reply)
+  File "/Users/MortenImac/anaconda3/lib/python3.6/site-packages/nbclient/client.py", line 735, in _check_raise_for_error
+    raise CellExecutionError.from_cell_and_msg(cell, exec_reply['content'])
+nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
+------------------
+import pandas as pd
+import os
+import numpy as np
+
+
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.compose import ColumnTransformer
+from sklearn.preprocessing import StandardScaler, OneHotEncoder
+from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score
+
+# Trying to set the seed
+np.random.seed(0)
+import random
+random.seed(0)
+
+# Reading file into data frame
+cwd = os.getcwd()
+filename = cwd + '/default of credit card clients.xls'
+nanDict = {}
+df = pd.read_excel(filename, header=1, skiprows=0, index_col=0, na_values=nanDict)
+
+df.rename(index=str, columns={"default payment next month": "defaultPaymentNextMonth"}, inplace=True)
+
+# Features and targets 
+X = df.loc[:, df.columns != 'defaultPaymentNextMonth'].values
+y = df.loc[:, df.columns == 'defaultPaymentNextMonth'].values
+
+# Categorical variables to one-hot's
+onehotencoder = OneHotEncoder(categories="auto")
+
+X = ColumnTransformer(
+    [("", onehotencoder, [3]),],
+    remainder="passthrough"
+).fit_transform(X)
+
+y.shape
+
+# Train-test split
+trainingShare = 0.5 
+seed  = 1
+XTrain, XTest, yTrain, yTest=train_test_split(X, y, train_size=trainingShare, \
+                                              test_size = 1-trainingShare,
+                                             random_state=seed)
+
+# Input Scaling
+sc = StandardScaler()
+XTrain = sc.fit_transform(XTrain)
+XTest = sc.transform(XTest)
+
+# One-hot's of the target vector
+Y_train_onehot, Y_test_onehot = onehotencoder.fit_transform(yTrain), onehotencoder.fit_transform(yTest)
+
+# Remove instances with zeros only for past bill statements or paid amounts
+'''
+df = df.drop(df[(df.BILL_AMT1 == 0) &
+                (df.BILL_AMT2 == 0) &
+                (df.BILL_AMT3 == 0) &
+                (df.BILL_AMT4 == 0) &
+                (df.BILL_AMT5 == 0) &
+                (df.BILL_AMT6 == 0) &
+                (df.PAY_AMT1 == 0) &
+                (df.PAY_AMT2 == 0) &
+                (df.PAY_AMT3 == 0) &
+                (df.PAY_AMT4 == 0) &
+                (df.PAY_AMT5 == 0) &
+                (df.PAY_AMT6 == 0)].index)
+'''
+df = df.drop(df[(df.BILL_AMT1 == 0) &
+                (df.BILL_AMT2 == 0) &
+                (df.BILL_AMT3 == 0) &
+                (df.BILL_AMT4 == 0) &
+                (df.BILL_AMT5 == 0) &
+                (df.BILL_AMT6 == 0)].index)
+
+df = df.drop(df[(df.PAY_AMT1 == 0) &
+                (df.PAY_AMT2 == 0) &
+                (df.PAY_AMT3 == 0) &
+                (df.PAY_AMT4 == 0) &
+                (df.PAY_AMT5 == 0) &
+                (df.PAY_AMT6 == 0)].index)
+
+from sklearn.linear_model import LogisticRegression
+from sklearn.model_selection import GridSearchCV
+
+lambdas=np.logspace(-5,7,13)
+parameters = [{'C': 1./lambdas, "solver":["lbfgs"]}]#*len(parameters)}]
+scoring = ['accuracy', 'roc_auc']
+logReg = LogisticRegression()
+gridSearch = GridSearchCV(logReg, parameters, cv=5, scoring=scoring, refit='roc_auc')
+------------------
+
+---------------------------------------------------------------------------
+ImportError                               Traceback (most recent call last)
+<ipython-input-4-4edad39a9543> in <module>
+     19 filename = cwd + '/default of credit card clients.xls'
+     20 nanDict = {}
+---> 21 df = pd.read_excel(filename, header=1, skiprows=0, index_col=0, na_values=nanDict)
+     22 
+     23 df.rename(index=str, columns={"default payment next month": "defaultPaymentNextMonth"}, inplace=True)
+
+~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_base.py in read_excel(io, sheet_name, header, names, index_col, usecols, squeeze, dtype, engine, converters, true_values, false_values, skiprows, nrows, na_values, keep_default_na, verbose, parse_dates, date_parser, thousands, comment, skipfooter, convert_float, mangle_dupe_cols, **kwds)
+    302 
+    303     if not isinstance(io, ExcelFile):
+--> 304         io = ExcelFile(io, engine=engine)
+    305     elif engine and engine != io.engine:
+    306         raise ValueError(
+
+~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_base.py in __init__(self, io, engine)
+    822         self._io = stringify_path(io)
+    823 
+--> 824         self._reader = self._engines[engine](self._io)
+    825 
+    826     def __fspath__(self):
+
+~/anaconda3/lib/python3.6/site-packages/pandas/io/excel/_xlrd.py in __init__(self, filepath_or_buffer)
+     18         """
+     19         err_msg = "Install xlrd >= 1.0.0 for Excel support"
+---> 20         import_optional_dependency("xlrd", extra=err_msg)
+     21         super().__init__(filepath_or_buffer)
+     22 
+
+~/anaconda3/lib/python3.6/site-packages/pandas/compat/_optional.py in import_optional_dependency(name, extra, raise_on_missing, on_version)
+     90     except ImportError:
+     91         if raise_on_missing:
+---> 92             raise ImportError(msg) from None
+     93         else:
+     94             return None
+
+ImportError: Missing optional dependency 'xlrd'. Install xlrd >= 1.0.0 for Excel support Use pip or conda to install xlrd.
+ImportError: Missing optional dependency 'xlrd'. Install xlrd >= 1.0.0 for Excel support Use pip or conda to install xlrd.
+