FYS-STK4155/doc/pub/NeuralNet/ipynb/.ipynb_checkpoints/NeuralNet-checkpoint.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- dom:TITLE: Data Analysis and Machine Learning: Elements of machine learning -->\n",
    "# Data Analysis and Machine Learning: Elements of machine learning\n",
    "<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->\n",
    "<!-- Author: -->  \n",
    "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
    "\n",
    "Date: **Sep 28, 2018**\n",
    "\n",
    "Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
    "\n",
    "\n",
    "\n",
    "<!-- add own code for DNN -->\n",
    "\n",
    "## Neural networks\n",
    "\n",
    "Artificial neural networks are computational systems that can learn to\n",
    "perform tasks by considering examples, generally without being\n",
    "programmed with any task-specific rules. It is supposed to mimic a\n",
    "biological system, wherein neurons interact by sending signals in the\n",
    "form of mathematical functions between layers. All layers can contain\n",
    "an arbitrary number of neurons, and each connection is represented by\n",
    "a weight variable.\n",
    "\n",
    "\n",
    "## Artificial neurons\n",
    "\n",
    "The field of artificial neural networks has a long history of\n",
    "development, and is closely connected with the advancement of computer\n",
    "science and computers in general. A model of artificial neurons was\n",
    "first developed by McCulloch and Pitts in 1943 to study signal\n",
    "processing in the brain and has later been refined by others. The\n",
    "general idea is to mimic neural networks in the human brain, which is\n",
    "composed of billions of neurons that communicate with each other by\n",
    "sending electrical signals.  Each neuron accumulates its incoming\n",
    "signals, which must exceed an activation threshold to yield an\n",
    "output. If the threshold is not overcome, the neuron remains inactive,\n",
    "i.e. has zero output.\n",
    "\n",
    "This behaviour has inspired a simple mathematical model for an artificial neuron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"artificialNeuron\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y = f\\left(\\sum_{i=1}^n w_ix_i\\right) = f(u)\n",
    "\\label{artificialNeuron} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the output $y$ of the neuron is the value of its activation function, which have as input\n",
    "a weighted sum of signals $x_i, \\dots ,x_n$ received by $n$ other neurons.\n",
    "\n",
    "Conceptually, it is helpful to divide neural networks into four\n",
    "categories:\n",
    "1. general purpose neural networks for supervised learning,\n",
    "\n",
    "2. neural networks designed specifically for image processing, the most prominent example of this class being Convolutional Neural Networks (CNNs),\n",
    "\n",
    "3. neural networks for sequential data such as Recurrent Neural Networks (RNNs), and\n",
    "\n",
    "4. neural networks for unsupervised learning such as Deep Boltzmann Machines.\n",
    "\n",
    "In natural science, DNNs and CNNs have already found numerous applications. In\n",
    "statistical physics, they have been applied to detect phase\n",
    "transitions in 2D Ising and Potts models, lattice gauge theories, and\n",
    "different phases of polymers, or solving the Navier-Stokes equation in weather forecasting.\n",
    "Deep learning has also found interesting applications in quantum\n",
    "physics. Various quantum phase transitions can be detected and studied\n",
    "using DNNs and CNNs,\n",
    "topological phases, and even non-equilibrium many-body\n",
    "localization. Representing quantum states as DNNs quantum state\n",
    "tomography are among some of the impressive\n",
    "achievements to reveal the potential of DNNs to facilitate the study\n",
    "of quantum systems.\n",
    "\n",
    "In quantum information theory, it has been shown that one can perform\n",
    "gate decompositions with the help of neural. In lattice quantum chromodynamics,\n",
    "DNNs have been used to learn action parameters in regions of parameter\n",
    "space where PCA fails. \n",
    "\n",
    "The applications are not limited to the natural sciences. There is a plethora of applications in essentially all disciplines, from the humanities to life science and medicine.\n",
    "\n",
    "## Neural network types\n",
    "\n",
    "An artificial neural network (NN), is a computational model that\n",
    "consists of layers of connected neurons, or *nodes*.  It is supposed\n",
    "to mimic a biological nervous system by letting each neuron interact\n",
    "with other neurons by sending signals in the form of mathematical\n",
    "functions between layers.  A wide variety of different NNs have been\n",
    "developed, but most of them consist of an input layer, an output layer\n",
    "and eventual layers in-between, called *hidden layers*. All layers can\n",
    "contain an arbitrary number of nodes, and each connection between two\n",
    "nodes is associated with a weight variable.\n",
    "\n",
    "Neural networks (also called neural nets) are neural-inspired\n",
    "nonlinear models for supervised learning.  As we will see, neural nets\n",
    "can be viewed as natural, more powerful extensions of supervised\n",
    "learning methods such as linear and logistic regression and soft-max\n",
    "methods.\n",
    "\n",
    "\n",
    "## Feed-forward neural networks\n",
    "\n",
    "The feed-forward neural network (FFNN) was the first and simplest type of NN devised. In this network, \n",
    "the information moves in only one direction: forward through the layers.\n",
    "\n",
    "Nodes are represented by circles, while the arrows display the connections between the nodes, including the \n",
    "direction of information flow. Additionally, each arrow corresponds to a weight variable, not displayed here. \n",
    "We observe that each node in a layer is connected to *all* nodes in the subsequent layer, \n",
    "making this a so-called *fully-connected* FFNN. \n",
    "\n",
    "\n",
    "\n",
    "A different variant of FFNNs are *convolutional neural networks* (CNNs), which have a connectivity pattern\n",
    "inspired by the animal visual cortex. Individual neurons in the visual cortex only respond to stimuli from\n",
    "small sub-regions of the visual field, called a receptive field. This makes the neurons well-suited to exploit the strong\n",
    "spatially local correlation present in natural images. The response of each neuron can be approximated mathematically \n",
    "as a convolution operation. \n",
    "\n",
    "CNNs emulate the behaviour of neurons in the visual cortex by enforcing a *local* connectivity pattern\n",
    "between nodes of adjacent layers: Each node\n",
    "in a convolutional layer is connected only to a subset of the nodes in the previous layer, \n",
    "in contrast to the fully-connected FFNN.\n",
    "Often, CNNs \n",
    "consist of several convolutional layers that learn local features of the input, with a fully-connected layer at the end, \n",
    "which gathers all the local data and produces the outputs. They have wide applications in image and video recognition\n",
    "\n",
    "## Recurrent neural networks\n",
    "\n",
    "So far we have only mentioned NNs where information flows in one direction: forward. *Recurrent neural networks* on\n",
    "the other hand, have connections between nodes that form directed *cycles*. This creates a form of \n",
    "internal memory which are able to capture information on what has been calculated before; the output is dependent \n",
    "on the previous computations. Recurrent NNs make use of sequential information by performing the same task for \n",
    "every element in a sequence, where each element depends on previous elements. An example of such information is \n",
    "sentences, making recurrent NNs especially well-suited for handwriting and speech recognition.\n",
    "\n",
    "## Other types of networks\n",
    "\n",
    "There are many other kinds of NNs that have been developed. One type that is specifically designed for interpolation\n",
    "in multidimensional space is the radial basis function (RBF) network. RBFs are typically made up of three layers: \n",
    "an input layer, a hidden layer with non-linear radial symmetric activation functions and a linear output layer (''linear'' here\n",
    "means that each node in the output layer has a linear activation function). The layers are normally fully-connected and \n",
    "there are no cycles, thus RBFs can be viewed as a type of fully-connected FFNN. They are however usually treated as\n",
    "a separate type of NN due the unusual activation functions.\n",
    "\n",
    "## Multilayer perceptrons\n",
    "\n",
    "One uses often so-called fully-connected feed-forward neural networks\n",
    "with three or more layers (an input layer, one or more hidden layers\n",
    "and an output layer) consisting of neurons that have non-linear\n",
    "activation functions.\n",
    "\n",
    "Such networks are often called *multilayer perceptrons* (MLPs)\n",
    "\n",
    "## Why multilayer perceptrons?\n",
    "\n",
    "According to the *Universal approximation theorem*, a feed-forward neural network with just a single hidden layer containing \n",
    "a finite number of neurons can approximate a continuous multidimensional function to arbitrary accuracy, \n",
    "assuming the activation function for the hidden layer is a **non-constant, bounded and monotonically-increasing continuous function**.\n",
    "\n",
    "Note that the requirements on the activation function only applies to\n",
    "the hidden layer, the output nodes are always assumed to be linear, so\n",
    "as to not restrict the range of output values.\n",
    "\n",
    "We note that this theorem is only applicable to an NN with *one* hidden\n",
    "layer.  Therefore, we can easily construct an NN that employs\n",
    "activation functions which do not satisfy the above requirements, as\n",
    "long as we have at least one layer with activation functions that\n",
    "*do*. Furthermore, although the universal approximation theorem lays\n",
    "the theoretical foundation for regression with neural networks, it\n",
    "does not say anything about how things work in practice: A neural\n",
    "network can still be able to approximate a given function reasonably\n",
    "well without having the flexibility to fit *all other* functions.\n",
    "\n",
    "\n",
    "\n",
    "## Mathematical model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"artificialNeuron2\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y = f\\left(\\sum_{i=1}^n w_ix_i + b_i\\right) = f(u)\n",
    "\\label{artificialNeuron2} \\tag{2}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In an FFNN of such neurons, the *inputs* $x_i$ are the *outputs* of\n",
    "the neurons in the preceding layer. Furthermore, an MLP is\n",
    "fully-connected, which means that each neuron receives a weighted sum\n",
    "of the outputs of *all* neurons in the previous layer.\n",
    "\n",
    "## Mathematical model\n",
    "\n",
    "First, for each node $i$ in the first hidden layer, we calculate a weighted sum $u_i^1$ of the input coordinates $x_j$,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " u_i^1 = \\sum_{j=1}^2 w_{ij}^1 x_j  + b_i^1 \n",
    "\\label{_auto1} \\tag{3}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This value is the argument to the activation function $f_1$ of each neuron $i$,\n",
    "producing the output $y_i^1$ of all neurons in layer 1,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"outputLayer1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y_i^1 = f_1(u_i^1) = f_1\\left(\\sum_{j=1}^2 w_{ij}^1 x_j  + b_i^1\\right)\n",
    "\\label{outputLayer1} \\tag{4}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where we assume that all nodes in the same layer have identical\n",
    "activation functions, hence the notation $f_l$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"generalLayer\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y_i^l = f_l(u_i^l) = f_l\\left(\\sum_{j=1}^{N_{l-1}} w_{ij}^l y_j^{l-1} + b_i^l\\right)\n",
    "\\label{generalLayer} \\tag{5}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where $N_l$ is the number of nodes in layer $l$. When the output of\n",
    "all the nodes in the first hidden layer are computed, the values of\n",
    "the subsequent layer can be calculated and so forth until the output\n",
    "is obtained.\n",
    "\n",
    "\n",
    "\n",
    "## Mathematical model\n",
    "\n",
    "The output of neuron $i$ in layer 2 is thus,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto2\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y_i^2 = f_2\\left(\\sum_{j=1}^3 w_{ij}^2 y_j^1 + b_i^2\\right) \n",
    "\\label{_auto2} \\tag{6}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"outputLayer2\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    " = f_2\\left[\\sum_{j=1}^3 w_{ij}^2f_1\\left(\\sum_{k=1}^2 w_{jk}^1 x_k + b_j^1\\right) + b_i^2\\right]\n",
    "\\label{outputLayer2} \\tag{7}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where we have substituted $y_m^1$ with. Finally, the NN output yields,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto3\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y_1^3 = f_3\\left(\\sum_{j=1}^3 w_{1m}^3 y_j^2 + b_1^3\\right) \n",
    "\\label{_auto3} \\tag{8}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto4\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    " = f_3\\left[\\sum_{j=1}^3 w_{1j}^3 f_2\\left(\\sum_{k=1}^3 w_{jk}^2 f_1\\left(\\sum_{m=1}^2 w_{km}^1 x_m + b_k^1\\right) + b_j^2\\right)\n",
    "  + b_1^3\\right]\n",
    "\\label{_auto4} \\tag{9}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mathematical model\n",
    "\n",
    "We can generalize this expression to an MLP with $l$ hidden\n",
    "layers. The complete functional form is,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"completeNN\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "y^{l+1}_1\\! = \\!f_{l+1}\\!\\left[\\!\\sum_{j=1}^{N_l}\\! w_{1j}^3 f_l\\!\\left(\\!\\sum_{k=1}^{N_{l-1}}\\! w_{jk}^2 f_{l-1}\\!\\left(\\!\n",
    " \\dots \\!f_1\\!\\left(\\!\\sum_{n=1}^{N_0} \\!w_{mn}^1 x_n\\! + \\!b_m^1\\!\\right)\n",
    " \\!\\dots \\!\\right) \\!+ \\!b_k^2\\!\\right)\n",
    " \\!+ \\!b_1^3\\!\\right] \n",
    "\\label{completeNN} \\tag{10}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which illustrates a basic property of MLPs: The only independent\n",
    "variables are the input values $x_n$.\n",
    "\n",
    "## Mathematical model\n",
    "\n",
    "This confirms that an MLP, despite its quite convoluted mathematical\n",
    "form, is nothing more than an analytic function, specifically a\n",
    "mapping of real-valued vectors $\\vec{x} \\in \\mathbb{R}^n \\rightarrow\n",
    "\\vec{y} \\in \\mathbb{R}^m$.  In our example, $n=2$ and\n",
    "$m=1$. Consequentially, the number of input and output values of the\n",
    "function we want to fit must be equal to the number of inputs and\n",
    "outputs of our MLP.\n",
    "\n",
    "Furthermore, the flexibility and universality of a MLP can be\n",
    "illustrated by realizing that the expression is essentially a nested\n",
    "sum of scaled activation functions of the form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto5\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " h(x) = c_1 f(c_2 x + c_3) + c_4\n",
    "\\label{_auto5} \\tag{11}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where the parameters $c_i$ are weights and biases. By adjusting these\n",
    "parameters, the activation functions can be shifted up and down or\n",
    "left and right, change slope or be rescaled which is the key to the\n",
    "flexibility of a neural network.\n",
    "\n",
    "### Matrix-vector notation\n",
    "\n",
    "We can introduce a more convenient notation for the activations in a NN. \n",
    "\n",
    "Additionally, we can represent the biases and activations\n",
    "as layer-wise column vectors $\\vec{b}_l$ and $\\vec{y}_l$, so that the $i$-th element of each vector \n",
    "is the bias $b_i^l$ and activation $y_i^l$ of node $i$ in layer $l$ respectively. \n",
    "\n",
    "We have that $\\mathrm{W}_l$ is a $N_{l-1} \\times N_l$ matrix, while $\\vec{b}_l$ and $\\vec{y}_l$ are $N_l \\times 1$ column vectors. \n",
    "With this notation, the sum in  becomes a matrix-vector multiplication, and we can write\n",
    "the equation for the activations of hidden layer 2 in"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto6\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " \\vec{y}_2 = f_2(\\mathrm{W}_2 \\vec{y}_{1} + \\vec{b}_{2}) = \n",
    " f_2\\left(\\left[\\begin{array}{ccc}\n",
    "    w^2_{11} &w^2_{12} &w^2_{13} \\\\\n",
    "    w^2_{21} &w^2_{22} &w^2_{23} \\\\\n",
    "    w^2_{31} &w^2_{32} &w^2_{33} \\\\\n",
    "    \\end{array} \\right] \\cdot\n",
    "    \\left[\\begin{array}{c}\n",
    "           y^1_1 \\\\\n",
    "           y^1_2 \\\\\n",
    "           y^1_3 \\\\\n",
    "          \\end{array}\\right] + \n",
    "    \\left[\\begin{array}{c}\n",
    "           b^2_1 \\\\\n",
    "           b^2_2 \\\\\n",
    "           b^2_3 \\\\\n",
    "          \\end{array}\\right]\\right).\n",
    "\\label{_auto6} \\tag{12}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matrix-vector notation  and activation\n",
    "\n",
    "The activation of node $i$ in layer 2 is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto7\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " y^2_i = f_2\\Bigr(w^2_{i1}y^1_1 + w^2_{i2}y^1_2 + w^2_{i3}y^1_3 + b^2_i\\Bigr) = \n",
    " f_2\\left(\\sum_{j=1}^3 w^2_{ij} y_j^1 + b^2_i\\right).\n",
    "\\label{_auto7} \\tag{13}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is not just a convenient and compact notation, but also a useful\n",
    "and intuitive way to think about MLPs: The output is calculated by a\n",
    "series of matrix-vector multiplications and vector additions that are\n",
    "used as input to the activation functions. For each operation\n",
    "$\\mathrm{W}_l \\vec{y}_{l-1}$ we move forward one layer.\n",
    "\n",
    "\n",
    "### Activation functions\n",
    "\n",
    "A property that characterizes a neural network, other than its\n",
    "connectivity, is the choice of activation function(s).  As described\n",
    "in, the following restrictions are imposed on an activation function\n",
    "for a FFNN to fulfill the universal approximation theorem\n",
    "\n",
    "  * Non-constant\n",
    "\n",
    "  * Bounded\n",
    "\n",
    "  * Monotonically-increasing\n",
    "\n",
    "  * Continuous\n",
    "\n",
    "### Activation functions, Logistic and Hyperbolic ones\n",
    "\n",
    "The second requirement excludes all linear functions. Furthermore, in\n",
    "a MLP with only linear activation functions, each layer simply\n",
    "performs a linear transformation of its inputs.\n",
    "\n",
    "Regardless of the number of layers, the output of the NN will be\n",
    "nothing but a linear function of the inputs. Thus we need to introduce\n",
    "some kind of non-linearity to the NN to be able to fit non-linear\n",
    "functions Typical examples are the logistic *Sigmoid*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"sigmoidActivationFunction\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " f(x) = \\frac{1}{1 + e^{-x}},\n",
    "\\label{sigmoidActivationFunction} \\tag{14}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and the *hyperbolic tangent* function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"tanhActivationFunction\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    " f(x) = \\tanh(x)\n",
    "\\label{tanhActivationFunction} \\tag{15}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relevance\n",
    "\n",
    "The *sigmoid* function are more biologically plausible because the\n",
    "output of inactive neurons are zero. Such activation function are\n",
    "called *one-sided*. However, it has been shown that the hyperbolic\n",
    "tangent performs better than the sigmoid for training MLPs.  has\n",
    "become the most popular for *deep neural networks*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEWCAYAAACdaNcBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8leXdx/HPj2wymCGMMBwgsmREXE8tKraotbbWumdtqa1bu6x9tHbYaofaavWhzjpAba2iUrepqypEUGSHPU1IIGSQ/Xv+OMcSMZAAJ7nP+L5fr/PinPtc59y/c3HyzZXrXubuiIhIfOkSdAEiIhJ5CncRkTikcBcRiUMKdxGROKRwFxGJQwp3EZE4pHCXTmFmPzWze6NtvWa2yswm7+K5DDN71swqzOzJjquy1XUvMLNJnblOiS/JQRcgicHdb47B9Z4G5AG93L0xQiV9jpk9CKxz9599uszdR3bU+iQxaOQusmuDgaUdGewiHUXhLhFlZj82s/VmVmlmS8zsuPDyn5vZIy3anW9mq82szMz+t+X0SLjtk2b2SPh95pvZMDO7zsxKzGytmX2pxXv1N7OZZlZuZsVm9p0Wz+283vNarPf63XyOm4AbgDPMrMrMLm7lvYaYmZtZcvhxoZn90szeDtf9kpn1btH+f8zsHTPbGv4MF5rZVOAc4Efh9TwbbtuyP9LM7HYz2xC+3W5maeHnJpnZOjO7Ntw3G83sor39/5P4oXCXiDGzg4DLgEPdPRv4MrCqlXYjgL8QCrV+QDdgwE7NTgYeBnoAc4EXCX1fBwC/AP6vRdsZwDqgP6GplJvN7NhdrPdu4Lxw215Afmufxd1vBG4GHnf3LHe/r80OCDkbuAjoA6QCPwivezDwL+DPQC4wFpjn7tOAR4Fbw+s5uZX3vB44PPyaQ4CJwM9aPN+XHX14MXCXmfVoZ70SpxTuEklNQBowwsxS3H2Vuy9vpd1pwLPu/pa71xMaIe98kqM33f3F8JTIk4QC8bfu3kAozIeYWXczGwgcBfzY3WvdfR5wL3D+Ltb7nLu/4e51wP8Czfv+sT/jAXdf6u7bgScIBTKEQv8Vd5/u7g3uXhautT3OAX7h7iXuXgrcROgX1Kcaws83uPssoAo4KDIfR2KVwl0ixt2LgauAnwMlZjbDzPq30rQ/sLbF62qAsp3afNLi/nZgs7s3tXgMkBV+r3J3r2zRfjWf/0ugtfVWt7LefbWpxf2acI0AA4HWftG1R39Cn+lTq8PLPlW203aBluuVBKVwl4hy98fc/X8IbYx04JZWmm2kxXSImWUQmiLZGxuAnmaW3WLZIGD9LtY7sMV6u+7hequBri0e992D164FDtjFc22dmnUDof781KDwMpFdUrhLxJjZQWZ2bHhjXy2hEXZr0x5/B042syPNLJXQSN/2Zp3uvhZ4B/iNmaWb2RhC886PtNL878BXwhs2UwnN3e/Jz8A84GgzG2Rm3YDr9uC1jwKTzex0M0s2s15m9umUzSfA/rt57XTgZ2aWG95AewOtfz6R/1K4SySlAb8FNhOanuhDKwHo7guAywnNnW8kNEdcAtTt5XrPAoYQGs3+E7jR3V/ZxXovBR4Lr3cLoQ2x7eLuLwOPAx8BRcBze/DaNcCJwLVAOaFfFIeEn76P0HaKrWb2dCsv/xUwJ7ze+cAH4WUiu2S6WIcEzcyygK3AUHdfGXQ9IvFAI3cJhJmdbGZdzSwT+D2hEemqYKsSiR8KdwnKKYSmUTYAQ4EzXX9GikSMpmVEROKQRu4iInEosLNC9u7d24cMGRLU6v+rurqazMzMoMuICuqLEPXDDuqLHaKlL4qKija7e25b7QIL9yFDhjBnzpygVv9fhYWFTJo0KegyooL6IkT9sIP6Yodo6QszW912K03LiIjEJYW7iEgcUriLiMQhhbuISBxSuIuIxCGFu4hIHFK4i4jEIYW7iEgcUriLiMQhhbuISBxSuIuIxCGFu4hIHFK4i4jEoTbD3czuN7MSM/t4F8+bmf3JzIrN7CMzGx/5MkVEZE+0Z+T+IDBlN8+fQOgyaUOBqcDd+16WiIjsizbD3d3fAMp30+QU4G8e8i7Q3cz6RapAERHZc5G4WMcAYG2Lx+vCyzbu3NDMphIa3ZOXl0dhYWEEVr9vqqqqoqKOaKC+CFE/7KC+2CHW+qJTr8Tk7tOAaQAFBQUeDVc1iZarq0QD9UWI+mEH9cUOe9oX7k5lXSPbtjdQWRv6d1ttI1V1DVTVNlJZ10hVbSPVdY1U1TVRU99IdX0T2+sbqa5rorahiZr60PJ7zp3AkQf23qN6IxHu64GBLR7nh5eJiMSNhmZn/dbtbK6so7SyjvLqesqq6ymrqqO8pp6tNQ2UV9eztaaeinCQNzX7bt8zqYuRmZpEVloymWnJdE1LJjM1if7dU8hITSYjpQtdU5PJzU7b43ojEe4zgcvMbAZwGFDh7p+bkhERiVbVdY2s37qd9Vu2s6FiO5sqatlYUcsn2z691VGxvQFeeu1zr01P6UKvzDS6d02hZ2YqA3t2pXtGCt1a3HIykslJTyErPZns9BSy05PJSksmLbkLZtYhn6nNcDez6cAkoLeZrQNuBFIA3P0eYBZwIlAM1AAXdUilIiJ7yd0prapjZWk1KzdXs7q8hjXlNawN37bUNHymfReDPtnp5HVLZ7/emRy+fy+qNm9g4ujh5Gan0TsrjV5ZqfTKTCMjNSmgT7V7bYa7u5/VxvMOXBqxikRE9pK7s7GiliWbKlnySSXFJVUsK6liRUkVlXWN/22X3MXI75HBwJ5dGTW6H/k9MhjQPYP8Hhn0755BblYayUmf3ZmwsHAzkyYO6uyPtNc6dYOqiEikNDU7K0qr+HhDBR+v38bH6ytYtHEb22p3hHif7DSG5mVx6vgB7J+bxZDemezfO5N+3dI/F97xRuEuIjGhvLqeotVbKFq9hQ/XbmX++gqqwqPx9JQuHNwvh68c0p+D+2YzvF8Ow/Ky6ZaREnDVwVG4i0hUKq2s4z8ryvjP8jJmryqnuKQKgJQk4+B+OXx93AAOGdidMfnd2L93ZtyPxPeUwl1EokJtQxPvryznjaWlvLGslKWfhMI8Oy2ZgiE9+Pq4ARw6pCdj8ruRnhKdGzGjicJdRAJTsq2WVxeX8MrCT3h7+WZqG5pJTe7CxCE9+fq4fI48oBcj++doVL4XFO4i0qnWb93Ov+Zv5Pn5G5m7ZisA+T0yOPPQQXzxoFwO369X1O5eGEsU7iLS4cqr63n2ww08M289H4QDfWT/HK49fhjHj8zjoLzsDjuYJ1Ep3EWkQzQ0NfPqohL+XrSWwiWlNDY7w/tm88MvH8RJo/sxpHdm0CXGNYW7iETUmrIaZsxew5NF6yitrCMvJ42L/2c/vj5+AMP75gRdXsJQuIvIPnN33i4u48F3VvLq4hIMOHZ4H86aOIgvDsvVBtEAKNxFZK/VNzbz9Nz1/PXNFSwrqaJXZiqXH3MgZx02iH7dMoIuL6Ep3EVkj1XVNTL9vTXc99ZKNm2rZUS/HP7wzUP4yiH9SEvWni7RQOEuIu1WXdfI3/6zmmlvLGdLTQNH7N+LW08bwxeG9tbeLlFG4S4ibaptaOKRd1dzd+Fyyqrr+eKwXK6cPJTxg3oEXZrsgsJdRHap2Z2nPljHH15ayvqt2/mfA3tz9fHDmDBYoR7tFO4i0qp3V5Tx83dqWVP5IaMG5PC708bs8XU8JTgKdxH5jI0V2/n184t47qON9Eo37jhzLCeP6U+XLppTjyUKdxEBQkeU3vfWSu54ZRnN7lx53FBGdFnPl8cOCLo02QsKdxHho3Vb+fE/5rNo4zYmH5zHjSePYGDPrhQWbgi6NNlLCneRBFbb0MTvX1zC/W+vJDc7jXvOncCUUX2DLksiQOEukqDmrtnCtU9+yIrSas45bBA/PmE4OemJe1m6eKNwF0kw9Y3N3PHqUu4uXE7fnHQe/fZhHKW9YOKOwl0kgazaXM0VM+by0boKvjkhn/89eYRG63FK4S6SIJ6Zt57r//kxXQzuOXc8U0b1C7ok6UAKd5E4V9vQxI3PLODxOWspGNyD288cS36PrkGXJR1M4S4Sx9aW1/C9R4v4eP02Lj3mAK6ePEznVk8QCneROPXvpaVcOWMuTc3OvecXMHlEXtAlSSdSuIvEGXfn3jdXcvO/FnFQXjb3nDtB1ytNQO36+8zMppjZEjMrNrOftPL8IDN73czmmtlHZnZi5EsVkbbUNTbxo79/xK9nLWLKyL489f0jFewJqs2Ru5klAXcBxwPrgNlmNtPdF7Zo9jPgCXe/28xGALOAIR1Qr4jsQnl1Pd99eA6zV23himMP5KrJw3SyrwTWnmmZiUCxu68AMLMZwClAy3B34NPLmncDdEIKkU60uqyaC+5/nw0Vtdxx5lhO0cm+Ep65++4bmJ0GTHH3b4cfnwcc5u6XtWjTD3gJ6AFkApPdvaiV95oKTAXIy8ubMGPGjEh9jr1WVVVFVlZW0GVEBfVFSKz1w4qKJm4rqqXZ4arx6QztEblrmMZaX3SkaOmLY445psjdC9pqF6kNqmcBD7r7H8zsCOBhMxvl7s0tG7n7NGAaQEFBgU+aNClCq997hYWFREMd0UB9ERJL/fDa4k/43atz6Z2dwYMXTeSA3MiGTyz1RUeLtb5oT7ivBwa2eJwfXtbSxcAUAHf/j5mlA72BkkgUKSKf98y89VzzxIcc3C+b+y88lD7Z6UGXJFGkPXvLzAaGmtl+ZpYKnAnM3KnNGuA4ADM7GEgHSiNZqIjs8PC7q7nq8XkUDO7B9O8crmCXz2lz5O7ujWZ2GfAikATc7+4LzOwXwBx3nwlcC/zVzK4mtHH1Qm9rMl9E9srdhcu55YXFHDe8D3edM570lMjNsUv8aNecu7vPIrR7Y8tlN7S4vxA4KrKlicjObn9lKbe/soyvHtKfP5x+CCk6lYDsgo5QFYkB7s5tLy/lT68Vc9qEfG75xhiStA+77IbCXSTKuTu/f2kJd72+nDMKBvKbU0fr4CRpk8JdJMr94aWl3PX6cs6aOIhff22Ugl3aRRN2IlHszteWcefrxZw1caCCXfaIwl0kSt375gp+/9JSTh03gF9/TVMxsmcU7iJR6NH3VvOr5xdx0uh+3HraGAW77DGFu0iUee6jDfzs6Y85bngfbjtjrK6cJHtF3xqRKPLG0lKufnwehw7pyV3njCc1WT+isnf0zRGJEnPXbOG7DxdxYJ9s7r2gQEeeyj5RuItEgeKSKi56cDZ9ctJ46FuHkpOeEnRJEuMU7iIBK6ms5cIH3ie5i/G3b03UScAkInQQk0iAqusaufjBOZRV1fP4dw9ncC9d71QiQyN3kYA0NjVz6WMfsGBDBXedM44x+d2DLkniiEbuIgFwd26YuYDCJaXc/PXRHDs8L+iSJM5o5C4SgPveWslj763hki8ewNmHDQq6HIlDCneRTvbKwk/49axFTBnZlx99+aCgy5E4pXAX6UQLNlRwxYy5jB7QjdvOGKvTCkiHUbiLdJLSyjq+/dAcumWkcO/5BWSk6iAl6TjaoCrSCeoam7jkkSK21jTw5CVH0CdH+7JLx1K4i3Qwd+dn//yYotVbuOvs8Ywa0C3okiQBaFpGpIM98PYqnixaxxXHDeWkMf2CLkcShMJdpAO9tWwzv3p+IV8emcdVxw0NuhxJIAp3kQ6ytryGy6Z/wIF9svjj6dozRjqXwl2kA2yvb+K7DxfR1OxMO6+AzDRt3pLOpW+cSIS5O9c99RGLNm3jvgsKGNJbJwOTzqeRu0iEPfD2Kp6et4FrJg/TOWMkMAp3kQiavaqcm2ct4vgReVx6zIFBlyMJrF3hbmZTzGyJmRWb2U920eZ0M1toZgvM7LHIlikS/Uoqa7n00Q/I75HBH04/RBtQJVBtzrmbWRJwF3A8sA6YbWYz3X1hizZDgeuAo9x9i5n16aiCRaJRY1Mzlz82l221DTz0rYm6TJ4Erj0j94lAsbuvcPd6YAZwyk5tvgPc5e5bANy9JLJlikS3W19cwnsry/nNqaM5uF9O0OWItGtvmQHA2haP1wGH7dRmGICZvQ0kAT939xd2fiMzmwpMBcjLy6OwsHAvSo6sqqqqqKgjGqgvQva0H4o+aWTa3DqOHZRMj4piCguLO664TqbvxA6x1heR2hUyGRgKTALygTfMbLS7b23ZyN2nAdMACgoKfNKkSRFa/d4rLCwkGuqIBuqLkD3phzVlNVxe+CaH5Hfj7qlHkJYcX2d61Hdih1jri/ZMy6wHBrZ4nB9e1tI6YKa7N7j7SmApobAXiVu1DU1879Eiuphx59nj4y7YJba1J9xnA0PNbD8zSwXOBGbu1OZpQqN2zKw3oWmaFRGsUyTq3PTsQhZs2MYfTz+EgT27Bl2OyGe0Ge7u3ghcBrwILAKecPcFZvYLM/tquNmLQJmZLQReB37o7mUdVbRI0J6Zt57p74eugXrcwTpQSaJPu+bc3X0WMGunZTe0uO/ANeGbSFwrLqniuqfmc+iQHvzgS8OCLkekVTpCVWQPbK9v4tJHPyA9JYk/nTWO5CT9CEl00onDRPbATc8uYMknlTx40aH065YRdDkiu6Rhh0g7PT13PTNmr+X7kw5g0kE6CFuim8JdpB2Wl1bx03+G5tmvOV7z7BL9FO4ibahtCM2zpyV30Ty7xAzNuYu04ZfPLWTxpkoeuFDz7BI7NAQR2Y3nPtrAo++t4btH788xwzXPLrFD4S6yC6vLqrnuH/MZN6g7P/jyQUGXI7JHFO4irahvbOby6XMxgz+dOY4UzbNLjNGcu0grbnlhMR+tq+Cec8frvDESkxTuIjuZW9LIfR+s5PwjBjNlVL+gyxHZK/pbU6SFjRXbuXd+HSP65fDTEw8OuhyRvaZwFwlrbGrmiulzaWyGO88eR3qKzs8usUvhLhJ2x6vLmL1qCxeMTGP/3KygyxHZJwp3EeCtZZu58/VivjkhnyP7a1OUxD6FuyS80so6rnp8HgfkZnHTKSODLkckIjREkYTW3Oxc88Q8KmsbeOTbE+maqh8JiQ8auUtCu/vfy3lz2WZuPHkkw/vmBF2OSMQo3CVhzV5Vzh9fXsrJh/TnrIkDgy5HJKIU7pKQyqvrufyxuQzskcHNXx+FmQVdkkhEaYJREk5zs/PDJz+kvLqep75/JNnpKUGXJBJxGrlLwrn3rRW8uriE6086mFEDugVdjkiHULhLQilaXc4tLyzhxNF9Of+IwUGXI9JhFO6SMLaE59kHdM/gt98Yo3l2iWuac5eE0NzsXPvkh2yuqucf3zuSHM2zS5zTyF0Swv+9sYLXFpfws68czOh8zbNL/FO4S9x7b0UZv39pCSeN6cd5h2ueXRJDu8LdzKaY2RIzKzazn+ym3TfMzM2sIHIliuy90so6Lp8+l8E9u3KL5tklgbQZ7maWBNwFnACMAM4ysxGttMsGrgTei3SRInujqdm5csZcttU28Jdzx5OVpk1MkjjaM3KfCBS7+wp3rwdmAKe00u6XwC1AbQTrE9lrt7+ylHeWl/HLU0bpvDGScNozlBkArG3xeB1wWMsGZjYeGOjuz5vZD3f1RmY2FZgKkJeXR2Fh4R4XHGlVVVVRUUc0iKe+mFfSyJ8/qOMLA5LJrVpOYeHydr82nvphX6kvdoi1vtjnv1PNrAvwR+DCttq6+zRgGkBBQYFPmjRpX1e/zwoLC4mGOqJBvPTFmrIarvjzm4zsn8NfLzlyjy+XFy/9EAnqix1irS/aMy2zHmh5yrz88LJPZQOjgEIzWwUcDszURlUJQm1DE5c8UoSZcc+5E3QdVElY7Qn32cBQM9vPzFKBM4GZnz7p7hXu3tvdh7j7EOBd4KvuPqdDKhbZBXfnZ09/zKJN27j9jLEM7Nk16JJEAtNmuLt7I3AZ8CKwCHjC3ReY2S/M7KsdXaBIez3y3hr+XrSOy48dyjHD+wRdjkig2jXn7u6zgFk7LbthF20n7XtZIntm9qpybpq5gGMOyuWq44YGXY5I4HSEqsS8TRW1fO+RD8jvkcHtZ46jSxcdqCSiozokptU1NvG9R4uoqW/kse8cRrcMnRBMBBTuEsPcnRueXsDcNVv5yznjGZaXHXRJIlFD0zISsx56ZxWPz1nL5cceyImj+wVdjkhUUbhLTHpr2WZ++fwijh+Rx9WThwVdjkjUUbhLzFm1uZpLH/uAA3OzuO2MsdqAKtIKhbvElIrtDVz80Gy6GNx7QYHO9CiyC/rJkJjR0NTM9x8tYk15DQ9ffJiOQBXZDYW7xAR354ZnFvB2cRm//+YhHL5/r6BLEolqmpaRmHDfWyuZ/v4avj/pAE6bkB90OSJRT+EuUe+Fjzfy61mLOGFUX37wpYOCLkckJijcJaoVrS7nyhnzGDewu/aMEdkDCneJWstLq7j4oTn0757BvRccqnOzi+wBhbtEpdLKOi584H2SzHjwokPpmZkadEkiMUV7y0jU2VbbwIUPvE9pZR0zph7B4F6ZQZckEnM0cpeoUtvQxHcemsOSTZXcfe4Exg7sHnRJIjFJI3eJGo1NzVwxfS7vrSznjjPHcsxBupqSyN7SyF2iQnOzc91T83lp4Sf8/OQRnDJ2QNAlicQ0hbsEzt25ceYCnixaxxXHDeXCo/YLuiSRmKdwl0C5OzfPWsTD765m6tH7c/VkXf9UJBIU7hKo215eyl/fXMkFRwzmuhOGY6aDlEQiQRtUJRDuzu2vLONPrxVzekE+N548UsEuEkEKd+l07s4fX17Kn18r5psT8vnNqWN0WgGRCFO4S6dyd3734hL+UricMwoG8ptTRyvYRTqAwl06zacbT//65krOmjiIX39tlIJdpIMo3KVTNDU7P31qPo/PWcv5Rwzm5yePVLCLdCCFu3S4+sZmrn58Hs/P38jlxx7INccP08ZTkQ7Wrl0hzWyKmS0xs2Iz+0krz19jZgvN7CMze9XMBke+VIlF1XWNfPtvc3h+/kZ+euJwrv3SQQp2kU7QZribWRJwF3ACMAI4y8xG7NRsLlDg7mOAvwO3RrpQiT0llbWcMe0/vF28mVu+MZqpRx8QdEkiCaM9I/eJQLG7r3D3emAGcErLBu7+urvXhB++C+gilwmuuKSKU//yDstLqrn3/ALOOHRQ0CWJJBRz9903MDsNmOLu3w4/Pg84zN0v20X7O4FN7v6rVp6bCkwFyMvLmzBjxox9LH/fVVVVkZWVFXQZUSFSfbGorIk759WSZHD1hHT26xZbV1DSd2IH9cUO0dIXxxxzTJG7F7TVLqIbVM3sXKAA+GJrz7v7NGAaQEFBgU+aNCmSq98rhYWFREMd0SASfTH9/TX84aWPGdwrkwcunMigXl0jU1wn0ndiB/XFDrHWF+0J9/XAwBaP88PLPsPMJgPXA19097rIlCexorGpmZtnLeb+t1dy9LBc7jx7HDnpKUGXJZKw2hPus4GhZrYfoVA/Ezi7ZQMzGwf8H6Hpm5KIVylRrayqjitnzOOt4s1cdNQQrj/xYJKTdE46kSC1Ge7u3mhmlwEvAknA/e6+wMx+Acxx95nA74As4Mnwbm5r3P2rHVi3RIkP127le48Usbm6nlu/MYbTDx3Y9otEpMO1a87d3WcBs3ZadkOL+5MjXJdEOXfnsffXcNPMheRmp/GPS45kdH63oMsSkTAdoSp7rGJ7Az99aj7Pz9/I0cNyueOMsfTITA26LBFpQeEue+SDNVu4YvpcNlbU8uMpw/nu0fvrHDEiUUjhLu3S0NTMna8Vc+frxfTrls6TlxzB+EE9gi5LRHZB4S5tWvZJJdc88SHz11fwtbH9uemUUXTL0G6OItFM4S671NDUzL1vruS2V5aSlZbM3eeM54TR/YIuS0TaQeEurfpo3VZ+/I/5LNq4jS+PzONXXxtNbnZa0GWJSDsp3OUzttU2cPvLy3jwnZXkZqdxz7kTmDKqb9BlicgeUrgLAM3NzpvrGvjB7wspq67nnMMG8aMpw3UKAZEYpXAXilZv4VfPL2TumnrGD+rOAxdO1AFJIjFO4Z7AVpRWcesLS3hhwSZys9P4zuhUrjvrSO23LhIHFO4JaP3W7dz5WjFPzFlLenIXrjl+GN/+wn68/85bCnaROKFwTyAbK7bzl9eXM2P2GgzjnMMGcfmxQ7UXjEgcUrgngOWlVUz79wqemrsOdzj90IFcesyBDOieEXRpItJBFO5xyt2Zs3oL9725khcXbiI1qQtnHjqIqUfvz8CesXd1JBHZMwr3OFPb0MSzH27gwXdWsWDDNrplpHDZMQdywZFD6J2l6ReRRKFwjxPLPqlk+vtreWruOrbWNDAsL4ubvz6ar43rT9dU/TeLJBr91MewipoGnpu/gac+WE/R6i2kJBlfGtmXcyYO4ogDehG+KpaIJCCFe4ypqW/k9cWlPPvhBl5bXEJ9UzND+2Rx3QnDOW1CPr009SIiKNxjwrbaBv69pJQXPt7Ea4tL2N7QRO+sNM49fDCnjh/AyP45GqWLyGco3KOQu7OqrIZ/LynhlUUlvLuijMZmp3dWKt+YMICTRvdn4n49SdIBRyKyCwr3KLGlup53V5Tx9vLN/HtpKWvLtwNwQG4mF39hP740Io+xA3so0EWkXRTuASnZVsuc1VuYs2oL764oY9GmbbhD19QkjjygF1O/sD9HD8tlcK/MoEsVkRikcO8EtQ1NLNiwjQ/XbmVe+LamvAaA9JQujBvYg2smD+PIA3sxJr87KUldAq5YRGKdwj3CNlfVsXhjJYs3bWPhhm18vKGC5aXVNDU7AP26pXNIfnfOO3wwBUN6MLJ/N1KTFeYiElkK973Q1Oxs2LqdlZurWVFaRXFpFcs+qWJ5aRWbq+r/2y43O43RA7oxZWRfRg7oxtiB3cnLSQ+wchFJFAr3Vrg7W2saWLdlO+u31rBuy3bWltewJnxbW76d+qbm/7bPSU9maF42xw3PY1jfbIb3zeagvtk63F9EApNQ4e7uVNU1UlpZx+aqejZX1fH2qgbe/ddiSipr2VQRum2sqGV7Q9NnXpuVlsygnl0Z2iebySPy2L93JkN6ZbJfbia5WWnaz1xEokpMhru7U1PfRGVtI9tqG9i2vYGKFretNQ0eyiqDAAAF3UlEQVRsralnS00DW2rqKauqp7y6nvKaeuobmz/3finLVpCblUbfbukc3C+HY4f3oW+3dPJ7dCW/RwYDumfQvWuKAlxEYka7wt3MpgB3AEnAve7+252eTwP+BkwAyoAz3H3V7t5zW20Dz364ge0NTdQ2NLG9voma+tD96vpGaurC/9Y3UVXXSHVdI9V1TVTWNlBV10h4++Qu5aQn0zMzle5dU+nbLZ2R/XPomZVKr8xUcrPT6J0VuhXPL+KkyZN0BSIRiStthruZJQF3AccD64DZZjbT3Re2aHYxsMXdDzSzM4FbgDN2976ry2q4fPrczy1PT+lC19RkuqYmkZmaTEZqEtnpyeRlp5OZlkx2euiWlZZMTkYK2enJ5KSnkJORQreMFLqHlyW3c3fCT5aYgl1E4k57Ru4TgWJ3XwFgZjOAU4CW4X4K8PPw/b8Dd5qZufsux9cH5mbxzDVHk5acRHpKEl1Tk8hISVLQiohEQHvCfQCwtsXjdcBhu2rj7o1mVgH0Aja3bGRmU4GpAHl5eaxbWLSXZUdOVVUVhYWFQZcRFdQXIeqHHdQXO8RaX3TqBlV3nwZMAygoKPBJkyZ15upbVVhYSDTUEQ3UFyHqhx3UFzvEWl+0Z2J6PTCwxeP88LJW25hZMtCN0IZVEREJQHvCfTYw1Mz2M7NU4Exg5k5tZgIXhO+fBry2u/l2ERHpWG1Oy4Tn0C8DXiS0K+T97r7AzH4BzHH3mcB9wMNmVgyUE/oFICIiAWnXnLu7zwJm7bTshhb3a4FvRrY0ERHZWzodoYhIHFK4i4jEIYW7iEgcUriLiMQhhbuISBxSuIuIxCGFu4hIHFK4i4jEIYW7iEgcUriLiMQhhbuISBxSuIuIxCEL6sy8ZlYKrA5k5Z/Vm52uGJXA1Bch6ocd1Bc7REtfDHb33LYaBRbu0cLM5rh7QdB1RAP1RYj6YQf1xQ6x1healhERiUMKdxGROKRwD1+wWwD1xafUDzuoL3aIqb5I+Dl3EZF4pJG7iEgcUriLiMQhhXuYmV1rZm5mvYOuJShm9jszW2xmH5nZP82se9A1dTYzm2JmS8ys2Mx+EnQ9QTGzgWb2upktNLMFZnZl0DUFzcySzGyumT0XdC3toXAn9EUGvgSsCbqWgL0MjHL3McBS4LqA6+lUZpYE3AWcAIwAzjKzEcFWFZhG4Fp3HwEcDlyawH3xqSuBRUEX0V4K95DbgB8BCb112d1fcvfG8MN3gfwg6wnARKDY3Ve4ez0wAzgl4JoC4e4b3f2D8P1KQqE2INiqgmNm+cBJwL1B19JeCR/uZnYKsN7dPwy6lijzLeBfQRfRyQYAa1s8XkcCB9qnzGwIMA54L9hKAnU7oQFgc9CFtFdy0AV0BjN7BejbylPXAz8lNCWTEHbXF+7+TLjN9YT+LH+0M2uT6GNmWcA/gKvcfVvQ9QTBzL4ClLh7kZlNCrqe9kqIcHf3ya0tN7PRwH7Ah2YGoWmID8xsortv6sQSO82u+uJTZnYh8BXgOE+8gyDWAwNbPM4PL0tIZpZCKNgfdfengq4nQEcBXzWzE4F0IMfMHnH3cwOua7d0EFMLZrYKKHD3aDjzW6czsynAH4Evuntp0PV0NjNLJrQh+ThCoT4bONvdFwRaWAAsNNp5CCh396uCridahEfuP3D3rwRdS1sSfs5dPuNOIBt42czmmdk9QRfUmcIbky8DXiS0AfGJRAz2sKOA84Bjw9+FeeGRq8QIjdxFROKQRu4iInFI4S4iEocU7iIicUjhLiIShxTuIiJxSOEuIhKHFO4iInFI4S4SZmaXtDhgZ6WZvR50TSJ7SwcxiewkfE6V14Bb3f3ZoOsR2RsauYt83h3Aawp2iWUJcVZIkfYKnxVzMKFzzIjELE3LiISZ2QRCZ0L8grtvCboekX2haRmRHS4DegKvhzeqxswl1UR2ppG7iEgc0shdRCQOKdxFROKQwl1EJA4p3EVE4pDCXUQkDincRUTikMJdRCQO/T/+UeY0C/tmnQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGnhJREFUeJzt3X2QZXV95/H3h+FBEzSAaIvDCBjYrPiEoQNumY0NKhmNC6ldE2E3ChvJbFJiYhI3YtxSi8SEmN0lm8KsThFKfIhjdNVMUhMRhV438YnBIMIkwIBEZsQQAR86Kjj0d/+4p6fvabpneuYe+va9835VdfU9z9/+Vc98+pzfOb+TqkKSpDkHDbsASdLqYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJD2QZIfS3Jjku8k+dUVPO5Tk8wkWbNSx9SBy2DQWEny1iTvexQP8VvAdVX1uKr640frIEnuSvKiuemq+mpVHV5VDz9ax5TmGAzSvjkOuGXYRUiPJoNBIynJG5LsbC7p3JrkhUnWA78NvKK57PKlZt0fSfKnSe5ptvnduUsySS5I8rdJLk/yrST/kOSFSxzzWuAM4PJm//8qyXSSC/vWuSDJ3/RNV5JfTnJ7km8meUeS9C3/pSR/3/wc25L8eJL3Ak8F/rI5zm8lOb7Z18HNdk9JsjnJ/Um2J/mlvn2+NcmfJ3lPs99bkkx22f4abwaDRk6SHwMuAn6iqh4H/DRwV1V9HPg94IPNZZfnNJu8G9gFnAg8FzgLuLBvl6cDdwBHA28BPpLkqIXHraozgf8HXNTs/7Zllvwy4CeAZwM/39RLkp8D3gq8Cng8cDZwX1W9Evgq8O+a47x9kX1uAnYATwFeDvxekjP7lp/drHMEsBm4fJm1SgaDRtLDwGHAyUkOqaq7quqOxVZMMgG8FHhdVf1LVd0LXAac27favcAfVdUPquqDwK3Az3RY76VV9c2q+ipwHXBKM/9C4O1VdX31bK+qf9zbzpKsA54PvKGqvl9VNwJX0AuYOX9TVVuaPon3As9ZZFfSogwGjZyq2g68jt5f2/cm2ZTkKUusfhxwCHBPcynnm8C7gCf1rbOz2qNJ/iO9v8S78vW+z98FDm8+r6N3prKvngLcX1Xf6Zv3j8DaPRzzMXOXoaS9MRg0kqrqz6rqJ+n9x1/AH8wtWrDq3cCDwNFVdUTz9fiqekbfOmv7r/vTu77/tWWW8i/AD/VNP3nZP0Svth9dYtmehj3+GnBUksf1zXsqsHMfji0tyWDQyGmeJTgzyWHA94HvAbPN4n8Cjk9yEEBV3QN8AvgfSR6f5KAkP5rkBX27fBLwq0kOaa77Px3YssxybgT+fZIfSnIi8Op9+FGuAF6f5NT0nJjkuL6f42mLbVRVdwOfAX4/yWOSPLs57qN5m64OIAaDRtFhwKXAN+hdMnkS8MZm2Yea7/cl+WLz+VXAocA24AHgw8Axffv7PHBSs7+3AS+vqvuWWctlwEP0/iO/Cnj/cn+IqvpQc7w/A74DfAyY6/T+feC/NZe/Xr/I5ucBx9M7e/go8Jaq+uRyjy3tSXxRjw5kSS4ALmwuS0nCMwZJ0gKdBEOSK5Pcm+TmJZZPNQ8P3dh8vblv2frmAaXtSS7uoh5J0v7r5FJSkp8CZoD3VNUzF1k+Bby+ql62YP4a4DbgxfQe1rkeOK+qtg1clCRpv3RyxlBVnwbu349NTwO2V9WdVfUQvSc1z+miJknS/lnJB17+TTN2zdfonT3cQu+BnLv71tlBb3iCR0iyAdgA8NjHPvbUdevWPcrl7tns7CwHHWQXDdgW/WyLebbFvNXSFrfddts3quqJe1tvpYLhi8BxVTWT5KX0bss7aV92UFUbgY0Ak5OTtXXr1u6r3AfT09NMTU0NtYbVwraYZ1vMsy3mrZa2SLLXIVdghe5KqqpvV9VM83kLcEiSo+k9qdn/p/+x+PSmJA3VigRDkifPDTmQ5LTmuPfR62w+KckJSQ6lN7DZ5pWoSZK0uE4uJSX5ADAFHJ1kB72hiw8BqKp30hsW+FeS7KI3fMG5zaBlu5JcBFwNrAGubPoeJElD0kkwVNV5e1l+OUuMB99cWlruuDSSpEfZ8LvJJUmrisEgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJaukkGJJcmeTeJDcvsfw/JbkpyZeTfCbJc/qW3dXMvzHJ1i7qkSTtv67OGN4NrN/D8q8AL6iqZwG/A2xcsPyMqjqlqiY7qkeStJ8O7mInVfXpJMfvYfln+iY/BxzbxXElSd0bRh/Dq4G/7psu4BNJbkiyYQj1SJL6pKq62VHvjOGvquqZe1jnDOBPgJ+sqvuaeWurameSJwHXAK+tqk8vsu0GYAPAxMTEqZs2beqk7v01MzPD4YcfPtQaVgvbYp5tMc+2mLda2uKMM864YTmX7Du5lLQcSZ4NXAG8ZC4UAKpqZ/P93iQfBU4DHhEMVbWRpm9icnKypqamVqLsJU1PTzPsGlYL22KebTHPtpg3am2xIpeSkjwV+Ajwyqq6rW/+Dyd53Nxn4Cxg0TubJEkro5MzhiQfAKaAo5PsAN4CHAJQVe8E3gw8AfiTJAC7mtOZCeCjzbyDgT+rqo93UZMkaf90dVfSeXtZfiFw4SLz7wSe88gtJEnD4pPPkqQWg0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIklo6CYYkVya5N8nNSyxPkj9Osj3JTUl+vG/Z+Ulub77O76IeSdL+O7ij/bwbuBx4zxLLXwKc1HydDvxv4PQkRwFvASaBAm5IsrmqHuioLmlFzVbx/R88POwyVoWHHrYt5qyGtjgoWfa6nQRDVX06yfF7WOUc4D1VVcDnkhyR5BhgCrimqu4HSHINsB74QBd1SSvtshse5Bev/viwy1g9rrEtdhtyW5zxY09c9rpdnTHszVrg7r7pHc28peY/QpINwAaAiYkJpqenH5VCl2tmZmboNawWtsW8e2Z28dTHreG0Y9YMu5She+jBhzj0sEOHXcaqsBra4kmP/fay112pYBhYVW0ENgJMTk7W1NTUUOuZnp5m2DWsFrZFn/+7hckTj+G/v+KUYVcydP5ezFstbfGG85a33krdlbQTWNc3fWwzb6n50kiqApZ/KVdalVYqGDYDr2ruTnoe8K2quge4GjgryZFJjgTOauZJI6nYt04+aTXq5FJSkg/Q60g+OskOencaHQJQVe8EtgAvBbYD3wX+c7Ps/iS/A1zf7OqSuY5oaVQdZC5oxHV1V9Ier1w1dyO9ZollVwJXdlGHNGyzBfFakkacTz5LHTvIf1Uacf4KSx2aLbD3WaPOYJA6VJR9DBp5BoPUpfKuJI0+g0Hq0CxgLmjUGQxSxzxj0KgzGKQO9TqfpdFmMEgd84xBo85gkDpUZR+DRp/BIHVoFofE0OgzGKQuebuqxoDBIHVoFnzwWSPPYJC65BmDxoDBIHXI9/RoHBgMUod8UY/GgcEgdcjbVTUODAapQwXEZNCIMxikjvReVOhzDBp9nQRDkvVJbk2yPcnFiyy/LMmNzddtSb7Zt+zhvmWbu6hHGoa5cZJ8tadG3cDvfE6yBngH8GJgB3B9ks1VtW1unar69b71Xws8t28X36uqUwatQxo2zxg0Lro4YzgN2F5Vd1bVQ8Am4Jw9rH8e8IEOjiutKrvPGAwGjbiBzxiAtcDdfdM7gNMXWzHJccAJwLV9sx+TZCuwC7i0qj62xLYbgA0AExMTTE9PD175AGZmZoZew2phW/T8oEmGu77yFaandw65muHz92LeqLVFF8GwL84FPlxVD/fNO66qdiZ5GnBtki9X1R0LN6yqjcBGgMnJyZqamlqRgpcyPT3NsGtYLWyLnu//4GH4xMd52o8+jampE4ddztD5ezFv1Nqii0tJO4F1fdPHNvMWcy4LLiNV1c7m+53ANO3+B2lkzO7uY/BakkZbF8FwPXBSkhOSHErvP/9H3F2U5F8DRwKf7Zt3ZJLDms9HA88Hti3cVhoFTS7Y+ayRN/ClpKraleQi4GpgDXBlVd2S5BJga1XNhcS5wKaau3Wj5+nAu5LM0gupS/vvZpJGydwZg7eratR10sdQVVuALQvmvXnB9FsX2e4zwLO6qEEatrm/eLySpFHnk89SR2q2990hMTTqDAapI4UPuGk8GAxSR+aHxJBGm8EgdWT37aqeMmjEGQxSR2r3kBgGg0abwSB1pHbfriqNNoNB6sjc7ao++axRZzBIHdn9gJu5oBFnMEgdcUgMjQuDQeqIQ2JoXBgMUkfKF/VoTBgMUkfmLyWZDBptBoPUETufNS4MBqkj3q6qcWEwSB3xjEHjwmCQOrL7yWeTQSPOYJA6Uo6uqjFhMEgdmfWuJI2JToIhyfoktybZnuTiRZZfkOSfk9zYfF3Yt+z8JLc3X+d3UY80DL6oR+Ni4Hc+J1kDvAN4MbADuD7J5qratmDVD1bVRQu2PQp4CzBJ76aOG5ptHxi0Lmmlze5+tedw65AG1cUZw2nA9qq6s6oeAjYB5yxz258Grqmq+5swuAZY30FN0oqbO2Ow81mjbuAzBmAtcHff9A7g9EXW+w9Jfgq4Dfj1qrp7iW3XLnaQJBuADQATExNMT08PXvkAZmZmhl7DamFb9Nz1rYcBuOXmmznsn/9hyNUMn78X80atLboIhuX4S+ADVfVgkv8CXAWcuS87qKqNwEaAycnJmpqa6rzIfTE9Pc2wa1gtbIuem3Z8Ez77tzz7Wc9i6uSJYZczdP5ezBu1tujiUtJOYF3f9LHNvN2q6r6qerCZvAI4dbnbSqNi91hJ3uunEdfFr/D1wElJTkhyKHAusLl/hSTH9E2eDfx98/lq4KwkRyY5EjirmSeNHIfd1rgY+FJSVe1KchG9/9DXAFdW1S1JLgG2VtVm4FeTnA3sAu4HLmi2vT/J79ALF4BLqur+QWuShmFurCT7njXqOuljqKotwJYF897c9/mNwBuX2PZK4Mou6pCGySExNC68Gip1xFd7alwYDFJHZnePlWQyaLQZDFJH5jqfPWPQqDMYpI7Mv/PZZNBoMxikjpQv6tGYMBikjvhqT40Lg0HqiK/21LgwGKSOeLuqxoXBIHVk7ozBl3tq1BkMUkc8Y9C4MBikjsy/2tNk0GgzGKSO+GpPjQuDQeqIt6tqXBgMUkfmO5+l0WYwSB2Z73z2jEGjzWCQOuKQGBoXBoPUkVnPGDQmDAapI/O3qw65EGlAnQRDkvVJbk2yPcnFiyz/jSTbktyU5FNJjutb9nCSG5uvzV3UIw3D7hf1GAwacQO/8znJGuAdwIuBHcD1STZX1ba+1f4OmKyq7yb5FeDtwCuaZd+rqlMGrUMaNt/5rHHRxRnDacD2qrqzqh4CNgHn9K9QVddV1Xebyc8Bx3ZwXGlV2f2inuGWIQ1s4DMGYC1wd9/0DuD0Paz/auCv+6Yfk2QrsAu4tKo+tthGSTYAGwAmJiaYnp4epOaBzczMDL2G1cK26Nn2tV0AXP+FL/DVH7b7zt+LeaPWFl0Ew7Il+QVgEnhB3+zjqmpnkqcB1yb5clXdsXDbqtoIbASYnJysqamplSh5SdPT0wy7htXCtui574YdcNOXeN7zTue4J/zwsMsZOn8v5o1aW3TxZ81OYF3f9LHNvJYkLwLeBJxdVQ/Oza+qnc33O4Fp4Lkd1CStuLknn71dVaOui2C4HjgpyQlJDgXOBVp3FyV5LvAueqFwb9/8I5Mc1nw+Gng+0N9pLY2M3W9jMBc04ga+lFRVu5JcBFwNrAGurKpbklwCbK2qzcAfAocDH2ru2PhqVZ0NPB14V5JZeiF16YK7maSR4V1JGhed9DFU1RZgy4J5b+77/KIltvsM8KwuapCGzRf1aFx464TUkd0PuHnDqkacwSB1xCExNC4MBqkjs7t7n4dahjQwg0HqSHm7qsaEwSB1xBf1aFwYDFJH5h5wMxY06gwGqSOeMWhcGAxSR2YdXlVjwmCQOuIDbhoXBoPUkbnnGBwSQ6POYJA6MusZg8aEwSB1xM5njQuDQerI7s5nacQZDFLHPGPQqDMYpI7Mzs51Pg+5EGlABoPUkVn7GDQmDAapIw67rXFhMEgd2f2iHs8YNOI6CYYk65PcmmR7kosXWX5Ykg82yz+f5Pi+ZW9s5t+a5Ke7qEcaiipHw9BYGDgYkqwB3gG8BDgZOC/JyQtWezXwQFWdCFwG/EGz7cnAucAzgPXAnzT7k0bOrHerakwc3ME+TgO2V9WdAEk2AecA2/rWOQd4a/P5w8Dl6Z1vnwNsqqoHga8k2d7s77N7OuC3v/cDPnHL1zsoff/d/E+7eGjINawWtkXPnd+YsX9BY6GLYFgL3N03vQM4fal1qmpXkm8BT2jmf27BtmsXO0iSDcAGgEOffCIb3ntDB6UP6O9WQQ2rhW0BwOGHFNPT08MuY1WYmZmxLRqj1hZdBMOKqKqNwEaAZzznufXB1/7kUOu54YatnHrq5FBrWC1si3l33PxFpqamhl3GqjA9PW1bNEatLboIhp3Aur7pY5t5i62zI8nBwI8A9y1z20d47CFreObaHxmk5oF94/bh17Ba2BbzvnG715I0+rq4K+l64KQkJyQ5lF5n8uYF62wGzm8+vxy4tnpvTt8MnNvctXQCcBLwhQ5qkiTtp4HPGJo+g4uAq4E1wJVVdUuSS4CtVbUZ+FPgvU3n8v30woNmvT+n11G9C3hNVT08aE2SpP3XSR9DVW0BtiyY9+a+z98Hfm6Jbd8GvK2LOiRJg/PJZ0lSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJahkoGJIcleSaJLc3349cZJ1Tknw2yS1Jbkryir5l707ylSQ3Nl+nDFKPJGlwg54xXAx8qqpOAj7VTC/0XeBVVfUMYD3wR0mO6Fv+X6vqlObrxgHrkSQNaNBgOAe4qvl8FfCzC1eoqtuq6vbm89eAe4EnDnhcSdKjZNBgmKiqe5rPXwcm9rRyktOAQ4E7+ma/rbnEdFmSwwasR5I0oFTVnldIPgk8eZFFbwKuqqoj+tZ9oKoe0c/QLDsGmAbOr6rP9c37Or2w2AjcUVWXLLH9BmADwMTExKmbNm3a80/2KJuZmeHwww8fag2rhW0xz7aYZ1vMWy1tccYZZ9xQVZN7XbGq9vsLuBU4pvl8DHDrEus9Hvgi8PI97GsK+KvlHPfUU0+tYbvuuuuGXcKqYVvMsy3m2RbzVktbAFtrGf/HDnopaTNwfvP5fOAvFq6Q5FDgo8B7qurDC5Yd03wPvf6JmwesR5I0oEGD4VLgxUluB17UTJNkMskVzTo/D/wUcMEit6W+P8mXgS8DRwO/O2A9kqQBHTzIxlV1H/DCReZvBS5sPr8PeN8S2585yPElSd3zyWdJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQMFQ5KjklyT5Pbm+5FLrPdwkhubr819809I8vkk25N8MMmhg9QjSRrcoGcMFwOfqqqTgE8104v5XlWd0nyd3Tf/D4DLqupE4AHg1QPWI0ka0KDBcA5wVfP5KuBnl7thkgBnAh/en+0lSY+OgwfcfqKq7mk+fx2YWGK9xyTZCuwCLq2qjwFPAL5ZVbuadXYAa5c6UJINwIZmcibJrQPWPqijgW8MuYbVwraYZ1vMsy3mrZa2OG45K+01GJJ8EnjyIove1D9RVZWkliqmqnYmeRpwbZIvA99aToF9+98IbNyXbR5NSbZW1eSw61gNbIt5tsU822LeqLXFXoOhql601LIk/5TkmKq6J8kxwL1L7GNn8/3OJNPAc4H/AxyR5ODmrOFYYOd+/AySpA4N2sewGTi/+Xw+8BcLV0hyZJLDms9HA88HtlVVAdcBL9/T9pKklTVoMFwKvDjJ7cCLmmmSTCa5olnn6cDWJF+iFwSXVtW2ZtkbgN9Isp1en8OfDljPSlo1l7VWAdtinm0xz7aYN1Jtkd4f7pIk9fjksySpxWCQJLUYDB1I8ptJqulcPyAl+cMk/5DkpiQfTXLEsGtaaUnWJ7m1GeJlqVEAxl6SdUmuS7ItyS1Jfm3YNQ1TkjVJ/i7JXw27luUyGAaUZB1wFvDVYdcyZNcAz6yqZwO3AW8ccj0rKska4B3AS4CTgfOSnDzcqoZmF/CbVXUy8DzgNQdwWwD8GvD3wy5iXxgMg7sM+C3ggO7Fr6pP9D3F/jl6z6UcSE4DtlfVnVX1ELCJ3pAxB5yquqeqvth8/g69/xSXHNVgnCU5FvgZ4Iq9rbuaGAwDSHIOsLOqvjTsWlaZXwT+ethFrLC1wN1903sc4uVAkeR4eg+0fn64lQzNH9H7w3F22IXsi0HHShp7exkS5LfpXUY6IOypLarqL5p13kTvUsL7V7I2rT5JDqc3wsHrqurbw65npSV5GXBvVd2QZGrY9ewLg2EvlhoSJMmzgBOAL/UGiuVY4ItJTquqr69giStmT8OjACS5AHgZ8MI68B6Q2Qms65s+oId4SXIIvVB4f1V9ZNj1DMnzgbOTvBR4DPD4JO+rql8Ycl175QNuHUlyFzBZVathBMUVl2Q98D+BF1TVPw+7npWW5GB6ne4vpBcI1wP/sapuGWphQ9AMqX8VcH9VvW7Y9awGzRnD66vqZcOuZTnsY1BXLgceB1zTvKnvncMuaCU1He8XAVfT62z98wMxFBrPB14JnNn35saXDrsoLZ9nDJKkFs8YJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgdSDJL/c9zPWVJNcNuyZpf/mAm9ShZoyga4G3V9VfDrseaX94xiB1638B1xoKGmWOrip1pBld9jh6YyZJI8tLSVIHkpxKb0TRf1tVDwy7HmkQXkqSunERcBRwXdMBPVKvcpT6ecYgSWrxjEGS1GIwSJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLX8f0kuJFspbuVaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd83Hd9+PHXW9vW3h7y0PB2bMdLnrETZ5lAAhRKCCMp0ABtaEspJYEWKC39hZaWQqGFNIQEAhkEEpzpOLZlJ952vLckL8lD27Yka929f3/oZGRH+8b3xvv5eNzDd/dd7691d+/vZ35FVTHGGGO6RDkdgDHGmOBiicEYY8w1LDEYY4y5hiUGY4wx17DEYIwx5hqWGIwxxlzDEoMJKyLydRF53E/7/qKIXBCRRhHJ9Mcxejmu387JmJ6IjWMwpn8iEgtcAhao6l4/Hmc58LSq5vnrGMb0x0oMxgxMLpAAHHQ6EGP8zRKDCUki8jURqRSRyyJyVERWeN7/tog87Xk+XkRURO4XkdMiUiMi3+i2jygReVhEykSkVkSeF5GMHo41ETjqedkgIuu67Tum23olIvI5z/MHROQdEfm+iNSLyAkRWdlt3QwR+YWInPUsf0lEEoHXgVGe6qpGERnV/Zw8294tIgdFpMFzzCndlp0Ukb8TkX0iclFEnhORBF/9v5vIYInBhBwRmQQ8BMxT1WTgDuBkH5ssASYBK4Bvdvsh/RLwQWAZMAqoB35y/caqegyY5nmZpqq3DDDUYjoTShbwb8DPRUQ8y34FDPfsNwf4gao2ASuBs6qa5Hmcve7cJwLPAH8DZAOvAS+LSFy31f4UuBPIB2YADwwwXmMASwwmNLmAeGCqiMSq6klVLetj/X9S1SuetoG9wEzP+18AvqGqFaraCnwb+Ej3UoCXTqnq/6mqC3gKGAnkishIOhPAF1S1XlXbVXXDAPf5MeBVVV2jqu3A94FhwKJu6/xIVc+qah3wMjDLR+djIoQlBhNyVLWUzivmbwNVIvKsiIzqY5Pz3Z43A0me5+OAFz1VMg3AYTqTTq6PQr16XFVt9jxNAsYAdapaP4R9jgJOdduvGzgDjO7puFx7vsYMiCUGE5JU9TequoTOH3cFvjeE3ZwBVqpqWrdHgqpWDmDbJs+/w7u9N2IQx80QkbQelvXXTfAsnecMgKdqagwwkJiNGRBLDCbkiMgkEblFROKBFuAK4B7Crn4KfFdExnn2my0i9wxkQ1WtpvPH+JMiEi0inwEKB7jtOTobmf9HRNJFJFZEbvIsvgBkikhqL5s/D9wlIis8XWi/ArQCmwdybGMGwhKDCUXxwKNADZ3VJjnAI0PYzw+BVcCbInIZ2Epng/FA/TnwVaCWzkbkwfw4fwpoB44AVXRWjaGqR+hsXC73VHFdU0WmqkeBTwL/Tef5fwD4gKq2DeLYxvTJBrgZY4y5hpUYjDHGXMMniUFEnhCRKhE50MtyEZEfiUipZ+DN7G7L7heR457H/b6IxxhjzND5qsTwJJ0DanqzEpjgeTwI/C90jv4EvkVnve584Fsiku6jmIwxxgyBTxKDqm4E6vpY5R7gl9ppK5DmGeRzB7BGVbv6dK+h7wRjjDHGz3w1wrM/o+nsu92lwvNeb++/h4g8SGdpg2HDhs0ZM2ZMjwdyu91ERQW+6cSl0NCqNLbpezqiJ0RDcpyQGCs9bjsQTp2XP9k5BYYCjW3K5Tal7bpOvTFRkBwrJMUJ0X18PIPxvLwVied07NixGlXN7m8/gUoMXlPVx4DHAObOnas7d+7scb2SkhKWL18eyLj473Wl/HhdKWmqfHZ2HssnZZOZFA/AuiNVvH7gHKdqm7ltxkj+9cM3kJIQO+jjBPq8AsHOyf+OXbjM557ayem6ZpaMTOFTC8YxPms4sdFRnKlr5rkdZ9h2oo60pDh+ct9sigt6vs1EsJ2XL0TiOYnIqV4XdhOoxFBJ5+jMLnme9yqB5de9XxKgmLzW7nLzjRf38/zOCj4wcxRfu3MSeenDr1lnfn4GX71jEj/bWMZ/vHmMPWca+Okn5zB9dG/jl4zxjbWHL/BXz+xmeHwMT3+2mMVFmfxxDj+YNz6DD8/O49DZSzz0m3f5xOPb+MZdU3hg0fhr1jORJ1DlqFXApz29kxYAFz2jP1cDt3tGf6YDt3veC3ot7S4e/OVOnt9ZwV+tmMCP7p31nqTQJTpK+IvlRfz2Cwtxu5X7n9jOyZqmHtc1xhde2FXB5365k/zsRFY9tJglE7J6/bGfOiqFlx5azPJJOfzTy4f44drjAY7WBBtfdVd9BtgCTBKRChH5rIh8QUS+4FnlNaAcKAX+D/gLAM/sj/8M7PA8vuN5L6ipKl///X5KjlXz3Q9N529vmzigK6zZY9P51eeKcaty/y+2U9PYGoBoTaTZebKOR36/j0WFmfz284sYmTqs321SEmJ57FNz+JPZefzXW8d5ee/Zfrcx4csnVUmq+vF+livwl70sewJ4whdxBMrT207z+92VfPnWiXyieFz/G3RTmJ3Ezx+Yx33/t5XPPrmD5z6/kITYaD9FaiJNZcMVvvD0LkanDeMn981mWNzAP1tRUcK/fng6Z+qa+bvf7iUvfRg3jrXe45EovJrkA2D36Xq+8/JBbp6UzZduKRrSPmaPTedH997I3oqL/OeaYz6O0ESq1g4Xn//VTlra3Tx+/1zShsf1v9F14mOi+emn5pCTEs+Dv9pFfZNNwRSJLDEMQlNrBw/9Zje5KQn84GOziIoaegPd7dNGcF/xWB5/u5x3Tw9lWn5jrvXTknIOVF7iP/90JkU5yUPeT0ZiHD/75Fzqm9r47muHfRihCRWWGAbhv9eVUtlwhR/eO2tIV2PXe2TlZEamDuOrv91LS7vLBxGaSFVe3chP1pfygZmjuH3aQG8L0bupo1L4/LICXthVwabSGh9EaEKJJYYBKq26zONvl/PROXnMGfee+8UPSXJCLP/vwzdQVt3Ef71lPUHM0Kgq//DSAeJjo/jH90/pf4MB+tItExifOZyvv7ifNpfNwhxJLDEMgKryrVUHGR4XzddWTvbpvm+amM1H5+Tx+NvlnK5t7n8DY67z4u5KNpfV8rU7J5OTnOCz/SbERvOvH76BU7XNrCpr99l+TfCzxDAAr+0/z6bSWv7ujklkeUY0+9Lf3TGJ6CjhP9cc9fm+TXhraXfx6OtHmDUmjfvmj/X5/hcVZvGhG0ez+mQ7VZdafL5/E5wsMfTD5Va+/+ZRJo9IHnTX1IHKTUngzxbn84e9Zzl09pJfjmHC06+3nabqcisPr5zsVWeIvvzNrRNwK/xkfalf9m+CjyWGfryy7ywnapr4m1snEu2nLx7AF5cVkhwfw/fftFKDGZjmtg7+t6SURYWZLOhljiNfGJeZyNLRMfxm+2kq6q26MxJYYuiDy905Qd7kEcncPjXXr8dKHR7LF5cXse5IFdtPBP3gbxMEnt56iprGNr5820S/H+sDhbEIwo/XWakhElhi6MPrB85RWtXIl26Z4LdiencPLBpPVlK8FdlNv5paO/jphnKWTshi3njf9JLrS+awKO4rHstvd1XYPF8RwBJDL9xu5UdrjzMhJ4mV073vFz4Qw+KieWDRODYcq+bYhcsBOaYJTb/Zdpq6psCUFrr8xc2FREcJj79THrBjGmdYYujF2iNVHLvQyEO3FAWktNDlE8XjSIiN4udvnwjYMU1ocbmVJzefZH5+BrMDOJdRTnICH5w1it/tquRis3VfDWeWGHrx1OaTjEpN4K4bRgb0uOmJcXxkTh4v7q6k+rLNvmrea82hC1Q2XOEzi8cH/Nh/tjifK+0unt1xOuDHNoFjiaEHpVWXeae0hk8sGEdMdOD/iz6zOJ92t5tfbR3QzZZMhHly8wlGpw3j1in+7RDRkykjU1hYkMlTm0/S4XL3v4EJSZYYevCrLaeIi47iY/N6vq+0vxVkJ7Fici5Pbz1lcyiZaxw+d4mt5XV8eqEzFy0An1mSz9mLLaw+eMGR4xv/s8RwncbWDn73biXvnzHSL6OcB+qzS/Kpa2rjtf3nHIvBBJ8nN50kIda5ixaAWybnMC5zOE9ssnawcGWJ4TovvltBY2sHn1403tE4FhRkkJ+VyLM7zjgahwkeDc1tvLSnkg/PzvPJ7L5DFR0lfHrheHadqufwORupH44sMXSjqvxyyylm5qUya0yao7GICH86dwzbT9RRXt3oaCwmOPxhz1laO9x8otj3cyIN1odvHE1cdBS/3VnhdCjGD3x1z+c7ReSoiJSKyMM9LP+BiOzxPI6JSEO3Za5uy1b5Ip6h2n2mgeNVjdwXBF88gD+ZM5roKOG5nVZqMPDbXWeYOjKFaaNSnQ6F9MQ4bpuay4u7K2jrsEbocON1YhCRaOAnwEpgKvBxEZnafR1V/bKqzlLVWcB/A7/vtvhK1zJVvdvbeLzxu10VJMRG8b4Ad1HtTU5yAism5/C7XRV0uG0+/Eh2+NwlDlRe4qNz85wO5aqPzs2jvrmdtYetETrc+KLEMB8oVdVyVW0DngXu6WP9jwPP+OC4PtXS7uLlvWe5Y9oIkhNinQ7nqnvnj6GmsY291dY7KZL9dmcFsdHCPbNGOx3KVUsnZDMiJYHnrUQbdnyRGEYD3T8ZFZ733kNExgH5wLpubyeIyE4R2SoiH/RBPEOy9nAVl1o6+JPZwXNFBnCT58u3oaLD6VCMQ9o63Ly0p5Jbp+SSkehco/P1oqOEj8zJY8Oxas5ftHs1hJOYAB/vXuAFVe1++TtOVStFpABYJyL7VbXs+g1F5EHgQYDc3FxKSkp6PEBjY2Ovy/ry2K4W0uOFjsoDlJwN3BQYAzE3y8Vr5R2sWr2elPjgis0bQ/1bBTN/nNOuCx3UNbUxOb7esf+v3s5rTIcbt8L3X9jI+wuDJ2kNhH3++qCqXj2AhcDqbq8fAR7pZd3dwKI+9vUk8JH+jjlnzhztzfr163td1puqSy1a8Mir+v9eOzzobQPh8LmLOu5rr+hTm084HYpPDeVvFez8cU6ffXKHzvuXNdre4fL5vgeqr/P66P9u1hX/UaJutztwAflAJH7+gJ06gN91X1Ql7QAmiEi+iMTRWSp4T+8iEZkMpANbur2XLiLxnudZwGLgkA9iGpQ/7KnE5VY+Mid46m+7mzwihbwkYdWes06HYgLsYnM7G45VcffMUY6NdO7PB2aNorSqkaM2I3DY8PqTpqodwEPAauAw8LyqHhSR74hI915G9wLPerJWlynAThHZC6wHHlXVgCeGl/ee5YbRqRTlJAf60ANWPDKGnafq7Q5aEWb1ofO0u5QPzBzldCi9Wjl9BNFRwst77cIlXPjkEkRVX1PViapaqKrf9bz3TVVd1W2db6vqw9dtt1lVb1DVmZ5/f+6LeAbjTF0zeysu8v4ZwdFFtTfFIzubg17ea1NkRJJX9p1jbMZwZuQ5P3ahN1lJ8SwqzOTlvee49rrPhKrgLJsG0KueuYiCZexCb3KGRzFrTBqr7KosYtQ1tbGptIa7ZoxEJLg7HXxgxihO1zWzv/Ki06EYH7DEsO8cM8ekMSZjuNOh9OueWaM4fO4Sx60uNyK8ceA8LrcGfWkW4I5pI4iNtuqkcBHRieF0becVzvuDvLTQ5a4ZI4kS7MsXIV7ee5aCrESmjkxxOpR+pQ6PZdnEbF7Zdw63jdIPeRGdGLqqkVbeEJh7OnsrJzmBeeMzeOPgeadDMX5WdbmFbSdqeX8IVCN1+cDMUZy72MKu0/VOh2K8FOGJ4SyzxqSRlx781Uhd7pg2gmMXGm3G1TD3xoHzuBXeH8S9ka63YkoucTFRvHHALlxCXcQmhlO1TRyovBQS9bfd3TG9s3Rjd88Kb28evEBBdiITc4O3C/X1kuJjWFKUxeqD5613UoiL2MSw2lMdc+f00KhG6jI6bRg3jE69Gr8JPxeb29laXsvtU0Prswlw+9RcKuqvcPicdZAIZRGbGNYcusDUkSkhVY3U5c7pI9hzpsEmLgtT649W0eFWbp+W63Qog3br1FxE4M1DduESyiIyMdQ2trLrVD23TQ29Lx7AHZ4fDPvyhac1hy6QnRzPrDxn7yI4FFlJ8cwdl25VnSEuIhPD2iNVuJWQTQxFOckUZidadVIYaml3UXK0itum5hIVFRq9ka53x7QRHD53iTN1Nn1LqIrIxLDm0AVGpSYwbVTw9w/vzR3TRrC1vI76pjanQzE+tKWslqY2F7eH6EUL/PGC681DVmoIVRGXGK60uXj7eLWnLjQ0r8gAbp82Apdb2XCs2ulQjA+9eeg8SfExLCzMdDqUIRuXmcjkEclWog1hEZcY3imtoaXdHbLVSF1mjE4lKymOtUeqnA7F+Ijbraw5VMXySdnEx0Q7HY5Xbp+ay86TddRZiTYkRVxiWHPoPMnxMRTnh+4VGUBUlHDzpBw2HK2iw+V2OhzjA3srGqhpbA35ixaAW6bk4lbYaCXakBRRicHtVtYdqWLZpGziYkL/1G+ZnMOllg52nbIpCMLB+qPVRAksm5jtdChe6yrRrrMSbUgK/V/HQdhfeZGaxjZWTMlxOhSfWDIhi9hosS9fmCg5WsXssemkDQ+teyf3JCpKWDYxhw3HqnHZpHohJ6ISQ8nRakTgpgmhf0UGkJwQy/z8DEsMYaDqcgv7Ki5y8+TwuGiBzhLtxSvt7LZJ9UJORCWG9UermJmXRmZSvNOh+Mwtk3M5XtVofcZD3IajnXXxN08Kn8SwdGIW0VFWog1FEZMYahtb2VvRwPJJ4VFa6LLCc4VpX77Qtv5oFbkp8UwZGTqT5vUnJSGWuePS7bMZgnySGETkThE5KiKlIvJwD8sfEJFqEdnjeXyu27L7ReS453G/L+LpydvHa1ANrysygPFZiRRkJVq31RDW7nLz9rEabp6UE9Jja3pyy+Qcjpy/zNmGK06HYgbB68QgItHAT4CVwFTg4yIytYdVn1PVWZ7H455tM4BvAcXAfOBbIpLubUw9WX+0iszEOG4YHbw3VR+q5ZNy2Fpey5U2l9OhmCHYdaqey60dYdW+0OUWzzmVHLVuq6HEFyWG+UCpqparahvwLHDPALe9A1ijqnWqWg+sAe70QUzXcLmVjceqWTYxO2Tnn+nLsknZtHW42Xai1ulQzBCsP1JFbLSwuCjL6VB8rignibz0YVadFGJifLCP0cCZbq8r6CwBXO9PROQm4BjwZVU908u2o3s6iIg8CDwIkJubS0lJSY/BNDY2vmdZaYOL+uZ2ctw1vW4X7Ho6ry5tLiU2Cn6zbjecC52G9b7OKVQN5ZxefbeZCWnCzi3v+CcoH/DmbzUhqZ13jl3grXXriQmiCzP7/PXOF4lhIF4GnlHVVhH5PPAUcMtgdqCqjwGPAcydO1eXL1/e43olJSVcv+zdNceIkuN84Z6bQraPeE/n1d2Ck9sov9jC8uXLAheUl/o7p1A02HO6cKmFijfW8sjSSSxfVui/wLzkzd+qJes865/eRfL4GRQXBM+MA/b5650vqpIqgTHdXud53rtKVWtVtdXz8nFgzkC39YWNx6qZOSYtZJPCQCybmE1pVSOV1sgXUt4+XgPA0jAZW9OTRUWZREfJ1XM1wc8XiWEHMEFE8kUkDrgXWNV9BRHpfmPlu4HDnuergdtFJN3T6Hy75z2fudjczr6KhrAZ1NabmzzTKNjcNKFl47FqspLCq5vq9VISYrlxTBobj9tnM1R4nRhUtQN4iM4f9MPA86p6UES+IyJ3e1b7KxE5KCJ7gb8CHvBsWwf8M53JZQfwHc97PrO5rAa3wtIJ4dew192EnCRGpCRYYgghbrfyTmkNN03ICrtuqte7aWI2+ysv2myrIcIn4xhU9TVVnaiqhar6Xc9731TVVZ7nj6jqNFWdqao3q+qRbts+oapFnscvfBFPdxuP15AcH8PMMaF3m8TBEBGWTczmndIam201RBw6d4m6pjaWTgzvixbovDBT7Zz23gS/sB75rKq8fbyahYWZxEaH9akCnVdll1s62HOmwelQzAB03WRpSVF4V3MCzMhLI3VYrJVoQ0RY/1qeqm2mov5K2FcjdVlSlEWUWDtDqHj7eDVTR6aQnRw6XYyHKjpKWFKUxdvHq1G12VaDXVgnhrc9jV3h3OOju9ThsdyQl8amMhvoFuyaWjvvoxEJ1UhdbpqYxYVLrRy70Oh0KKYfYZ0YNh6vYUzGMMZlDnc6lIBZUpTJnjMNXG5pdzoU04et5bW0uzTse8t113WB9rb1Tgp6YZsY2l1utpbVsqQoO+x7fHS3uCgLl1vZVu7Tzl3Gx94prSE+Joo54/wyNVhQGpU2jIKsRDZZA3TQC9vEsPdMA5dbOyKmfaHL7LHpJMRGsanMvnzBbHNpLfPzM0iIjXY6lIBaVJTJthN1tFvPuaAWtolhU2ktIrCoMHiG4AdCQmw088Zn2FVZEKu63MLRC5dZVBhZFy3Q2UGiuc1lPeeCXPgmhrIapo9KDetpMHqzpCiLYxcaqbrU4nQopgdbPJ0DFhdF1kULwIKCTESwC5cgF5aJobmtg92n6yOutNCla/pmq04KTptKa0hJiGHaqPC7N0h/0oZ33hPFEkNwC8vEsONkPe0uZVEYzm8/EFNHppA+PJZ3jlu31WCjqmwqrWVRYef9kCPRosIsdp9uoKm1w+lQTC/CMjFsLq0hNlqYNz5yenx0FxUlLCrKYlNpjQ0mCjKn65qpbLgSkdVIXZYUZdHhVrafsJ5zwSosE8OmshpuHJvO8LhA3W4i+CwpyuL8pRbKqpucDsV0s6m0sxQXqaVZgLnj04mLibLqpCAWdomhsU05ePYSiyOwx0d3Xe0rW8qtOimYbCqtYURKAgVZiU6H4piE2Gjmjku3CfWCWNglhiN1LlQjs8dHd2MzhjMqNYEt1gAdNNxuZXNZDYuLwn+a7f4sLsriyPnL1Da29r+yCbiwSwyH6lwMj4sO+2m2+yMiLCzMYmt5HW63tTMEgyPnL1Pf3B6xveW6W+C5xedWG6EflMIvMdS6mJ+fERHTbPdnYWEmdU1tHL1w2elQDH+s1lsU4aVZgBl5qSTGRbOl3Eq0wSisfj0vXGrhfJPaFZnHwq52BpttNShsKatlfOZwRqYOczoUx8VGRzEvP8M+m0EqrBLDVs8V2cKCyG547jI6rXNm2c325XOcy61sO1F7NVkbWFiQSVl1k43QD0I+SQwicqeIHBWRUhF5uIflfysih0Rkn4isFZFx3Za5RGSP57HKmzi2lNUyPAamjkrxZjdhZVFhJttO1OKydgZHHTx7kcstHVfr1k23Eq31nAs6XicGEYkGfgKsBKYCHxeRqdetthuYq6ozgBeAf+u27IqqzvI87vYmli3ltUzKiI7YEaU9WVCQyeWWDg6eveh0KBGtq8pkoSWGq6aNSiU5Icaqk4KQL0oM84FSVS1X1TbgWeCe7iuo6npVbfa83Ark+eC41zjbcIVTtc1MzoisaYz70/VDZNVJztpSXkthdiI5KQlOhxI0oqOE4vxMKzEEIV8MDR4NnOn2ugIo7mP9zwKvd3udICI7gQ7gUVV9qaeNRORB4EGA3NxcSkpKrlm+qbLzjmXjElrfsywcNDY2Dvm8RiUKr+w4zmQ90//KAeTNOQWrns6pw61sLW1m0aiYkD1ff/2tsrWdt2rb+N3r68gcFtgmz0j5/A1FQOeMEJFPAnOBZd3eHqeqlSJSAKwTkf2qWnb9tqr6GPAYwNy5c3X58uXXLH/1t3tJHXaBibmxXL8sHJSUlAz5vFY0HOB371aweOlNQdWN15tzClY9ndO7p+tpeXMzf7J0BstnjHQmMC/562+Vc/YSzxx5G3ImsnyOzysS+hQpn7+h8MWvRCUwptvrPM971xCRW4FvAHer6tXhjqpa6fm3HCgBbhxKEFvKaynOzyAqwkeU9mRBQSbNbS72V1o7gxO66tAXFGQ4HEnwmTwimfThsVadFGR8kRh2ABNEJF9E4oB7gWt6F4nIjcDP6EwKVd3eTxeReM/zLGAxcGiwAZypa6ai/op1BexFsecHaat9+RyxtbyWSbnJZCbFOx1K0InytDPYZzO4eJ0YVLUDeAhYDRwGnlfVgyLyHRHp6mX070AS8NvruqVOAXaKyF5gPZ1tDINODFfHL1hi6FFWUjwTcpLYZtMPBFy7y83Ok/VWWujDgoIMKuqvUFHf3P/KJiB80sagqq8Br1333je7Pb+1l+02Azd4e/yt5XWkD49lYk4y5494u7fwtKAgk9+/W0G7yx1U7Qzhbl/FRa60u2z8Qh+KPf8328rryJsz3OFoDITJyOet5bUU52cSZeMXerWgIJOmNhcHrJ0hoLpKs/PzrcTQm0m5yaQNj7XqpCAS8onhjOeOWFZU79sf2xmsOimQtpbXMjE3ydoX+tDZzpDB1hOWGIJFyCeGbZ7bAxZbUb1PXe0MdlUWOO0uN7tO1Vs10gAU52dypu4KlQ1XnA7FEA6JobyWtOGxTMpNdjqUoLegIJOdJ+tod7mdDiUi7K+8SHObi+J8Swz9WXC1ncEuXIJByCeGrSdqmT8+w9oXBsDaGQKrqxdYsVVz9mvyiGRSh1k7Q7AI6cRQ2XCFM3VXrKg+QNbOEFhby2uZkJNElrUv9CsqSpifn3G1atg4K6QTQ1ex067IBiYrKZ6inCS2WSOf33W43Ow8WWefzUFYUJDJqdpmzlo7g+NCPDHUkToslikj7P4LA1Wcn8HOk/V0WDuDXx04e4mmNhu/MBjFni69duHivJBODFtP1DLP2hcGpbggk8bWDg6du+R0KGHNxi8M3pSRKSQnxNgI/SAQsonh3MXO+y/Y+IXBWdB1VWZfPr/aVl5LQXYiOcl2/4WBio4S5o+3doZgELKJYbvnw2NF9cHJSUkgPyvRiut+5HIrO0/WWzfVISguyOBEjd0H2mkhmxi2lteRnBDDlJHWvjBYxfkZbD9RZ/eB9pNDZy9xubXDSrND0JVMt1qpwVEhmxi2edoX7P7Og1dckMGllg6OnLd2Bn/oKo1ZiWHwpo1KISk+xga6OSwkE0OHWymvbrrai8EMztWrMmtn8IttJ+oYlzmcEanWvjBYMdFRzBmXbu0MDgvJxNDU2gFYj48GcN8TAAAgAElEQVShGpU2jDEZw+yqzA/cquw4WWcXLV4oLsigtKqRmsbW/lc2fhGyiWF4XDTTR6c6HUrIKs7PZPvJOtzWzuBTlY1KQ3O7VSN5oev/bruVGhwTkomhsbWDOePS7YYzXijOz6ChuZ1jVZedDiWsHKlzATYa3xsz8lIZFhttJVoHheQva2uH27qpeqnr/8+uynzraJ2L0WnDyEu3O5ENVay1MzjOJ4lBRO4UkaMiUioiD/ewPF5EnvMs3yYi47ste8Tz/lERuWOgx7Q6XO/kpQ9jVGqCDXTzIVXlaL3LSgs+UJyfwZHzl2lobnM6lIjkdWIQkWjgJ8BKYCrwcRGZet1qnwXqVbUI+AHwPc+2U4F7gWnAncD/ePbXzzFhRl6at6FHNJGu2SxrUbV2Bl8orWrkcptdtPhCsZVoHeWLEsN8oFRVy1W1DXgWuOe6de4BnvI8fwFYISLief9ZVW1V1RNAqWd/fUqMiyEuJiRrwYJKcUEmNY1tlFU3OR1KWLh6N0FrePbazDGpxMVEWXWSQ2J8sI/RwJluryuA4t7WUdUOEbkIZHre33rdtqN7OoiIPAg8CJCaO4aSkpIeg2lsbOx1WSjzx3lJU+cMq0+v3sLyMbE+3fdAhNvf6uU9LaTGKSf2b+ekhNfASyf+VvnJ8Na+UyxNqvLL/sPt8we+OydfJIaAUNXHgMcA5s6dq8uXL+9xvZKSEnpbFsr8cV6qyn/sWUt9bCbLl9/o030PRDj9rVSVv9+0limZcPPNNzsdjs858bd6t/0YP153nNkLFpOS4PsLl3D6/HXx1Tn5oj6mEhjT7XWe570e1xGRGCAVqB3gtsZPrrYzlNdZO4OXTtY2U3W5lUnp/TaRmQFakJ+BW2HXyXqnQ4k4vkgMO4AJIpIvInF0Niavum6dVcD9nucfAdZp5y/RKuBeT6+lfGACsN0HMZkBWpCfwflLLZyua3Y6lJDW1ed+UoYlBl+5cWw6sdHCVpsJOOC8rkrytBk8BKwGooEnVPWgiHwH2Kmqq4CfA78SkVKgjs7kgWe954FDQAfwl6rq8jYmM3BdvT865/dJdDia0LX9RB1ZSXGMTAyvtgUnDYuLZkZemnWpdoBP2hhU9TXgteve+2a35y3AR3vZ9rvAd30Rhxm8CTlJZCTGsa28jj+dO6b/DUyPtp2oY35+BiI2ktyXivMz+NnGcppaO0iMD5km0ZBnfT4jnEjXXbOsuD5UZ+qaqWy4Yt1U/aC4IBOXW3n3tLUzDNUvt5zkneM1g9rGEoOhuCCDivorVDZccTqUkHR1/IKNePa5OePSiY4Sq04aonaXm0dfP8Kbh84PajtLDObqla5NWjY028prSRsey8ScZKdDCTtJ8TFMH51qJdoh2l95keY216BLs5YYDJNHJJM6LNauyoZo24k65o3PIMruJugXC/Iz2HvmIlfarF/KYHV9pwd77xpLDIaoKGGetTMMybmLVzhd12yz/fpRcUEGbS43u62dYdC2nailMDuR7OT4QW1nicEAsKAgg5O1zZy/2OJ0KCGl64rMJs7zn7njM4gS2GrzJg1Kh8vNzpP1Q7poscRggG7tDFZqGJSt5bWkJMQwZWSK06GErZSEWKaNSrU2sEE6dO4Sja0dV8cqDYYlBgPA1FEpJMfHsNXaGQala/xCtLUv+FVxfga7zzTQ0m7tDAO11ZNIFwyhNGuJwQAQHSXMHZ9uJYZBuHCphRM1TTZ+IQAWFGTS1uFmz5kGp0MJGdvK68jPSiQnJWHQ21piMFcVF2RSXt1E1WVrZxiIq1dk1vDsd/PyMxDBes4NkMutbD9ZN+S2L0sM5iq7D/TgbDtRR3J8DFNHWfuCv6UOi2XKiBQr0Q7Q4XOXuNzSMeSLFksM5qrpo1JIjIu+eiVs+ratvJa549OtfSFAFhRk8u7pelo7rJ2hP96OxrfEYK6KiY5i7vgMa4AegKrLLZRVN1k1UgAVF2TQ0u5mX8VFp0MJelvLaxmbMZyRqcOGtL0lBnONhYWZlFY1Un251elQgtr2q1dklhgCpfhqO4OVaPvidivbT9Sx0IvPpiUGc40FBTaeYSC2ltd2zuNj7QsBkzY8jskjUthiiaFPh85d4uKVdhYUDn3QpSUGcw1rZxiYLWW1zBufTky0fYUCaUFBBrtOWTtDX3zRW84+1eYaMdFRzMu3doa+WPuCcxYWZNLS7mbvGWtn6M3W8jrGZw69fQEsMZgeLCiwdoa+dCXNhYWWGAJtvqedwUq0PXOrsv1ErdcXLV4lBhHJEJE1InLc8296D+vMEpEtInJQRPaJyMe6LXtSRE6IyB7PY5Y38Rjf6Gq0si9fz7aW13aOX7D5kQIubXgcU0ak2GezF6cvubnkxfiFLt6WGB4G1qrqBGCt5/X1moFPq+o04E7gv0Qkrdvyr6rqLM9jj5fxGB+YNiqFpPgY+/L1YmtZLfPzM6x9wSELCzOtnaEXR+rcgPej8b39ZN8DPOV5/hTwwetXUNVjqnrc8/wsUAVke3lc40cx0VHMG59uiaEHFy61UF5j7QtOWlCQSWuHmz2nbd6k6x2pc5GflciI1MHPj9RdjJdx5KrqOc/z80BuXyuLyHwgDijr9vZ3ReSbeEocqtpjxbaIPAg8CJCbm0tJSUmPx2hsbOx1WSgL9Hll08b66nZeemMdaQn+uTIOxb/VlrMdAMTWn6Ck5PR7lofiOQ1EMJ1XW7siwDPrdnHldNyQ9xNM5+QLblWO1nUwf2Sr9+elqn0+gLeAAz087gEarlu3vo/9jASOAguue0+AeDpLHN/sLx5VZc6cOdqb9evX97oslAX6vPaeqddxX3tFX9pd4bdjhOLf6uHf7dXp33pDO1zuHpeH4jkNRLCd110/2qgf+9lmr/YRbOfkrX1nGvr9zgI7dQC/sf1eCqrqrao6vYfHH4ALIjISwPNvVU/7EJEU4FXgG6q6tdu+z3nibQV+AcwfXFoz/jJtVCrJCTFsKbPqpO62lNVSbPdfcNyC/EzePW33Z+huc1kNgFcjnrt4W0ewCrjf8/x+4A/XryAiccCLwC9V9YXrlnUlFaGzfeKAl/EYH4mOEhYUZNoo027OXbzCyVq7v3MwWFTUeX+Gd0/ZfaC7bC6rZVSiDOn+C9fzNjE8CtwmIseBWz2vEZG5IvK4Z50/BW4CHuihW+qvRWQ/sB/IAv7Fy3iMDy0qzORUbTMV9c1OhxIUNpd2JslFhVkOR2Lmje8stW22Ei0AbR1udpysY0pmtE/251Xjs6rWAit6eH8n8DnP86eBp3vZ/hZvjm/8q+sHcEtZLR+dO9zhaJy3qayGjMQ4Jo9IdjqUiJecEMuMvFRP9ckkp8Nx3L6KBprbXEzJiPfJ/qwjtunVxNwkMhPjrJ2Bzk4aW8pqWViQSZS1LwSFxYVZ7K24yOWWdqdDcdzmslpEYHKGb0oMlhhMr0SEBYWZbC6r7epFFrFO1DRx7mKLTYMRRBYVZuJyKztO2rxem8tqmDoyhaQ431y0WGIwfVpUmMl5z03vI1lXXfbiImtfCBazx6UTFxN1te0nUrW0u3j3VAOLfHjRYonB9KmrnSHSG/k2l9UwMjWB8ZnW1hIsEmKjmTM2nU0R/tncdaqeNpfbp50iLDGYPnVO35sQ0e0MbrenfaEwk86e1SZYLCrM5PC5S9Q1tTkdimM2l9UQHSXMyx/6jXmuZ4nB9ElEWFiYyeayGtzuyGxnOHL+MvXN7Sy2bqpBZ5Gnai+S5/XaXFbLzLxUkuK9neHojywxmH4tKcqivrmdQ+cuOR2KI7pGlC4qsobnYDMjL5XEuGg2ldY4HYojLrW0s/dMg8/H1lhiMP1a4rkqeydCv3ybSmvIz0r06o5Yxj9io6MoLsiM2MSwpawWt8KSCZYYTIDlpCQwMTcpIr98bR1utp2oY7GVFoLWkqIsTtY2c6Yu8kbobyqtYXhcNLPHvuceaV6xxGAGZHFRFttP1EXcpGXvnq6nuc3F0gl2C5FgtXRC5JZo3zleQ3F+BnExvv0pt8RgBmTphCxaO9zsirBJy9453tnjwwa2Ba+inCRyU+IjLjFUNlyhvKbJL2NrLDGYAZmfn0lMlETcl+/t0hpm5qWSkhDrdCimFyLC4qIsNpdGVs+5Tcc7v4v+KM1aYjADkhQfw+yx6RHVznCxuZ39FQ0ssWqkoLd0QmfPuYNnI6fn3NulNWQnxzMxN8nn+7bEYAZscVEW+ysvUh8hg4k2l9Xg1j/WYZvg1VWd8nZptcORBIbbrWwurWFJUZZfBl1aYjADtmRCJqqRMz3G26U1JMXHMGtMmtOhmH7kJCcweUQy7xyPjBLt4fOXqG1qu9qV3NcsMZgBm5mXRnJ8DO9EyFXZO8drWFCQQWy0fU1CwZKiLHaerOdKW/j3nOtKgL4ev9DFPvFmwGKio1hclMWGo9VhPw33qdomTtc1WzfVELJkQhZtLjfbI2Aa7o3Hq5mYm0SuD27j2RNLDGZQlk3K5uzFFkqrGp0Oxa/e9vMVmfG94vxM4mKi2HgsvEu0Ta0d7DhRz/JJOX47hleJQUQyRGSNiBz3/Nvj8DsRcXW73/Oqbu/ni8g2ESkVkedEJM6beIz/3TSx8wp6Q5h/+UqOVpOXPoyCrESnQzEDNCwumuL8DEqOVjkdil9tLa+lzeVm2UT/lWa9LTE8DKxV1QnAWs/rnlxR1Vmex93d3v8e8ANVLQLqgc96GY/xs9Fpw5iQkxTWiaG1w8XmshqWTcy2abZDzPJJOZRVN4X19BgbjlUzLDaaueN9Ow1Gd94mhnuApzzPnwI+ONANpfMbdwvwwlC2N85ZNjGbbSfqwraRb9fJzmkw/FlUN/6xLAJKtBuOVbOoMJP4GN/c37kn4k0joog0qGqa57kA9V2vr1uvA9gDdACPqupLIpIFbPWUFhCRMcDrqjq9l2M9CDwIkJubO+fZZ5/tMabGxkaSknw/4MNpwXReB2pcfH9nC387J54Z2UOfAz6Yzqm7Z4+08dapdn68YjgJMYMrMQTrOXkrVM5LVfnqxiuMSY7ir2f33TAbKufU3YUmN197+wqfmhrHirHvHY3f3zndfPPNu1R1br8HUtU+H8BbwIEeHvcADdetW9/LPkZ7/i0ATgKFQBZQ2m2dMcCB/uJRVebMmaO9Wb9+fa/LQlkwndeVtg6d9A+v6bf+cMCr/QTTOXV323+W6H3/t2VI2wbrOXkrlM7rGy/u06n/+Lq2trv6XC+UzqnLk5tO6LivvaKnapp6XN7fOQE7dQC/sf1WJanqrao6vYfHH4ALIjISwPNvj60+qlrp+bccKAFuBGqBNBHpuuTMAyr7zWTGcQmx0SwoyGTj8fArrp9tuMKxC40sn2jVSKFq2cQcmtpc7DwVft1WNxyrJj8rkbF+vve4t20Mq4D7Pc/vB/5w/Qoiki4i8Z7nWcBi4JAne60HPtLX9iY4LZuYTXl1E6drw6uRr+RoZ7JbPsnGL4SqRYWZxEYLG46G14VLS/sfO0X4m7eJ4VHgNhE5DtzqeY2IzBWRxz3rTAF2isheOhPBo6p6yLPsa8DfikgpkAn83Mt4TIDcMrnzinrdkQsOR+JbJUerGJ02jKKc0Kp7Nn+UGB/DvPEZYdcAve1EHS3t/u2m2sWru0erai2woof3dwKf8zzfDNzQy/blwHxvYjDOGJeZSGF2ImuPVPHA4nynw/GJtg43m8tquXvWKOumGuJunpTDd187TEV9M3np/q12CZR1hy+QEBsVkHuD2MhnM2S3Tslla3ktja0dTofiE9tP1NHY2sHN1k015N0ypatEGx6D3VSVtw5XsaQom4RY/3VT7WKJwQzZLZNzaHcpb4dJkf2twxeIj4ny24yVJnAKs5MoyErkrcPhkRiOXrhMZcMVbp0SmIsWSwxmyOaMSyd1WCxrw+CqTFVZc+gCSydkMSzO/1dkxv9WTMlha1l4lGjXehJcV9uev1liMEMWEx3F8knZrD9ShSvEb6n4xyuyXKdDMT5y65Rc2lzusCjRvnX4AjPzUsnx02yq17PEYLyyYkoutU1t7K1ocDoUr7x1qLN31S0BKqob/+sq0YZ6dVJNYyt7zjSwIoAXLZYYjFeWTcgmOkpYezi0u62uOVzFrDFp5CQH5orM+F9MdBQ3T8pm/dHQLtGuO1KFamfVWKBYYjBeSR0ey9xx6bx1KHSvyqoutbD3TAO3TbVqpHBz69Rc6pra2H263ulQhmzt4QuMTE1g6siUgB3TEoPx2h3TRnD0wmVO1DQ5HcqQdDWeW/tC+LlpYjYxUcKaEC3RXmlzsfFYDSum5AR0bI0lBuO1O6ePAOCNA+cdjmRo1hy6QF76MCbm2mjncJOSEMvCwkxWHzgfkrej3XCsmivtLlZOHxnQ41piMF4blTaMmXmpvHHgnNOhDNqllnbeOV7DndNG2GjnMLVy+khO1jZz5Pxlp0MZtDcOnCN9eCzF+RkBPa4lBuMTd04fyd6Ki1Q2XHE6lEFZe/gCbS43K28I7BWZCZzbp+USJfD6/tC6cGntcLH2cBW3Tc0lJjqwP9WWGIxPdFUnrQ6x6qRX951nZGoCN455z/2lTJjISoqnOD+T10Lss7mptIbLrR0Br0YCSwzGR/KzEpk8Ijmk2hkut7Sz8Xg1d04fQVSUVSOFs/fdMILSqkaOXwid6qTX958nOT6GRUX+nzTvepYYjM/cOX0EO07VUXW5xelQBmTdkSraOtzcZdVIYe+OaSMQgdf2h8aFS7vLzZrDF1gxJcev93bujSUG4zN3Th+BKqw+GBpdA1/bf47clHhmj013OhTjZzkpCcwbl8HrIdJBYlt5HQ3N7dzpQDUSWGIwPjQpN5nC7ERe3nPW6VD61dTaQcnRalZOH2nVSBFi5Q0jOHL+MmXVjU6H0q9X959leFx0QG7K0xNLDMZnRIQPzhrN9pN1Qd87ae2RKlo73Kz0NJqb8Ldy+khE4OW9wX3h0trh4tV957hj2gjHZvq1xGB86p5ZowFYFeSlhpd2VzIyNYG54wPbP9w4Z0RqAgsLMnlpd2VQD3Zbf6SaSy0d3DNrlGMxeJUYRCRDRNaIyHHPv++prBWRm0VkT7dHi4h80LPsSRE50W3ZLG/iMc4bmzmcG8em8Yc9lU6H0qvqy61sOFbNPbNGE23VSBHlgzeO5mRtM7vPBO9swC/triQrKc7RG0Z5W2J4GFirqhOAtZ7X11DV9ao6S1VnAbcAzcCb3Vb5atdyVd3jZTwmCHxw1miOnL/MkfOXnA6lRy/vPYvLrXx49minQzEBtnL6COJjonjx3eC8cLl4pZ11R6r4wMxRAR/U1p23R74HeMrz/Cngg/2s/xHgdVVt9vK4JojdNWMk0VHCS7uDszrpxd2VTB+dwsTcZKdDMQGWnBDLbVNzeWXfWTqCcCru1/efo83l5kM3OnvR4m1iyFXVrv5f54H+pqe8F3jmuve+KyL7ROQHIhLvZTwmCGQlxbN0Qhar9lTiDrIvX2nVZfZXXuRDN+Y5HYpxyIduHE19czv7a1xOh/IeL+6upCA7kRtGpzoah/TXCCMibwE9dd34BvCUqqZ1W7deVXvsFC4iI4F9wChVbe/23nkgDngMKFPV7/Sy/YPAgwC5ublznn322R7jbWxsJCkp/GbJDLXz2nK2g5/ta+Xv5yUwNbPnnhVOnNMLx9p47UQ7P1g+nNR437cvhNrfaaDC6bw63MqX1zdTlKr89dzgOafaK26+suEKHyqK5Z6iuCHto7+/080337xLVef2uyNVHfIDOAqM9DwfCRztY92/Bh7rY/ly4JWBHHfOnDnam/Xr1/e6LJSF2nldaevQGd9erX/56129rhPoc3K53LrwX9/S+5/Y5rdjhNrfaaDC7by++dJ+LXzkFa1rbHU6lKv+482jOv7hV/R0bdOQ99Hf3wnYqQP4jfW2KmkVcL/n+f3AH/pY9+NcV43kKTEgnfMdfxA44GU8JkgkxEbz4dmjWX3wPLWNrU6HA8CG49WcvdjCn8y2aqRI97F5Y+lww+/erXA6FAA6XG6e23GamyZkMyZjuNPheJ0YHgVuE5HjwK2e14jIXBF5vGslERkPjAE2XLf9r0VkP7AfyAL+xct4TBD5+PyxtLs0aL58T285RVZSPHdMs0FtkW7qqBSK0qL4zbbTQTGmYd2RKi5cauW+4rFOhwJ4mRhUtVZVV6jqBFW9VVXrPO/vVNXPdVvvpKqOVlX3ddvfoqo3qOp0Vf2kqgb/WHUzYBNzk5k7Lp1ntp9x/Mt3pq6ZdUeruHfeGOJibFyngZvHxFBe08SWslqnQ+E320+TmxLPisk5TocC2Mhn42cfnz+WEzVNbC2vczSOZ7afRoCPB8kVmXHevBExpA2P5dfbTjsax5m6ZjYcq+Zjc8c4Onahu+CIwoStu2aMJCUhht9sd+7L19rh4vmdZ7hlci6j04Y5FocJLnHRwkdm57H64HlHp4p/bscZBPjY/OC5aLHEYPwqITaaj84dw+v7zzk2sd4bB85T09jGpxaOc+T4JnjdVzyWDrfy/I4zjhy/pd3FszvOsHxSTlBdtFhiMH73mSX5KPDEOycCfmxV5cnNJxmXOZylDs49Y4JTQXYSSydk8dSWU7S0B37A2+/fraSmsZXPLckP+LH7YonB+N3otGHcPXMUz2w/zcXm9oAee0t5LbtPN/C5pQV23wXToy8uK6T6cisv7Aps7zmXW3lsYxkz8lJZWBj423f2xRKDCYg/X1pAc5uLp7edCuhx/2d9GdnJ8Xx0jo1dMD1bWJjJzDFp/GxjGR0ud/8b+MgbB85zsraZLy4rpHMoV/CwxGACYuqoFG6amM0vNp0MWJF9z5kG3imt4c+X5pMQ68wNT0zwExH+YnkhZ+qu8Or+wNz6U1X56YYy8rMSuT0Ix9VYYjAB84WbCqhpDFyR/SfrS0kdFst9xdbobPp225RcJuQk8b8lZQEZc7O5rJb9lRd58KaCoLwniCUGEzALCzOZMy6dH649TlNrh1+PdeT8JdYcusADi8aTFB/j12OZ0BcVJXxhWSFHzl9m9cELfj2WqvL9N4+SmxLv+PTavbHEYAJGRPj6+yZTfbmVx9/2Xw8lVeVfXztCckIMf7Z4vN+OY8LLPbNGMSEniUdfP0xbh//aGl7Zd47dpxv4yu2TgraK0xKDCag54zJYOX0EP9tYRkOrf758649WsfFYNX+9YgJpw4c2fbGJPDHRUXzjrimcrG3ml1tO+uUYLe0uvvfGEaaMTAnqyRwtMZiA+/s7J9PW4ealUt93XW3rcPPPrxymMDuR+xeN9/n+TXhbPimH5ZOy+eHa436ZFfipzSepqL/CP9w1JSjbFrpYYjABl5+VyCeKx7KxooMDlRd9uu8nN5/gRE0T//j+qcQGybwzJrT8w11TaG5z8YO3jvl0v9WXW/nxulJumZzD4iAfbGnfHOOIL982kZQ44cvP7fFZ99Uzdc38aG3nF2/5pOCYpdKEnqKcZD61YBy/2XaabeW+mXlVVfna7/bR6nLz9fdN8ck+/ckSg3FE2vA4Pjs9juNVjfz76qNe76/d5eZLz+xGgH+6e5r3AZqI9nd3TGJcZiJ/89weGprbvN7f09tOs+5IFY+snExRTvDcTrQ3lhiMY27IjuFTC8bx83dOsLmsxqt9fX/1UfacaeB7H5kRFHfAMqEtKT6GH917IzWNrXztd/u8GttQWtXId189xE0Ts7l/4XjfBelHlhiMox5532TysxL5q2f2cKKmaUj7WHfkAj/bWM4nF4zlfTeM9HGEJlLdkJfKV++YxOqDF3hq88kh7eNiczsP/eZdhsVG8/2PzAiZ+bosMRhHDY+L4f8+PRe3Kp98fNugp+Z+53gNf/Hrd5kyMoV/uGuqn6I0kepzSwq4dUoO3375EM8O8p4il1ra+fQT2yivbuJHH7+RnJQEP0Xpe5YYjOOKcpL45Wfmc6mlnU8+vo3zFwd205S1hy/wmad2MD4zkV9+Zn7QDhYyoSsqSvjxfbNZNjGbh3+/n98M8G5vja0dPPDEdg6evcT/fGI2Sydk+zlS3/IqMYjIR0XkoIi4RWRuH+vdKSJHRaRURB7u9n6+iGzzvP+ciNhopAg1fXQqT/7ZPC5cauH2H2zgd7sqeq3XvdLm4sfrjvP5X+1i8ohknvnzBWQnxwc4YhMpEmKj+dmn5nDzpGy+/uJ+vv7ifuqbem+Q3nismvf98G32Vlzkx/fdyK1TcwMYrW94O4nMAeDDwM96W0FEooGfALcBFcAOEVmlqoeA7wE/UNVnReSnwGeB//UyJhOi5ozL4JUvLeHvX9jHV367l5f2VPKBmaNYWJBJZlIc5dVN7DnTwE/Wl3LuYgt3ThvBv310BikJsU6HbsJcQmw0P/3UHL73+lGe2nKS1/af4wvLCpk3PoPJI5K53NLBu6freXX/OV7dd46CrER+/bliFhQE130WBsqrxKCqh4H+5hKfD5Sqarln3WeBe0TkMHALcJ9nvaeAb2OJIaIVZCfx3OcX8tTmk/xPSSlvH39vb6UZean818dmURyiXzoTmuJjovnmB6bysXlj+Paqgzz6+pH3rJMQG8VfrZjAXywvDOmqzUBMOzka6H5D1QqgGMgEGlS1o9v7vU41KCIPAg96XjaKSG+d37MA7/o+BqdwPK8hndMp4OUv+T4YHwnHvxOE53n55Zy+4nk4pL9zGtAc9P0mBhF5C+jpThLfUNU/DOQgvqCqjwGP9beeiOxU1V7bO0JVOJ6XnVPoCMfzsnPqXb+JQVVv9fIYlcCYbq/zPO/VAmkiEuMpNXS9b4wxxkGB6K66A5jg6YEUB9wLrNLOLifrgY941rsfCFgJxBhjTM+87a76IRGpABYCr4rIas/7o0TkNQBPaeAhYDVwGHheVQ96dvE14G9FpJTONoefexOPR7/VTSEqHM/Lzil0hON52Tn1QgJxf1NjjDGhwzbdtQcAAAPXSURBVEY+G2OMuYYlBmOMMdcI28QgIl8SkSOeKTv+zel4fEVEviIiKiLBfQuoARKRf/f8nfaJyIsikuZ0TEPV29QvoUpExojIehE55Pke/bXTMfmKiESLyG4RecXpWHxFRNJE5AXP9+mwiCwc6r7CMjGIyM3APcBMVZ0GfN/hkHxCRMYAtwODm+YxuK0BpqvqDOAY8IjD8QxJt6lfVgJTgY+LSKhP99oBfEVVpwILgL8Mg3Pq8td0doYJJz8E3lDVycBMvDi/sEwMwBeBR1W1FUBVqxyOx1d+APw9EDY9BlT1zW6j37fSOZ4lFF2d+kVV24Bn6bw4CVmqek5V3/U8v0znD02vsxOEChHJA+4CHnc6Fl8RkVTgJjw9O1W1TVUbhrq/cE0ME4GlnplbN4jIPKcD8paI3ANUqupep2Pxo88ArzsdxBD1NPVLyP+IdhGR8cCNwDZnI/GJ/6LzAsvtdCA+lA9UA7/wVJE9LiKJQ91ZIOZK8ou+puqg87wy6Cz+zgOeF5ECDfK+uf2c09fprEYKOQOZVkVEvkFn1cWvAxmb6Z+IJAG/A/5GVS85HY83ROT9QJWq7hKR5U7H40MxwGzgS6q6TUR+CDwM/ONQdxaS+pqqQ0S+CPzekwi2i4ibzsmlqgMV31D0dk4icgOdVwR7PTPZ5gHvish8VT0fwBCHpL9pVUTkAeD9wIpgT9596G3ql5AmIrF0JoVfq+rvnY7HBxYDd4vI+4AEIEVEnlbVTzocl7cqgApV7SrRvUBnYhiScK1Kegm4GUBEJgJxhPDMkKq6X1VzVHW8qo6n80MwOxSSQn9E5E46i/V3q2qz0/F4ocepXxyOySvSeRXyc+Cwqv6n0/H4gqo+oqp5nu/RvcC6MEgKeH4LzojIJM9bK4BDQ91fyJYY+vEE8ISIHADagPtD+Eo03P0YiAfWeEpDW1X1C86GNHiq2iEiXVO/RANPdJv6JVQtBj4F7BeRPZ73vq6qrzkYk+ndl4Bfey5MyoE/G+qObEoMY4wx1wjXqiRjjDFDZInBGGPMNSwxGGOMuYYlBmOMMdewxGCMMeYalhiMMcZcwxKDMcaYa1hiMMYHROQLIrLH8zghIuudjsmYobIBbsb4kGduoXXAv6nqy07HY8xQWInBGN/6IZ3z71hSMCErXOdKMibgPLPEjgMecjgUY7xiVUnG+ICIzAGeApaqar3T8RjjDatKMsY3HqLz5lDrPQ3QYXPbSBN5rMRgjDHmGlZiMMYYcw1LDMYYY65hicEYY8w1LDEYY4y5hiUGY4wx17DEYIwx5hqWGIwxxlzj/wPrOOPwGlxvcwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW9//HXh10IO2HfhbAJVUFwuxVckargVtFWRaVoW27rvW0F61r1urW11avW4lZtrUoBERFlkcR9AwsJOxGQfZcl7Ek+vz/mcH9pmpCQOTNzQt7Px2MeOWfOd873Mycw7znbN+buiIiIVEt1ASIiEg0KBBERARQIIiISUCCIiAigQBARkYACQUREAAWCVGJmlmdmnYPp48zsLTPbaWb/MLMfmNmMCq53hJl9VMqyjmbmZlYjmH/HzK6v+LuIhmPlfUh8aqS6ADm2mNkqoAVQAOQB7wKj3T0vzvVmAX9z9+cOP+fuaUWaXBH029Td84PnXomnz/Jw9wsT3UcyFH0fZjYCGOnuZ6auIkkF7SFIIlwcfFifCJwE3J6EPjsAy4qEwTHr8N6JSNgUCJIw7r4RmE4sGAAws9pm9jszW21mm8zsGTM7rsjyoWY2z8x2mdnXZjbYzP4H+A/gyeAw0ZNBWzezLmb2G+Bu4Kpg+U3FD/uYWXczm2lm281sqZl9v8iypmY2JejzC+D48r5HM8sys5HB9Agz+yh4f9+a2UozK/rNu6GZPW9mG8xsnZk9YGbVg2XHm9lsM9tmZlvN7BUza1TktavMbIyZZQN7iodC8UNZFagty8xGmlkP4BngtGBb7ijvtpDKT4EgCWNmbYELgdwiTz8MZBALiS5AG2If5phZf+Bl4FdAI+C7wCp3vwP4kNihpzR3H120H3e/B3gQeD1Y/nyxOuoBM4G/A82B4cDTZtYzaPIUsB9oBdwYPCpqALAUaAY8CjxvZhYs+wuQH7zvk4DzgZGHywQeAloDPYB2wL3F1n018D2gUQX3hI5UGwDuvhi4Bfg02JaN/n01cqxSIEgiTDaz3cAaYDNwD0Dw4TMK+C933+7uu4l9kA8PXncT8IK7z3T3Qndf5+5LQqjnImLB8qK757v7P4GJwJXBN/TLgbvdfY+7LwBeiqOvb9z9WXcvCNbTCmhhZi2AIcCtQT+bgT8QvHd3zw3e9wF33wI8BpxVbN1PuPsad98XZm0VXJccg3QsUhJhmLvPMrOziH0rbwbsANKBusDcIl9MDageTLcDpiWgng7AgGKHP2oAfw1qqkEsvA77Jo6+Nh6ecPe9wftMA5oANYENRd57tcP9BoHxOLFDY/WDZd8WW/ca4lNabSKAAkESyN3fN7O/AL8DhgFbgX1AL3dfV8JL1lD68ft4huVdA7zv7ucVXxDsIeQTC6PDeyPt4+jrSDUcAJqVcrjnQWLvsbe7bzezYcCTxdocaRvsCX7WBXYF0y0rWKuGQK6idMhIEu2PwHlm9h13LwSeBf5gZs0BzKyNmV0QtH0euMHMzjGzasGy7sGyTUDnCtYwFcgws2vNrGbwOMXMegSHTyYB95pZ3eC8QujX47v7BmAG8HszaxC8v+ODvSiI7RXkATvNrA2x8yhHs/4twDrgh2ZW3cxu5ChOjhezCWhrZrUq+HqppBQIklDBB9XLBCeOgTHETjJ/Zma7gFlAt6DtF8ANxI6t7wTeJ3a4B2KHU64IrpB54ihr2E3sBO5wYD2xQyePALWDJqOJHTrZSOzE74tH+z7L6TqgFrCI2OGgCcSO4wP8BjiZ2Pt+m1hIHa0fEQuSbUAv4JMK1jkbWAhsNLOtFVyHVEKmP5AjIiKgPQQREQnEHQhm1s7MMs1skZktNLOfl9DGzOwJM8s1s2wzOznefkVEJFxhXGWUD/zC3b8ys/rELimc6e6LirS5EOgaPAYAfwp+iohIRMS9h+DuG9z9q2B6N7CY2N2nRQ0FXvaYz4BGZtYKERGJjFDvQzCzjsRuyf+82KI2/OtNNWuD5zaUsI5RxO5mpU6dOn3bt0/EJeHhKSwspFq16J+KUZ3hUp3hUp3hWbZs2VZ3T6/Qi909lAexy/bmApeVsGwqcGaR+feAfmWtMyMjw6MuMzMz1SWUi+oMl+oMl+oMDzDHK/g5HkrUmVlNYmPDvOLuJV0/vY7YnaCHtQ2eExGRiAjjKiMjdofpYnd/rJRmU4DrgquNTgV2euzOTRERiYgwziGcAVwL5JjZvOC5XxOMB+PuzxAbsGwIsTtU9xK7G1VERI5CYaGTdzCfBnVqJmT9cQeCu39EbMTKI7Vx4Kfx9iUiUlXlbs5j7MRs0urU4MURp1DsT1mEItqny0VEqrhDBYU8lZnLkMc/JHdLHhf3aZ2wvjT8tYhIRC1Yt5PbJmSzaMMuvte7Ffde0ov0+rXLfmEFKRBERCJm/6ECHn9vOeM+WEGTerV45od9GXxCRf+8RfkpEEREIuTLVdsZMyGbFVv38P1+bbljSE8a1k3MSeTiFAgiIhGQdyCfR99dwsuffkPbxsfxt5sGcGbXZkmtQYEgIpJimUs3c8ekHDbs2s8NZ3TkVxd0o26t5H88KxBERFLk2z0HuX/qIib9cx1dmqcx4ZbT6duhccrqUSCIiCSZuzMtZyP3TFnAjr2H+NnZXfjp2V2oXaN6SutSIIiIJNHmXfu5c/ICZizaRO82DXn5xgH0bN0g1WUBCgQRkaRwd/4xZy33v72Ig/mF3H5hd246sxM1qkfn/mAFgohIgq3ZvpfbJ+XwUe5W+ndqwsOX9aZzelqqy/o3CgQRkQQpKHRe+mQVv52+lOrVjPuHncAP+renWrXwxyEKgwJBRCQBlm/azZiJ2Xy1egcDu6Xz4KW9ad3ouFSXdUQKBBGREB0qKOSZrK/539m51KtdnT9edSJDT2ydkNFJw6ZAEBEJSc7anfxqwnyWbNzNRX1ig9E1S0vcYHRhUyCIiMRp/6EC/jBrGc9+sIJmabUZd21fzu+V+MHowqZAEBGJw+crtjF2Ug4rt+7hqn7t+PX3etDwuOQMRhc2BYKISAXs3n+IR95dwt8+W037JnV5ZeQAzuiS3MHowhZKIJjZC8BFwGZ3P6GE5QOBN4GVwVOT3P2+MPoWEUm2zCWb+fUbOWzatZ+RZ3biv8/PSMlgdGEL6x38BXgSePkIbT5094tC6k9EJOl2H3Rufe2fTJ63nq7N03j6x6dzUvvUDUYXtlACwd0/MLOOYaxLRCRq3J23sjdwx4d72V+4j5+f05WfDDo+5YPRhS2Z+zinmdl8YD3wS3dfmMS+RUQqZOPO2GB0sxZvolPDavzphjPo3jIag9GFzdw9nBXF9hCmlnIOoQFQ6O55ZjYEeNzdu5aynlHAKID09PS+48ePD6W+RMnLyyMtLXpjkhSnOsOlOsMVxTrdnffX5vP60oMUFMKlXWtxRrMDNKgfrTqLGzRo0Fx371ehF7t7KA+gI7CgnG1XAc3KapeRkeFRl5mZmeoSykV1hkt1hitqda7amufD//ypdxgz1a/68ye+ckueu0evzpIAc7yCn+NJOWRkZi2BTe7uZtYfqAZsS0bfIiLlVVDovPjxSn43Yyk1q1Xjoct6M/yUdpVi2IkwhHXZ6avAQKCZma0F7gFqArj7M8AVwI/NLB/YBwwPkkxEJBKWbtzNbROzmb9mB+f2aM4Dw3rTsmGdVJeVVGFdZXR1GcufJHZZqohIpBzML+TprFyeysylfp2aPHH1SVzcp1WV2SsoqvLfSSEiUkHz1uxgzIRslm7azSXfac09F/ekaSUajC5sCgQRqXL2HSzg9zOW8sLHK2levw7PX9+Pc3q0SHVZKadAEJEq5ZOvtzJ2Yg6rt+/lmgHtGXthdxrUqZyD0YVNgSAiVcKu/Yd4aNpiXv1iDR2b1uXVH53Kacc3TXVZkaJAEJFj3sxFm7hzcg5bdh/g5u925tZzMziu1rE17EQYFAgicszamneAe6csZGr2Brq3rM+z1/WjT9tGqS4rshQIInLMcXfenLee37y1kLwD+fz3eRncctbx1KpRLdWlRZoCQUSOKet37OPOyQuYvWQzJ7ZrxKNX9CGjRf1Ul1UpKBBE5JhQWOi8+uVqHpq2hIJC566LejLi9I5Ur1b1bjCrKAWCiFR6K7fuYezEbD5fuZ0zujTloUv70L5p3VSXVekoEESk0sovKOSFj1fy+xnLqFWjGo9e3ocr+7WtksNOhEGBICKV0uINuxgzMZvstTs5r2cLHhh2Ai0aVK3B6MKmQBCRSuVAfgFPzc7l6ayvaVS3Jk9dczJDerfUXkEIFAgiUml8tfpbxkzIZvnmPC47qQ13XdSTxvVqpbqsY4YCQUQib+/BfH43fRkvfrKSVg3q8OINpzCoW/NUl3XMUSCISKR9nLuVsZOyWbN9H9ee2oHbBnejvgajSwgFgohE0s59h3jw7cW8PmcNnZrV4/VRpzKgswajSyQFgohEzvSFG7lr8gK27TnILWcdz63ndqVOTQ1Gl2gKBBGJjC27Y4PRvZ2zgR6tGvD89afQu23DVJdVZYQSCGb2AnARsNndTyhhuQGPA0OAvcAId/8qjL5FpPJzdyZ9tZb7pi5i74ECfnVBN0Z9tzM1q2swumQKaw/hL8CTwMulLL8Q6Bo8BgB/Cn6KSBW3bsc+Hpt7gJyt8+nboTGPXN6HLs3TUl1WlRRKILj7B2bW8QhNhgIvu7sDn5lZIzNr5e4bwuhfRCqfwkLnlc+/4eF3lpBfUMC9F/fkutM6Uk2D0aWMxT6jQ1hRLBCmlnLIaCrwsLt/FMy/B4xx9zkltB0FjAJIT0/vO378+FDqS5S8vDzS0qL/bUZ1hkt1xmdDXiEvLjzAsm8L6dW0Gt/vVECHZtGrs7iobs+iBg0aNNfd+1XktZE7qezu44BxAN26dfOBAwemtqAyZGVlEfUaQXWGTXVWTH5BIeM+XMEfP1tOnRrV+O0VJ3BF37a8//77kaqzNFHbnmFLViCsA9oVmW8bPCciVcTC9TsZMzGbBet2MbhXS+4b1ovm9TUYXZQkKxCmAKPN7DViJ5N36vyBSNWw/1AB/zt7Oc+8v4LGdWvxpx+czIW9W6W6LClBWJedvgoMBJqZ2VrgHqAmgLs/A0wjdslpLrHLTm8Io18Riba532zntgnZfL1lD5ef3Ja7LupBo7oajC6qwrrK6Ooyljvw0zD6EpHo23Mgn99OX8pLn66idcPjeOnG/pyVkZ7qsqQMkTupLCKV2wfLtnD7pBzW79zHdad24FeDu5NWWx81lYF+SyISip17D3H/24uYMHctndPrMf7m0zilY5NUlyVHQYEgInF7d8EG7npzIdv3HOSng47nP8/WYHSVkQJBRCps8+793PPmQt5ZsJFerRvwlxtOoVdrDUZXWSkQROSouTsTv1rH/VMXse9QAbcN7saP/kOD0VV2CgQROSprtu/l12/k8OHyrfTr0JiHNRjdMUOBICLlUljovPzpKh6dvhQD7hvaix8O6KDB6I4hCgQRKVPu5t2MmZjD3G++5ayMdP7n0hNo27huqsuSkCkQRKRUhwoKGffBCh6ftZy6tavz2Pe/w6UntSH2N6/kWKNAEJESLVi3k9smZLNowy6+16cV917ci/T6tVNdliSQAkFE/sX+QwX8cdZynv1wBU3q1eLP1/blgl4tU12WJIECQUT+zxcrtzN2YjYrtu7hqn7t+PWQHjSsWzPVZUmSKBBEhLwD+TzyzhL++tk3tG18HH+7aQBndm2W6rIkyRQIIlVc5tLN3DEphw279nPjGZ345QUZ1K2lj4aqSL91kSrq2z0HuX/qIib9cx1dmqcx4ZbT6duhcarLkhRSIIhUMe7OtJyN3DNlATv2HuJnZ3fhp2d3oXYNDUZX1SkQRKqQTbv2c9fkBcxYtInebRry15sG0KNVg1SXJRGhQBCpAtyd8XPW8MDbizmYX8jtF3bnpjM7UUOD0UkRofxrMLPBZrbUzHLNbGwJy0eY2RYzmxc8RobRr4iUbfW2vfzw+c8ZMzGHHq0a8O6t3+Xms45XGMi/iXsPwcyqA08B5wFrgS/NbIq7LyrW9HV3Hx1vfyJSPgWFzvRVh5j83gdUr2Y8MOwErunfXoPRSanCOGTUH8h19xUAZvYaMBQoHggikiTLN+3mtonZ/HP1Qc7u3pwHhp1A60bHpbosiThz9/hWYHYFMNjdRwbz1wIDiu4NmNkI4CFgC7AM+C93X1PK+kYBowDS09P7jh8/Pq76Ei0vL4+0tOiPBa86wxXVOvMLnbdXHOKtrw9RpwZc3skZ2Kle5Aeji+r2LK4y1Dlo0KC57t6vQi9297gewBXAc0XmrwWeLNamKVA7mL4ZmF2edWdkZHjUZWZmprqEclGd4YpinfNWf+sX/OF97zBmqo/++1e+dff+SNZZEtUZHmCOV/DzPIxDRuuAdkXm2wbPFQ2dbUVmnwMeDaFfEQH2HSzgj7OW8eyHK0ivX5tnr+vHeT1bpLosqYTCCIQvga5m1olYEAwHrinawMxaufuGYPYSYHEI/YpUeZ+t2MbYidms2raXq/u34/YhPWhQR4PRScXEHQjunm9mo4HpQHXgBXdfaGb3Edt1mQL8zMwuAfKB7cCIePsVqcp27z/Ew+8s4ZXPV9O+SV3+PnIAp3fRYHQSn1BuTHP3acC0Ys/dXWT6duD2MPoSqepmL9nEHW8sYNOu/Yw8sxO/OL8bx9XSsBMSP92pLFJJbN9zkPveWsjkeevJaJHG0z84nZPaazA6CY8CQSTi3J23sjdw75SF7N5/iFvP7cpPBnahVg3daSzhUiCIRNjGnfu5c/ICZi3exHfaNeLRy/vQrWX9VJclxygFgkgEuTuvfbmGB99ezKHCQu78Xg9uOKMT1TXshCSQAkEkYr7ZtoexE3P4dMU2TuvclIcv702HpvVSXZZUAQoEkYgoKHRe/Hglv5uxlJrVqvHgpb25un+7yA87IccOBYJIBCzdGBuMbv6aHZzbozkPDOtNy4Z1Ul2WVDEKBJEUOphfyFOZuTydlUv9OjV54uqTuLhPK+0VSEooEERSZN6aHdw2YT7LNuUx7MTW3H1xL5rUq5XqsqQKUyCIJNneg/k8NmMZL3y8khYN6vDCiH6c3V2D0UnqKRBEkuiT3K2MnZTD6u17uWZAe26/sDv1NRidRIQCQSQJdu47xEPTFvPal2vo2LQur406lVM7N011WSL/QoEgkmAzF23izsk5bNl9gJu/25lbz83QYHQSSQoEkQTZlneAe99axFvz19O9ZX2eva4ffdo2SnVZIqVSIIiEzN2ZMn89905ZyJ4DBfzivAxuPut4DUYnkadAEAnR+h37uHPyAmYv2cxJ7WOD0XVtocHopHJQIIiEoLDQ+fsXq3n4nSUUFDp3XdSTEad31GB0UqkoEETitHLrHsZOzObzlds5o0tTHrq0D+2b1k11WSJHLZRAMLPBwOPE/qbyc+7+cLHltYGXgb7ANuAqd18VRt8iqZJfUMjzH63ksZnLqFWjGo9e3ocr+7XVsBNSacUdCGZWHXgKOA9YC3xpZlPcfVGRZjcB37p7FzMbDjwCXBVv3yKpsnpXAZc+/Qk563ZyXs8WPDDsBFo00GB0UrmFsYfQH8h19xUAZvYaMBQoGghDgXuD6QnAk2Zm7u5HWvHmvc6P/zY3hBITZ8uW/by+Nto1guoM06GCQjKX7KdxvUKeuuZkhvRuqb0COSaEEQhtgDVF5tcCA0pr4+75ZrYTaApsLb4yMxsFjAKo06IT2as2hVBi4hQWFrJhT7RrBNUZtv7NnR/0qkG97Ut5//2lqS6nVHl5eWRlZaW6jDKpzmiI3Elldx8HjAPo1q2bf3znkBRXdGRZWVkMHDgw1WWUSXWGS3WGS3VGQxh3yqwD2hWZbxs8V2IbM6sBNCR2cllERCIijED4EuhqZp3MrBYwHJhSrM0U4Ppg+gpgdlnnD0REJLniPmQUnBMYDUwndtnpC+6+0MzuA+a4+xTgeeCvZpYLbCcWGiIiEiGhnENw92nAtGLP3V1kej9wZRh9iYhIYmi0LRERARQIIiISUCCIiAigQBARkYACQUREAAWCiIgEFAgiIgIoEEREJKBAEBERQIEgIiIBBYKIiAAKBBERCSgQREQEUCCIiEhAgSAiIoACQUREAgoEEREBFAgiIhKIKxDMrImZzTSz5cHPxqW0KzCzecFjSjx9iohIYsS7hzAWeM/duwLvBfMl2efuJwaPS+LsU0REEiDeQBgKvBRMvwQMi3N9IiKSIubuFX+x2Q53bxRMG/Dt4fli7fKBeUA+8LC7Tz7COkcBowDS09P7jh8/vsL1JUNeXh5paWmpLqNMqjNcqjNcqjM8gwYNmuvu/Sr0Ync/4gOYBSwo4TEU2FGs7belrKNN8LMzsAo4vqx+3Z2MjAyPuszMzFSXUC6qM1yqM1yqMzzAHC/H52tJjxrlCIxzS1tmZpvMrJW7bzCzVsDmUtaxLvi5wsyygJOAr8uRVyIikiTxnkOYAlwfTF8PvFm8gZk1NrPawXQz4AxgUZz9iohIyOINhIeB88xsOXBuMI+Z9TOz54I2PYA5ZjYfyCR2DkGBICISMWUeMjoSd98GnFPC83OAkcH0J0DvePoREZHE053KIiICKBBERCSgQBAREUCBICIiAQWCiIgACgQREQkoEEREBFAgiIhIQIEgIiKAAkFERAIKBBERARQIIiISUCCIiAigQBARkYACQUREAAWCiIgEFAgiIgIoEEREJBBXIJjZlWa20MwKzazfEdoNNrOlZpZrZmPj6VNERBIj3j2EBcBlwAelNTCz6sBTwIVAT+BqM+sZZ78iIhKyGvG82N0XA5jZkZr1B3LdfUXQ9jVgKLAonr5FRCRc5u7xr8QsC/ilu88pYdkVwGB3HxnMXwsMcPfRpaxrFDAKID09ve/48ePjri+R8vLySEtLS3UZZVKd4VKd4VKd4Rk0aNBcdy/1EP6RlLmHYGazgJYlLLrD3d+sSKdH4u7jgHEA3bp184EDB4bdRaiysrKIeo2gOsOmOsOlOqOhzEBw93Pj7GMd0K7IfNvgORERiZBkXHb6JdDVzDqZWS1gODAlCf2KiMhRiPey00vNbC1wGvC2mU0Pnm9tZtMA3D0fGA1MBxYD4919YXxli4hI2OK9yugN4I0Snl8PDCkyPw2YFk9fIiKSWLpTWUREAAWCiIgEFAgiIgIoEEREJKBAEBERQIEgIiIBBYKIiAAKBBERCSgQREQEUCCIiEhAgSAiIoACQUREAgoEEREBFAgiIhJQIIiICKBAEBGRgAJBREQABYKIiATi/ZvKV5rZQjMrNLN+R2i3ysxyzGyemc2Jp08REUmMuP6mMrAAuAz4cznaDnL3rXH2JyIiCRJXILj7YgAzC6caERFJmWSdQ3BghpnNNbNRSepTRESOgrn7kRuYzQJalrDoDnd/M2iTBfzS3Us8P2Bmbdx9nZk1B2YC/+nuH5TSdhQwCiA9Pb3v+PHjy/teUiIvL4+0tLRUl1Em1Rku1Rku1RmeQYMGzXX3Us/pHpG7x/0AsoB+5Wx7L7HwKLNtRkaGR11mZmaqSygX1Rku1Rku1RkeYI5X8LM84YeMzKyemdU/PA2cT+xktIiIREi8l51eamZrgdOAt81sevB8azObFjRrAXxkZvOBL4C33f3dePoVEZHwxXuV0RvAGyU8vx4YEkyvAL4TTz8iIpJ4ulNZREQABYKIiAQUCCIiAigQREQkoEAQERFAgSAiIgEFgoiIAAoEEREJKBBERARQIIiISECBICIigAJBREQCCgQREQEUCCIiElAgiIgIoEAQEZGAAkFERAAFgoiIBBQIIiICxBkIZvZbM1tiZtlm9oaZNSql3WAzW2pmuWY2Np4+RUQkMeLdQ5gJnODufYBlwO3FG5hZdeAp4EKgJ3C1mfWMs18REQlZXIHg7jPcPT+Y/QxoW0Kz/kCuu69w94PAa8DQePoVEZHw1QhxXTcCr5fwfBtgTZH5tcCA0lZiZqOAUcHsATNbEFqFidEM2JrqIspBdYZLdYZLdYanW0VfWGYgmNksoGUJi+5w9zeDNncA+cArFS3kMHcfB4wL1jvH3fvFu85Eqgw1guoMm+oMl+oMj5nNqehrywwEdz+3jM5HABcB57i7l9BkHdCuyHzb4DkREYmQeK8yGgzcBlzi7ntLafYl0NXMOplZLWA4MCWefkVEJHzxXmX0JFAfmGlm88zsGQAza21m0wCCk86jgenAYmC8uy8s5/rHxVlfMlSGGkF1hk11hkt1hqfCNVrJR3lERKSq0Z3KIiICKBBERCQQqUCoDENhmNmVZrbQzArNrNTLz8xslZnlBOdWKnwZWEUdRZ0pHVbEzJqY2UwzWx78bFxKu4JgW84zs6RdlFDW9jGz2mb2erD8czPrmKzaitVRVp0jzGxLkW04MgU1vmBmm0u7t8hingjeQ7aZnZzsGoM6yqpzoJntLLIt705Bje3MLNPMFgX/z39eQpuj357uHpkHcD5QI5h+BHikhDbVga+BzkAtYD7QM4k19iB240cW0O8I7VYBzVK4LcusM9XbMqjhUWBsMD22pN95sCwvBduwzO0D/AR4JpgeDrwe0TpHAE8mu7ZiNXwXOBlYUMryIcA7gAGnAp9HtM6BwNQUb8tWwMnBdH1iQwcV/50f9faM1B6CV4KhMNx9sbsvTVZ/FVXOOqMwrMhQ4KVg+iVgWJL7P5LybJ+i9U8AzjEzS2KNEI3fY5nc/QNg+xGaDAVe9pjPgEZm1io51f1/5agz5dx9g7t/FUzvJnYFZ5tizY56e0YqEIq5kVi6FVfSUBjFN0QUODDDzOYGw3FEURS2ZQt33xBMbwRalNKujpnNMbPPzCxZoVGe7fN/bYIvMzuBpkmproQaAqX9Hi8PDh1MMLN2JSxPtSj8eyyv08xsvpm9Y2a9UllIcJjyJODzYouOenuGOZZRuSR7KIyKKE+N5XCmu68zs+bE7tNYEnzzCE1IdSbckeosOuPubmalXQfdIdienYHZZpbj7l+HXesx7C3gVXc/YGY3E9urOTvFNVVWXxH795hnZkOAyUDXVBRiZmnAROBWd98V7/qSHgheCYa7Mtu4AAACZUlEQVTCKKvGcq5jXfBzs5m9QWy3PtRACKHOpAwrcqQ6zWyTmbVy9w3B7uzmUtZxeHuuMLMsYt+IEh0I5dk+h9usNbMaQENgW4LrKq7MOt29aE3PETt3EzWVYpiboh+87j7NzJ42s2buntRB78ysJrEweMXdJ5XQ5Ki3Z6QOGdkxMhSGmdUzs/qHp4mdLI/iqK1R2JZTgOuD6euBf9uzMbPGZlY7mG4GnAEsSkJt5dk+Reu/AphdyheZRCqzzmLHji8hdsw5aqYA1wVXx5wK7CxyODEyzKzl4fNEZtaf2OdoUr8EBP0/Dyx298dKaXb02zOVZ8pLOHOeS+yY17zgcfjqjdbAtGJnz5cR+4Z4R5JrvJTYsbgDwCZgevEaiV3tMT94LEx2jeWtM9XbMui/KfAesByYBTQJnu8HPBdMnw7kBNszB7gpifX92/YB7iP2pQWgDvCP4N/uF0DnZG/Dctb5UPBvcT6QCXRPQY2vAhuAQ8G/zZuAW4BbguVG7I9pfR38nku9ii/FdY4usi0/A05PQY1nEjtPmV3k83JIvNtTQ1eIiAgQsUNGIiKSOgoEEREBFAgiIhJQIIiICKBAEBGRgAJBREQABYKIiAQUCCJHwcxuKTIO/kozy0x1TSJh0Y1pIhUQjCMzG3jU3d9KdT0iYdAegkjFPE5s3CKFgRwzkj7aqUhlF4zI24HYmDYixwwdMhI5CmbWl9jfEvgPd/821fWIhEmHjESOzmigCZAZnFh+LtUFiYRFewgiIgJoD0FERAIKBBERARQIIiISUCCIiAigQBARkYACQUREAAWCiIgE/h8n8ijWtV6/qgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "\"\"\"The sigmoid function (or the logistic curve) is a \n",
    "function that takes any real number, z, and outputs a number (0,1).\n",
    "It is useful in neural networks for assigning weights on a relative scale.\n",
    "The value z is the weighted sum of parameters involved in the learning algorithm.\"\"\"\n",
    "\n",
    "import numpy\n",
    "import matplotlib.pyplot as plt\n",
    "import math as mt\n",
    "\n",
    "z = numpy.arange(-5, 5, .1)\n",
    "sigma_fn = numpy.vectorize(lambda z: 1/(1+numpy.exp(-z)))\n",
    "sigma = sigma_fn(z)\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.plot(z, sigma)\n",
    "ax.set_ylim([-0.1, 1.1])\n",
    "ax.set_xlim([-5,5])\n",
    "ax.grid(True)\n",
    "ax.set_xlabel('z')\n",
    "ax.set_title('sigmoid function')\n",
    "\n",
    "plt.show()\n",
    "\n",
    "\"\"\"Step Function\"\"\"\n",
    "z = numpy.arange(-5, 5, .02)\n",
    "step_fn = numpy.vectorize(lambda z: 1.0 if z >= 0.0 else 0.0)\n",
    "step = step_fn(z)\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.plot(z, step)\n",
    "ax.set_ylim([-0.5, 1.5])\n",
    "ax.set_xlim([-5,5])\n",
    "ax.grid(True)\n",
    "ax.set_xlabel('z')\n",
    "ax.set_title('step function')\n",
    "\n",
    "plt.show()\n",
    "\n",
    "\"\"\"Sine Function\"\"\"\n",
    "z = numpy.arange(-2*mt.pi, 2*mt.pi, 0.1)\n",
    "t = numpy.sin(z)\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.plot(z, t)\n",
    "ax.set_ylim([-1.0, 1.0])\n",
    "ax.set_xlim([-2*mt.pi,2*mt.pi])\n",
    "ax.grid(True)\n",
    "ax.set_xlabel('z')\n",
    "ax.set_title('sine function')\n",
    "\n",
    "plt.show()\n",
    "\n",
    "\"\"\"Plots a graph of the squashing function used by a rectified linear\n",
    "unit\"\"\"\n",
    "z = numpy.arange(-2, 2, .1)\n",
    "zero = numpy.zeros(len(z))\n",
    "y = numpy.max([zero, z], axis=0)\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.plot(z, y)\n",
    "ax.set_ylim([-2.0, 2.0])\n",
    "ax.set_xlim([-2.0, 2.0])\n",
    "ax.grid(True)\n",
    "ax.set_xlabel('z')\n",
    "ax.set_title('Rectified linear unit')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- !split  -->\n",
    "## Setting up a Multi-layer perceptron model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import optimize\n",
    "\n",
    "class Neural_Network(object):\n",
    "    def __init__(self, Lambda=0):        \n",
    "        #Define Hyperparameters\n",
    "        self.inputLayerSize = 2\n",
    "        self.outputLayerSize = 1\n",
    "        self.hiddenLayerSize = 3\n",
    "        \n",
    "        #Weights (parameters)\n",
    "        self.W1 = np.random.randn(self.inputLayerSize,self.hiddenLayerSize)\n",
    "        self.W2 = np.random.randn(self.hiddenLayerSize,self.outputLayerSize)\n",
    "        \n",
    "        #Regularization Parameter:\n",
    "        self.Lambda = Lambda\n",
    "        \n",
    "    def forward(self, X):\n",
    "        #Propogate inputs though network\n",
    "        self.z2 = np.dot(X, self.W1)\n",
    "        self.a2 = self.sigmoid(self.z2)\n",
    "        self.z3 = np.dot(self.a2, self.W2)\n",
    "        yHat = self.sigmoid(self.z3) \n",
    "        return yHat\n",
    "        \n",
    "    def sigmoid(self, z):\n",
    "        #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "        return 1/(1+np.exp(-z))\n",
    "    \n",
    "    def sigmoidPrime(self,z):\n",
    "        #Gradient of sigmoid\n",
    "        return np.exp(-z)/((1+np.exp(-z))**2)\n",
    "    \n",
    "    def costFunction(self, X, y):\n",
    "        #Compute cost for given X,y, use weights already stored in class.\n",
    "        self.yHat = self.forward(X)\n",
    "        J = 0.5*sum((y-self.yHat)**2)/X.shape[0] + (self.Lambda/2)*(np.sum(self.W1**2)+np.sum(self.W2**2))\n",
    "        return J\n",
    "        \n",
    "    def costFunctionPrime(self, X, y):\n",
    "        #Compute derivative with respect to W and W2 for a given X and y:\n",
    "        self.yHat = self.forward(X)\n",
    "        \n",
    "        delta3 = np.multiply(-(y-self.yHat), self.sigmoidPrime(self.z3))\n",
    "        #Add gradient of regularization term:\n",
    "        dJdW2 = np.dot(self.a2.T, delta3)/X.shape[0] + self.Lambda*self.W2\n",
    "        \n",
    "        delta2 = np.dot(delta3, self.W2.T)*self.sigmoidPrime(self.z2)\n",
    "        #Add gradient of regularization term:\n",
    "        dJdW1 = np.dot(X.T, delta2)/X.shape[0] + self.Lambda*self.W1\n",
    "        \n",
    "        return dJdW1, dJdW2\n",
    "    \n",
    "    #Helper functions for interacting with other methods/classes\n",
    "    def getParams(self):\n",
    "        #Get W1 and W2 Rolled into vector:\n",
    "        params = np.concatenate((self.W1.ravel(), self.W2.ravel()))\n",
    "        return params\n",
    "    \n",
    "    def setParams(self, params):\n",
    "        #Set W1 and W2 using single parameter vector:\n",
    "        W1_start = 0\n",
    "        W1_end = self.hiddenLayerSize*self.inputLayerSize\n",
    "        self.W1 = np.reshape(params[W1_start:W1_end], \\\n",
    "                             (self.inputLayerSize, self.hiddenLayerSize))\n",
    "        W2_end = W1_end + self.hiddenLayerSize*self.outputLayerSize\n",
    "        self.W2 = np.reshape(params[W1_end:W2_end], \\\n",
    "                             (self.hiddenLayerSize, self.outputLayerSize))\n",
    "        \n",
    "    def computeGradients(self, X, y):\n",
    "        dJdW1, dJdW2 = self.costFunctionPrime(X, y)\n",
    "        return np.concatenate((dJdW1.ravel(), dJdW2.ravel()))\n",
    "    \n",
    "        \n",
    "class trainer(object):\n",
    "    def __init__(self, N):\n",
    "        #Make Local reference to network:\n",
    "        self.N = N\n",
    "        \n",
    "    def callbackF(self, params):\n",
    "        self.N.setParams(params)\n",
    "        self.J.append(self.N.costFunction(self.X, self.y))\n",
    "        self.testJ.append(self.N.costFunction(self.testX, self.testY))\n",
    "        \n",
    "    def costFunctionWrapper(self, params, X, y):\n",
    "        self.N.setParams(params)\n",
    "        cost = self.N.costFunction(X, y)\n",
    "        grad = self.N.computeGradients(X,y)\n",
    "        return cost, grad\n",
    "        \n",
    "    def train(self, trainX, trainY, testX, testY):\n",
    "        #Make an internal variable for the callback function:\n",
    "        self.X = trainX\n",
    "        self.y = trainY\n",
    "        \n",
    "        self.testX = testX\n",
    "        self.testY = testY\n",
    "\n",
    "        #Make empty list to store training costs:\n",
    "        self.J = []\n",
    "        self.testJ = []\n",
    "        \n",
    "        params0 = self.N.getParams()\n",
    "\n",
    "        options = {'maxiter': 200, 'disp' : True}\n",
    "        _res = optimize.minimize(self.costFunctionWrapper, params0, jac=True, method='BFGS', \\\n",
    "                                 args=(trainX, trainY), options=options, callback=self.callbackF)\n",
    "\n",
    "        self.N.setParams(_res.x)\n",
    "        self.optimizationResults = _res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- !split  -->\n",
    "## Two-layer Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output after training:  [[6.55109972e-03 9.93684857e-01 9.93925710e-01 6.62304973e-03]\n",
      " [1.71082162e-03 9.97516440e-01 9.97766376e-01 1.82685927e-03]\n",
      " [2.05800960e-03 9.98268211e-01 9.97548919e-01 1.77362990e-03]\n",
      " [5.35659849e-04 9.99320839e-01 9.99100767e-01 4.87503198e-04]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "#sigmoid\n",
    "def nonlin(x, deriv=False):\n",
    "    if (deriv==True):\n",
    "        return x*(1-x)\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "#input data\n",
    "x=np.array([[0,0,1],[0,1,1],[1,0,1],[1,1,1]])\n",
    "\n",
    "#output data\n",
    "y=np.array([0,1,1,0]).T\n",
    "\n",
    "#seed random numbers to make calculation\n",
    "np.random.seed(1)\n",
    "\n",
    "#initialize weights with mean=0\n",
    "syn0=2*np.random.random((3,4))-1\n",
    "\n",
    "for iter in range(10000):\n",
    "    #forward propogation\n",
    "    l0=x\n",
    "    l1=nonlin(np.dot(l0,syn0))\n",
    "    l1_error=y-l1\n",
    "    #multiply error by slope of sigmoid at values of l1\n",
    "    l1_delta=l1_error*nonlin(l1,True)\n",
    "    #update weights\n",
    "    syn0+=np.dot(l0.T, l1_delta)\n",
    "    \n",
    "print(\"Output after training: \",l1 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "class Network(object):\n",
    "    \n",
    "    def _init_(self, sizes):\n",
    "        self.num_layers=len(sizes)\n",
    "        self.sizes=sizes\n",
    "        self.biases=[np.random.randn(y,1) for y in sizes[1:]]\n",
    "        self.weights=[np.random.randn(y,x) for x,y in zip(sizes[:-1], sizes[1:])]\n",
    "\n",
    "#sizes is the number of neurons in each layer\n",
    "#for example, say n_1st_layer=3, n_2nd_layer=3, n_3rd_layer=1, then net=Network([3,3,1])\n",
    "\n",
    "#The biases and weights are initialized randomly, using Gaussian distributions of mean=0, stdev=1\n",
    "#z is a vector (or a np.array)\n",
    "\n",
    "    def feedforward(self,a):\n",
    "        #returns output w/ 'a' as an input\n",
    "        for b, w in zip(self.biases, self.weights):\n",
    "            a=sigmoid(np.dot(w,b)+b)\n",
    "        return a\n",
    "    \n",
    "#Apply a Stochastic Gradient Descent (SGD) method:\n",
    "    def SGD(self, training_data, epochs, mini_batch_size, eta, test_data=None):\n",
    "        \"\"\"Trains network using batches incorporating SGD. The network will be evaluated against the\n",
    "        test data after each epoch, with partial progress being printed out (this is useful for tracking,\n",
    "        but slows the process.)\"\"\"\n",
    "        if test_data: n_test=len(test_data)\n",
    "        n=len(training_data)\n",
    "        for j in xrange(epochs):\n",
    "            random.shuffle(training_data)\n",
    "            mini_batches=[training_data[k:k+mini_batch_size] for k in xrange(o,n,mini_batch_size)]\n",
    "            for mini_batch in mini_batches:\n",
    "                self.update_mini_batch(mini_batch, eta)\n",
    "            if test_data:\n",
    "                print (\"Epoch {0}: {1}/{2}\".format(j, self.evaluate(test_data), n_test))\n",
    "            else:\n",
    "                print (\"Epoch {0} complete\".format(j))\n",
    "            \n",
    "        \n",
    "    def update_mini_batch(self, mini_batch, eta):\n",
    "        #updates w and b using backpropagation to a single mini batch. eta is the learning rate.\"\n",
    "        nabla_b=[np.zeros(b.shape) for b in self.biases]\n",
    "        nabla_w=[np.zeros(w.shape) for w in self.weights]\n",
    "        for x,y in mini_batch:\n",
    "            delta_nabla_b, delta_nabla_w=self.backprop(x,y)\n",
    "            nabla_b=[nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]\n",
    "            nabla_w=[nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]\n",
    "        self.weights=[w-(eta/len(mini_batch))*nw for w, nw in zip(self.weights, nabla_w)]\n",
    "        self.biases=[b-(eta/len(mini_batch))*nb for b, nb in zip(self.biases, nabla_b)]\n",
    "        \n",
    "    def backprop(self, x, y):\n",
    "        \"\"\"Return a tuple ``(nabla_b, nabla_w)`` representing the\n",
    "        gradient for the cost function C_x.  ``nabla_b`` and\n",
    "        ``nabla_w`` are layer-by-layer lists of numpy arrays, similar\n",
    "        to ``self.biases`` and ``self.weights``.\"\"\"\n",
    "        nabla_b = [np.zeros(b.shape) for b in self.biases]\n",
    "        nabla_w = [np.zeros(w.shape) for w in self.weights]\n",
    "        # feedforward\n",
    "        activation = x\n",
    "        activations = [x] # list to store all the activations, layer by layer\n",
    "        zs = [] # list to store all the z vectors, layer by layer\n",
    "        for b, w in zip(self.biases, self.weights):\n",
    "            z = np.dot(w, activation)+b\n",
    "            zs.append(z)\n",
    "            activation = sigmoid(z)\n",
    "            activations.append(activation)\n",
    "        # backward pass\n",
    "        delta = self.cost_derivative(activations[-1], y) * \\\n",
    "            sigmoid_prime(zs[-1])\n",
    "        nabla_b[-1] = delta\n",
    "        nabla_w[-1] = np.dot(delta, activations[-2].transpose())\n",
    "        # Note that the variable l in the loop below is used a little\n",
    "        # differently to the notation in Chapter 2 of the book.  Here,\n",
    "        # l = 1 means the last layer of neurons, l = 2 is the\n",
    "        # second-last layer, and so on.  It's a renumbering of the\n",
    "        # scheme in the book, used here to take advantage of the fact\n",
    "        # that Python can use negative indices in lists.\n",
    "        for l in xrange(2, self.num_layers):\n",
    "            z = zs[-l]\n",
    "            sp = sigmoid_prime(z)\n",
    "            delta = np.dot(self.weights[-l+1].transpose(), delta) * sp\n",
    "            nabla_b[-l] = delta\n",
    "            nabla_w[-l] = np.dot(delta, activations[-l-1].transpose())\n",
    "        return (nabla_b, nabla_w)\n",
    "\n",
    "    def evaluate(self, test_data):\n",
    "        \"\"\"Return the number of test inputs for which the neural\n",
    "        network outputs the correct result. Note that the neural\n",
    "        network's output is assumed to be the index of whichever\n",
    "        neuron in the final layer has the highest activation.\"\"\"\n",
    "        test_results = [(np.argmax(self.feedforward(x)), y)\n",
    "                        for (x, y) in test_data]\n",
    "        return sum(int(x == y) for (x, y) in test_results)\n",
    "\n",
    "    def cost_derivative(self, output_activations, y):\n",
    "        \"\"\"Return the vector of partial derivatives \\partial C_x /\n",
    "        \\partial a for the output activations.\"\"\"\n",
    "        return (output_activations-y)\n",
    "    \n",
    "    \n",
    "    \n",
    "#Functions\n",
    "def sigmoid(z):\n",
    "        return 1.0/(1.0+np.exp(-z))\n",
    "\n",
    "def sigmoid_prime(z):\n",
    "    return sigmoid(z)*(1-sigmoid(z))\n",
    "\n",
    "network=Network()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'network' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-5-4ac16fae6ccb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     86\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 88\u001b[0;31m \u001b[0mnet\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnetwork\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mNetwork\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m784\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m30\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m30\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     89\u001b[0m \u001b[0mnet\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSGD\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtraining_data\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m30\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mtest_data\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtest_data\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'network' is not defined"
     ]
    }
   ],
   "source": [
    "# %load neural-networks-and-deep-learning/src/mnist_loader.py\n",
    "\"\"\"\n",
    "mnist_loader\n",
    "~~~~~~~~~~~~\n",
    "\n",
    "A library to load the MNIST image data.  For details of the data\n",
    "structures that are returned, see the doc strings for ``load_data``\n",
    "and ``load_data_wrapper``.  In practice, ``load_data_wrapper`` is the\n",
    "function usually called by our neural network code.\n",
    "\"\"\"\n",
    "\n",
    "#### Libraries\n",
    "# Standard library\n",
    "import pickle\n",
    "import gzip\n",
    "\n",
    "# Third-party libraries\n",
    "import numpy as np\n",
    "\n",
    "def load_data():\n",
    "    \"\"\"Return the MNIST data as a tuple containing the training data,\n",
    "    the validation data, and the test data.\n",
    "\n",
    "    The ``training_data`` is returned as a tuple with two entries.\n",
    "    The first entry contains the actual training images.  This is a\n",
    "    numpy ndarray with 50,000 entries.  Each entry is, in turn, a\n",
    "    numpy ndarray with 784 values, representing the 28 * 28 = 784\n",
    "    pixels in a single MNIST image.\n",
    "\n",
    "    The second entry in the ``training_data`` tuple is a numpy ndarray\n",
    "    containing 50,000 entries.  Those entries are just the digit\n",
    "    values (0...9) for the corresponding images contained in the first\n",
    "    entry of the tuple.\n",
    "\n",
    "    The ``validation_data`` and ``test_data`` are similar, except\n",
    "    each contains only 10,000 images.\n",
    "\n",
    "    This is a nice data format, but for use in neural networks it's\n",
    "    helpful to modify the format of the ``training_data`` a little.\n",
    "    That's done in the wrapper function ``load_data_wrapper()``, see\n",
    "    below.\n",
    "    \"\"\"\n",
    "    f = gzip.open('../data/mnist.pkl.gz', 'rb')\n",
    "    training_data, validation_data, test_data = cPickle.load(f)\n",
    "    f.close()\n",
    "    return (training_data, validation_data, test_data)\n",
    "\n",
    "def load_data_wrapper():\n",
    "    \"\"\"Return a tuple containing ``(training_data, validation_data,\n",
    "    test_data)``. Based on ``load_data``, but the format is more\n",
    "    convenient for use in our implementation of neural networks.\n",
    "\n",
    "    In particular, ``training_data`` is a list containing 50,000\n",
    "    2-tuples ``(x, y)``.  ``x`` is a 784-dimensional numpy.ndarray\n",
    "    containing the input image.  ``y`` is a 10-dimensional\n",
    "    numpy.ndarray representing the unit vector corresponding to the\n",
    "    correct digit for ``x``.\n",
    "\n",
    "    ``validation_data`` and ``test_data`` are lists containing 10,000\n",
    "    2-tuples ``(x, y)``.  In each case, ``x`` is a 784-dimensional\n",
    "    numpy.ndarry containing the input image, and ``y`` is the\n",
    "    corresponding classification, i.e., the digit values (integers)\n",
    "    corresponding to ``x``.\n",
    "\n",
    "    Obviously, this means we're using slightly different formats for\n",
    "    the training data and the validation / test data.  These formats\n",
    "    turn out to be the most convenient for use in our neural network\n",
    "    code.\"\"\"\n",
    "    tr_d, va_d, te_d = load_data()\n",
    "    training_inputs = [np.reshape(x, (784, 1)) for x in tr_d[0]]\n",
    "    training_results = [vectorized_result(y) for y in tr_d[1]]\n",
    "    training_data = zip(training_inputs, training_results)\n",
    "    validation_inputs = [np.reshape(x, (784, 1)) for x in va_d[0]]\n",
    "    validation_data = zip(validation_inputs, va_d[1])\n",
    "    test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]\n",
    "    test_data = zip(test_inputs, te_d[1])\n",
    "    return (training_data, validation_data, test_data)\n",
    "\n",
    "def vectorized_result(j):\n",
    "    \"\"\"Return a 10-dimensional unit vector with a 1.0 in the jth\n",
    "    position and zeroes elsewhere.  This is used to convert a digit\n",
    "    (0...9) into a corresponding desired output from the neural\n",
    "    network.\"\"\"\n",
    "    e = np.zeros((10, 1))\n",
    "    e[j] = 1.0\n",
    "    return e\n",
    "\n",
    "net=network.Network([784,30,30])\n",
    "net.SGD(training_data,30,10,3,test_data=test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}